Tuesday, 2016-03-29

*** IlyaG has joined #openstack-infra00:00
*** davideagnello has joined #openstack-infra00:01
Clintjeblair: trying to diff change 298389 makes gertty asplode00:02
*** IlyaG has quit IRC00:02
*** EricGonczer_ has quit IRC00:02
*** sputnik13 has joined #openstack-infra00:03
openstackgerritTony Breeds proposed openstack-infra/irc-meetings: Move OpenStack-Salt team meeting  https://review.openstack.org/29833500:03
openstackgerritMerged openstack-infra/irc-meetings: Update DefCore Meeting description  https://review.openstack.org/29841600:03
*** sflanigan has joined #openstack-infra00:06
*** Qiming has quit IRC00:06
openstackgerritMerged openstack-infra/irc-meetings: Move OpenStack-Salt team meeting  https://review.openstack.org/29833500:07
*** mylu has joined #openstack-infra00:07
*** resker has joined #openstack-infra00:08
*** mylu has quit IRC00:08
*** mylu has joined #openstack-infra00:09
openstackgerritKATO Tomoyuki proposed openstack-infra/project-config: Add release notes translation scripts  https://review.openstack.org/29704500:10
openstackgerritSachi King proposed openstack-infra/git-review: Fix no_git_dir UnboundLocalError in except block  https://review.openstack.org/29847300:10
*** IlyaG has joined #openstack-infra00:10
*** lucasagomes has quit IRC00:10
*** ociuhandu has quit IRC00:10
*** asettle has joined #openstack-infra00:10
*** lucasagomes has joined #openstack-infra00:11
*** fawadkhaliq has quit IRC00:12
*** esker has quit IRC00:12
*** ociuhandu has joined #openstack-infra00:12
*** fawadkhaliq has joined #openstack-infra00:13
*** IlyaG has quit IRC00:13
*** davideag_ has joined #openstack-infra00:15
*** davideagnello has quit IRC00:15
*** tphummel has quit IRC00:16
*** harlowja has quit IRC00:16
*** chuck_ has joined #openstack-infra00:17
*** kzaitsev_mb has joined #openstack-infra00:17
*** SumitNaiksatam has quit IRC00:19
openstackgerritClint Adams proposed openstack-infra/project-config: Add periodic job to autogenerate tempest plugins list  https://review.openstack.org/29151700:19
openstackgerritClint Adams proposed openstack-infra/project-config: Add periodic job to autogenerate grenade plugins list  https://review.openstack.org/29151400:19
*** maishsk_ has joined #openstack-infra00:23
*** amitgandhinz has joined #openstack-infra00:23
*** maishsk has quit IRC00:23
*** maishsk_ is now known as maishsk00:23
*** sarob has joined #openstack-infra00:23
*** salv-orlando has joined #openstack-infra00:24
*** kzaitsev_mb has quit IRC00:24
*** IlyaG has joined #openstack-infra00:26
*** gyee has quit IRC00:26
*** sridhar_ram has quit IRC00:27
*** LaT0rtue has quit IRC00:28
*** piet has joined #openstack-infra00:28
*** salv-orlando has quit IRC00:28
*** mtanino_ has joined #openstack-infra00:29
*** mtanino has quit IRC00:31
*** sputnik13 has quit IRC00:34
*** IlyaG has quit IRC00:35
*** sdake_ has joined #openstack-infra00:35
*** amitgandhinz has quit IRC00:36
*** roxanaghe has quit IRC00:37
*** sdake has quit IRC00:38
*** sarob has quit IRC00:40
*** angdraug has quit IRC00:40
*** fawadkhaliq has quit IRC00:40
*** fawadkhaliq has joined #openstack-infra00:40
*** achanda has joined #openstack-infra00:41
*** roxanaghe has joined #openstack-infra00:42
*** apoorvad has quit IRC00:43
*** Sukhdev has quit IRC00:44
*** Swami has quit IRC00:44
*** zeih has joined #openstack-infra00:45
*** tiswanso has joined #openstack-infra00:45
*** tiswanso has quit IRC00:46
*** achanda has quit IRC00:46
*** sdake_ has quit IRC00:47
*** tiswanso has joined #openstack-infra00:47
*** esker has joined #openstack-infra00:47
*** sdake has joined #openstack-infra00:47
*** armax has left #openstack-infra00:48
*** zeih has quit IRC00:49
*** akshai has joined #openstack-infra00:50
*** tphummel has joined #openstack-infra00:50
*** resker has quit IRC00:51
*** harlowja has joined #openstack-infra00:51
*** Sukhdev has joined #openstack-infra00:51
*** roxanaghe has quit IRC00:52
*** asettle has quit IRC00:54
*** tphummel has quit IRC00:54
*** _amrith_ is now known as amrith00:57
*** akshai_ has joined #openstack-infra00:58
*** NobodyCam has joined #openstack-infra00:58
*** eliqiao has quit IRC00:59
*** LaT0rtue has joined #openstack-infra00:59
*** eliqiao has joined #openstack-infra01:00
*** achanda has joined #openstack-infra01:01
*** akshai has quit IRC01:01
*** fawadkhaliq has quit IRC01:02
*** asettle has joined #openstack-infra01:04
*** rkukura has quit IRC01:05
*** Liuqing has joined #openstack-infra01:06
*** fawadkhaliq has joined #openstack-infra01:07
*** pvaneck has quit IRC01:07
*** asettle has quit IRC01:08
mordreddougwig: I don't konw if anyone followed up with you - but I would expect nodepool to do the right thing WRT your provider network with the shade patch01:12
mordreddougwig: I'd love to debug what's going wrong with you there01:12
mordreddougwig: whenver is convenient for you01:12
*** mriedem has quit IRC01:13
mordreddougwig: the easiest way to start that is for you to just run "shade-inventory --list --yaml" as a user who has access to your clouds.yaml file and send me the output so I can see what your network config looks like that we have not taken in to account01:13
*** tqtran has quit IRC01:14
*** fawadkhaliq has quit IRC01:14
*** esker has quit IRC01:14
*** SumitNaiksatam has joined #openstack-infra01:15
*** Liuqing has quit IRC01:15
*** Qiming has joined #openstack-infra01:15
*** armax has joined #openstack-infra01:16
*** armax has left #openstack-infra01:17
openstackgerritamrith proposed openstack/requirements: Blacklist Routes 2.3  https://review.openstack.org/29848201:18
*** chuck_ is now known as zu01:18
*** zu is now known as zul01:18
amrithhi folks, could I get some help with  https://review.openstack.org/29848201:18
amrithI'd sent email earlier today to the ML about this issue; it impacts Trove and Neutron at least (as far as I know).01:19
amriththis is breaking Trove in mitaka and master, and I assume liberty as well.01:19
amrithcould I get some help getting this merged, and also some help from someone who knows more about this than me to undertand why there are two consecutive lines for Routes in g-r.txt01:20
mordredamrith: they are for different python versions01:20
mordredone of them applies to python 2.701:20
mordredone applies to versions of python that are not01:20
amrithsorry, missed the !01:20
amrithok, I'll resubmt to block this for both01:21
mordredI'm going to -1 the patch just to record the reason01:21
amrithyes plz01:21
*** kzaitsev_mb has joined #openstack-infra01:21
mordred(also, I get so little stackalytics credit for -1s because I always just talk to people in IRC :) )01:21
openstackgerritamrith proposed openstack/requirements: Blacklist Routes 2.3  https://review.openstack.org/29848201:22
amrithmordred, did I beat you to it ?01:22
mordrednope! I got it in :)01:22
*** ddieterly has joined #openstack-infra01:22
*** asettle has joined #openstack-infra01:23
amriththx mordred I pushed one up for both, and I will go find my glasses.01:24
mordredamrith: cool. +2 from me01:24
*** Sukhdev has quit IRC01:25
openstackgerritfeipeng proposed openstack/requirements: Add crc16 lib to requirments  https://review.openstack.org/29848501:25
amriththx mordred will push for other stable's as well01:26
*** efoley_ has quit IRC01:26
*** ybathia has quit IRC01:26
*** kzaitsev_mb has quit IRC01:26
*** EricGonczer_ has joined #openstack-infra01:28
*** jamielennox|away is now known as jamielennox01:29
*** LaT0rtue has quit IRC01:32
anteayamordred: I know stackalytics gives no credit for irc discussions01:33
*** amitgandhinz has joined #openstack-infra01:33
mordredanteaya: it's sad01:33
mordredanteaya: I'd win that one for sure ;)01:33
anteayapleia2: thanks, I underestimated the water flow so at this point don't think I'll get sleep until it slows down01:33
anteayamordred: you would indeed01:33
pleia2anteaya: that's unfortunate01:33
anteayapleia2: on the bright side I hope that means it is over faster01:33
pleia2anteaya: hopefully!01:34
anteayapleia2: it's typical01:34
pleia2spring in canada, eh?01:34
anteayaI had just thought I missed it this year01:34
anteayaand my house01:34
anteayaand where the plow piles the snow01:34
anteayauntil I clear the driveway myself this is where the person who plows the drive will put it01:34
*** aspiers has quit IRC01:35
anteayaso I'm not surprised, I have my routine well oiled, it is just how much rest I get between freshening all the towels01:36
*** jamesmcarthur has joined #openstack-infra01:36
amrithmordred, would you take a look at https://review.openstack.org/298486 and https://review.openstack.org/298488 as well, I'll go to openstack-stable and see if I can get them reviewed there as well.01:37
dougwigmordred: sure thing.01:39
*** aspiers has joined #openstack-infra01:40
openstackgerritfeipeng proposed openstack/requirements: Add crc16 lib to requirments  https://review.openstack.org/29848501:41
*** mtanino_ has quit IRC01:43
*** tphummel has joined #openstack-infra01:45
dougwigmordred: sent you a couple of private gists. let me know what else i can do. when i walked through shade's code, the check for whether the newly spawned instance had a "public" ip was failing, hence it trying for a floating ip.01:46
*** amitgandhinz has quit IRC01:46
dougwigmordred: (specifically, in shade, the call to get_server_public_ip was returning non-true.)01:48
dougwigmordred: i stopped looking at that point and did the revert, but i can fire it back up if needed.01:48
*** rkukura has joined #openstack-infra01:50
*** Liuqing has joined #openstack-infra01:51
*** roxanaghe has joined #openstack-infra01:53
*** EricGonczer_ has quit IRC01:55
*** jamesmcarthur has quit IRC01:56
*** edmondsw has quit IRC01:56
*** woodster_ has quit IRC01:57
*** roxanaghe has quit IRC01:58
*** amrith is now known as _amrith_01:59
*** harlowja has quit IRC01:59
*** piet has quit IRC01:59
*** thorst has quit IRC01:59
*** yamahata has quit IRC02:05
*** sbalukoff has quit IRC02:05
*** jamesmcarthur has joined #openstack-infra02:07
*** LaT0rtue has joined #openstack-infra02:09
*** tqtran has joined #openstack-infra02:12
*** ddieterly is now known as ddieterly[away]02:15
*** tqtran has quit IRC02:17
*** jamesmcarthur has quit IRC02:18
*** jamesmcarthur has joined #openstack-infra02:19
*** ddieterly[away] is now known as ddieterly02:20
*** jamesmcarthur has quit IRC02:21
*** ddieterly is now known as ddieterly[away]02:21
*** kzaitsev_mb has joined #openstack-infra02:22
*** ddieterly[away] is now known as ddieterly02:23
*** ddieterly is now known as ddieterly[away]02:23
*** zhurong has joined #openstack-infra02:25
*** kzaitsev_mb has quit IRC02:28
openstackgerritJane.zhang proposed openstack/diskimage-builder: Set tgtd not auto-start on OS boot time.  https://review.openstack.org/29754802:31
sdakeany idea what is going on with this gte job here; https://jenkins04.openstack.org/job/gate-kolla-dsvm-build-ubuntu-source/544/console02:37
sdakeis it running ansible-playbook, or is it doing something else?02:37
*** baoli has quit IRC02:40
*** baoli has joined #openstack-infra02:40
*** amitgandhinz has joined #openstack-infra02:43
*** lifeless has quit IRC02:43
*** lifeless has joined #openstack-infra02:44
openstackgerritKomei Shimamura proposed openstack/requirements: Update global-requirements.txt for python-kafka version  https://review.openstack.org/29799502:45
*** zeih has joined #openstack-infra02:46
sdakeis vexxhost.ca a new hosting provider for our gates?02:47
sdakeI don't seem to recall seeing it previously02:47
*** harlowja has joined #openstack-infra02:47
*** burgerk has joined #openstack-infra02:48
*** armax has joined #openstack-infra02:49
clarkbits about a month old02:49
sdakeclarkb does that link above look odd to you?02:49
sdakeits like stuck in ansible or something02:50
sdakenot really sure what to make of it02:50
clarkbmaybe ansible is slow?02:50
sdakeit would have kicked out output by now02:50
*** zeih has quit IRC02:50
sdakethat job takes about 20 seconds to run not 45 minutes02:50
clarkbunless something went wrong02:50
sdakeperhaps the nodepool inf owe use is not the same output a the rest of the  gates02:51
clarkbnot sure what that playbook does02:51
clarkbiirc infras use of ansible is all explicit commands to aif debugging and readability02:51
*** mtanino has joined #openstack-infra02:51
*** harlowja has quit IRC02:52
sdakeclarkb if it becomes a problem i'll debug and provide output02:53
sdakethis is the first time i've seen it thus far02:53
sdakeand i check every timed out job02:53
*** kdas_ has quit IRC02:54
*** LaT0rtue has quit IRC02:54
clarkbcould be ssh could be file copies I dunno impossible to tell02:54
*** yamahata has joined #openstack-infra02:55
*** amitgandhinz has quit IRC02:56
clarkbits possible somethibg disrupted jenkins' connection and jenkins hasnt noticed02:56
sdakewe use nsible to ssh copy a file the setup script02:57
sdakefor the distro type02:57
openstackgerritIan Wienand proposed openstack-infra/project-config: Don't use neutron with centos7-minimal  https://review.openstack.org/29850302:57
*** _amrith_ is now known as amrith02:59
*** zhurong has quit IRC02:59
*** binbincong has quit IRC02:59
*** baoli has quit IRC03:00
*** baoli has joined #openstack-infra03:01
*** zhurong has joined #openstack-infra03:01
*** IlyaG has joined #openstack-infra03:02
*** rlandy has quit IRC03:02
*** gildub has quit IRC03:03
*** fedexo has joined #openstack-infra03:03
*** erlon has quit IRC03:07
krotscheckvexxhost nodes are just... slow, ye?03:07
clarkbno they should be roughly on par with the others03:09
clarkbthere is a spread though03:09
openstackgerritfeipeng proposed openstack/requirements: Add crc16 lib to requirments  https://review.openstack.org/29848503:10
*** tqtran has joined #openstack-infra03:13
*** links has joined #openstack-infra03:13
*** yamamoto has joined #openstack-infra03:14
*** IlyaG has quit IRC03:15
*** harlowja_at_home has quit IRC03:15
*** Qiming has quit IRC03:16
*** Qiming has joined #openstack-infra03:17
*** tqtran has quit IRC03:17
*** baoli has quit IRC03:20
*** rfolco_ has quit IRC03:21
*** sbalukoff has joined #openstack-infra03:23
*** ddieterly[away] has quit IRC03:23
*** kzaitsev_mb has joined #openstack-infra03:25
*** harlowja has joined #openstack-infra03:29
*** eliqiao has quit IRC03:29
*** bpokorny has joined #openstack-infra03:29
*** jamesmcarthur has joined #openstack-infra03:30
*** kzaitsev_mb has quit IRC03:30
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: Enable gate-networking-midonet-v2-rally-dsvm-nv for liberty  https://review.openstack.org/29851003:31
krotscheckWe just had a build timeout on bindep, and previous runs on vexxhost nodes have been unexpectedly slow.03:32
krotscheck40 minutes to run bindep seems wrong03:32
*** fedexo has quit IRC03:34
*** achanda has quit IRC03:38
*** tiswanso has quit IRC03:40
openstackgerritTony Breeds proposed openstack-infra/elastic-recheck: Add Query for 1563157  https://review.openstack.org/29851203:40
NakatoSo, I just fat-fingered a +W button instead of the reply button in gerrit, is there a way to stop the merge?  https://review.openstack.org/#/c/296061/03:42
Nakatolifeless: ^03:42
*** LaT0rtue has joined #openstack-infra03:42
*** jerryz has quit IRC03:43
*** armax has quit IRC03:44
*** ayoung has quit IRC03:46
jeblairNakato: just remove the +w vote03:47
Nakatojeblair: Thanks.  I'll be murdering that button from the UI now.03:48
*** achanda has joined #openstack-infra03:48
*** ramishra has quit IRC03:49
jeblairClint: in what way does 298389 break gertty?  noting that gertty does not display permissions, i see an empty diff, as expected.  is there something else wrong, or is it just the lack of permissions display?03:49
*** zhurong has quit IRC03:50
*** Sukhdev has joined #openstack-infra03:51
*** ramishra has joined #openstack-infra03:51
*** amitgandhinz has joined #openstack-infra03:53
*** NobodyCam has quit IRC03:54
Clintjeblair: if i hit 'd' it goes boom with  KeyError: "Blob or Tree named 'Unknown File' not found"03:58
*** asettle has quit IRC03:58
jeblairClint: git master?03:58
lifelessNakato: yes03:59
lifelessNakato: as jeblair says03:59
*** thorst has joined #openstack-infra04:00
Clintjeblair: well, apparently not. sorry04:00
*** roxanaghe has joined #openstack-infra04:00
jeblairClint: whew; cause that was sounding awful familiar :)04:00
NakatoButton murdered now too. - $('button[title="Apply score with one click"]').remove()04:01
*** oanson has joined #openstack-infra04:02
*** dimtruck is now known as zz_dimtruck04:03
*** achanda has quit IRC04:04
*** harlowja has quit IRC04:06
*** amitgandhinz has quit IRC04:06
*** harlowja has joined #openstack-infra04:09
*** thorst has quit IRC04:09
*** jcooley_ has joined #openstack-infra04:11
openstackgerritTony Breeds proposed openstack-infra/devstack-gate: Log locale during setup_host  https://review.openstack.org/29852104:11
*** LaT0rtue has quit IRC04:15
*** jamesmcarthur has quit IRC04:15
*** piet has joined #openstack-infra04:16
*** IlyaG has joined #openstack-infra04:16
*** salv-orlando has joined #openstack-infra04:16
*** gildub has joined #openstack-infra04:17
*** NobodyCam has joined #openstack-infra04:18
*** fawadkhaliq has joined #openstack-infra04:20
*** fawadk has joined #openstack-infra04:21
openstackgerritCraige McWhirter proposed openstack-infra/infra-specs: Updated my IRC nick.  https://review.openstack.org/29852304:21
*** IlyaG has quit IRC04:21
*** fawadkhaliq has quit IRC04:25
*** kzaitsev_mb has joined #openstack-infra04:26
*** salv-orlando has quit IRC04:28
*** akshai_ has quit IRC04:29
openstackgerritIan Wienand proposed openstack-infra/project-config: Revert "move fedora23 devstack to experimental"  https://review.openstack.org/29852504:30
*** maishsk has quit IRC04:31
*** kzaitsev_mb has quit IRC04:31
*** maishsk has joined #openstack-infra04:42
*** abregman has joined #openstack-infra04:42
*** LaT0rtue has joined #openstack-infra04:42
openstackgerritMerged openstack/requirements: Cap Routes < 2.3  https://review.openstack.org/29845804:43
*** bpokorny has quit IRC04:45
*** mylu has quit IRC04:46
*** zeih has joined #openstack-infra04:46
*** yamamoto has quit IRC04:47
*** mylu has joined #openstack-infra04:47
*** vgridnev has joined #openstack-infra04:48
*** maishsk has quit IRC04:48
*** zeih has quit IRC04:52
*** asselin has quit IRC04:52
*** fawadk has quit IRC05:00
*** roxanaghe has quit IRC05:02
*** amitgandhinz has joined #openstack-infra05:03
*** mylu has quit IRC05:03
*** zhurong has joined #openstack-infra05:03
*** roxanaghe has joined #openstack-infra05:05
*** thorst has joined #openstack-infra05:06
*** pcaruana has quit IRC05:09
*** achanda has joined #openstack-infra05:12
*** asettle has joined #openstack-infra05:12
*** thorst has quit IRC05:14
*** LaT0rtue has quit IRC05:16
*** amitgandhinz has quit IRC05:16
*** Nikolay_St has quit IRC05:16
*** yamamoto has joined #openstack-infra05:17
*** mylu has joined #openstack-infra05:27
*** kzaitsev_mb has joined #openstack-infra05:28
*** ybathia has joined #openstack-infra05:28
*** baoli has joined #openstack-infra05:32
*** piet has quit IRC05:32
*** kzaitsev_mb has quit IRC05:33
*** roxanaghe has quit IRC05:35
*** baoli has quit IRC05:36
*** zhurong has quit IRC05:38
*** fawadkhaliq has joined #openstack-infra05:38
*** asettle has quit IRC05:39
*** sputnik13 has joined #openstack-infra05:44
*** nadya has joined #openstack-infra05:44
*** burgerk has quit IRC05:44
*** zhurong has joined #openstack-infra05:44
*** burgerk has joined #openstack-infra05:44
*** sputnik13 has quit IRC05:48
*** zhurong has quit IRC05:49
*** ybathia has quit IRC05:50
*** sdake_ has joined #openstack-infra05:50
*** sdake has quit IRC05:52
*** zhurong has joined #openstack-infra05:52
*** yamamoto has quit IRC05:53
*** Guest79255 has quit IRC05:53
*** tphummel has quit IRC05:53
*** asettle has joined #openstack-infra05:54
*** burgerk_ has joined #openstack-infra05:56
*** burgerk has quit IRC05:56
openstackgerritMerged openstack-infra/project-config: remove sahara-bandit job  https://review.openstack.org/29727105:57
*** oanson has quit IRC05:57
*** Sukhdev has quit IRC05:57
*** asettle has quit IRC05:59
*** thorst has joined #openstack-infra06:01
AJaegerNakato: I feel that we should murder that button globally... Patches welcome ;)06:04
openstackgerritThomas Herve proposed openstack-dev/pbr: Fix wsgiref script use with oslo.config  https://review.openstack.org/29606106:05
*** zeih has joined #openstack-infra06:05
*** yamamoto has joined #openstack-infra06:06
*** salv-orlando has joined #openstack-infra06:07
*** jaosorior has joined #openstack-infra06:08
*** kushal has joined #openstack-infra06:08
*** skraynev has quit IRC06:08
*** thorst has quit IRC06:09
*** skraynev has joined #openstack-infra06:09
*** zeih has quit IRC06:10
*** mylu has quit IRC06:11
*** yolanda has quit IRC06:12
*** amitgandhinz has joined #openstack-infra06:12
*** cody-somerville has quit IRC06:14
*** cody-somerville has joined #openstack-infra06:15
*** cody-somerville has quit IRC06:15
*** cody-somerville has joined #openstack-infra06:15
*** vgridnev has quit IRC06:15
*** ramineni has joined #openstack-infra06:15
*** burgerk_ has quit IRC06:16
*** salv-orlando has quit IRC06:16
*** Nikolay_St has joined #openstack-infra06:18
*** yolanda has joined #openstack-infra06:19
*** tesseract has joined #openstack-infra06:19
*** harlowja has quit IRC06:19
*** tesseract is now known as Guest5878206:19
*** vgridnev has joined #openstack-infra06:20
*** LaT0rtue has joined #openstack-infra06:20
*** markvoelker has joined #openstack-infra06:22
*** nadya has quit IRC06:22
*** markvoelker_ has joined #openstack-infra06:23
*** skraynev has quit IRC06:26
*** amitgandhinz has quit IRC06:26
*** vgridnev has quit IRC06:26
*** vgridnev has joined #openstack-infra06:27
*** skraynev has joined #openstack-infra06:27
*** markvoelker has quit IRC06:27
*** MarkAtwood has joined #openstack-infra06:27
*** kzaitsev_mb has joined #openstack-infra06:29
*** maishsk has joined #openstack-infra06:30
*** binbincong has joined #openstack-infra06:30
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/29854806:31
*** zhurong has quit IRC06:31
*** harlowja has joined #openstack-infra06:32
yolandagood morning, back from holiday06:34
*** zhurong has joined #openstack-infra06:34
*** kzaitsev_mb has quit IRC06:34
AJaegeryolanda: good morning. HAve you had a great holiday?06:35
yolandaAJaeger, yest, mostly at home but is always great to spend time with the family. How about you?06:36
openstackgerritThomas Herve proposed openstack-dev/pbr: Fix wsgiref script use with oslo.config  https://review.openstack.org/29606106:36
*** roxanaghe has joined #openstack-infra06:36
*** scheuran has joined #openstack-infra06:38
AJaegeryolanda: similar to yours ;)06:39
AJaegertogether with some unexpected sunshine it was great ;)06:40
yolandait was a bit rainy here yesterday06:41
*** roxanaghe has quit IRC06:41
*** nmagnezi has joined #openstack-infra06:43
*** markvoelker_ has quit IRC06:43
*** markvoelker has joined #openstack-infra06:44
*** lmiccini has quit IRC06:46
*** cgross- has quit IRC06:46
*** mkoderer has quit IRC06:46
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/28884906:46
*** cgross has joined #openstack-infra06:48
*** mkoderer has joined #openstack-infra06:49
*** fawadkhaliq has quit IRC06:49
*** jtomasek has joined #openstack-infra06:50
*** lmiccini has joined #openstack-infra06:50
*** flepied has joined #openstack-infra06:51
ramineniyolanda: could you please review https://review.openstack.org/#/c/296380/06:51
ramineniyolanda: its needed  by a patch which should go to mitaka, so it would be great if you could have alook at it06:52
yolandahi ramineni . Yes, i'll start with the review queue in a while06:52
ramineniyolanda: sure, thanks a lot06:52
*** LaT0rtue has quit IRC06:53
*** GheRiver1 has quit IRC06:55
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/29854806:55
openstackgerritVasyl Saienko proposed openstack-infra/devstack-gate: Allow to pass TEMPEST_OS_TEST_TIMEOUT from the job  https://review.openstack.org/29844606:56
*** GheRiver1 has joined #openstack-infra06:57
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Set ZUUL_REFNAME for propose-devstack-plugins  https://review.openstack.org/29856406:59
*** gnuoy_ has joined #openstack-infra06:59
AJaegeryolanda: the change above is needed to fix the job - and is related to https://review.openstack.org/#/c/287415 . Reviews are welcome but not urgent ;) Thanks!06:59
*** arxcruz has joined #openstack-infra07:01
*** GheRivero has quit IRC07:03
*** GheRiver1 is now known as GheRivero07:05
*** LaT0rtue has joined #openstack-infra07:06
*** thorst has joined #openstack-infra07:06
*** GheRivero has quit IRC07:06
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Increase OS_TEST_TIMEOUT for ironic-multitenant  https://review.openstack.org/29856907:06
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Increase OS_TEST_TIMEOUT for ironic-multitenant  https://review.openstack.org/29856907:07
*** Hal has joined #openstack-infra07:07
*** GheRivero has joined #openstack-infra07:07
*** Hal is now known as Guest3833207:07
*** dtardivel has joined #openstack-infra07:08
*** gnuoy_ has quit IRC07:09
*** samuelBartel has joined #openstack-infra07:10
*** eliqiao has joined #openstack-infra07:12
*** mrmartin has joined #openstack-infra07:12
*** markvoelker has quit IRC07:12
*** vgridnev has quit IRC07:13
*** thorst has quit IRC07:13
*** tqtran has joined #openstack-infra07:14
*** shardy has joined #openstack-infra07:14
*** vgridnev has joined #openstack-infra07:18
*** tqtran has quit IRC07:18
*** maishsk has quit IRC07:19
*** jordanP has joined #openstack-infra07:21
openstackgerritMerged openstack-infra/infra-manual: Document other-requirements  https://review.openstack.org/29828107:22
*** amitgandhinz has joined #openstack-infra07:22
*** GheRivero has quit IRC07:22
*** sdake_ has quit IRC07:22
*** esikachev has joined #openstack-infra07:23
*** sputnik13 has joined #openstack-infra07:24
*** sshnaidm has quit IRC07:25
*** sdake has joined #openstack-infra07:25
*** salv-orlando has joined #openstack-infra07:25
openstackgerritMerged openstack-infra/project-config: Enable Murano service as part of congress job  https://review.openstack.org/29638007:25
*** sputnik13_ has joined #openstack-infra07:27
*** jlanoux has joined #openstack-infra07:28
openstackgerritMerged openstack-infra/project-config: Update build-image.sh for new images, add EXTRA_ELEMENTS  https://review.openstack.org/29846307:29
*** GheRivero has joined #openstack-infra07:30
openstackgerritMerged openstack-infra/project-config: [new project] Create a OpenStack related project "daisycloud-core"  https://review.openstack.org/29611707:30
*** kzaitsev_mb has joined #openstack-infra07:31
*** maishsk has joined #openstack-infra07:31
*** amitgandhinz has quit IRC07:36
*** kzaitsev_mb has quit IRC07:36
*** dmellado|off is now known as dmellado07:36
*** rcernin has joined #openstack-infra07:36
openstackgerritMerged openstack-infra/system-config: Remove unused nspr dependency  https://review.openstack.org/29831907:37
*** zhurong has quit IRC07:37
*** yuanying has quit IRC07:38
*** zhurong has joined #openstack-infra07:38
*** achanda has quit IRC07:39
openstackgerritMerged openstack-infra/puppet-jenkins: Add support for ubuntu xenial  https://review.openstack.org/29797407:39
openstackgerritMerged openstack-infra/system-config: Remove old pypi vhost template  https://review.openstack.org/29722407:39
*** LaT0rtue has quit IRC07:39
*** yuanying has joined #openstack-infra07:41
*** camunoz has quit IRC07:41
*** matrohon has joined #openstack-infra07:43
*** lezbar has joined #openstack-infra07:43
*** achanda has joined #openstack-infra07:44
*** ifarkas has joined #openstack-infra07:46
*** sputnik13net has joined #openstack-infra07:46
*** bauwser is now known as bauzas07:47
cody-somervillejeblair: mordred: clarkb: Apparently git now supports cloning using rsync. Is that something that would be useful to us?07:47
*** _degorenko|afk is now known as degorenko07:48
*** pcaruana has joined #openstack-infra07:48
*** GheRivero has quit IRC07:49
*** GheRivero has joined #openstack-infra07:49
*** daemontool has joined #openstack-infra07:50
*** sputnik13 has quit IRC07:50
AJaegeryolanda: please review also https://review.openstack.org/#/c/298307/07:52
AJaegerAnd thanks a lot for going throught all these!07:53
*** zeih has joined #openstack-infra07:54
yolandasure, no problme07:55
*** gnuoy_ has joined #openstack-infra07:57
openstackgerritMerged openstack-infra/shade: Support InsecureRequestWarning == None  https://review.openstack.org/29779107:57
*** GheRivero has quit IRC07:57
*** gnuoy__ has joined #openstack-infra07:57
*** GheRivero has joined #openstack-infra07:57
jd__AJaeger: turns out that sbcl is now too old I guess for cl-openstack-client to work, we'll have to wait for the new LTS probably with a new sbcl that brings ASDF 307:57
*** daemontool_ has joined #openstack-infra07:58
AJaegerjd__: so, should I remove my depends-on on https://review.openstack.org/#/c/298229/ ? And leave https://review.openstack.org/#/c/298226/ open?08:00
AJaegerjd__: I don't want to install sbcl on each and every job like we do right now...08:01
*** daemontool has quit IRC08:02
*** gnuoy_ has quit IRC08:02
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove sbcl from bindep-fallback.txt  https://review.openstack.org/29822908:03
AJaegerjd__: removed depends on ^, a +1 from you would be appreciated.08:03
*** gnuoy__ has quit IRC08:03
openstackgerritMerged openstack-infra/system-config: Add support for Ubuntu Xenial  https://review.openstack.org/29796408:03
sputnik13_anyone having issues with diskimage-builder generated images?  I'm trying to generate a fresh ubuntu image and it keeps dropping in to grub rather than booting08:04
sputnik13_just running disk-image-create vm ubuntu08:04
*** achanda has quit IRC08:06
*** akaszuba has joined #openstack-infra08:07
openstackgerritMerged openstack-infra/project-config: Set ZUUL_REFNAME for propose-devstack-plugins  https://review.openstack.org/29856408:07
*** achanda has joined #openstack-infra08:07
openstackgerritMerged openstack-infra/project-config: Make gate-sahara-tests-python34 job voting  https://review.openstack.org/29830408:08
openstackgerritMerged openstack-infra/project-config: Set gate-bindep-fallback-debian-jessie voting  https://review.openstack.org/29798008:08
anteayayolanda: for the infra meeting today, do you feel like adding an agenda item to update folks on your progress with the new gerrit node?08:08
openstackgerritMerged openstack-infra/project-config: Remove libnspr4-dev and nspr-devel  https://review.openstack.org/29832108:09
yolandaanteaya, so i have no progress on it, because i was told nothing was needed. The node is created, puppet running, but everything is disabled until the migration date08:09
openstackgerritMerged openstack-infra/project-config: Move `keystone-coverage-db` job to check queue  https://review.openstack.org/29735108:09
anteayayolanda: okay great, so is that no, you don't want to add an item?08:09
anteayabecause April 11th is coming up and someone has to have some sort of plan for the migration08:09
*** LaT0rtue has joined #openstack-infra08:10
anteayaso if that isn't you, and it doesn't have to be, I'd like to know who it is08:10
*** nadya has joined #openstack-infra08:10
yolandaok what i can do is add that item, and confirm that the intention is to do the volume change, start gerrit, etc... on the same maintenance day08:10
anteayagreat thank you08:10
AJaegeryolanda: and then discuss open points.08:10
*** thorst has joined #openstack-infra08:11
*** derekh has joined #openstack-infra08:11
*** kzaitsev_mb has joined #openstack-infra08:11
*** markvoelker has joined #openstack-infra08:13
openstackgerritMerged openstack-infra/project-config: Remove experimental-openstackci-beaker-ubuntu-trusty  https://review.openstack.org/29743108:14
*** tojuvone has quit IRC08:15
anteayagoing offline again, talk to you later08:15
yolandaanteaya, bye08:15
*** tojuvone has joined #openstack-infra08:15
jd__AJaeger: yep, looks good08:15
jd__AJaeger: it'll merge… at some points when sbcl can be updated I guess :]08:16
jd__it's not like the project is uber-active anyway08:16
*** dizquierdo has joined #openstack-infra08:16
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Run bindep on project-config only if needed  https://review.openstack.org/29780908:16
*** achanda has quit IRC08:16
*** kzaitsev_mb has quit IRC08:17
AJaegerthanks, jd__ . Could you +1 https://review.openstack.org/#/c/298229/ as well, please?08:17
*** markvoelker has quit IRC08:17
AJaegeranteaya: good night!08:17
AJaegeranteaya: hope your flooding is under control again!08:18
*** thorst has quit IRC08:18
*** snikitin has joined #openstack-infra08:21
*** rossella_s has joined #openstack-infra08:21
*** hashar has joined #openstack-infra08:22
openstackgerritMerged openstack-infra/project-config: Add experimental centos-7 to windmill  https://review.openstack.org/29780008:22
*** elo has quit IRC08:23
*** elo has joined #openstack-infra08:23
*** jistr has joined #openstack-infra08:23
*** YorikSar_ is now known as YorikSar08:23
*** ogelbukh_ is now known as ogelbukh08:24
*** mtanino has quit IRC08:24
*** roxanaghe has joined #openstack-infra08:24
*** mtanino has joined #openstack-infra08:25
snikitinetoews: Hi, Everett! Can I ask you to review this API-WG patch? https://review.openstack.org/#/c/276709/ It has a lot of +1 (even from Michael McCune) but it's not approved yet :(08:25
*** pc_m has quit IRC08:25
jtomasekHi, can anyone please help with the phantomjs issue we're hitting here https://review.openstack.org/#/c/295882/ ?08:25
openstackgerritMerged openstack-infra/project-config: set stable/mitaka permissions for senlin projects  https://review.openstack.org/29772108:25
openstackgerritMerged openstack-infra/project-config: remove pypi publishing jobs for cloudkitty  https://review.openstack.org/29830708:25
*** dims has quit IRC08:26
jtomasekthe error is here: http://logs.openstack.org/82/295882/1/check/gate-tripleo-ui-npm-run-test/ad951b5/console.html.gz#_2016-03-24_14_47_34_40608:26
jtomasekit seems like this issue: https://github.com/Medium/phantomjs/issues/430#issuecomment-16774702608:26
*** asettle has joined #openstack-infra08:26
*** nijaba has quit IRC08:26
*** pc_m has joined #openstack-infra08:26
*** tojuvone has quit IRC08:27
*** nijaba has joined #openstack-infra08:27
*** tojuvone has joined #openstack-infra08:27
*** yamahata has quit IRC08:27
*** dkehn has quit IRC08:27
*** dkehn has joined #openstack-infra08:28
*** yamahata has joined #openstack-infra08:28
mrmartinyolanda, you need to approve this one also: https://review.openstack.org/#/c/295208/ to make this work: https://review.openstack.org/#/c/295231/08:29
*** roxanaghe has quit IRC08:29
yolandaoh, this only has my +208:29
AJaegeryolanda: what do you think of  https://review.openstack.org/274701 ? That is too late for translators for mitaka but would be great to have in to get it up soonish so that they can use it.08:29
yolandamrmatin, it will need to wait then08:29
AJaegermrmartin: Good morning! does that mean we do not use php5-cli anymore?08:30
yolandaAJaeger i was just looking at it now,i defered to the last because it was complex08:30
AJaegeryolanda: ;) Thanks!08:30
mrmartinAJaeger: I think both groups.o.o and openstackid is using php5-cli08:31
mrmartinand openstackid also needs php5-mcrypt08:31
*** salv-orlando has quit IRC08:31
*** dims has joined #openstack-infra08:31
mrmartinso we are using it08:31
AJaegermrmartin: the question is: For running or for testing with Jenkins?08:32
mrmartinfor building the dist tarballs08:32
*** amitgandhinz has joined #openstack-infra08:32
mrmartingroups is using drush, and openstackid is using composer, both tool depends on php5-cli08:33
*** tojuvone has quit IRC08:33
*** tojuvone has joined #openstack-infra08:33
*** maishsk has quit IRC08:34
AJaegerstrange, gate-openstackid-unittests does not install php5-cli currently08:34
AJaegermrmartin: I'll -1 now https://review.openstack.org/#/c/298407/ - and let's work on getting this solved properly.08:34
*** lyz has quit IRC08:35
*** lyz has joined #openstack-infra08:35
AJaegermrmartin: thanks for that extra information!08:35
*** tojuvone has quit IRC08:36
*** tojuvone has joined #openstack-infra08:36
*** maishsk has joined #openstack-infra08:37
openstackgerritAndreas Jaeger proposed openstack-infra/groups: Add other-requirements.txt  https://review.openstack.org/29860008:41
AJaegermrmartin: here's a very minimal file ^, let's see whether the tests work ;)08:41
*** daemontool_ has quit IRC08:43
*** LaT0rtue has quit IRC08:43
*** daemontool_ has joined #openstack-infra08:43
*** sshnaidm has joined #openstack-infra08:45
openstackgerritMerged openstack-infra/project-config: Add octavia experimental tempest job  https://review.openstack.org/28487508:45
*** kzaitsev_mb has joined #openstack-infra08:45
*** IlyaG has joined #openstack-infra08:46
*** amitgandhinz has quit IRC08:46
*** IlyaG has quit IRC08:50
*** LaT0rtue has joined #openstack-infra08:52
*** sambetts has quit IRC08:55
*** sputnik13_ has quit IRC08:55
matrohonyolanda, hi08:55
*** sputnik13 has joined #openstack-infra08:55
matrohonyolanda, whose approval from neutron do you need for https://review.openstack.org/#/c/258522/4?08:56
*** sputnik13 has quit IRC08:56
rcarrillocruzmatrohon: https://wiki.openstack.org/wiki/CrossProjectLiaisons08:57
yolandawhat rcarrillocruz said ^08:57
matrohonyolanda, rcarrillocruz : ok thanks. net-bgpvpn is a release-independent project, so I'm not sure that ihrachyshka will be able to vouch for this change08:59
rcarrillocruzmatrohon: you want to check this section instead08:59
rcarrillocruzihar is on oslo section09:00
matrohonrcarrillocruz, yep, I meant mestery, armax or dougwig09:00
matrohonrcarrillocruz, sorry :)09:00
matrohonrcarrillocruz, yolanda, I'll ping them when they'll be awake09:01
yolandamatrohon, i normally wait for that +1 from neutron liaisons for all networking projects09:01
matrohonyolanda, ack09:01
yolandayours is not related to neutron?09:01
*** tojuvone has quit IRC09:02
matrohonyolanda, it's in the neutron stadium, under the governance of neutron's PTL, but none of the neutron core are part of net-bgpvpn...09:03
*** tojuvone has joined #openstack-infra09:03
yolandaif it's under governance of neutron, we need to wait for that +109:03
*** tojuvone has quit IRC09:03
matrohonyolanda, ack, np09:03
*** tojuvone has joined #openstack-infra09:03
*** zhurong has quit IRC09:06
*** tosky has joined #openstack-infra09:06
*** tojuvone has quit IRC09:07
*** tojuvone has joined #openstack-infra09:07
*** sileht has quit IRC09:07
*** rm_work has quit IRC09:08
*** mgagne has quit IRC09:08
*** khappone_ has joined #openstack-infra09:09
AJaegermatrohon: We have a very simple rule - if it's under governance of neutron's PTL, it needs neutron infra liason +1 - as requested by the neutron PTL for all such changes.09:09
*** jbryce has quit IRC09:10
*** dtroyer has quit IRC09:10
vgridnevHi, folks. Is there something that should be done so that "Closes-bug" statement will take affect for fix in particular project?09:11
*** mjb has quit IRC09:11
*** khappone has quit IRC09:11
*** comstud has quit IRC09:11
*** greghaynes has quit IRC09:12
vgridnevWe have some issues with sahara-tests repo and appropriate launchpad project associated with that09:12
AJaegervgridnev: your project needs to be setup correctly ;)09:12
*** sileht has joined #openstack-infra09:12
AJaegervgridnev: show me a bug URL and a patch and then I'll double check09:12
vgridnev1 min09:12
openstackgerritMerged openstack-infra/release-tools: improve check_library_constraints.sh to edit the constraints file  https://review.openstack.org/29839809:13
vgridnevfor example: https://review.openstack.org/#/c/297757/ and https://bugs.launchpad.net/sahara-tests/+bug/155899409:13
openstackLaunchpad bug 1558994 in OpenStack Data Processing ("Sahara") sahara-tests "Default variables for edp.yaml and credentials.yaml not applied" [Undecided,In progress] - Assigned to Evgeny Sikachev (esikachev)09:13
vgridnevAJaeger, ^^09:13
*** jbryce has joined #openstack-infra09:13
*** dtroyer has joined #openstack-infra09:13
*** comstud has joined #openstack-infra09:13
*** rm_work has joined #openstack-infra09:13
*** mjb has joined #openstack-infra09:14
*** mgagne has joined #openstack-infra09:14
*** mgagne is now known as Guest6891009:14
AJaegervgridnev: ah, that alone is enough information ;)09:14
*** greghaynes has joined #openstack-infra09:14
AJaegervgridnev: you configured in http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerrit/projects.yaml#n3391 with the groups line that bugs are in bugs.launchpad.net/sahara/ not in sahara-tests09:14
*** tojuvone has quit IRC09:15
AJaegerAnd so sahara-tests changes only close bugs reported agains "sahara" project09:15
*** tojuvone has joined #openstack-infra09:15
vgridnevcool, I've got it09:15
AJaegervgridnev: so, consider what you as team want and change either launchpad setup or the setting in project-config09:16
*** thorst has joined #openstack-infra09:16
*** achanda has joined #openstack-infra09:17
*** e0ne has joined #openstack-infra09:18
*** tojuvone has quit IRC09:18
*** tojuvone has joined #openstack-infra09:18
openstackgerritKoteswara Rao Kelam proposed openstack-infra/project-config: Adding networking-hpe project.  https://review.openstack.org/29577509:20
openstackgerritEvgeny Sikachev proposed openstack-infra/project-config: Add sahara-tests to sahara-tests group  https://review.openstack.org/29861409:20
esikachevAJaeger: ^^09:20
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Change gerrit acl for shotgun project  https://review.openstack.org/29861509:20
*** salv-orlando has joined #openstack-infra09:20
openstackgerritMarkus Zoeller (markus_z) proposed openstack-infra/project-config: Add job to test Nova with specific libvirt and kvm packages  https://review.openstack.org/28925509:21
*** tojuvone has quit IRC09:21
*** achanda has quit IRC09:21
*** tojuvone has joined #openstack-infra09:21
*** thorst has quit IRC09:24
*** asettle has quit IRC09:27
*** dtantsur|afk is now known as dtantsur09:29
*** zeih has quit IRC09:30
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Change gerrit acl for shotgun project  https://review.openstack.org/29861509:34
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: ssh less  https://review.openstack.org/29597309:35
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Don't execute the output from wait_for  https://review.openstack.org/29529709:37
openstackgerritMerged openstack-infra/tripleo-ci: Add support for mitaka jobs  https://review.openstack.org/29837009:38
*** lifeless has quit IRC09:39
*** lifeless has joined #openstack-infra09:40
openstackgerritKoteswara Rao Kelam proposed openstack-infra/project-config: Adding networking-hpe project.  https://review.openstack.org/29577509:41
*** amitgandhinz has joined #openstack-infra09:42
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Run tempest tests with non-HA jobs  https://review.openstack.org/29703809:43
*** tojuvone has quit IRC09:43
openstackgerritAndreas Jaeger proposed openstack-infra/system-config: Do not install sbcl  https://review.openstack.org/29862709:43
*** tojuvone has joined #openstack-infra09:43
*** Liuqing has quit IRC09:47
*** zeih has joined #openstack-infra09:50
*** kushal has quit IRC09:52
*** asettle has joined #openstack-infra09:52
openstackgerritMerged openstack-infra/project-config: Add sahara-tests to sahara-tests group  https://review.openstack.org/29861409:53
*** oanson has joined #openstack-infra09:54
*** daemontool_ has quit IRC09:55
*** tojuvone has quit IRC09:55
*** daemontool_ has joined #openstack-infra09:55
*** amitgandhinz has quit IRC09:56
*** tojuvone has joined #openstack-infra09:56
*** daemontool_ has quit IRC09:56
*** daemontool_ has joined #openstack-infra09:56
*** tojuvone has quit IRC09:57
*** asettle has quit IRC09:57
*** tojuvone has joined #openstack-infra09:57
*** salv-orlando has quit IRC09:57
*** cdent has joined #openstack-infra09:57
*** gildub has quit IRC09:58
*** LaT0rtue has quit IRC09:58
*** tojuvone has quit IRC09:58
*** tojuvone has joined #openstack-infra09:58
*** hichihara has quit IRC10:04
*** yamada-h has joined #openstack-infra10:06
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861510:08
*** salv-orlando has joined #openstack-infra10:10
*** sdake has quit IRC10:10
kozhukalovAJaeger: when you said "And I suggest to change the order of the depends-on" in this review https://review.openstack.org/#/c/298615, did you mean that in governance patch it should be 'needed-by' tag and in infra patch it should 'depends-on'?10:11
*** placko has joined #openstack-infra10:12
*** roxanaghe has joined #openstack-infra10:12
*** pblaho has joined #openstack-infra10:14
trashSpamapS: hi, again for https://review.openstack.org/#/c/280178/13 - the current version does not work at all - can you verify that? I can reproduce the error locally.10:14
*** markvoelker has joined #openstack-infra10:14
*** tqtran has joined #openstack-infra10:15
*** roxanaghe has quit IRC10:17
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861510:18
*** tqtran has quit IRC10:20
*** markvoelker has quit IRC10:20
*** thorst has joined #openstack-infra10:21
*** Qiming has quit IRC10:24
*** eliqiao_ has joined #openstack-infra10:24
*** tojuvone has quit IRC10:26
*** tojuvone has joined #openstack-infra10:27
*** Liuqing has joined #openstack-infra10:27
*** eliqiao has quit IRC10:27
*** thorst has quit IRC10:29
*** esikachev has quit IRC10:30
openstackgerrityolanda.robla proposed openstack-infra/shade: Use keystoneauth1.betamax for shade mocks  https://review.openstack.org/29864710:31
openstackgerrityolanda.robla proposed openstack-infra/shade: (WIP) Use keystoneauth1.betamax for shade mocks  https://review.openstack.org/29864710:31
AJaegerkozhukalov: correct10:32
kozhukalovAJaeger: thanks, will update10:32
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861510:35
*** tojuvone has quit IRC10:36
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 network isolation on the ha job  https://review.openstack.org/28944510:36
*** tojuvone has joined #openstack-infra10:36
*** flaper87 has quit IRC10:36
*** flaper87 has joined #openstack-infra10:36
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861510:36
*** sputnik13net has quit IRC10:41
*** LaT0rtue has joined #openstack-infra10:44
openstackgerritAthlan-Guyot sofer proposed openstack-infra/project-config: Re-activate Ubuntu beaker for puppet-pacemaker.  https://review.openstack.org/29865010:45
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 and pacemaker on the upgrades job  https://review.openstack.org/28944510:46
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861510:47
openstackgerritSergey Kraynev proposed openstack/requirements: Bump up upper-constraint for heatclient  https://review.openstack.org/29865510:48
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861510:49
*** achanda has joined #openstack-infra10:50
*** amitgandhinz has joined #openstack-infra10:52
*** ihrachys has joined #openstack-infra10:53
*** mescanef has joined #openstack-infra10:53
*** tojuvone has quit IRC10:53
*** tojuvone has joined #openstack-infra10:53
*** kushal has joined #openstack-infra10:54
*** yamamoto has quit IRC10:54
*** achanda has quit IRC10:55
*** yamamoto has joined #openstack-infra10:55
*** Liuqing has quit IRC10:56
*** asettle has joined #openstack-infra10:56
*** asettle has quit IRC10:56
*** nadya has quit IRC10:56
*** ociuhandu has quit IRC10:57
*** esikachev has joined #openstack-infra10:57
*** tojuvone has quit IRC10:58
*** tojuvone has joined #openstack-infra10:58
*** skraynev has quit IRC10:59
*** lucasagomes is now known as lucas-hungry11:00
*** skraynev has joined #openstack-infra11:01
*** amitgandhinz has quit IRC11:05
*** yamamoto has quit IRC11:07
*** yamada-h has quit IRC11:07
*** tojuvone has quit IRC11:07
*** tojuvone has joined #openstack-infra11:07
*** jcooley_ has quit IRC11:08
*** sdague has joined #openstack-infra11:12
openstackgerrityolanda.robla proposed openstack-infra/shade: (WIP) Use keystoneauth1.betamax for shade mocks  https://review.openstack.org/29864711:13
*** zeih has quit IRC11:15
*** markvoelker has joined #openstack-infra11:16
*** yamamoto has joined #openstack-infra11:18
*** Qiming has joined #openstack-infra11:18
*** markvoelker has quit IRC11:21
*** |-paul-| has joined #openstack-infra11:22
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861511:23
*** yamamoto has quit IRC11:23
kozhukalovAJaeger: seems it does not work this way. once I put 'depends-on' tag to the infra patch, i immediately get 'Verified11:26
kozhukalov-1 Jenkins'11:26
*** tpsilva has joined #openstack-infra11:26
*** ociuhandu has joined #openstack-infra11:26
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861511:27
AJaegerkozhukalov: wait until the governance change has been submitted and recheck after 2 mins...11:27
AJaegerkozhukalov: let me check...11:28
AJaegerkozhukalov: patch set 8 succeeded...11:28
kozhukalovAJaeger: yes, but that was without tags at all11:29
AJaegerkozhukalov: let me edit and double check...11:29
kozhukalovAJaeger: i removed tags in both patches, then waited until they are green, then put 'depends-on' tag into infra patch and got '-1' from jenkins11:29
AJaegerkozhukalov: governance change has a depends-on, it needs to be needed-by on that one11:30
AJaegerkozhukalov: strange, let me try to reproduce...11:30
*** gordc has joined #openstack-infra11:30
kozhukalovAJaeger: i added this depends-on tag to governance patch just 1 min ago, it was after i got '-1'11:30
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Decouple Shotgun project from Fuel  https://review.openstack.org/29861511:31
*** kushal has quit IRC11:31
*** smarcet has joined #openstack-infra11:32
*** dprince has joined #openstack-infra11:32
AJaegerkozhukalov: 298615,11 is currently building - see status.openstack.org/zuul Let's wait for it to finish...11:33
AJaegerkozhukalov: this looks all fine right now AFAIU11:33
kozhukalovAJaeger: ok, let's wait, but i see -1 from jenkins there11:34
*** kushal has joined #openstack-infra11:35
*** Liuqing has joined #openstack-infra11:35
kozhukalovAJaeger: is it going to become +1 once tests have successfully passed?11:35
AJaegerkozhukalov: I hope ;)11:36
*** yamada-h has joined #openstack-infra11:38
*** sc68cal has joined #openstack-infra11:41
*** nadya has joined #openstack-infra11:41
fricklerdid anything change regarding the log-publisher target recently? it worked here http://logs.openstack.org/39/291939/47/check/gate-ubuntu-trusty-openstack-chef-repo-chef-rake-integration-nv/9e3d032/ and failed a couple of hours later http://logs.openstack.org/39/291939/49/check/gate-ubuntu-trusty-openstack-chef-repo-chef-rake-integration-nv/1769625/ and has stayed in the latter state since then11:42
*** asselin has joined #openstack-infra11:42
openstackgerrityolanda.robla proposed openstack-infra/shade: (WIP) Use keystoneauth1.betamax for shade mocks  https://review.openstack.org/29864711:42
*** yamada-h has quit IRC11:42
AJaegerkozhukalov: I think the -1 came from a previous edit of yours and was copied over. Now it'S a +1 and everything fine...11:42
*** huzhj_ has joined #openstack-infra11:43
*** thorst has joined #openstack-infra11:44
*** thorst has quit IRC11:45
*** thorst has joined #openstack-infra11:45
*** tojuvone has quit IRC11:47
*** tojuvone has joined #openstack-infra11:47
jrollmordred: could you recommend a public cloud that is somewhat inexpensive, easy to sign up for, and accepts qcow2 images? :)11:48
*** thorst has quit IRC11:51
*** ldnunes has joined #openstack-infra11:52
*** mrmartin has quit IRC11:52
openstackgerritVasyl Saienko proposed openstack-infra/devstack-gate: Allow to pass TEMPEST_OS_TEST_TIMEOUT from the job  https://review.openstack.org/29844611:53
*** weshay has joined #openstack-infra11:53
*** burgerk_ has joined #openstack-infra11:54
*** ddieterly has joined #openstack-infra11:55
*** flepied has quit IRC11:55
*** asettle has joined #openstack-infra11:56
openstackgerritKoteswara Rao Kelam proposed openstack-infra/project-config: Adding networking-hpe project.  https://review.openstack.org/29577511:56
openstackgerritJames Slagle proposed openstack-infra/project-config: Add a periodic mitaka job for TripleO  https://review.openstack.org/29836911:59
*** bdemers has quit IRC12:00
*** vikram_ has joined #openstack-infra12:00
*** thorst has joined #openstack-infra12:01
*** roxanaghe has joined #openstack-infra12:01
vikram_Hi all12:01
vikram_Can someone plz help me in understanding the jerkin failure for https://review.openstack.org/#/c/268727/312:01
vikram_can't find the logs archieved12:01
*** amitgandhinz has joined #openstack-infra12:02
openstackgerritJames Slagle proposed openstack-infra/project-config: Add notifications for TripleO stable/mitaka  https://review.openstack.org/29868912:02
*** aysyd has joined #openstack-infra12:03
AJaegervikram_: some the log upload failed. Best to recheck the build12:04
*** roxanaghe has quit IRC12:05
vikram_AJaeger: Ok.. What tox cmd will "gate-governance-docs" check? I can check locally12:05
*** vgridnev has quit IRC12:05
*** dizquierdo has quit IRC12:06
openstackgerritSergey Kraynev proposed openstack/requirements: Bump up upper-constraint for heatclient  https://review.openstack.org/29865512:06
AJaegervikram_: run "tox -e docs" or "python setup.py build_sphinx"12:06
vikram_AJaeger: Ok..12:07
vikram_AJaeger: Thanks for your help12:07
*** rfolco_ has joined #openstack-infra12:07
openstackgerritThomas Herve proposed openstack-dev/pbr: Fix wsgiref script use with oslo.config  https://review.openstack.org/29606112:07
*** EricGonczer_ has joined #openstack-infra12:08
*** ddieterly is now known as ddieterly[away]12:08
*** flepied has joined #openstack-infra12:08
*** jcooley_ has joined #openstack-infra12:09
*** salv-orlando has quit IRC12:09
*** kgiusti has joined #openstack-infra12:09
*** EricGonczer_ has quit IRC12:10
*** placko has quit IRC12:11
*** jcooley_ has quit IRC12:13
*** oanson has quit IRC12:14
*** vsaienko has quit IRC12:15
*** amitgandhinz has quit IRC12:15
openstackgerritRodrigo Duarte proposed openstack-infra/project-config: Enable non-voting keystone tempest plugin tests  https://review.openstack.org/29869612:16
*** salv-orlando has joined #openstack-infra12:17
*** markvoelker has joined #openstack-infra12:17
*** rfolco_ has quit IRC12:20
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869912:20
openstackgerritvikram.choudhary proposed openstack-infra/project-config: Adding neutron-bgp as a part of neutron project.  https://review.openstack.org/26872612:21
kozhukalovAJaeger: ok, nice, it is green now, but i put WF-1 until we have a consensus in ML thread. Thanks very much.12:21
*** rfolco has joined #openstack-infra12:22
*** markvoelker has quit IRC12:22
AJaegerkozhukalov: sure, go ahead...12:23
*** pradk has joined #openstack-infra12:24
*** oanson has joined #openstack-infra12:26
*** mrmartin has joined #openstack-infra12:29
jrollmordred: I wound up going with ovh, having trouble with getting into the api, wondering if you have any tips or if I should hit support12:31
zigoAJaeger: Hi! Do you have a few minutes to discuss the deb packaging patch I'm trying to push?12:31
*** isaacb has joined #openstack-infra12:32
AJaegerzigo: a few minutes, yes.12:34
zigoAJaeger: Ok, I'll try to be quick.12:34
zigoAJaeger: In your review here: https://review.openstack.org/#/c/294022/12:34
zigoyou're saying that you'd like us to take care of the scripts, within a package.12:34
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869912:34
zigoWell, I'd like to do that, but I can't, since there's no hosted package (yet) by infra.12:34
AJaegerzigo: That'S a suggestion which I think will benefit all of us - so you can have a much faster turnaround for this ;)12:35
zigoAJaeger: So, to bootstrap the process, I thought having the scripts hosted *temporarily* on infra, so that deb-openstack-pkg-tools can be build, then published.12:35
zigoAJaeger: THEN, when we got that ready, move the scripts to openstack-pkg-tools, and use them to build all packages.12:35
zigoIsn't this a good plan?12:35
zigoOr should I do otherwise?12:35
zigoAJaeger: It has always been my intention the way you suggested, but the issue is boostraping the process.12:36
AJaegerzigo: Especially in the bootstrap phase you need fast turnaround ;)12:36
AJaegerand I cannot review your scripts properly so won't be able to help with those12:36
zigoIn other words: how can I build openstack-pkg-tools if it needs itself ...12:36
igorbelikovzigo: you could still build/publish deb-openstack-pkg-tools if the scripts will be hosted in that repo)12:36
AJaegerzigo: that's the wrong idea.12:36
*** ddieterly[away] is now known as ddieterly12:36
AJaegerigorbelikov is right. You can just publish a script with pypi if you want12:37
AJaegerOr you can add it to your repo and make a source checkout12:37
zigoAJaeger: openstack-pkg-tools isn't just a tool for building, it's also a build-dependency of all OpenStack packages.12:37
zigoAJaeger: I would like to avoid having a special case for building openstack-pkg-tools itself.12:37
zigoAJaeger: Having a tox.ini isn't an option (as upstream sources have one themselves, and I don't want to overwrite it...)12:38
*** jaosorior has quit IRC12:38
*** huzhj has joined #openstack-infra12:38
AJaegerTAke a step back. I'm not talking about openstack-pkg-tools, I'm talking about your scripts that you currently push, they can leave everywhere ;)12:38
*** jaosorior has joined #openstack-infra12:39
*** yamamoto has joined #openstack-infra12:39
zigoAJaeger: So my plan was: 1/ get openstack-pkg-tools to build with the scripts I sent against infra12:39
zigo2/ get the package to be published on the OpenStack infra deb repo12:39
zigo3/ change the build process to use the scripts from the openstack-pkg-tools .deb instead of the scripts from infra12:39
zigo4/ re-use that for all other packages12:39
zigoAJaeger: Do you see another way to bootstrap things?12:40
AJaegerzigo: I hate cyclic dependencies ;)12:40
*** xyang1 has joined #openstack-infra12:40
*** dmellado is now known as dmellado|lunch12:40
zigoIt's *bad*.12:40
zigoBut I don't see how we could do otherwise.12:40
igorbelikovzigo: just change 1/ to 'get openstack-pkg-tools to build with the scripts hosted in openstack-pkg-tools or anywhere else'12:40
AJaegerI'm not sure we want an OpenStack Infra deb repo - but that'S a different discussion.12:40
zigoigorbelikov: How then?12:40
*** degorenko is now known as _degorenko|afk12:41
*** huzhj_ has quit IRC12:41
zigoAJaeger: Others (like mordred, pabelanger and fungi) agreed already that having a deb repo is the way to go.12:41
zigoAnd they already have one for other stuff anyways.12:41
igorbelikovzigo: maybe it’s worth bringing the repo question to the infra meeting today?12:41
zigoigorbelikov: Of course!12:42
zigoigorbelikov: Will you be at the meeting?12:42
igorbelikovzigo: yep12:42
*** jaosorior has quit IRC12:42
zigoigorbelikov: Unfortunately, it's not a good time for me, it's dinner time for the kids, and I have to be there... :(12:42
zigoigorbelikov: Please be our voice then.12:42
*** jaosorior has joined #openstack-infra12:43
smarcetfungi: morning, https://review.openstack.org/#/c/298699/ is the proposal to change openstackid config from mpm prefork to mpm events12:44
*** lucas-hungry is now known as lucasagomes12:44
smarcetfungi: after reviewed, could we only apply to dev and disabled production just for testing ?12:44
AJaegerzigo: perhaps it helps if you write some kind of design document or spec on what you want.12:46
*** rlandy has joined #openstack-infra12:46
AJaegerzigo: please note: I'm not holding up your change, I'm just offering suggestions on my limited understanding of what you want to achieve12:46
AJaegerdesign document can be a spec as well12:46
*** baoli has joined #openstack-infra12:46
*** sfinucan has joined #openstack-infra12:48
*** baoli_ has joined #openstack-infra12:48
*** |-paul-| has quit IRC12:50
*** baoli has quit IRC12:51
huzhjHi all,12:52
huzhjI encountered an issue when doing http://docs.openstack.org/infra/manual/creators.html#preparing-a-new-git-repository-using-cookiecutter12:52
huzhjI have a new project called "daisycloud-core", so I deal with cookiecutter as the following:12:52
huzhj1. $git clone https://git.openstack.org/openstack/daisycloud-core12:52
huzhjHere it create directory in pwd named "daisycloud-core"12:52
huzhj2. $pip install cookiecutter12:52
huzhj3. $cookiecutter https://git.openstack.org/openstack-dev/cookiecutter12:52
*** annegentle has joined #openstack-infra12:52
huzhjmodule_name [replace with the name of the python module]: daisycloud-core12:52
huzhjrepo_group [openstack]:12:52
huzhjrepo_name [replace with the name for the git repo]: daisycloud-core12:52
huzhjlaunchpad_project [replace with the name of the project on launchpad]: daisycloud-core12:52
huzhjproject_short_description [OpenStack Boilerplate contains all the boilerplate you need to create an OpenStack package.]: ...12:52
huzhjError: "daisycloud-core" directory already exists12:52
huzhjSee, at the end of step 3,  it complain Error: "daisycloud-core" directory already exists, becasue we have already created that directory in step 1.12:52
huzhjSo, I do not quite understand why it told me to do like this. May be the description is wrong? Or I misunderstood it?12:52
*** achanda has joined #openstack-infra12:52
*** amitgandhinz has joined #openstack-infra12:52
openstackgerritsebastian marcet proposed openstack-infra/openstackid: Fix on deleted member  https://review.openstack.org/29873112:53
*** julim has joined #openstack-infra12:54
*** amitgandhinz has quit IRC12:57
*** infra-red has joined #openstack-infra12:58
*** achanda has quit IRC12:58
*** LaT0rtue has quit IRC13:00
*** krtaylor has quit IRC13:00
*** LaT0rtue has joined #openstack-infra13:01
*** dprince has quit IRC13:01
*** links has quit IRC13:02
*** huzhj_ has joined #openstack-infra13:02
*** dprince has joined #openstack-infra13:03
openstackgerritKATO Tomoyuki proposed openstack-infra/project-config: Add release notes translation scripts  https://review.openstack.org/29704513:04
*** ddieterly is now known as ddieterly[away]13:04
zigoAJaeger: Your help is very much appreciated, thanks.13:05
zigoAJaeger: It's been my plan to write a spec indeed, but I had no time to do so, so far.13:05
*** ddieterly[away] is now known as ddieterly13:05
zigoAJaeger: I don't think I'll have time before the meeting tonight.13:05
*** ddieterly is now known as ddieterly[away]13:05
*** huzhj has quit IRC13:06
*** piet has joined #openstack-infra13:07
*** tojuvone has quit IRC13:07
*** tojuvone has joined #openstack-infra13:07
*** huzhj_ has quit IRC13:07
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869913:08
*** huzhj_ has joined #openstack-infra13:08
*** edmondsw has joined #openstack-infra13:09
*** ayoung has joined #openstack-infra13:09
*** zeih has joined #openstack-infra13:10
*** kushal has quit IRC13:12
*** huzhj__ has joined #openstack-infra13:12
*** tiswanso has joined #openstack-infra13:12
*** kushal has joined #openstack-infra13:12
*** tiswanso has quit IRC13:13
*** tiswanso has joined #openstack-infra13:14
*** krtaylor has joined #openstack-infra13:14
*** kushal has joined #openstack-infra13:14
*** huzhj_ has quit IRC13:15
*** markvoelker has joined #openstack-infra13:18
*** huzhj_ has joined #openstack-infra13:21
*** markvoelker has quit IRC13:22
openstackgerritMerged openstack-infra/openstackid: Fix on deleted member  https://review.openstack.org/29873113:23
*** markvoelker has joined #openstack-infra13:23
*** claudiub has joined #openstack-infra13:24
openstackgerritMarton Kiss proposed openstack-infra/puppet-drupal: Support Apache 2.4  https://review.openstack.org/29874213:24
*** huzhj__ has quit IRC13:25
*** tojuvone has quit IRC13:25
*** tojuvone has joined #openstack-infra13:25
*** kushal has quit IRC13:25
*** tojuvone has quit IRC13:25
*** tojuvone has joined #openstack-infra13:26
*** zeih has quit IRC13:26
zigoAJaeger: How do I get a new debian/mitaka branch created on the remote at ssh://thomas-goirand@review.openstack.org:29418/openstack/deb-openstack-pkg-tools.git ?13:28
*** mrmartin has quit IRC13:28
*** markvoelker_ has joined #openstack-infra13:28
*** jsavak has joined #openstack-infra13:30
*** mylu has joined #openstack-infra13:30
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Make job networking-generic-switch-dsvm voting  https://review.openstack.org/29721813:31
*** mriedem has joined #openstack-infra13:32
openstackgerritIgor Belikov proposed openstack-infra/project-config: Add create option to deb-openstack-pkg-tools ACL  https://review.openstack.org/29874613:32
*** markvoelker has quit IRC13:32
*** woodster_ has joined #openstack-infra13:32
igorbelikovzigo: this ^^ should help13:32
*** mylu has quit IRC13:32
*** amitgandhinz has joined #openstack-infra13:32
*** mylu has joined #openstack-infra13:32
zigoOh !13:32
zigoigorbelikov: Thanks a lot.13:34
*** amrith is now known as _amrith_13:35
*** bdemers has joined #openstack-infra13:35
pabelangeryolanda: thanks for the merges, testing ubuntu-xenial dib again now13:39
*** mrmartin has joined #openstack-infra13:39
yolandapabelanger, nice, the changes made sense13:40
rcarrillocruzheya paul13:41
*** diana_clarke has quit IRC13:44
*** pradk has quit IRC13:45
*** pradk has joined #openstack-infra13:45
*** yamahata has quit IRC13:45
*** yamahata has joined #openstack-infra13:45
*** roxanaghe has joined #openstack-infra13:49
*** mtanino has quit IRC13:49
*** asettle has quit IRC13:49
*** mylu has quit IRC13:50
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Add Rally role to OpenStack-Ansible  https://review.openstack.org/29875413:51
*** tonytan4ever has joined #openstack-infra13:51
*** roxanaghe has quit IRC13:53
isaacbianw: here?13:54
*** asettle has joined #openstack-infra13:55
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Add Rally role to OpenStack-Ansible  https://review.openstack.org/29875413:56
*** burgerk_ has quit IRC13:56
SpamapStrash: I see that it already merged, but anyway, thanks so much for the great change, and for working hard in the face of my unit test pedantry. :)13:57
*** akshai has joined #openstack-infra13:57
*** sigmavirus24_awa is now known as sigmavirus2413:57
*** mylu has joined #openstack-infra13:58
*** akshai_ has joined #openstack-infra13:59
*** mkoderer__ has quit IRC13:59
*** andrewbogott has joined #openstack-infra13:59
*** asettle has quit IRC14:00
*** bdemers has quit IRC14:00
AJaegerzigo: AFAIK via https://review.openstack.org/#/admin/projects/openstack/deb-openstack-pkg-tools,branches if the ACLs are setup properly14:01
*** amotoki has joined #openstack-infra14:01
AJaegerzigo: you need create permissions - and those are not setup for it14:01
AJaegerigorbelikov: ah, you're fast. Thanks!14:01
zigoAJaeger: Can you review his patch?14:01
*** IlyaG has joined #openstack-infra14:01
andrewbogottI could use a hand with an improperly failing tempest test.  Had a few thoughts about what might have been wrong but now I'm out of ideas :(   Here's the output:  http://logs.openstack.org/02/298002/4/check/gate-tempest-dsvm-neutron-src-django_openstack_auth/8375bb0/14:02
AJaegerzigo: sure, later ;)14:02
*** regXboi has joined #openstack-infra14:02
*** akshai has quit IRC14:02
*** Liuqing has quit IRC14:02
*** mkoderer__ has joined #openstack-infra14:02
*** piet has quit IRC14:03
*** tojuvone has quit IRC14:03
*** tojuvone has joined #openstack-infra14:03
*** _ari_ has joined #openstack-infra14:03
*** ddieterly has joined #openstack-infra14:04
*** ddieterly has quit IRC14:04
*** daemontool_ is now known as daemontool14:06
isaacbclarkb: here?14:06
*** IlyaG has quit IRC14:06
*** hichihara has joined #openstack-infra14:08
*** rbrndt_ has joined #openstack-infra14:10
*** jcooley_ has joined #openstack-infra14:10
*** zz_dimtruck is now known as dimtruck14:10
*** eharney has joined #openstack-infra14:13
*** Goneri has joined #openstack-infra14:14
*** jcooley_ has quit IRC14:15
AJaegerpabelanger: thanks for your reviews. Could you review https://review.openstack.org/298627 as well, please? That sbcl removal from system-config14:15
*** markvoelker_ has quit IRC14:15
pabelangerAJaeger: done14:16
AJaegerthanks, pabelanger !14:16
pabelangerAJaeger: fungi: So, I wonder it we should first convert system-config (jenkins.pp) to use bindep.  Then we avoid the multi project patches14:17
*** tqtran has joined #openstack-infra14:17
fungipabelanger: we basically already did. now that we're not building bare-.* systems we should be able to strip a lot of that down14:18
AJaegerkeep in mind the proposal slaves, I doubt that they use install-distro-packages ;)14:19
pabelangerYa, I noticed when I did the wheel-mirror is was using that code14:20
pabelangeroh, and we still have bare-precise too :(14:20
pabelangerOh, I remember14:21
AJaegerfungi, pabelanger: Also, I think we have maven on ubuntu-trusty - how is that getting added?14:21
pabelangerfungi: so, if I want to stand up a new puppetdb.o.o with ubuntu-trusty, ansible will know it is a 2nd server right? And we won't break the original?14:21
*** dmellado|lunch is now known as dmellado14:21
*** tqtran has quit IRC14:21
AJaegerAlso, looking at openstack-infra/groups which according to mrmartin needs php5-cli: The tests do not install distro packages, so do we have php5-cli on the nodes? Or will the tests work without?14:22
openstackgerritIgor Belikov proposed openstack-infra/project-config: Add create option to deb-openstack-pkg-tools ACL  https://review.openstack.org/29874614:22
pabelangerAJaeger: I suspect we'll need some dependencies in puppet, but a lot from bindep-fallback could be removed14:22
AJaegerpabelanger: yolanda had some fun with ansible when she set up the second gerrit. Best talk with here.14:22
AJaegerpabelanger: so far my approach has been: Remove piece by piece everywhere where possible.14:23
mrmartinAJaeger: openstackid also depends on php5-cli14:23
yolandawell yes, we were hitting a bug with ansible inventory, bug this was fixed14:23
*** burgerk has joined #openstack-infra14:23
fungioh, right, bare-precise workers still. so no we can't easily trim down the package lists in puppet but it's probably safer to let them diverge if infra puppet testing is the only consumer of bare-trusty now14:23
pabelangerAJaeger: Right, I guess we keep down that path for now.14:23
*** kushal has joined #openstack-infra14:23
fungias for long-lived workers, the list of packages they need for their jobs can probably be trimmed substantially14:24
AJaegermrmartin: same problem - yes, openstackid uses bare-precise for testing14:24
pabelangerWhy doesn't nodepool manage long lived workers? It is because we want them pinned to jenkins.o.o and nodepool cannot do that?14:24
*** _amrith_ is now known as amrith14:24
fungipabelanger: it's not a feature we implemented in nodepool, but it's on the roadmap with zuul v314:25
AJaegerfungi, so what shall we do with php5-cli? REmove it from bindep since it'S only needed for groups and openstackci which run on bare-precise?14:25
fungibut the new model would be to rebuild and replace them periodically to get them updated since we wouldn't be able to update them through puppet any longer14:26
openstackgerritEric Harney proposed openstack-infra/project-config: devstack-plugin-nfs: Make tempest non-voting  https://review.openstack.org/29877414:27
*** sdake has joined #openstack-infra14:27
fungiAJaeger: there's not much urgency in trimming down the bindep fallback list. and once those servers get upgraded to run on trusty (hopefully soon after the summit) we'll need to be able to test them on ubuntu-trusty workers14:28
pabelangerYa, I want to see if I can do puppetdb.o.o replacement this week. Should be straightforward to do14:29
pabelangerno migration, already confirmed it works14:29
*** sdake_ has joined #openstack-infra14:30
clarkbisaacb: usually not for another hour or so14:30
openstackgerrityolanda.robla proposed openstack-infra/shade: (WIP) Use keystoneauth1.betamax for shade mocks  https://review.openstack.org/29864714:30
fungipabelanger: you worked on puppet-httpd quite a bit. do you happen to know how to get it to switch the apache2 mpm in use? needed by 29869914:30
AJaegerfungi: but shouldn't we give them a other-requirements file once we move to newer images?14:31
openstackgerritRossella Sblendido proposed openstack-infra/reviewday: Add a link for Neutron Gerrit dashboard  https://review.openstack.org/29877914:32
*** sdake has quit IRC14:32
pabelangerfungi: I suspect we need to migrate to puppetlabs-apache to do that14:32
AJaegerfungi, I know there's no urgency, it just involves many steps and thus I'd like to go through it one by one...14:32
pabelangersince it include apache::mpm class14:32
*** achanda has joined #openstack-infra14:33
*** ZZelle has joined #openstack-infra14:33
pabelangerfungi: is an example of the migration: https://review.openstack.org/#/c/205596/14:33
pabelangerthat is for nodepool14:33
pabelangerI could add code to puppet-openstackid today if needed14:33
*** mtanino has joined #openstack-infra14:34
smarcetfungi:AJaeger: httpd module works as expected i tested locally and when install php5-fpm also install apache2.4 with pmp events14:34
fungiAJaeger: we can, i'm just worried about losing track of what things we're installing with puppet on bare-precise that we need to remember to add when switching to ubuntu-trusty14:35
fungismarcet: right, what i'm discussing with pabelanger is how to get it to actually use apache2-mpm-event rather than apache2-mpm-prefork (which seems to be the default for the puppet-httpd module)14:35
*** MarkAtwood has quit IRC14:35
AJaegerfungi: yes, that's a challenge. Ok, I'll keep that in mind and will not work on those now that I realized the situation.14:36
*** sdake_ is now known as sdake14:37
anteayaAJaeger: thanks it is, it was like someone turned the tap off at midnight, so I got some sleep after all14:37
openstackgerritMarton Kiss proposed openstack-infra/groups: Remove map panel title from home  https://review.openstack.org/29878214:37
AJaegeranteaya: great!14:37
*** doug-fish has joined #openstack-infra14:38
anteayathanks, I think so too14:38
*** kushal has quit IRC14:39
openstackgerrityolanda.robla proposed openstack-infra/shade: (WIP) Use keystoneauth1.betamax for shade mocks  https://review.openstack.org/29864714:40
AJaegerzaro, fungi: I noticed that we have for gerrit only a single job in check and gate and only run this on specific branches. So, a change against master like https://review.openstack.org/#/c/298346/ will never merge. Do we need to run gate-gerrit-buck on other branches as well?14:40
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Re-allow list of networks for FIP assignment  https://review.openstack.org/29878914:41
*** Nikolay_St has quit IRC14:43
*** piet has joined #openstack-infra14:44
*** andymaier has joined #openstack-infra14:45
openstackgerritMerged openstack-infra/project-config: Add notifications for TripleO stable/mitaka  https://review.openstack.org/29868914:47
*** cloudtrainme has joined #openstack-infra14:48
*** mrmartin has quit IRC14:49
*** mrmartin has joined #openstack-infra14:49
*** roxanaghe has joined #openstack-infra14:49
fungiAJaeger: oh, yep, we've never actually used that branch14:50
*** kozhukalov has quit IRC14:51
*** kozhukalov has joined #openstack-infra14:51
fungiAJaeger: we should probably consider fixing up the master branch to just have a readme saying to look at the other branches14:52
AJaegerfungi: Ah...14:52
openstackgerritTom Cocozzello proposed openstack-infra/project-config: Keystone Experimental patch_cover gate job  https://review.openstack.org/29680814:52
*** isaacb has quit IRC14:52
*** bdemers has joined #openstack-infra14:53
*** jaypipes has joined #openstack-infra14:53
*** tosky has quit IRC14:53
*** mrmartin has quit IRC14:54
*** _degorenko|afk is now known as degorenko14:54
*** roxanaghe has quit IRC14:54
anteayaEmilienM: I'm so glad for the conclusion to the conversation on https://review.openstack.org/#/c/29786414:55
anteayaI agree, skipping jobs just leads to confusion14:55
anteayaI know I'm confused by it14:55
EmilienManteaya: me too14:55
*** maishsk has quit IRC14:55
EmilienMin a meeting atm but I want to re-add the skip-if one day14:56
* AJaeger will be back later...14:56
*** EricGonczer_ has joined #openstack-infra14:56
*** tosky has joined #openstack-infra14:57
openstackgerritAlexander Kislitsky proposed openstack-infra/project-config: fuel-stats: use db setup for python jobs  https://review.openstack.org/29880014:57
anteayaEmilienM: sorry to bother you in the meeting14:58
*** Douhet has quit IRC14:58
*** mylu has quit IRC14:58
anteayaif we are counting votes, I'm of the same opinion expressed by fungi and clarkb14:58
EmilienManteaya: no worries14:58
anteayaif the test is important enough to exist, it should run14:58
*** mylu has joined #openstack-infra14:59
*** Douhet has joined #openstack-infra14:59
EmilienManteaya: if we can find a skip-if rule that would work for sure, and we don't touch it much, I would be happy14:59
EmilienMit makes CI faster and also save nodepool resources14:59
*** dizquierdo has joined #openstack-infra14:59
*** cloudtrainme has quit IRC14:59
anteayaI think the provisions you put on the skip-if regex is the problem14:59
*** mylu_ has joined #openstack-infra15:00
*** cloudtrainme has joined #openstack-infra15:01
*** mylu has quit IRC15:01
*** knikolla has joined #openstack-infra15:01
pabelangerSo, looking at system-config, do we actually use ptrace on our jenkins slaves?15:02
pabelangera temporary solutions15:02
pabelangerwhich looks to be since 201215:02
*** pblaho has quit IRC15:05
dhellmannfungi : I need to recreate the stable/mitaka branch for openstack/manila-ui :-(15:06
dhellmannfungi : can you delete the current branch, please?15:06
openstackgerritPaul Belanger proposed openstack-infra/system-config: Remove 'Temporary for debugging glance' on jenkins slaves  https://review.openstack.org/29880615:06
dhellmannfungi : it only has the .gitreview update and the g-r sync on it right now15:06
*** openstackgerrit has quit IRC15:06
pabelangerokay, added patch^. With The logic that projects can use JJB today15:06
*** openstackgerrit has joined #openstack-infra15:06
dhellmannfungi : and there are no open reviews15:07
*** amotoki has quit IRC15:08
*** jcooley_ has joined #openstack-infra15:11
openstackgerrityolanda.robla proposed openstack-infra/shade: (WIP) Use keystoneauth1.betamax for shade mocks  https://review.openstack.org/29864715:12
fungidhellmann: okay, will do shortly15:13
dhellmannfungi : ty15:14
*** claudiub|2 has joined #openstack-infra15:15
*** jcooley_ has quit IRC15:16
*** baoli has joined #openstack-infra15:16
*** kushal has joined #openstack-infra15:16
*** baoli_ has quit IRC15:16
openstackgerritMerged openstack-infra/tripleo-ci: Use instack-qcow2 from cache if available and appropriate  https://review.openstack.org/28525915:17
*** claudiub has quit IRC15:18
*** arxcruz has quit IRC15:19
*** baoli_ has joined #openstack-infra15:20
*** ociuhandu_ has joined #openstack-infra15:20
*** salv-orl_ has joined #openstack-infra15:21
*** ociuhandu has quit IRC15:21
*** ociuhandu_ is now known as ociuhandu15:21
*** claudiub|2 has quit IRC15:22
*** baoli has quit IRC15:23
*** gordc has quit IRC15:23
*** claudiub|2 has joined #openstack-infra15:24
*** salv-orlando has quit IRC15:24
*** infra-red has quit IRC15:26
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Deploy ceph in the nonha job  https://review.openstack.org/29305115:26
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the upgrades job  https://review.openstack.org/28944515:26
*** isaacb has joined #openstack-infra15:26
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use --overcloud-update in the upgrades job  https://review.openstack.org/26046615:26
*** rcernin has quit IRC15:27
dansmithclarkb: hey, the latest patchset on this: https://review.openstack.org/#/c/293155/ has failed job, but I don't know enough to know if it's related or not15:28
clarkbdansmith: I probably failed at puppet, will dig into it15:29
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Deploy ceph in the nonha job  https://review.openstack.org/29305115:29
dansmithclarkb: okay, once that is worked out, is that good to go or is it blocked on anything else?15:29
clarkbI need to make an afs volume15:30
clarkband "mount" it to the correct location15:30
*** Swami has joined #openstack-infra15:30
dansmithclarkb: okay is that an involved process?15:30
dansmithclarkb: just trying to get some idea of how long until we can get this voting again15:31
clarkbI have no idea. never done it before15:31
dansmithheh, okay15:31
dansmithclarkb: can we make mordred do it?15:31
clarkbbut pabelanger got the centos wheel mirror volume up in a day so hopefully not bad15:31
dansmithcan we make pabelanger do it? :D15:31
mordredit's easy15:32
*** links has joined #openstack-infra15:32
mordredbut I'm in board meeting all day today15:32
mordredso don't block on me15:32
*** bdemers has quit IRC15:32
pabelangerYa, I can help15:32
*** jcooley_ has joined #openstack-infra15:33
clarkbok pabelanger maybe in an hour or so after lunch/breakfast we can sort it out!15:33
*** bdemers has joined #openstack-infra15:33
pabelangerclarkb: WFM15:33
dansmithclarkb: pabelanger thanks dudes15:33
*** amitgandhinz has quit IRC15:33
fungitoday it's clarkb's turn to mix up sentence punctuation15:35
* fungi ceremoniously passes the trophy along15:35
clarkbI accept this trophy¡15:36
anteayaI consider that yoda speak15:37
anteayaand picture anyone using it sporting green ears15:37
*** gyee has joined #openstack-infra15:41
*** harlowja has quit IRC15:41
anteayaso zuul restart15:42
anteayait is heavily into swap15:42
anteayaany infra-root have any thoughts on that?15:42
*** jamesmcarthur has joined #openstack-infra15:42
*** huzhj__ has joined #openstack-infra15:43
*** piet has quit IRC15:43
*** derekh has quit IRC15:44
fungianteaya: jeblair was working on disentangling the suspect changes to try a manual revert of one along with a restart and see if it identified the source of te leak. on a positive note, the paging doesn't seem to be adversely impacting cpu/load http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2064&rra_id=all15:44
*** piet has joined #openstack-infra15:44
*** yamahata has quit IRC15:44
anteayafungi: oh good, thanks15:44
fungibut we do need to get it sorted before we exhaust virtual memory15:44
anteayaI'll wait to hear from jeblair15:45
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use --overcloud-update in the upgrades job  https://review.openstack.org/26046615:45
*** nmagnezi has quit IRC15:45
fungier, those were the wrong graphs15:45
*** gordc has joined #openstack-infra15:46
*** angdraug has joined #openstack-infra15:46
fungianyway, steady state even after paging began15:46
*** GheRivero has quit IRC15:46
*** GheRivero has joined #openstack-infra15:46
*** huzhj_ has quit IRC15:46
fungifrom the current rate of growth in memory utilization, i estimate we have roughly a week before we exhaust all available virtual memory15:47
anteayaokay thanks15:47
*** amitgandhinz has joined #openstack-infra15:47
*** dprince has quit IRC15:47
*** dprince has joined #openstack-infra15:47
anteayaglad to have that estimate as an outside perameter15:47
anteayaperhaps I am mis-interpretting swap15:48
anteayait is my understanding that usage of swap is indicative of a decrease in performance15:48
*** roxanagh_ has joined #openstack-infra15:50
*** huzhj_ has joined #openstack-infra15:50
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Tempest for tripleo CI  https://review.openstack.org/29584415:51
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Run tempest tests with non-HA jobs  https://review.openstack.org/29703815:51
*** dtantsur is now known as dtantsur|afk15:53
*** armax has joined #openstack-infra15:54
openstackgerritDean Troyer proposed openstack/requirements: Bump python-openstackclient to 2.2.1 in u-c  https://review.openstack.org/29884115:54
*** huzhj__ has quit IRC15:54
*** roxanagh_ has quit IRC15:55
*** kushal has quit IRC15:55
*** isaacb has quit IRC15:57
fungianteaya: it's an indicator that the system has run out of ram to allocate more memory, and so is starting to move some memory allocations to disk. if it's not accessing those allocations regularly, then it may not impact performance at all15:58
*** ilyashakhat has joined #openstack-infra15:58
*** hashar has quit IRC15:59
*** gokrokve has joined #openstack-infra15:59
*** samuelBartel has quit IRC16:00
*** ilyashakhat__ has joined #openstack-infra16:01
anteayaah, thank you for clearing up my mis-understanding16:01
*** Qiming has quit IRC16:03
*** esker has joined #openstack-infra16:03
*** Guest58782 has quit IRC16:03
*** links has quit IRC16:04
*** vikram_ has quit IRC16:05
*** jlanoux has quit IRC16:06
*** jtomasek has quit IRC16:07
*** maishsk has joined #openstack-infra16:08
mriedemanyone know if there is a problem with the g-r sync bot being busted on kilo?16:08
fungimriedem: the job being run in the post pipeline might have been broken by something on that branch. i'll take a look16:09
*** LaT0rtue_ has joined #openstack-infra16:10
*** scheuran has quit IRC16:11
*** hichihara has quit IRC16:11
*** baoli has joined #openstack-infra16:12
*** LaT0rtue has quit IRC16:12
*** matrohon has quit IRC16:12
*** esker has quit IRC16:13
*** mylu_ has quit IRC16:13
*** bdemers has quit IRC16:13
anteayadoes a channel op have time to refresh the channel topic in -meeting?16:13
*** mylu has joined #openstack-infra16:13
*** baoli__ has joined #openstack-infra16:13
*** esker has joined #openstack-infra16:13
*** kzaitsev_mb has quit IRC16:13
*** jistr has quit IRC16:14
*** maishsk_ has joined #openstack-infra16:14
*** GheRivero has quit IRC16:14
*** baoli_ has quit IRC16:14
*** bdemers has joined #openstack-infra16:15
*** maishsk has quit IRC16:15
*** maishsk_ is now known as maishsk16:15
*** bdemers has quit IRC16:16
*** baoli has quit IRC16:16
*** GheRivero has joined #openstack-infra16:16
*** bdemers has joined #openstack-infra16:17
*** yamahata has joined #openstack-infra16:19
*** ilyashakhat__ has quit IRC16:21
fungianteaya: done--thanks for spotting16:23
*** flepied1 has joined #openstack-infra16:23
*** IlyaG has joined #openstack-infra16:23
*** eil397 has joined #openstack-infra16:24
pleia2good morning16:24
anteayafungi: thank you16:24
anteayamorning pleia216:24
*** kzaitsev_mb has joined #openstack-infra16:24
abregmangood morning pleia216:25
anteayaabregman: hello there16:25
abregmananteaya: hello :)16:25
fungidhellmann: i've deleted the errant stable/mitaka branch from openstack/manila-ui as requested (head was 850a6130bc754229fce22c76f0ca288e54f53240)16:25
anteayaabregman: I've been thinking about you, how are you doing?16:25
*** cloudtrainme has quit IRC16:25
*** flepied has quit IRC16:25
clarkbpabelanger: ok back from breakfast16:25
clarkbI will start looking at why thta patch failed first so it can get tested again soon16:25
pabelangerclarkb: ack16:26
*** dizquierdo has quit IRC16:26
*** jsavak has quit IRC16:26
anteayaclarkb: looks to me like puppet failures as you say, looks like cloning or dependency issues16:26
*** maishsk has quit IRC16:26
*** Nikolay_St has joined #openstack-infra16:27
abregmananteaya: well..it's complicated :). I think you were right about infra-ansible. not much activity there :|. currently I'm just looking on infra projects to implement in our internal environment...I'm still not able to find practical small tasks in infra projects to work on.16:27
abregmananteaya: thanks for asking16:27
clarkbError: Parameter key_id failed on Gnupg_key[Ceph Archive]: Invalid key id d41d8cd98f00b204e9800998ecf8427e what makes it invalid16:28
*** nadya has quit IRC16:28
clarkbmaybe it only wants the last 16 'digits'16:28
*** harlowja has joined #openstack-infra16:29
*** ilyashakhat__ has joined #openstack-infra16:29
anteayaabregman: have you spent any more time reading backscroll?16:29
*** jsavak has joined #openstack-infra16:29
anteayaabregman: having a big picture concept really does help the small choices16:29
*** derekh has joined #openstack-infra16:29
fungiclarkb: is this still that gowhatsit/pgp puppet module? it sounds like a security nightmare16:29
anteayaabregman: even though some folks don't believe it is worth the time16:29
huzhj_Hi, anyone know how to login https://openstackid.org/accounts/user/login with ubuntuID? I want to login and make myself a fundation member16:29
*** ilyashakhat__ has quit IRC16:29
anteayamorning zaro16:29
abregmananteaya: you think I should keep reading backscroll? every day?16:30
anteayaabregman: yes16:30
*** tphummel has joined #openstack-infra16:30
*** lezbar has quit IRC16:30
*** tqtran has joined #openstack-infra16:30
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add ubuntu-xenial DIB to nodepool  https://review.openstack.org/29798116:30
anteayaabregman: you ask really good questions when you read the backscroll16:30
fungihuzhj_: it's a completely separate account you need to create. you can't log into it with an ubuntuid16:30
pabelangerand like that, ubuntu-xenial dibs are working ^16:30
*** arxcruz has joined #openstack-infra16:30
anteayaabregman: you don't have to do the exercise of writing everying down and prioritizing16:31
pabelangerjust booted it locally and things seem to work as expected16:31
*** yamamoto has quit IRC16:31
clarkbfungi: its https://github.com/n1tr0g/golja-gnupg16:31
anteayaabregman: that was an exercise so I could give you feedback on how you were reading things, and I think you have good instincts16:31
clarkbfungi: I am just reusing what reprepro already uses16:31
huzhj_thanks fungi. I currenlty encountered this problem: https://ask.openstack.org/en/question/56720/cannot-store-contact-information-when-updating-info-in-openstack-gerrit/ with my ubuntuID16:31
*** e0ne has quit IRC16:31
openstackgerritClark Boylan proposed openstack-infra/system-config: Apt mirror for ceph hammer release  https://review.openstack.org/29315516:31
clarkblets see if ^ does any better16:32
fungiclarkb: got it. i'm sorry i either missed reviewing that implementation in the reprepro module or neglected to make my objection to the golja-gnupg terribleness strong/lasting enough16:32
abregmananteaya: I guess I can try keeping doing that...at least until my number of questions per day will be very little16:32
*** ramishra has quit IRC16:32
abregmananteaya: I'll do it. thank you again. :)16:32
fungihuzhj_: please read and follow http://docs.openstack.org/infra/manual/developers.html#account-setup16:33
huzhj_fungi: Thanks.16:33
anteayaabregman: you only learn if you are willing to ask questions16:33
anteayaabregman: and you are asking very good ones16:33
anteayaeveryone wants to jump to the tangibleness of a patch, but often can land in a hole and not realize it16:34
*** mylu has quit IRC16:34
*** jaypipes has quit IRC16:34
*** mylu has joined #openstack-infra16:34
anteayaget the big picture, it is of far more value to both you and the rest of infra if you do16:34
*** rossella_s has quit IRC16:34
anteayaeven if you can't justify it to yourself at the moment16:34
*** mylu has quit IRC16:34
*** sshnaidm has quit IRC16:34
*** mylu has joined #openstack-infra16:34
*** mylu has quit IRC16:35
*** rossella_s has joined #openstack-infra16:35
*** GheRivero has quit IRC16:36
abregmananteaya: I have a feeling you are right. someone already told you are a good motivator? :)16:36
DuncanTHi, I'm doing a bit of scripting against the wiki, and I've found we have  a*lot* of spam pages. Is anybody interested in a list so that they can be pruned? We're currently boosting the pagerank of a bunch of scammers16:37
*** achanda has quit IRC16:37
*** ihrachys has quit IRC16:37
*** armax has quit IRC16:37
anteayaabregman: ha ha ha16:38
Guest68910DuncanT: I think infra is already aware as there is a very lengthy thread about the subject on the list: http://lists.openstack.org/pipermail/openstack-infra/2016-March/004038.html16:38
*** Guest68910 is now known as mgagne16:38
fungiDuncanT: absolutely. we've got a discussion going for a while on the openstack-infra@lists.openstack.org ml about ongoing cleanup and long-term plans to mitigate16:38
DuncanTI can't promise an exhaustive list, but I can give one of entries I've confirmed are highly likely spam (example: https://wiki.openstack.org/wiki/1-888-361-3731_Gmail_Customer_service_number_Helpline_Number)16:38
*** SumitNaiksatam has quit IRC16:38
*** mgagne has quit IRC16:38
*** mgagne has joined #openstack-infra16:38
anteayaabregman: the problem is that is it is tough, since you have to spend time in mental places where you don't feel comfortable, the trick is to keep doing that and get information so you do feel comfortable, not stay in old habits to avoid the discomfort16:39
fungii suspect that missed the 1-800 filter by being 1-88816:39
*** huzhj_ has quit IRC16:39
DuncanTThere's a bunch of 1800 pages, 1844, etc16:40
openstackgerritAndrey Nikitin proposed openstack-infra/puppet-gerrit: Order of the classes parameters is refactored  https://review.openstack.org/29472816:40
anteayaI wonder if we can regex on 1-8xx16:40
*** cloudtrainme has joined #openstack-infra16:40
DuncanTfungi: I'll email you a list when I have it if that's useful? I don't fancy scripting page deletion myself at the moment16:41
pabelangerDuncanT: fungi: Agreed, if you can follow up on the infra ML, that would be great16:41
bkeroWhy not just regex on a 10 digit number starting with 1 and having 2 optional dashes?16:41
fungiDuncanT: the wiki also keeps their history even after deletion, so depending on how you're searching you may also turn up already deleted pages (though that's one we definitely haven't nuked yet)16:41
*** derekh has quit IRC16:41
pabelangerI should also note, we haven't have any new spam in about 2 weeks16:41
pabelangerI haven't had to block any new accounts16:41
pabelangerexisting accounts16:41
DuncanTfungi: There are things like https://wiki.openstack.org/wiki/AMERICAN_aIRlINES_1%5E800%5E611%5E0092_rESERVAtiONS_nUMBER,_aMERICAN_aIRLINES too that are moderately hard to regex16:41
anteayapabelanger: what is the status of the wiki, as DuncanT we still seem to have lots of spam16:41
fungiDuncanT: pabelanger did some mass deletions via regular expression/globbing match not long ago so maybe helping him improve the matches he used would be the best benefit16:42
anteayapabelanger: is the conversation progressing or should we discuss it at the meeting today?16:42
fungibut yeah, some things just aren't safe to do broad matching on16:42
pabelangeranteaya: DuncanT: We have SmiteSpam installed but likely need to tweak our spam settings to get the outstanding pages16:42
anteayaso how many existing pages of spam do we have?16:42
DuncanTpabelanger: Do you regex against the content or just the title?16:42
*** LaT0rtue has joined #openstack-infra16:42
anteayawhat is the current health status of the wiki?16:42
pabelangerDuncanT: we use the defaults for SmiteSpam16:43
*** LaT0rtue_ has quit IRC16:43
DuncanTanteaya: About 100 pages I've spotted so far16:43
pabelangerand Nuke16:43
anteayaDuncanT: thanks16:43
anteayahow can someone delete those pages16:43
anteayaI had offered before and I'll offer again16:43
openstackgerritDoug Hellmann proposed openstack-infra/project-config: update tag ACLs for all official projects  https://review.openstack.org/29886616:43
AJaegeranteaya: do you see any reason to keep these files around in project-config? https://review.openstack.org/298410 I don't...16:43
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the upgrades job  https://review.openstack.org/28944516:43
anteayaAJaeger: looking16:43
*** armax has joined #openstack-infra16:44
anteayaAJaeger: that was added in the first place for historical reasons16:44
pabelangerDuncanT: are you using a mediawiki extension to delete the spam?16:44
anteayait was never added in the belief that it would be used a second time16:44
*** jerryz has joined #openstack-infra16:44
DuncanTpabelanger: No, I'm writing some scraping software, and stumbled across the spam16:45
AJaegeranteaya: so, time to remove it, isn't it?16:45
*** sfinucan has quit IRC16:45
*** Guest38332 has quit IRC16:45
anteayaAJaeger: what harm is it causing?16:45
anteayaI think it is useful for being a reference for a large operation we performed in the past16:46
AJaegeranteaya: I saw already a patch for it ;/16:46
AJaeger(a patch that wanted to change stuff there)16:46
anteayacare to share that patch?16:46
*** robcresswell is now known as grand-poobah-of-16:47
* AJaeger didn't bookmark it ;(16:47
DuncanTanteaya: pabelanger: I'm up to a few hundred probable spam items now16:47
*** grand-poobah-of- is now known as robcresswell16:47
anteayaDuncanT: I believe you, thanks for sharing your experience16:47
*** annegentle has quit IRC16:47
AJaegerfor me it's just cruft that we don't need - and for reference we have the history.16:47
anteayaI wish I could do something useful to help you and despite my attempts to try I don't see how I can16:47
*** jordanP has quit IRC16:47
pabelangerDuncanT: if we can add them to https://www.mediawiki.org/wiki/Extension:SpamBlacklist then we can delete them using an update script16:47
*** ifarkas has quit IRC16:48
*** pcaruana has quit IRC16:48
pabelangerI just haven't looked into the syntax for it yet16:48
*** jaypipes has joined #openstack-infra16:48
anteayaAJaeger: well delete it then, my response would be just -2 the patch that tries to change it16:48
clarkbpabelanger: apt-get is compiling afs client kernel modules for me currently16:49
pabelangerDuncanT: which should be this page: https://wiki.openstack.org/wiki/MediaWiki:Spam-blacklist16:49
*** jtomasek has joined #openstack-infra16:49
fungianteaya: would you mind if i added you to the wiki bureaucrat group? i wouldn't want you spending a lot of time deleting pages when we can get better matches in our existing filters/batching but for one-off things that slip through, more hands helping with cleanup there would be helpful16:49
clarkbI think I have a kerberos user and admin user already but don't know that it is part of the afs managemtn yet16:49
pabelangeror https://wiki.openstack.org/wiki/Spam_blacklist16:50
pabelangerclarkb: ack16:50
clarkbwill sort tht out as soon as my lap is no longer a space heater16:50
*** asselin_ has joined #openstack-infra16:50
AJaegeranteaya: I was looking for a reason to keep that besides "for historic" - and yes, I gave -1 to that change and can't find it anymore...16:51
DuncanTpabelanger: That only seems to cover links though, not free text - a lot of this spam is phone number spam, with no links - probably to get around that sort of blacklist16:51
anteayafungi: I'd be delighted to be added to the wiki bureaucrat group, thank you16:51
pabelangerDuncanT: I think it does wiki page titles too16:51
pabelangerDuncanT: but haven't tested it yet16:52
anteayafungi: I'd be grateful to be able to delete spam16:52
pabelangeranteaya: +116:52
anteayaAJaeger: okay, my only reason to keep it is historic16:52
*** sridhar_ram has joined #openstack-infra16:52
DuncanTpabelanger: I'll create a list of page titles, and put it on pastebin - that should let somebody have a crack at cleanup. I'll try to generate some regex too16:53
pabelangerDuncanT: WFM16:53
*** piet has quit IRC16:53
*** GheRivero has joined #openstack-infra16:53
DuncanTpabelanger: If somebody notices my scripting is hitting the wiki too hard, please shout. Should be reasonably gentle though16:54
fungiDuncanT: i seriously doubt it would be worse than the search engine crawlers, spambots trying to hammer their way in, et cetera16:54
DuncanTfungi: It shouldn't be, no - I was mostly being polite. I'm running from behind a fairly terrible ADSL line after all16:55
*** armax_ has joined #openstack-infra16:55
*** abregman has quit IRC16:55
anteayaDuncanT: thank you for being polite :)16:55
*** erlon has joined #openstack-infra16:56
anteayaand for sharing your findings16:56
*** jcooley_ has quit IRC16:56
*** armax has quit IRC16:56
*** armax_ is now known as armax16:56
fungianteaya: you're an admin/bureaucrat on the wiki now, so can help with various advanced tasks at your convenience16:58
*** harlowja_ has joined #openstack-infra16:58
*** dimtruck is now known as zz_dimtruck16:58
fungiDuncanT: is there any other kind of adsl line?16:58
*** jcooley_ has joined #openstack-infra16:58
DuncanTfungi: There are many less-terrible-than-this ADSL lines. In fact there are less-terrible-than-this pieces of wet string....16:59
anteayafungi: thanks, I'll take a look and read any guides I find about doing the right things16:59
DuncanT(to be fair, you can put a gigabit down a hundred metres of wet string if you don't mind making every radio spectrum user in the area cry)16:59
*** harlowja has quit IRC17:00
fungi(and possibly drawing the wrath of the fcc in the process)17:00
DuncanTfungi: Indeed17:00
*** mgagne_ has joined #openstack-infra17:06
*** tpsilva has quit IRC17:07
*** dtroyer has quit IRC17:07
*** zhiyan has quit IRC17:07
*** zz_dimtruck has quit IRC17:07
*** kambiz has quit IRC17:07
*** betherly has quit IRC17:07
*** ryanpetrello has quit IRC17:07
*** vaidy_ril has quit IRC17:07
*** isviridov_away has quit IRC17:07
*** rguillebert has quit IRC17:07
*** igormarnat has quit IRC17:07
*** fmccrthy has quit IRC17:07
*** thomasem has quit IRC17:07
*** dham1 has quit IRC17:07
*** igorbelikov has quit IRC17:07
*** skath_ has quit IRC17:07
*** sweston has quit IRC17:07
*** warpc has quit IRC17:07
*** dougwig has quit IRC17:07
*** monester has quit IRC17:07
*** baoli__ has quit IRC17:07
*** jamesmcarthur has quit IRC17:07
*** mtanino has quit IRC17:07
*** teran has quit IRC17:07
*** dmsimard has quit IRC17:07
*** vkramskikh has quit IRC17:07
*** blogan has quit IRC17:07
*** gsilvis has quit IRC17:07
*** zaro has quit IRC17:07
*** docaedo has quit IRC17:07
*** melwitt has quit IRC17:07
*** edwarnicke has quit IRC17:07
*** mkwiek has quit IRC17:07
*** sballe_ has quit IRC17:07
*** tpeoples has quit IRC17:07
*** adam_g has quit IRC17:07
*** jasondotstar has quit IRC17:07
*** akshai_ has quit IRC17:07
*** aysyd has quit IRC17:07
*** mjb has quit IRC17:07
*** mgagne has quit IRC17:07
*** comstud has quit IRC17:07
*** cody-somerville has quit IRC17:07
*** rkukura has quit IRC17:07
*** cj has quit IRC17:07
*** vtapia has quit IRC17:07
*** briancline has quit IRC17:07
*** hockeynut has quit IRC17:07
*** dhellmann has quit IRC17:07
*** zzzeek has quit IRC17:07
*** verdurin has quit IRC17:07
*** darrenc has quit IRC17:07
*** sigmavirus24 has quit IRC17:07
*** eglute has quit IRC17:07
*** kuromagi has quit IRC17:07
*** lewo has quit IRC17:07
*** gus has quit IRC17:07
*** mika has quit IRC17:07
*** markmcclain has quit IRC17:07
*** Nakato has quit IRC17:07
*** cloudnull has quit IRC17:07
*** flip214 has quit IRC17:07
*** bastelfreak has quit IRC17:07
*** mhu has quit IRC17:07
*** craige has quit IRC17:07
*** tbarron has quit IRC17:07
*** jamielennox has quit IRC17:07
*** vkmc has quit IRC17:07
*** xgerman has quit IRC17:07
*** fungi has quit IRC17:07
*** rbradfor has quit IRC17:07
*** Anticimex has quit IRC17:07
*** tosky has quit IRC17:07
*** mcarden has quit IRC17:07
*** phschwartz has quit IRC17:07
*** mfisch has quit IRC17:07
*** d34dh0r53 has quit IRC17:07
*** mmedvede has quit IRC17:07
*** tbarron has joined #openstack-infra17:07
*** eglute has joined #openstack-infra17:07
*** zz_dimtruck has joined #openstack-infra17:07
*** dtroyer has joined #openstack-infra17:07
*** comstud has joined #openstack-infra17:07
*** zz_dimtruck is now known as dimtruck17:07
*** d34dh0r53 has joined #openstack-infra17:07
*** kuromagi has joined #openstack-infra17:07
*** sdake_ has joined #openstack-infra17:07
pabelangerfungi: clarkb: initial thoughts on moving sysctl calls to ptrace to JJB tempaltes? https://review.openstack.org/#/c/298806/17:07
*** rkukura_ has joined #openstack-infra17:07
*** gyee has quit IRC17:07
*** baoli has joined #openstack-infra17:07
*** akshai has joined #openstack-infra17:07
*** mtanino_ has joined #openstack-infra17:07
*** Swami_ has joined #openstack-infra17:07
*** sdake has quit IRC17:07
*** jamesmcarthur_ has joined #openstack-infra17:07
*** tosky_ has joined #openstack-infra17:07
*** aysyd_ has joined #openstack-infra17:07
*** Nakato_ has joined #openstack-infra17:07
*** mcarden_ has joined #openstack-infra17:07
*** mikap has joined #openstack-infra17:07
*** kambiz_ has joined #openstack-infra17:07
*** flip214_ has joined #openstack-infra17:07
*** phschwartz_ has joined #openstack-infra17:07
*** hockeynut_afk has joined #openstack-infra17:07
*** dmsimard1 has joined #openstack-infra17:07
*** vtapia_ has joined #openstack-infra17:07
*** gsilvis_ has joined #openstack-infra17:07
*** ryanpetrello_ has joined #openstack-infra17:07
*** markmcclain1 has joined #openstack-infra17:07
*** briancli1e has joined #openstack-infra17:07
*** cody-somerville_ has joined #openstack-infra17:07
*** Anticime1 has joined #openstack-infra17:07
*** cj__ has joined #openstack-infra17:07
*** thomasem_ has joined #openstack-infra17:07
*** Swami_ has quit IRC17:07
*** kambiz_ is now known as kambiz17:07
*** kinrui has joined #openstack-infra17:07
*** jamesmcarthur_ is now known as jamesmcarthur17:07
*** markmcclain1 is now known as markmcclain17:07
*** rkukura_ is now known as rkukura17:07
*** ryanpetrello_ is now known as ryanpetrello17:07
*** adam_g` has joined #openstack-infra17:07
*** dhellmann has joined #openstack-infra17:07
*** mikap is now known as mika17:07
*** mjb has joined #openstack-infra17:07
*** hockeynut_afk is now known as hockeynut17:07
*** kinrui is now known as fungi17:07
*** mgagne_ has quit IRC17:07
*** mgagne_ has joined #openstack-infra17:07
*** craige has joined #openstack-infra17:08
clarkbpabelanger: is that bug still open? maybe if it hasn't had any updates in a couple years we just remove the things? and if glance wants them bakc they have sudo and can write them17:08
clarkbI am utterly failing at kerberos17:08
clarkbto kadmin.local do I need to be root on the kdc?17:08
clarkband whtas with the silly :s?17:08
*** melwitt has joined #openstack-infra17:08
*** aysyd_ is now known as aysyd17:08
*** _sigmavirus24 has joined #openstack-infra17:08
*** Guest28049 has joined #openstack-infra17:08
*** lewo has joined #openstack-infra17:08
*** zaro has joined #openstack-infra17:08
*** blogan has joined #openstack-infra17:08
*** rbradfor has joined #openstack-infra17:08
*** vkramskikh has joined #openstack-infra17:08
*** igormarnat has joined #openstack-infra17:08
*** cloudnull has joined #openstack-infra17:08
*** jasondotstar has joined #openstack-infra17:08
*** mriedem has quit IRC17:08
anteayahow long should the smitespam page take to load?17:08
*** adam_g` is now known as adam_g17:08
*** darrenc has joined #openstack-infra17:08
*** bastelfreak has joined #openstack-infra17:08
*** verdurin has joined #openstack-infra17:08
*** adam_g has quit IRC17:08
*** adam_g has joined #openstack-infra17:08
*** zzzeek has joined #openstack-infra17:08
*** gus has joined #openstack-infra17:09
*** apoorvad has joined #openstack-infra17:09
clarkbI just want to run listprincs to see if I have a principal or not17:09
pabelangerclarkb: it was eventually merged. I don't think we ever removed the hack. But I agree, now that people have sudo on the slaves, it makes more sense to let projects enable it if needed17:09
*** vkmc has joined #openstack-infra17:09
pabelangerclarkb: yes, you need root for kadmin.local17:09
*** lucasagomes is now known as lucas-dinner17:09
*** jaypipes has quit IRC17:11
*** SumitNaiksatam has joined #openstack-infra17:11
clarkbpabelanger: that attempts to auth with a password17:11
anteayait finally loaded17:12
clarkboh maybe I misread the output17:12
clarkbthis is a prompt? it ignored my command line request? woo17:12
*** isviridov_away has joined #openstack-infra17:12
*** jamielennox has joined #openstack-infra17:12
clarkbok I do not have a principal I sewar I had added one17:13
* clarkb works on adding these things17:13
*** yamamoto has joined #openstack-infra17:14
*** daemontool has quit IRC17:14
*** _sigmavirus24 is now known as sigmavirus2417:14
*** sigmavirus24 has joined #openstack-infra17:14
clarkbI am on kdc01 do I need to make sure I am on the master vs slave?17:14
anteayaI found a new page that is spam, not a previous page that was taken over, I am about to click Actions > Delete in my wikipage options17:14
anteayastop me if I should be doing something different than Actions > Delete17:14
*** cody-somerville_ has quit IRC17:14
*** mriedem has joined #openstack-infra17:15
*** vaidy has joined #openstack-infra17:15
*** cody-somerville has joined #openstack-infra17:15
fungisounds right to me17:15
anteayathank you17:15
fungikeep in mind deletes can be undone (at least until we decide whether to purge them)17:15
*** mgagne_ is now known as mgagne17:15
* anteaya proceeds17:15
*** tiswanso has quit IRC17:15
anteayaah thanks, I didn't know that17:15
pabelangerclarkb: I believe I used kdc0117:16
clarkbdocs don't say I need to check so just using 0117:16
anteayaI'm putting vandalism as the reason for deleting17:16
anteayashould I click watch this page?17:16
*** tiswanso has joined #openstack-infra17:17
*** monester has joined #openstack-infra17:17
*** rguillebert has joined #openstack-infra17:17
*** fmccrthy has joined #openstack-infra17:17
* anteaya checks "watch this page"17:17
*** MarkAtwood has joined #openstack-infra17:17
*** igorbelikov has joined #openstack-infra17:17
*** zhiyan has joined #openstack-infra17:17
*** tpsilva has joined #openstack-infra17:17
pabelangerclarkb: I then use afsdb01 for my pts / bos commands17:17
*** betherly has joined #openstack-infra17:18
*** EricGonczer_ has quit IRC17:18
*** xgerman has joined #openstack-infra17:18
clarkbI guess I could run all these commands locally on ^ and use -localauth too ?17:18
*** mkwiek has joined #openstack-infra17:19
*** tpeoples has joined #openstack-infra17:19
*** dham1 has joined #openstack-infra17:19
*** teran has joined #openstack-infra17:19
*** skath_ has joined #openstack-infra17:19
*** ajmiller has joined #openstack-infra17:20
clarkbpabelanger: does http://docs.openstack.org/infra/system-config/afs.html#adding-a-user work if your user isn't already added? eg do I need to use -localauth for that regardless?17:20
clarkbor is that more of a just map uids to usernames step?17:20
*** harlowja has joined #openstack-infra17:20
*** warpc has joined #openstack-infra17:21
clarkbanyways /me attemps such things in a moment17:21
*** sweston has joined #openstack-infra17:21
pabelangerclarkb: Right, you need to use -localauth regardless because you are creating the user17:21
pabelangerif I created your user, I don't need it17:21
*** sballe_ has joined #openstack-infra17:21
*** dmsimard1 is now known as dmsimard17:21
*** dougwig has joined #openstack-infra17:21
*** ilyashakhat has quit IRC17:21
*** igorbelikov has quit IRC17:22
*** sballe_ has quit IRC17:22
*** edwarnicke has joined #openstack-infra17:22
*** harlowja_ has quit IRC17:22
clarkbpabelanger: as root again?17:22
pabelangerclarkb: yes17:23
*** igorbelikov has joined #openstack-infra17:23
*** sballe_ has joined #openstack-infra17:23
*** ajmiller_ has joined #openstack-infra17:24
*** sshnaidm has joined #openstack-infra17:26
*** ajmiller has quit IRC17:27
*** esikachev has quit IRC17:28
anteayathey are also using tildes between numbers not just hyphens17:28
*** EricGonczer_ has joined #openstack-infra17:28
*** Daviey has joined #openstack-infra17:29
*** amrith is now known as _amrith_17:29
pabelangeranteaya: so what I tend to do, is once you find an account that is spamming I go to their contrib page. EG: https://wiki.openstack.org/wiki/Special:Contributions/Andoralifs17:29
anteayaI'll do that as well17:30
pabelangeranteaya: then I click block, then do a mass delete too17:30
*** docaedo_ has joined #openstack-infra17:30
pabelangerwhich should get all their pages they ever created17:30
anteayaawesome, much simpler than page at a time17:30
anteayathank you17:30
*** docaedo_ is now known as docaedo17:30
anteayacan I assume all pdfs are deleted?17:31
anteayaduckduckgo is returning a lot of hits on pdfs17:31
*** mmedvede has joined #openstack-infra17:31
*** eharney has quit IRC17:32
openstackgerritSean Dague proposed openstack-infra/project-config: create api-ref publishing pipeline  https://review.openstack.org/29888817:32
sdagueAJaeger: if you are still around ^^^17:32
pabelangeranteaya: For the most part. I've checked 1 or 2 in the past. But all likely spam17:32
AJaegersdague: again around ;)17:33
*** sdake has joined #openstack-infra17:33
*** achanda has joined #openstack-infra17:33
openstackgerritMonty Taylor proposed openstack-infra/shade: Support provider networks in public network detection  https://review.openstack.org/29889017:33
*** sdake_ has quit IRC17:34
sdagueAJaeger: my motivation is getting a thing which will build docs draft in check17:35
sdagueand we'll figure out the full publish story later17:35
*** amitgandhinz has quit IRC17:35
AJaegersdague: besides the swapped non-voting - see comments, it looks fine.17:36
*** amitgandhinz has joined #openstack-infra17:36
AJaegersdague: I have a review to trade in ;) https://review.openstack.org/#/c/298316/17:36
*** eil397 has quit IRC17:37
*** dimtruck is now known as zz_dimtruck17:37
anteayapabelanger: right, so my question is can I assume the pdfs that show up in my search engine results are already deleted?17:38
sdagueAJaeger: apparently the first one is non-voting17:38
*** roxanaghe has joined #openstack-infra17:38
pabelangeranteaya: Hmm, not sure about that. I never used a search engine to find spam17:39
AJaegersdague: you made the post job non-voting, I think you want the gate check non-voting17:39
*** gokrokve has quit IRC17:39
AJaegersdague: you're right, the first is non-voting already17:39
sdagueAJaeger: right17:39
openstackgerritSean Dague proposed openstack-infra/project-config: create api-ref publishing pipeline  https://review.openstack.org/29888817:39
sdagueI fixed the second one17:39
AJaegersdague: thanks! Then I'll +2...17:39
sdagueanyone else want to take a look here?17:40
*** cdent has quit IRC17:40
*** pvaneck has joined #openstack-infra17:40
sdaguehuh, ceilo used spider monkey?17:40
AJaegersdague: ages ago...17:41
*** cdent has joined #openstack-infra17:41
AJaegersdague: thanks17:41
*** ajmiller has joined #openstack-infra17:41
*** akaszuba has quit IRC17:41
anteayapabelanger: so the wiki still has pdfs that are not deleted?17:42
*** ajmiller_ has quit IRC17:42
anteayaI had thought you implimented a tool early on that removed pdfs17:42
anteayadid I mis-understand?17:42
*** smarcet has quit IRC17:43
*** MarkAtwood has quit IRC17:43
pabelangeranteaya: no, we just disabled pdf uploads.  And we also installed SmiteSpam17:43
*** Hal has joined #openstack-infra17:43
*** roxanaghe has quit IRC17:43
anteayaah okay so still plenty of spam pdfs floating around then is my conclusion17:43
pabelangerBut SmiteSpam will need to be tweaked via configuration to catch the current spam that exists17:44
*** smarcet has joined #openstack-infra17:44
*** Hal is now known as Guest50617:44
anteayayeah, it has found a lot of pages that aren't spam17:44
anteayawhere aer smitespams settings?17:44
*** kushal has joined #openstack-infra17:45
AJaegerfungi, we could soon remove gnome-doc-tools from bindep-fallback - but still need it on proposal slaves for those three repos that need it for DocBook XML translation extraction. Shall I remove it from the list while keeping it in system-config so that we setup the slave properly? Or do you prefer to wait?17:45
*** esikachev has joined #openstack-infra17:46
*** Nikolay_St has quit IRC17:46
clarkbpabelanger: ok I think I am all done configuring my user creds17:46
clarkbpabelanger: when you created your volume you did so as your admin user against the dfw fileserver?17:47
*** e0ne has joined #openstack-infra17:47
*** nelsnels_ has joined #openstack-infra17:48
*** nelsnelson has quit IRC17:48
*** annegentle has joined #openstack-infra17:48
clarkblooks like my puppet change is still failing, I wonder if that gpg module is trying to contact a keyserver about that key17:48
clarkband its not in any keyserver so fails \o/17:48
*** Guest506 has quit IRC17:48
*** andymaier has quit IRC17:49
pabelangerclarkb: Yup, admin user17:49
anteaya<-- lunch17:50
*** rlandy has quit IRC17:50
*** bpokorny has joined #openstack-infra17:52
*** bpokorny has quit IRC17:52
*** Sukhdev has joined #openstack-infra17:52
*** bpokorny has joined #openstack-infra17:53
*** tosky_ has quit IRC17:53
fungiAJaeger: if you're sure that one's safe to remove, feel free to propose a patch. i'm just unconvinced that we really need to bother cleaning up the bindep fallback list as it's an incentive to projects to provide their own other-requirements.txt files17:53
*** esikachev has quit IRC17:54
pabelangerclarkb: I used afsdb01 to create the volumes too, as unprivileged but kerberos admin17:54
fungiwe're going to cache most packages jobs need anyway, so removing a few packages from the default list to install will at most shave a few seconds off jobs17:55
AJaegeryou mean, we should add some more cruft to it? ;)17:55
fungiAJaeger: we could add a sleep 300 to it and impose a 5-minute runtime penalty (not really!)17:55
AJaegerfungi: ok. Will finish what I started and then let projects do it...17:56
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869917:56
AJaegerfungi: for gnome-doc-tools I just did that - adding other-requirements to all doc repos ;)17:56
*** jamesmcarthur has quit IRC17:57
*** sdake has quit IRC17:57
openstackgerritZara proposed openstack-infra/python-storyboardclient: Update Docs  https://review.openstack.org/29891017:57
*** e0ne has quit IRC17:58
*** kzaitsev_mb has quit IRC17:59
openstackgerritMerged openstack-infra/tripleo-ci: Don't execute the output from wait_for  https://review.openstack.org/29529717:59
*** asettle has joined #openstack-infra18:00
*** cloudtrainme has quit IRC18:02
*** pcaruana has joined #openstack-infra18:02
*** sdake has joined #openstack-infra18:02
* rcarrillocruz just realized infra meeting is in an hour18:03
* Clint nods.18:04
anteayarcarrillocruz: time change on the weekend apparently, eh?18:04
rcarrillocruz /facepalm18:05
*** arxcruz has quit IRC18:05
rcarrillocruzback in a bit18:05
*** degorenko is now known as _degorenko|afk18:06
anteayaso here is the inclusion of the smitespam extention in puppet-mediawiki: http://git.openstack.org/cgit/openstack-infra/puppet-mediawiki/commit/templates/Settings.php.erb?id=933853e0306a8fc8a8b9276b7bbed00db25f19ef any one have any thoughts on where I can find its configuration?18:06
anteayaso far I'm not seeing anything with smitespam in the message in system-config18:06
dansmithdoes anyone think that 40 minutes for running devstack on an ovh node is out of reasonable bounds? https://jenkins05.openstack.org/job/gate-tempest-dsvm-full/31250/console18:07
*** sdake has quit IRC18:08
*** sdake_ has joined #openstack-infra18:08
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove libtidy from bindep  https://review.openstack.org/29891318:08
*** dtardivel has quit IRC18:08
openstackgerritAndreas Jaeger proposed openstack-infra/system-config: Remove libtidy install  https://review.openstack.org/29891418:08
mordreddansmith: just for devstack, not for tempest?18:08
*** dizquierdo has joined #openstack-infra18:08
dansmithI don't think it's anything to do with my code, because it's happening on two separate jobs (postgres and mysql)18:08
dansmithmordred: I think that's what the console is telling me, right?18:08
*** EricGonczer_ has quit IRC18:08
*** hashar has joined #openstack-infra18:09
anteayadansmith: that is my interpretation of 'this should take 10 or 15 minutes'18:09
*** asettle has quit IRC18:10
dansmithanteaya: heh, yeah, that's my read18:10
dansmithand it does't say "except for dansmith"18:11
anteayadansmith: we should fix the logs18:11
dansmithanteaya: well, sdague said 10-15 minutes is right18:12
dansmithwhich is why I'm asking18:12
openstackgerritMichael Johnson proposed openstack-infra/devstack-gate: Add archiving of nova instance console logs  https://review.openstack.org/29891518:12
anteayasomewhere in backscroll mnaser had a mention of wondering if folks were having issues on ovh and mentioning new hardware was expected at his location18:12
sdaguemore than double that is definitely of concern18:12
anteayabut those were the extent of the details I read in backscroll18:12
openstackgerritDean Troyer proposed openstack/requirements: Bump python-openstackclient to 2.3.0 in u-c  https://review.openstack.org/29884118:13
*** e0ne has joined #openstack-infra18:13
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove gnome-doc-utils from bindep  https://review.openstack.org/29891718:13
fungianteaya: presumably vexxhost not ovh?18:14
*** Douhet has quit IRC18:14
anteayaah true18:14
anteayasorry I'm mashing them together18:14
anteayasorry vexxhost18:14
anteayayeah this is the first information I have of an issue with ovh18:15
*** Douhet has joined #openstack-infra18:15
*** sputnik13 has joined #openstack-infra18:15
*** piet has joined #openstack-infra18:16
*** MarkAtwood has joined #openstack-infra18:17
fungiinteresting. a node i held in nodepool yesterday became unreachable today (in osic), but nodepool still thinks it's held18:17
fungican't even ping it18:17
*** jaosorior has quit IRC18:18
anteayawhat criteria is nodepool using to consider it held?18:18
funginova list claims the instance is still running18:18
*** _amrith_ is now known as amrith18:19
fungianteaya: nodepool relies on an entry in its database for node state (building, ready, used, delete, hold)18:19
anteayaah so if another entry doens't update the database, the old entry is it18:19
dansmithsdague: fungi: so is there any reason to wait for this test to timeout.. for forensics or something? otherwise I'll push something up to restart it so I can move on18:19
anteayadansmith: I see nothing to be gained by making you watch the thing go round18:19
sdaguedansmith: if you have a different patch to test, I'd just push it18:20
dansmithanteaya: okay, just didn't know if someone wanted to jump on the box and look18:20
dansmithsdague: ack18:20
sdaguedansmith: any possibility that you made nova-manage go into an infinite loop?18:20
dansmithsdague: I don't think so18:20
anteayadansmith: yup, thanks for giving us the opportunity18:20
sdaguetrying to do it's upgrades18:20
dansmithsdague: and it is postgres only, but the mysql job hung the same way18:20
dansmithsdague: if it runs on another host and hangs again, we'll know I guess18:20
dansmithsdague: ohhhh18:21
dansmithyou know what18:21
dansmithI bet I did, lol18:21
sdaguedansmith: :)18:21
*** zz_dimtruck is now known as dimtruck18:21
sdaguedo I win a cookie18:21
dansmithI spent twenty minutes trying to prove I didn't18:21
*** jamesmcarthur has joined #openstack-infra18:21
*** eharney has joined #openstack-infra18:21
dansmithbut then you asked again and I just realized I bet I did18:21
fungibeer cookies18:21
fungii bet beer would make a good substitute for some of the water and sugar in certain cookie recipes. new project for next weekend18:22
anteayayou can already use beer in bread18:22
dansmithI hereby retract my complaint and all the bad words I said to my monitor18:22
anteayaI think it would make great cookies18:22
anteayawell you didn't type them18:22
fungianteaya: can and _do_18:22
anteayapoints for that18:22
jeblairi'm briefly online; anything urgently need my attention?18:22
anteayajeblair: how is the unwinding zuul so it may be restarted work coming along?18:23
fungijeblair: how far did you get with figuring out which patch you wanted to untangle in zuul to revert in a restart>?18:23
AJaegerfungi, I'm through my bindep-fallback cleanup for now, all patches send.18:23
fungirough estimate is that we'll run out of swap in a week (though the server seems to be humming along without significant impact from the additional paging for now)18:23
*** baoli has quit IRC18:24
*** jamesmcarthur has quit IRC18:24
*** cj__ is now known as cj18:24
fungijeblair: nothing extra urgent that i know of though18:24
*** jamesmcarthur has joined #openstack-infra18:25
*** e0ne has quit IRC18:25
fungimy held osic node showed nothing odd on the nova console-log, but returned to usability after a nova reboot call18:25
fungicurious how it managed to wedge itself as i wasn't really running anything on it overnight18:26
*** IlyaG has quit IRC18:27
funginothing looks out of sorts in its syslog18:28
*** EricGonczer_ has joined #openstack-infra18:28
fungiit was continuing to log mundane stuff up to the point where i rebooted18:28
jeblairfungi, anteaya: ah, zuul has a gate blockage; Nakato_ fixed it here: https://review.openstack.org/#/c/296180/18:30
jeblairso i'll approve that18:31
*** jsavak has quit IRC18:31
*** Sukhdev has quit IRC18:31
fungiyep, lgtm. thanks@18:31
jeblairfungi, anteaya: however, that's the only test failure on my change, so i think it's safe to actually deploy in production18:31
*** jsavak has joined #openstack-infra18:32
fungijeblair: awesome, so we just need a volunteer to temporarily disable puppet for zuul.o.o, install the patched version, then do the usual restart and reenqueue step (coordinating with release management to make sure we don't catch them unawares in the middle of running release jobs)18:32
jeblairfungi, anteaya: which is this one: https://review.openstack.org/#/c/298308/18:32
jeblairfungi: probably don't really need to stop puppet18:33
fungijust be... quick ;)18:33
openstackgerritMonty Taylor proposed openstack-infra/shade: Mutex protect internal/external network detection  https://review.openstack.org/29892718:33
fungii guess as long as the patch, upgrade, restart fit between puppet runs, should be fine18:33
*** abregman has joined #openstack-infra18:34
*** cdent has quit IRC18:34
fungipabelanger: you had previously offered to go through a zuul restart. when would be convenient for you? i can make sure i'm around to assist18:34
anteayajeblair: I support this direction, thank you18:34
*** kushal has quit IRC18:35
pabelangerfungi: sure, I don't mind helping. If needed18:36
jeblairfungi: puppet won't try to reinstall zuul unless a change lands18:36
jeblair(since it's driven by vcsrepo updates)18:36
*** piet has quit IRC18:36
fungijeblair: yep, i was considering if the patch was checked out in /opt/zuul on the server, but i suppose it's safer to not do that18:36
anteayadhellman has a patch in the gate he would like to get merged if he can please18:37
*** piet has joined #openstack-infra18:37
dhellmannanteaya : there's one change in our gate right now that I've been waiting for ages to get merged, so I'd like to let that one in if we can. after that, I can wait for a restart before tagging the release.18:37
fungiespecially since if you don't put it back puppet-vcsrepo may fail to pull new commits into it18:37
dhellmannanteaya :it looks like we're just a few minutes from it merging18:37
*** e0ne has joined #openstack-infra18:37
*** kushal has joined #openstack-infra18:37
anteayadhellmann: awesome, thanks for saying so18:38
dhellmannanteaya : thanks for asking :-)18:38
anteayais that the only patch in process at the moment?18:38
anteayadhellmann: of course :)18:38
fungidhellmann: well, we wouldn't probably do a restart until after the infra meeting, which is starting in a few minutes18:38
anteayagiven it is a non-emergency restart18:38
jeblairfungi: oh, yeah, i usually check out in ~corvus18:39
dhellmannanteaya, fungi : my patch landed18:39
anteayadhellmann: yay18:39
jeblairfungi: then "sudo pip install ."18:39
*** austin81 has joined #openstack-infra18:39
dhellmannok, if you're going to wait a while I'll go ahead and tag the release18:39
anteayadhellmann: how long does tagging usually take?18:39
*** eil397 has joined #openstack-infra18:40
dhellmannanteaya : just a few minutes, much less than the length of your meeting18:40
anteayadhellmann: yup, makes sense then18:40
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869918:40
funginote that we really only care about the release pipeline jobs up to the point where the release announcements are sent. the wheel builder/release jobs are fine to lose from the release pipeline since they'll just get rerun soon thereafter anyway18:40
*** achanda has quit IRC18:40
openstackgerritMerged openstack-infra/zuul: Fix test for new WebOb  https://review.openstack.org/29618018:41
*** cloudtrainme has joined #openstack-infra18:41
anteayafungi: so for instance the ironic-ui patch in the release pipeline, it is finished all jobs with the exception of two wheel jobs18:42
cloudnullfungi: -- re: osic node held but not reachable -- jthorne and I just got done dealing with an issue today where rabbitmq had a nice freaked out session. while its resolved now, its likely your node was impacted.18:42
anteayaso the wheel jobs will just get run later?18:43
cloudnullwas just reading and saw your osic post and figured i'd mention it here.18:43
fungicloudnull: thanks! luckily it wasn't doing anything important at all (i was actually about to delete it), i just got curious18:43
*** angdraug has quit IRC18:44
AJaegeranteaya: wheel jobs are run also in periodic pipeline every day18:44
anteayaah, awesome, thank you18:44
fungibut it's possible some of our job workers might have died mid-job so i'll keep an eye out for related issues on that front (luckily zuul just restarts the job in most cases like that)18:44
openstackgerritEric Harney proposed openstack-infra/project-config: devstack-plugin-nfs: Make tempest non-voting  https://review.openstack.org/29877418:45
anteayathank you18:45
cloudnull++ let me know if i can help -- or if it happens again18:45
fungicloudnull: will do--thanks again!18:45
AJaegeranteaya: the periodic pipeline covers any pypi releases. We run it as part of release pipeline for *our* releases to have the wheels directly instead of waiting for next periodic job18:45
cloudnullanytime -- sorry for the disruption18:45
*** david-lyle has quit IRC18:45
fungianteaya: yeah, we run those immediately after any release to speed up the availability of wheels in our mirror for them, but it's not critical and it gets rerun completely anytime something is released, in addition to periodically even if nothing is released18:46
*** david-lyle has joined #openstack-infra18:46
dhellmannfungi , AJaeger : I suspect that the announce job is installing way more packages than it really needs to do the work. IIRC you're looking at optimizations in other jobs, too? Let me know if I can help with analysis there.18:47
*** jaypipes has joined #openstack-infra18:47
*** ilyashakhat has joined #openstack-infra18:47
dhellmannfungi , anteaya : ok, the announce job is done. I'll hold off on any other release work until you give the all-clear that the restart is done.18:47
anteayadhellmann: thanks18:48
fungidhellmann: it's likely not worth optimizing that one job unless it really spends a lot of time installing packages. it shouldn't really be downloading any since we pre-cache them on disk18:48
dhellmannfungi : I'd have to look, but I wouldn't be surprised to find that it spends more than 50% of its time installing things18:48
anteayaAJaeger: fungi thank you, I appreciate the explaination18:48
dhellmannthe announce script itself runs pretty quickly18:48
*** achanda has joined #openstack-infra18:49
dhellmann(it doesn't do much)18:49
fungicody-somerville: woah! us west gear made it to texas finally? excellent news!18:49
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: WIP: Add migration to move attrs and tags into columns  https://review.openstack.org/29806918:51
AJaegerdhellmann: see http://docs.openstack.org/infra/manual/drivers.html#package-requirements on what you can do - but it's IMHO not worth for that job18:51
openstackgerritEric Harney proposed openstack-infra/project-config: devstack-plugin-nfs: Make tempest non-voting  https://review.openstack.org/29877418:51
dhellmannAJaeger, fungi : ok, if it's not going to buy much I won't worry about it18:51
*** phschwartz_ is now known as phschwartz18:51
*** ilyashakhat has quit IRC18:51
AJaegerdhellmann: looking at https://jenkins02.openstack.org/job/ironic-ui-announce-release/1/consoleFull, package install starts at 17:19:07 and finishes at 17:20:3018:53
AJaegerjob start at 17:18:59, finishes at 17:21:10, so 2:11 minutes runtime,  package install takes 1:23 mins18:53
dhellmannAJaeger : yeah, that's not worth worrying about.18:54
fungialso there would need to be some customization to do special package installation (or lack tehreof) for that job since it runs for multiple repos and our usual tuning is something the projects themselves would add/modify locally in their repos18:54
*** tiswanso has quit IRC18:55
*** tiswanso has joined #openstack-infra18:55
anteayaigorbelikov: hi, there seem to be two agenda items in this week's meeting agenda about baremetal in nodepool: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Weekly_Project_Infrastructure_team_meeting18:56
anteayaigorbelikov: did you want to maybe select one of them?18:56
*** piet has quit IRC18:57
*** jaypipes_ has joined #openstack-infra18:57
igorbelikovanteaya: hi, I removed the second one18:57
*** maishsk has joined #openstack-infra18:57
igorbelikovit was posted as a sub-item in previous week agenda, don’t know why it got this way:)18:57
anteayaigorbelikov: thanks18:57
anteayaigorbelikov: thanks for the clean up :)18:58
*** angdraug has joined #openstack-infra18:58
*** e0ne has quit IRC18:59
openstackgerritMerged openstack-infra/infra-specs: Add Nodepool Zookeeper spec  https://review.openstack.org/27877718:59
pleia2meeting time once keystone finishes up19:00
pleia2there we go19:00
*** Guest28049 is now known as mfisch19:01
openstackgerritMerged openstack/os-testr: Add pypi download + version badges into README.rst  https://review.openstack.org/29810219:01
*** sdake_ is now known as sdake19:01
openstackgerritMerged openstack-infra/infra-specs: Deploy-Stackviz spec  https://review.openstack.org/28737319:01
*** mfisch is now known as Guest8245419:01
*** Guest82454 has quit IRC19:02
*** Guest82454 has joined #openstack-infra19:02
*** Guest82454 is now known as mfisch19:03
*** eil397 has quit IRC19:03
*** e0ne has joined #openstack-infra19:03
*** e0ne has quit IRC19:03
openstackgerritMerged openstack/os-testr: py26/py33 are no longer supported by Infra's CI  https://review.openstack.org/26169219:04
*** e0ne has joined #openstack-infra19:04
openstackgerritMerged openstack/os-testr: remove python 3.3 trove classifier  https://review.openstack.org/26452119:04
*** eil397 has joined #openstack-infra19:05
anteayawaynr: you mentioned putting jjb things on https://etherpad.openstack.org/p/infra-newton-summit-planning on the mailing list19:05
*** eil397 has quit IRC19:06
*** eil397 has joined #openstack-infra19:06
anteayawaynr: don't delay we will be discussing slots next infra meeting19:06
*** e0ne has quit IRC19:07
*** krotscheck is now known as krotscheck_dcm19:07
*** jaypipes_ has quit IRC19:08
*** ilyashakhat has joined #openstack-infra19:08
*** doug-fis_ has joined #openstack-infra19:10
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Increase heat rpc_response_timeout in the overcloud  https://review.openstack.org/29894119:12
*** doug-fish has quit IRC19:12
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869919:12
cbader_anteaya, is there a document that describes how to remove a provider from the nodepool list so I can move it to another commonci server to balance loads?19:13
*** akscram has quit IRC19:14
anteayawell if we wanted to remove a provider from our nodepool we first would set min_servers to -1 in our nodepool.yaml19:14
anteayaso no images were created on that host provider19:14
anteayaand then we would remove the entry in nodepool.yaml in project-config19:14
cbader_anteaya, I have done that but it is still building images when I delete them19:14
anteayawell you have to look into why it is doing that19:15
*** akscram has joined #openstack-infra19:15
anteayasetting min_servers to -1 in our nodepool.yaml prevents images being created on hosts for our system19:15
cbader_anteaya, thank you for the information, I will look into more why mine is behaving strange19:15
*** ilyashakhat has quit IRC19:15
*** kzaitsev_mb has joined #openstack-infra19:16
anteayayou are welcome to post a build log or stacktrack to a paste service and share it in here if that is helpful for you19:16
cbader_anteaya, I will scrub the logs to see why it is happening and will post if I find something thank you.19:17
*** ociuhandu has quit IRC19:21
*** salv-orlando has joined #openstack-infra19:22
*** ericksonsantos has quit IRC19:23
*** salv-orl_ has quit IRC19:24
*** dizquierdo has quit IRC19:25
cloudnullhi infra folks -- one of the users of the osic cluster is about to do a very large VM spinnup, 600+ at the same time, this is what made rabbitmq freak out before.19:26
*** roxanaghe has joined #openstack-infra19:26
cloudnullwe'll be monitoring the cluster closely this go and working with the user to ensure they're not causing another rabbitmq freak out session but I wanted to let you all know.19:27
angdraugpabelanger: (taking this out of the meeting) bookwar will be in Austin, can you loop her into the tripleo/ironic/nodepool discussion?19:27
*** shardy is now known as shardy_afk19:28
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: WIP: Add migration to move attrs and tags into columns  https://review.openstack.org/29806919:28
pabelangerangdraug: sure19:28
*** EricGonc_ has joined #openstack-infra19:29
fungicloudnull: thanks for the heads up19:30
*** EricGonczer_ has quit IRC19:30
*** roxanaghe has quit IRC19:31
*** salv-orl_ has joined #openstack-infra19:32
*** jsavak has quit IRC19:32
*** mylu has joined #openstack-infra19:32
*** mylu has quit IRC19:33
*** salv-orlando has quit IRC19:35
*** MarkAtwood has quit IRC19:35
anteayaI wonder what the rabbitmq threshold is for freakout? apparently less than 600 nodes19:37
dimscloudnull : i'll be interested in the outcome :) rabbitmq/oslo.messaging if we can do stuff to fix things19:38
*** daemontool has joined #openstack-infra19:38
*** YorikSar has quit IRC19:38
cloudnull++ dims will let you know19:38
dimsthanks cloudnull19:38
cloudnullso far i think its a matter of proper tuning of the elang vm rather than tweaks we can make in oslo/the services19:39
cloudnullbut ill find out more soon19:39
dimscloudnull : ack. odyssey4me pointed me to the last set of tunings made19:41
*** mylu has joined #openstack-infra19:44
*** YorikSar has joined #openstack-infra19:45
*** nadya has joined #openstack-infra19:51
*** cdent has joined #openstack-infra19:52
bkeroIs the dstat log build artifact set up as a system service and then copied into the place where it's picked up as an artifact? I can't find much about it from project-config.19:53
clarkbbkero: it runs as a "devstack" service not a system service19:54
*** burgerk has quit IRC19:54
openstackgerritMichael Johnson proposed openstack-infra/devstack-gate: Add archiving of nova instance console logs  https://review.openstack.org/29891519:54
clarkbbkero: basically devstack does semi init duties and keeps track of pids and sets up log files19:54
bkeroI see19:54
clarkbbkero: if you look in devstack you sould be able to find out how it is configured19:54
bkeroI'm trying to get dstat-like logs to debug downstream ci19:54
bkeroI might just dstat > /tmp/dstat.log 2>&1, then kill it at the end and drop dstat.log into a place where it's picked up as a build artifact19:55
*** sdake_ has joined #openstack-infra19:55
clarkbthats basically what evstack does I think19:55
*** Rockyg has joined #openstack-infra19:55
bkeroI think it probably dumps a timestamp on it19:55
*** eil397 has quit IRC19:56
*** ociuhandu has joined #openstack-infra19:57
*** cdent has quit IRC19:57
*** sdake has quit IRC19:58
*** cdent has joined #openstack-infra19:58
*** rbrndt_ has quit IRC19:59
cody-somervillefungi: indeed!20:00
openstackgerritJamie Lennox proposed openstack-infra/project-config: Run the v3 only job in glance  https://review.openstack.org/29319920:00
*** EricGonc_ has quit IRC20:01
*** bdemers has quit IRC20:01
AJaegerpabelanger: could you review https://review.openstack.org/#/c/245297/ , please? You had a question on it and thus I'm waiting with approving...20:02
pleia2fungi: I should have brought up my agenda item much (like, weeks) sooner20:02
*** gokrokve has joined #openstack-infra20:02
*** sdake_ is now known as sdake20:02
pabelangerAJaeger: sure20:02
anteayapleia2: looks like the module patch has some reviews20:02
*** abregman has quit IRC20:02
pleia2with release next week, the checksite ceases to be valuable this cycle20:02
clarkbpuppet people http://logs.openstack.org/55/293155/5/check/gate-infra-puppet-apply-ubuntu-trusty/43fc2fd/console.html#_2016-03-29_16_56_08_864 compared to https://review.openstack.org/#/c/293155/5/modules/openstack_project/manifests/mirror_update.pp makes no sense because I do have a key_content parameter. looking at20:02
pleia2anteaya: yes, finally :)20:02
clarkbhttps://github.com/n1tr0g/golja-gnupg/blob/1.2.2/lib/puppet/provider/gnupg_key/gnupg.rb I can't figure out that error message20:02
anteayapabelanger: do you know where the config for smitespam is? I can't seem to find it20:02
clarkbanyone willing to help me debug that?20:02
pleia2anteaya: I had mostly been working on it myself on the infra side20:03
pabelangerAJaeger: will +2 but not confident on +A sorry20:03
anteayapleia2: yup, I know it20:03
*** salv-orlando has joined #openstack-infra20:03
fungibkero: clarkb: i thought there was a devstack "screen" log for its dstat service20:03
AJaegerpleia2: it still can be used for proof-reading, and changes go into next stable release.20:03
clarkbfungi: yes but we don't use "screen" :)20:03
AJaegerpleia2: but having it early for newton would be nice as well ;)20:03
clarkbfungi: it basically acts like an init system its good fun20:03
pleia2AJaeger: yeah, and getting it stabalized + proper error checking for rebuilds will take time anyway20:03
AJaegerpleia2: indeed.20:04
clarkbfungi: to reproduce the git review thing I have version 1.25.020:04
AJaegerpleia2: yolanda reviewed this morning and had quite a few comments...20:04
pleia2AJaeger: right now it has essentally no provisions for when devstack-goes-wrong20:04
*** maishsk has quit IRC20:04
pleia2AJaeger: yeah, I was thankful for some more eyes on it, thanks yolanda :)20:04
fungiclarkb: yeah, i can try to repro in a bit if you don't have time20:04
*** maishsk has joined #openstack-infra20:05
ianwbkero : enable_service dstat20:05
yolandait was a nice job, just some missing nits20:05
clarkbfungi: and I got the number wrong by transposing a couple digits and it mated two repos together20:05
bkeroianw: where's that defined? devstack?20:05
clarkbfungi: mostly trying to unravel puppet to figure out this gnupg_key error20:05
fungiclarkb: wow, that is really strange20:05
*** salv-orl_ has quit IRC20:05
AJaegerany other nodepool expert that wants to chime in on greghaynes's change to install from source: https://review.openstack.org/#/c/245297/ , please?20:05
zigofungi: Could you please grant me ACLs to create new branches in deb-openstack-pkg-tools? https://review.openstack.org/#/c/298746/20:06
zigoI need it go further on the packaging stuff ...20:06
ianwbkero: yes, it should probably be part of your default services.  i.e. already there20:06
clarkbthe best part is the error is telling me I have much to learn20:07
clarkbexcept the error makes no sense20:07
*** Goneri has quit IRC20:07
bkeroianw: I'm doing this on rdo-manager virthosts. It's not going to be there. :)20:07
anteayaclarkb: what does it want you to learn about?20:07
fungizigo: approved, though any of the project-config-core reviewers can approve those20:08
zigofungi: Thanks.20:08
clarkbanteaya: Error: Validation of Gnupg_key[Ceph Archive] failed: You need to specify at least one of key_source, key_server, key_content, much to learn, you still have20:08
pabelangeranteaya: we'll need to patch puppet-mediawiki to add it20:08
anteayapabelanger: oh okay that would explain why I couldn't find it20:08
clarkbbut i have key_content right there20:08
*** gokrokve has quit IRC20:08
anteayaclarkb: what an odd error message20:08
clarkbmaybe it didn't find the entry in hiera?20:08
anteayapabelanger: have you ever come across any documentation about what the config should look like?20:09
pabelangeranteaya: https://www.mediawiki.org/wiki/Extension:SmiteSpam20:10
*** rbrndt_ has joined #openstack-infra20:12
*** esker has quit IRC20:12
openstackgerritClark Boylan proposed openstack-infra/system-config: Apt mirror for ceph hammer release  https://review.openstack.org/29315520:12
clarkbhopefully that was it20:12
*** esker has joined #openstack-infra20:13
mmedvedepabelanger: have you already try to run d-g on xenial?20:13
*** jsavak has joined #openstack-infra20:13
*** Sukhdev has joined #openstack-infra20:13
pabelangermmedvede: not yet. Mostly focused on getting our CI tooling working first20:13
mmedvedepabelanger: I am asking because I found a problem with lvm on it20:13
clarkbfungi: I will work on reproducing the git review issue now that I pusehd latest 293155 patchset up20:14
mmedvedepabelanger: so tempest was failing with volume snapshot tests timing out20:14
openstackgerritMerged openstack-infra/project-config: Add create option to deb-openstack-pkg-tools ACL  https://review.openstack.org/29874620:14
mmedvedepabelanger: ok. So then fyi: https://bugs.launchpad.net/ubuntu/+source/lvm2/+bug/156122820:14
openstackLaunchpad bug 1561228 in lvm2 (Ubuntu) "dmevent can not open shared object file libdevmapper-event-lvm2snapshot.so leads to lvcreate/lvremove being slow" [Undecided,New]20:14
pabelangermmedvede: sure, going to try d-g today, if possible20:15
mgagneis logs.o.o a host running apache and proxying requests to swift when needed? my question is: what about huge file? is it working well?20:15
clarkbfungi: zaro jeblair http://paste.openstack.org/show/492356/ that look broken to you?20:15
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869920:15
clarkbmgagne: yes it proxies to swift for some files, many files are on disk though. It mostly works. The biggest issues we have had are that swift doesn't track mod times for the fake dir things20:16
mmedvedepabelanger: do you mind me following up with you later this week on how your d-g test went?20:16
jeblairclarkb: that looks weird.  that does not happen for me.20:16
mgagneright, I don't care much about modtime. I'm looking to upload binary to Swift and was looking at swift support in Zuul for that20:17
pabelangermmedvede: sure20:17
clarkbhuh I just tried the git review -d thing again and it worked, not sure which exact change I used when it broke20:17
fungiclarkb: i'm starting to think your computer has it in for you20:17
jesusaurclarkb: your paste looks like you may have hit a race condition, a change merged 2 min ago according to https://review.openstack.org/gitweb?p=openstack-infra/project-config.git;a=summary and your sha is HEAD~120:18
clarkbfungi: its possible I was also sleep deprived and misunderstanding the output of git reivew20:18
fungiclarkb: i ran the same git clone from your paste and it definitely did not try to checkout e86217d when it finished20:18
clarkbjesusaur: fun20:18
*** zeih has joined #openstack-infra20:19
*** ericksonsantos has joined #openstack-infra20:19
clarkbanyways I can no longer reproduce the git review -d thing so I was probably high or half alseep20:19
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869920:19
fungiinteresting. yeah i wonder if git clone gets confused if head moves while it's cloning20:19
waynranteaya: thanks for the reminder, i'll add a note now...$dayjob has been crazy20:20
*** imcsk8 is now known as imcsk8|afk20:20
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869920:21
*** achanda has quit IRC20:21
anteayawaynr: yup, it just came up at today's meeting that next week we start deciding slots20:21
anteayawaynr: so get on the board20:21
zaroclarkb: that git clone worked fine for me as well.20:21
clarkbjeblair: mordred pabelanger reading http://docs.openstack.org/infra/system-config/afs.html#creating-a-volume I think I want to use afs01.dfw for the rw volume because that is where the reprepro mirror server is. Also how does one check that the server hsa enough space? df ? or some afs thing?20:21
clarkbzaro: ya  Ithink jesusaur may be onto something, clones racing HEAD updates20:21
pabelangerclarkb: I was able to use afsdb01 IIRC20:23
pabelangerbut you need to use /afs/.openstack/...20:23
pabelangerfor RW20:23
clarkb/dev/mapper/main-vicepa  985G  349G  636G  36% /vicepa20:23
jeblairclarkb: yes, dfw is probably best for now for that reason.20:23
*** |-paul-| has joined #openstack-infra20:23
clarkbpabelanger: on the mkmount step? is there a docs update to fix the docs?20:24
jeblairclarkb: and yes, that works; also some afs commands will give you the same info (as partition % used)20:24
jeblairclarkb, pabelanger: i think clarkb was saying that he should locate the volume on afs01.dfw.  pabelanger is saying he could run the commands from afs01db.20:24
pabelangerclarkb: right, the fs mkmount command should use /afs/.openstack.org for RW.  I had a patch locally for it, I don't think I pushed it up20:25
jeblair(both things are correct, just wanted to make sure you weren'tt talking past each other)20:25
zarofungi: maybe something to keep on eye on for next time a git repo is unaccesible, https://mail.google.com/mail/u/0/#inbox/153bf62a41f784a920:26
*** eil397 has joined #openstack-infra20:26
clarkbok last question hopfeully, how do I list the volume names so that I use something similar to what is already in place?20:26
zaroit does seem to correlate to when the cron job runs repack.20:26
*** mcarden_ is now known as mcarden20:26
*** bdemers has joined #openstack-infra20:26
*** EricGonczer_ has joined #openstack-infra20:28
fungizaro: wow, to read that i need a google account20:29
fungizaro: is it a mailing list archive or something?20:29
zarooops. wrong link.20:29
*** gokrokve has joined #openstack-infra20:29
zarofungi: that was a copy/paste fail, here's the correct one https://groups.google.com/d/msg/repo-discuss/g2W-m12fwPs/EEsR3H5MEAAJ20:30
*** jtomasek_ has joined #openstack-infra20:30
*** eil397 has quit IRC20:30
*** eil397 has joined #openstack-infra20:30
clarkbjeblair: mordred pabelanger is listvldb's mirror.ubuntu mirror.pyi etc showing me the names of the volumes or is that just a path?20:31
fungizaro: thanks! i seem to be able to read that just fome20:31
fungier, fine20:31
anteayaI like fome20:31
pabelangerclarkb: name of volume20:31
fungianteaya: i do too sometimes20:31
anteayaha ha ha20:31
clarkbpabelanger: so in this case I want to call it mirror.deb.ceph?20:31
*** jtomasek_ has quit IRC20:32
*** kgiusti has left #openstack-infra20:32
*** gyee has joined #openstack-infra20:32
waynranteaya: okay, I have updated the etherpad let me know if any further clarification is required--I'll make a note to myself to attend next week's infra meeting20:32
fungizaro: the other missing detail is that the supposedly missing commit was still right there on disk, gerrit had just decided it wasn't for some reason20:32
*** annegentle has quit IRC20:32
*** Rocky_g has joined #openstack-infra20:32
*** bdemers_ has joined #openstack-infra20:33
*** annegentle has joined #openstack-infra20:33
anteayawaynr: yes be here at next week's infra meeting, I will not be here20:33
pabelangerclarkb: you create volume first, then use mkmount on the volume20:33
*** sridhar_ram1 has joined #openstack-infra20:33
anteayawaynr: but folks with questions probably won't ask them prior to the meeting20:33
clarkbpabelanger: yup, I am just trying to think that we might have a deb mirror and a rpm and so on20:33
*** sridhar_ram has quit IRC20:33
clarkbpabelanger: so the name would be mirror.deb.cpeh ?20:33
jeblairclarkb: name of volume20:33
fungizaro: but logged into gerrit locally in i could git show that sha just fine in the appropriate repo under ~gerrit2/review_site/git/20:33
anteayawaynr: so your attendance is helpful in you getting some sort of slot20:33
_odyBeen seening sporadic git clone issues today. http://logs.openstack.org/57/296557/24/check/gate-puppet-openstack-integration-3-scenario001-tempest-ubuntu-trusty/c74c92b/console.html#_2016-03-29_19_43_19_232 showing about 5 minutes a repo.20:33
clarkbthen the path could be /afs/.openstack.org/mirror/deb/ceph20:34
waynrthanks again for the heads up!20:34
*** gokrokve has quit IRC20:34
pabelangerclarkb: so, what will the debian volume look like when we bring it (if) online20:34
jeblairclarkb: do we need deb?20:34
clarkbjeblair: if we decide to have an rpm mirror at some point?20:34
jeblair(we have mirror/ubuntu)20:35
jeblair(so i would expect mirror/fedora; mirror/debian; mirror/ceph)20:35
clarkbjeblair: is it safe to write into mirror/ubuntu if not mirroring ubuntu??20:35
clarkboh I see20:35
anteayawaynr: welcome20:35
clarkbjeblair: so you are saying flip it around, mirror/ceph/deb ?20:35
jeblairclarkb: i think i see where're you are coming from20:35
jeblairyeah, maybe?20:36
*** bdemers has quit IRC20:36
*** Rockyg has quit IRC20:36
jeblairmirror/thing-being-mirrored/format(if needed) ?20:36
fungior mirror/debian-ceph?20:36
openstackgerritSean Dague proposed openstack-infra/project-config: create api-ref publishing pipeline  https://review.openstack.org/29888820:36
jeblairfungi: or that20:36
fungikeeping our package mirrors at a consistent hierarchy depth is appealing, if perhaps not entirely necessary20:37
clarkbya I can do it that way20:37
*** sigmavirus24 is now known as sigmavirus24_awa20:37
*** sigmavirus24_awa is now known as sigmavirus2420:37
fungimight make certain types of globbing across trees easier in the future for example20:37
fungibut i'll admit it leans toward bikeshedding or premature optimization so i don't feel strongly either way20:37
clarkbwell I just don't want to have to unravel a bunch of stuff in a week20:38
*** baoli has joined #openstack-infra20:39
ericksonsantosHi, everyone! Does someone know if there is a problem with tests I use a library that doesn't follow PEP8 guidelines inside OpenStack?20:39
ericksonsantosFor example, suppose I want to use a library, called A, inside a driver in one service, but there's a class named 'xpto' which isn't in CapWords convention, as suggested by PEP8 guidelines. So, if I create an instance of xpto inside the OpenStack driver, the PEP8 tests will crash?20:39
ericksonsantossorry if it is a dumb question.20:40
mgagnewhen using Zuul templates, can a list be used as a variable so it expands all the job names?20:40
clarkbericksonsantos: no shouldnt be a problem20:40
clarkbits even more complicated because ceph does a different repo per release20:41
ericksonsantosclarkb, thank you!20:41
clarkbso I think we want mirror.debian-ceph-hammer as the volume name and approximate path20:42
fungiclarkb: wow, that's some extra crazy to deal with20:42
*** asettle has joined #openstack-infra20:43
*** tiswanso_ has joined #openstack-infra20:43
*** baoli_ has joined #openstack-infra20:43
clarkbwould we need a different reprepro config for each of them?20:43
pabelangerclarkb: keep in mind there is a char limit on volume names20:45
*** baoli has quit IRC20:45
*** tiswanso has quit IRC20:45
pabelanger24 I think20:45
clarkbya each ceph release has a different repo20:47
clarkbso I think we would want a different reprepro setup for each20:47
clarkbwhich means I need to reorg this mirror change a bit to be more explicit20:47
jeblairclarkb: i think so... mordred ^ sound right?20:47
clarkband we will probably want a different volume for each and different mount path, fun20:47
sdaguefor the ceph mirror the ubuntu side of this is really the blocker from the nova team side, that's the job that will likely be voting20:47
*** pradk_ has joined #openstack-infra20:48
sdagueso it's fine to punt the rpm issue until later20:48
clarkbsdague: ya its not even the rpm issue20:48
*** gokrokve has joined #openstack-infra20:48
*** pradk has quit IRC20:48
clarkbsdague: each ceph release has a different ubunt repo20:48
sdagueah, ok20:48
clarkbso we hav eto solve this generally regardless20:48
mordredyah. I agree with teh above20:49
mordredthat said - we don't necessarily have to do more than one AFS volume20:49
pabelangerfungi: just stepping away for a few minutes, let me know how you would like to proceed with zuul.o.o20:49
mordredwe could put all of the ceph repos on one volume and release at the same time20:49
mordredor we might not want to do that20:49
mordredwe shoudl design it20:49
jeblairmordred, clarkb: the volume boundary shuold correspond to the build unit20:49
*** pradk has joined #openstack-infra20:49
openstackgerritMerged openstack-infra/project-config: Don't use neutron with centos7-minimal  https://review.openstack.org/29850320:50
jeblairso if we're going to run a bunch of reprerpros together, we can do them in one volume, but if we want them to be separate (parallel-ish?) then separate volumes20:50
jeblairbecause the release-to-read-only-volume action is at the volume level20:50
fungipabelanger: sure, can work through it as soon as you're back. was mostly trying to follow the tc meeting, and cross-project specs meeting up next20:51
*** jamesmcarthur has quit IRC20:51
*** kzaitsev_mb has quit IRC20:51
*** kzaitsev_mb has joined #openstack-infra20:51
*** rlandy has joined #openstack-infra20:52
clarkbright they should be separate in this case because hammer may update separately from whatever is after hammer20:52
*** pradk_ has quit IRC20:52
*** imcsk8|afk is now known as imcsk820:53
*** Goneri has joined #openstack-infra20:53
clarkband mirror.debian-ceph-hammer is too long20:54
clarkbmirror.deb.hammer ?20:54
clarkbor are .'s implicit /s?20:54
*** ayoung has quit IRC20:54
*** pradk_ has joined #openstack-infra20:54
*** pradk_ has quit IRC20:54
pabelangerI used . when I created the centos wheel volume20:56
pabelangerfungi: okay, back20:56
clarkbpabelanger: jeblair is there a similar tiny limit to path names?20:57
*** shardy_afk has quit IRC20:57
*** pradk has quit IRC20:57
jeblairclarkb: nope20:57
clarkbok I think mirror/debian-ceph-hammer for the path and mirror.deb-hammer for the volume name20:58
clarkbbut first let me push up the reprepro reconfigurations to make sure I am not crazy20:58
openstackgerritMatt Riedemann proposed openstack/requirements: Bump upper-constraint for python-novaclient to 3.4.0  https://review.openstack.org/29897920:59
*** Sukhdev has quit IRC21:01
*** jaypipes has quit IRC21:02
*** Sukhdev has joined #openstack-infra21:03
smarcetfungi: hey i am trying to understand the error here https://review.openstack.org/#/c/298699/ but it seems that is not related to puppet syntax o_021:03
fungimordred: heads up (ptl!), i added zigo to the deb-packaging-release group so that he can create release branches in openstack/deb-openstack-pkg-tools21:03
mordredfungi: great21:05
* fungi follows protocol21:05
lifelessdims: how long till mitaka is 'shipped' ?21:05
lifelessdims: also, relatedly, wsgi scripts in wheels is broken in pbr and mitaka kindof needs that ...21:06
fungismarcet: the errors i see look related "Error: Failed to parse template openstackid/vhost.erb..."21:06
dimslifeless : 4-8 APR (https://etherpad.openstack.org/p/mitaka-release-tracking)21:06
fungismarcet: "Could not find value for 'docroot' at /etc/p21:06
lifelesslets ask ttx and dhellmann their thoughts21:06
fungioops, stray newline21:06
smarcetfungi: oh ok i will take a look then21:06
smarcetthx :021:06
jeblairDetail: Could not find value for 'docroot' at /etc/puppet/modules/openstackid/templates/vhost.erb:3921:06
zigomordred: How do I set a branch as default?21:06
jeblairsmarcet: ah fungi beat me to it21:07
smarcetjeblair:fungi: its weird bc that part already exist on former version21:07
zigoI mean, how do I get HEAD to point at debian/mitaka for example?21:07
clarkbzigo: you don't21:07
zigoclarkb: ?!?21:07
*** sridhar_ram1 has quit IRC21:07
fungizigo: head stays as master, you checkout whichever branch you want after cloning if it's not master21:08
zigoclarkb: It's not possible to issue something like git symbolic-ref ?21:08
clarkbyes beacue gerrit (and git in general) does not replicate that on pushes21:08
zigoWell, it got debian/liberty as HEAD.21:08
clarkbso we avoid doing it entirely21:08
clarkbit just causes problems for little benefit when checkout foo is easy21:08
*** cloudtrainme has quit IRC21:09
fungizigo: if the repo which was imported had a non-master HEAD then that may have confused the import process21:09
openstackgerritClark Boylan proposed openstack-infra/system-config: Apt mirror for ceph hammer release  https://review.openstack.org/29315521:09
zigofungi: This is the case indeed.21:09
fungiit definitely breaks if master doesn't exist, but there may be a corner case there if it imports a repo with a non-master HEAD while master also exists21:09
*** nadya has quit IRC21:09
zigofungi: I don't use "master", because that's what upstream uses...21:09
dhellmannfungi , anteaya : was zuul restarted? is it safe to cut releases or should i wait?21:09
*** pradk has joined #openstack-infra21:09
jeblairsmarcet: http://logs.openstack.org/99/298699/9/check/gate-infra-puppet-apply-ubuntu-trusty/0469cd6/puppetapplytest43.final.out is the easiest place to see the error21:09
*** sridhar_ram has joined #openstack-infra21:10
zigofungi: There shouldn't be any master branch at all in fact.21:10
clarkbmordred: jeblair pabelanger ^ reviewing that for afs + reprepro sanity would likely be a good idea. the gnupg_key resource is still mad about my key for some reason but I can work on that separately21:10
smarcetjeblair: roger thx21:10
fungidhellmann: sorry, been absorbed in meetings. i was going to walk pabelanger through it when he gets back in a few21:10
dhellmannfungi : np, that's why I was checking :-)21:10
dhellmannfungi : these will keep until morning, so I'm just going to wait21:10
ttxlifeless: it's "shipped" on April 721:11
pabelangerfungi: back now, when you are ready21:11
ttxuntil then we may still update one thing or another21:11
ttxalthough less and less likely21:11
*** richm has joined #openstack-infra21:11
pleia2fungi: btw, I could use a once through myself if this can be a group zuul restart21:11
*** esikachev has joined #openstack-infra21:11
lifelessttx: we're considering a pbr release21:11
fungipleia2: pabelanger: may be easiest then if we share a screen session as root. i'll start one21:12
ttxlifeless: post-release ?21:12
lifelessttx: now21:12
*** edmondsw has quit IRC21:12
ttxto do what ?21:12
*** ybathia has joined #openstack-infra21:12
fungipleia2: pabelanger: ssh zuul; sudo su -; screen -x21:12
richmhello - this question isn't directly related to openstack infra, it relates to jenkins job builder + github plugin21:13
*** eil397 has quit IRC21:13
pleia2fungi: thanks21:13
*** eil397 has joined #openstack-infra21:13
pabelangerpleia2: fungi: connected21:13
fungii offered to start the screen session because my screen is likely the smallest of all of us ;)21:13
lifelesskey changes - wsgi_scripts in wheels and dealing with older versions of setuptools21:13
fungitmux handles that far better, but this will work for now21:13
richmthe jjb section for the github plugin is pretty much useless - http://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.github21:13
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869921:14
ttxlifeless: any reason why we'd need to have that pre-release rather than postrelease ?21:14
lifelesshmmm, I think there's another key change missing.21:14
pabelangerfungi: pleia2: I am fine to monitor this time around in screen21:14
richmhow can I find out how to use the github plugin with jjb?21:14
lifelessttx: mitaka introduced the wsgi_scripts thing didn't it? I figured it probably requires it ;)21:14
pleia2fungi: 80x24 forever21:14
*** Sukhdev has quit IRC21:14
fungipabelanger: pleia2: i'll let one of you drive. so we're all on the same page, the plan is this...21:15
fungipabelanger: pleia2: we want to restart zuul with this change manually patched in place https://review.openstack.org/29830821:15
ttxlifeless: well, it's a bit late for that imho21:16
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Wait for mysql cluster in HA job  https://review.openstack.org/29898621:16
dhellmannlifeless : we should move the pbr release discussion to #openstack-release21:16
ttxR-1 week is not the best moment to roll an openstack-wide dep like pbr21:16
*** ldnunes has quit IRC21:16
fungipabelanger: pleia2: basic process... make a private clone of zuul, checkout or cherry-pick that change (i don't know if the repo has moved since it was proposed), then pip install that directory21:16
pabelangerfungi: sure, did you want to drive pleia2 or me?21:16
pleia2pabelanger: go for it21:16
pleia2I'm pretty comfortable with manually applying changes like this21:17
pabelangerpleia2: fungi: any special place to clone?21:17
jeblairrichm: follow the link to "GitHub Plugin" at https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin to find out what those options mean21:17
pleia2pabelanger: can just do it in home21:18
fungipabelanger: pleia2: once we've got that installed, we'll see if the queue looks ripe for a dump and restart or if we should wait a minute or two for something to land21:18
jeblairrichm: we don't try to fully document the plugins themselves, just what yaml to use to get the configuration options in the jenkins gui21:18
*** smarcet has quit IRC21:18
richmjeblair: so is there some way I can go to Configure in the gui, and configure it there, then dump the yaml for that configuration?21:19
*** dprince has quit IRC21:19
jeblairrichm: if you look at the resulting xml in jenkins, you can probably get close.  i don't think there's a reverse-jjb though.21:19
pabelangerfungi: pleia2: that the right syntax for pip?21:19
*** devkulkarni has joined #openstack-infra21:20
richmjeblair: how can I look at the resulting xml?21:20
fungipabelanger: pleia2: yeah, the only thing to merge to zuul since that change was created was a minor fix to some tests, so irrelevant for this situation anyway21:20
fungipabelanger: not -e21:20
fungialso you didn't need to cd ..21:20
jeblairrichm: look on disk in jenkins21:20
fungiyou can just `pip install .` to install the pwd21:20
pabelangerfungi: pleia2: okay, ready to run the command?21:20
pleia2pabelanger: yep21:21
*** baoli_ has quit IRC21:21
clarkbjeblair: I think arch specifics are handled by reprepro/apt see http://download.ceph.com/debian/dists/trusty/21:21
*** esikachev has quit IRC21:21
*** achanda has joined #openstack-infra21:21
pabelangerpleia2: fungi guess we need pip -U ?21:21
*** baoli has joined #openstack-infra21:21
pabelangerpip install -U21:22
jeblairclarkb: yeah, i vaguely meant our "distro-ceph" release-release combo21:22
fungipabelanger: yeah, sorry, add -U21:22
jeblairclarkb: basically i'm just saying "do we want to put all the ceph stuff under one directory so that mirror/ stays cleanish"21:22
clarkbjeblair: gotcha21:23
jeblairclarkb: i do not have a strong opinion21:23
*** eil397 has quit IRC21:23
*** smarcet has joined #openstack-infra21:23
pabelangerfungi: pleia2: better21:23
pleia2pabelanger: cool, looks good21:23
clarkbI can think about that more whne I figure out why puppet is still being braindead with the gpg key21:23
richmjeblair: ok - thanks - do you know of an example of using a jenkins github plugin to trigger a job based on a github merge or tag push event?21:23
jeblairrichm: no, i don't use github21:23
*** IlyaG has joined #openstack-infra21:24
clarkbhttp://logs.openstack.org/55/293155/7/check/gate-infra-puppet-apply-ubuntu-trusty/3c09819/console.html#_2016-03-29_21_18_33_322 does not look like a public key, if I load the key directly using pyyaml and compare to the actual contents from ceph it looks correct... time to dig more21:24
devkulkarniHi infra team, solum's devstack gate is currently failing.. I want to chat/brainstorm with someone in this regard.. is this the right channel for that?21:24
fungipabelanger: pleia2: there's a slew of changes about to land in the gate pipeline, so maybe give that a sec before we go further21:26
pleia2fungi: are you just looking at the zuul status page for this?21:26
sdakeanyone know how i fin the channel owner in irc?21:26
pleia2sdake: /msg chanserv info #channel21:26
clarkbdevkulkarni: yes this is usually a decent place to start at least21:26
fungipleia2: correct21:26
devkulkarniclarkb: thanks21:26
*** mtanino__ has joined #openstack-infra21:27
*** achanda has quit IRC21:27
richmjeblair: thanks21:27
sdakejeblair looks like you were thinkingahead with openstack-kolla21:28
*** baoli has quit IRC21:28
jeblairsdake: once you find them, you can fin them, i guess, if it's legal in your jurisdiction...21:28
jeblairsdake: oh please don't fin me!21:28
*** julim has quit IRC21:28
sdakefinish tem ? :)21:28
*** baoli has joined #openstack-infra21:28
*** zeih has quit IRC21:28
sdakejeblair what i the magic incantation to make #kolla point to #openstack-kolla21:28
*** eil397 has joined #openstack-infra21:28
jeblairmortal kombat21:29
sdakejeblair or could you do that for me? :)21:29
jeblairsdake: oh pleia2 knows how to do that... i think maybe she documented it....21:29
clarkbnibalizer: jeblair the puppet apply test doesn't seem to use the built in hiera file. Is that something that we have done work to do?21:29
sdakei am not in the group for openstack-kolla21:29
*** ybathia has quit IRC21:29
clarkber jesusaur not jeblair21:29
jeblairpleia2: yep!  found it!21:30
*** ybathia has joined #openstack-infra21:30
*** ybathia has quit IRC21:30
nibalizerbuilt in?21:30
nibalizerthe locla hiera? it should be using21:30
clarkbnibalizer: system-config/hiera/common.yaml21:30
clarkbnibalizer: it is not using it21:30
nibalizeryou're getting the errror on the elasticsearch nodes?21:30
jeblairsdake, pleia2: i will follow those instructions now for #kolla -> #openstack-kolla21:30
clarkbnibalizer: that works fine, but there are seds for all hiera lookups21:30
devkulkarniclarkb: this is the gate failure log: http://logs.openstack.org/44/296644/2/check/gate-solum-devstack-dsvm/c6207b4/console.html#_2016-03-28_15_57_38_886 .. I have tried couple of things to fix the issue (this bug comment has the details: https://bugs.launchpad.net/solum/+bug/1563019) .. I want to know what would be the correct solution to address the gate failure21:31
sdakejeblair thanks if there is anythign you need let me know21:31
clarkbnibalizer: see tools/apply-test.sh21:31
openstackLaunchpad bug 1563019 in Solum "devstack-gate-failure-due-to-screen-n-dhcp-log-not-found" [Critical,New] - Assigned to Devdatta Kulkarni (devdatta-kulkarni)21:31
*** richm has left #openstack-infra21:31
sdakei dont hve permissions to #kolla either21:31
sdakebecause i don't know my damn freenode pwd21:31
clarkbnibalizer: anyways it seems that that is why my change is failing on the gpg key21:31
*** eharney has quit IRC21:31
nibalizerclarkb: change number?21:31
clarkbnibalizer: 29315521:31
clarkbdevkulkarni: /home/jenkins/workspace/gate-solum-devstack-dsvm/logs/screen-n-dhcp.txt does not exist21:32
clarkbdevkulkarni: so we need to work backward from there and figure out why21:32
*** thorst has quit IRC21:33
*** dimtruck is now known as zz_dimtruck21:33
pabelangerclarkb: nibalizer: I started on 239783 but need to loop back to it for intree testing of hiera21:33
*** EricGonczer_ has quit IRC21:33
*** thorst has joined #openstack-infra21:33
pabelangerbut requires we add testing hiera data into system-config21:33
*** _ari_ has quit IRC21:34
pabelangerfungi: pleia2: brb21:34
pleia2sdake: if you want to retrieve your password, can do it via: /msg nickserv help sendpass21:34
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869921:34
nibalizerclarkb: ah so your analysis is correct21:35
nibalizerthe seds futz it up21:35
*** sdake_ has joined #openstack-infra21:35
nibalizerits like we need a way of marking some hiera calls so they don't get sed'd (when we intend for local hiera to be used)21:35
*** cloudtrainme has joined #openstack-infra21:35
pabelangerfungi: pleia2: back21:36
openstackgerritRodrigo Duarte proposed openstack-infra/project-config: Enable non-voting keystone tempest plugin tests  https://review.openstack.org/29869621:36
pleia2pabelanger: this watched pot is not boiling ;)21:36
clarkbnibalizer: I would append dummy key values to common.yaml21:36
* pleia2 continues to wait for the queue21:36
clarkbnibalizer: basically if not in common.yaml add it21:36
fungipabelanger: pleia2: these couple of jobs are definitely running slow for some reason. if that top one fails we'll just go ahead with the restart21:36
pleia2fungi: nods21:36
clarkbdevkulkarni: in http://logs.openstack.org/44/296644/2/check/gate-solum-devstack-dsvm/c6207b4/logs/devstack-gate-cleanup-host.txt you can see it says that file doesn't exist either21:37
nibalizerclarkb: thats dangerous21:37
nibalizerit's like having the defaults21:37
devkulkarniclarkb: right.. so couple of things .. 1) the issue seems to have started happening only recently. for instance, if you check this log, which is from a merged patch, http://logs.openstack.org/40/293640/6/check/gate-solum-devstack-dsvm/dd0b1f5/logs/, notice that screen-n-dhcp.txt.gz is an empty file.  2) in our code, we are not doing anything related to n-dhcp. in fact, to resolve this issue, I explicitly tried add21:37
nibalizerif we screw up our private hiera we'll put dummy values in to sensitive things21:37
clarkbdevkulkarni: are you running devstack? if so I think devstack is repsonsible for that21:37
clarkbnibalizer: you only do it in the test run21:38
*** zeih has joined #openstack-infra21:38
*** thorst has quit IRC21:38
clarkbnibalizer: basically instead of running that sed in the test run you run a update common.yaml for testing step21:38
*** sdake has quit IRC21:38
*** amitgandhinz has quit IRC21:38
nibalizerbasically cat fake.yml >> common.yaml in the apply test?21:38
devkulkarniclarkb: yes, this is from the devstack gate log21:39
clarkbdevkulkarni: in this case I think that there is a symlink because apparently we only copy the symlinks21:40
*** cody-somerville has quit IRC21:40
clarkbdevkulkarni: but whatever is pointed to either does not exist or does not hae permissions allowing us to read it21:40
*** dizquierdo has joined #openstack-infra21:40
*** bdemers_ has quit IRC21:40
devkulkarniclarkb: I see.21:40
jeblairpleia2: should i kick everyone from the old channel once that's done?  (cc: sdake_ )21:41
devkulkarniclarkb: the part of the code where this is happening is probably devstack/lib/nova https://github.com/openstack-dev/devstack/blob/master/lib/nova#L63021:41
sdake_jeblair if its not too much trobule you might say "hey I'm kicking you from the hannel, please rejoin immidately and things willbe back to normal21:42
sdake_or use your own words21:42
sdake_whatever makes sense21:42
*** dizquierdo has quit IRC21:42
sdake_i dont have voice int he channel so i can't say anything atm21:42
pleia2jeblair: sometimes I nickname: everyone a message to avoid an immediate kick and wait a day, but that's kind of a preference thing21:42
*** dizquierdo has joined #openstack-infra21:42
sdake_pleia2 knows best practice here it sounds like21:42
*** esker has quit IRC21:43
sdake_but ya we need people in one channel soon ;)21:43
clarkbdevkulkarni: https://github.com/openstack-dev/devstack/blob/master/lib/nova#L638 is what we are relying on working there21:43
*** knikolla has quit IRC21:43
*** fawadkhaliq has joined #openstack-infra21:44
clarkbdevkulkarni: and the symlink does seem to be repsent otherwise we couldn't copy it at all with our find21:44
jeblairpleia2, sdake_: ok, i'll Nick: msg everyone for now21:44
clarkbdevkulkarni: I think the target side may be unreadable21:44
jeblairthen maybe tomorrow kick them21:44
*** regXboi has quit IRC21:45
clarkbdevkulkarni: if you use http://logs.openstack.org/44/296644/2/check/gate-solum-devstack-dsvm/c6207b4/logs/reproduce.sh you should be able to inspect it locally21:45
devkulkarniclarkb: probably you are right. the link target is probably unreadable. any ideas/suggestions on where/what to fix?21:45
devkulkarniclarkb: ah I see21:45
clarkbdevkulkarni: the fix should be in devstack21:46
clarkbif it ws readable in the past it should remian redable21:46
fungipabelanger: pleia2: i'm definitely going to look into this job once it finishes. i've logged onto the worker and ps says it's been on "apt-get install rabbitmq-server" for over 40 minutes21:46
devkulkarniclarkb: makes sense21:46
devkulkarniclarkb: I will try to reproduce it and see if that is the issue21:47
fungipabelanger: pleia2: i wonder if apt-get on (some?) vexxhost nodes is super slow21:47
pabelangerfungi: pleia221:47
pabelangerfungi: pleia2: I have seen that before, slowness21:47
pabelangeron vexxhost21:47
*** sigmavirus24 is now known as sigmavirus24_awa21:47
devkulkarniclarkb: I can run reproduce.sh and that will setup the devstack gate env, right? or are there any other steps that I need to do?21:48
fungimnaser: seeing very slow (disk i think) performance on c0f3c475-70b0-473e-8551-cfb3bb590f5b right now21:48
*** ayoung has joined #openstack-infra21:48
*** mylu has quit IRC21:49
*** |-paul-| has quit IRC21:49
fungimight be network. strace on the pid shows a select poll loop going on21:49
*** mylu has joined #openstack-infra21:50
pabelangerclarkb: first issue with devstack-gate: http://paste.openstack.org/show/492367/ from setup-host.txt21:50
clarkb  devkulkarni I think there was a change to add some prereqs to the comments of the scrpt but not sure if that has merged yet21:52
clarkbyou will need virtualenv and a couple of libs I think21:52
clarkbdevkulkarni: https://review.openstack.org/#/c/294308/21:53
devkulkarniclarkb: I see .. let me start and see how far I can go21:53
clarkbdevkulkarni: I will get that reviewed now21:53
*** xyang1 has quit IRC21:54
clarkbnibalizer: is fixing the hiera situation something you are willing wanting to work on or should I start poking at that?21:54
fungiinteresting... anyone know why we never added an aaaa record for mirror.ca-ymq-1.vexxhost.openstack.org?21:54
openstackgerritsebastian marcet proposed openstack-infra/puppet-openstackid: Apache MPM events + php5 fpm  https://review.openstack.org/29869921:54
devkulkarniclarkb: cool.. once that patch merges I will initiate a reproduce run on my side21:55
devkulkarniclarkb: I see that you have pushed it along21:55
*** smarcet has quit IRC21:55
clarkbdevkulkarni: well that patch merging won't affect yours21:55
clarkbdevkulkarni: since yours is done and written21:55
clarkbdevkulkarni: but you can refer to the contents of that patch to see what you need21:55
devkulkarniclarkb: ah I see21:55
devkulkarniclarkb: ok, will do that21:55
*** zeih has quit IRC21:55
*** cody-somerville has joined #openstack-infra21:56
clarkbfungi: because it didn't have ipv6 when we created it21:56
*** austin81 has quit IRC21:56
fungiclarkb: oh, right, that got added a couple days after21:56
openstackgerritThomas Goirand proposed openstack-infra/project-config: Add a job to build Debian packages  https://review.openstack.org/29402221:58
sdake_jeblair sorry for ninja dropping this on you21:58
sdake_you could have waited if you needed to ;)21:58
devkulkarniclarkb: ok.. looks like the patch is documenting the dependencies that are required. should be straightforward to install those.21:58
fungiit eventually moved on to apt-get install apache2 libapache2-mod-wsgi21:58
*** achanda has joined #openstack-infra21:58
devkulkarniclarkb: thanks for the pointer21:58
funginot seeing any packet loss21:58
nibalizerclarkb: either way21:59
clarkband now we have dns record limits iirc21:59
clarkbwhich we probably should sort out21:59
nibalizerits like a few line of code i think21:59
fungiclarkb: yeah, i'm thinking we should add a subdomain zone for ic.openstack.org21:59
*** jsavak has quit IRC21:59
fungii'm pretty sure it's all the infracloud nodes which popped us over the limit21:59
fungior close to anyway21:59
nibalizermost of that stuff can be deleted because ita gettong redone ya?22:00
clarkbnibalizer: not sure what you mean22:00
jeblairsdake_: no prob,  was actually a good time :)22:00
clarkboh thats infracloud not test stuff22:00
*** mriedem has quit IRC22:00
clarkbthe streams crossed22:00
*** fawadkhaliq has quit IRC22:01
clarkbnibalizer: ya  Ithink I see a simple ish way of doing this will work on a quick patch to see22:01
openstackgerritThomas Goirand proposed openstack-infra/project-config: Add a job to build openstack-pkg-tools  https://review.openstack.org/29402222:01
*** gokrokve has quit IRC22:01
funginibalizer: yeah, that's my thought. we delete all the existing *.ic.openstack.org records and then add them in a ic.openstack.org zone when we recreate them after we get the new ip addresses22:01
*** Sukhdev has joined #openstack-infra22:01
zigopabelanger: AJaeger: Does it look more like what you'd expect?22:01
openstackgerritPaul Belanger proposed openstack-infra/devstack-gate: Use service to stop / start rsyslog  https://review.openstack.org/29899822:01
*** kzaitsev_mb has quit IRC22:02
*** fawadkhaliq has joined #openstack-infra22:02
openstackgerritSteven Dake proposed openstack-infra/project-config: Change #kolla to point to point to #openstack-kolla  https://review.openstack.org/29899922:02
*** thorst has joined #openstack-infra22:02
zigopabelanger: AJaeger: I still have the issue that the repo isn't using master, but debian/mitaka, so I don't know how to get this checkout correctly before the build.22:02
*** gokrokve has joined #openstack-infra22:02
*** esker has joined #openstack-infra22:02
zigopabelanger: AJaeger: Writing "git checkout" within the shell script doesn't look like the way to go...22:03
*** austin81 has joined #openstack-infra22:04
*** cdent has quit IRC22:05
*** daemontool has quit IRC22:05
*** thorst has quit IRC22:07
*** tpsilva has quit IRC22:07
*** gokrokve has quit IRC22:07
*** gokrokve has joined #openstack-infra22:07
*** achanda has quit IRC22:07
openstackgerritSteven Dake proposed openstack-infra/project-config: Change #kolla to point to #openstack-kolla  https://review.openstack.org/29899922:08
jeblairfungi: ++ ic dns22:09
fungijeblair: still feels like a terrible workaround for an arbitrary limitation in rackspace's dns service22:09
*** cloudtrainme has quit IRC22:09
fungibut it's what we've got (for now at least)22:10
jeblairfungi: ya, probably worth raising it with rax, but hey, if we ran our adns we'd probably zone it anyway :)22:10
*** ybathia has joined #openstack-infra22:11
fungiif we wanted a separate mechanism updating records in it, maybe (or we might delegate the subdomain to something in/on infracloud itself?) but there's nothing wrong with long zonefiles per se22:12
clarkbnibalizer: the thing that makes this really ugly is the many hiera files :(22:12
clarkbnibalizer: because a key could be defined for one host over in a group file but not for another22:12
*** mylu has quit IRC22:12
clarkbnibalizer: and we in theory should test with that split properly in place22:12
*** markvoelker has joined #openstack-infra22:13
clarkbI think I will just do what pabelanger suggests and write the contents to a file and use that as the key_source instead22:14
clarkbmuch simpler for what I need right now22:14
pabelangerthat what I did in https://review.openstack.org/#/c/295560/22:14
jeblairfungi: (though in the brave new world of signed dns, there can be minor disadvantages)22:15
*** annegentle has quit IRC22:15
*** roxanaghe has joined #openstack-infra22:15
*** fawadkhaliq has quit IRC22:15
*** fawadkhaliq has joined #openstack-infra22:16
fungiyeah, the re-signing on update takes longer22:16
*** annegentle has joined #openstack-infra22:16
fungithough i've not seen that happen in practice, i also haven't tried to host a 1k record zonefile with signing22:16
jeblairme neither22:17
openstackgerritClark Boylan proposed openstack-infra/system-config: Apt mirror for ceph hammer release  https://review.openstack.org/29315522:18
clarkbpabelanger: ^ that look right?22:18
funginow this job is on "apt-get install libguestfs0"22:18
*** mylu has joined #openstack-infra22:19
fungii have a feeling it's going to hit the timeout22:19
*** aysyd has quit IRC22:19
clarkbpabelanger: jeblair I am going to wip that just to let mordred look it over once he has a chance since mordred did much of the initial reprepro work22:19
nibalizerclarkb: ya i think a file full of fakes is the way to go22:19
*** roxanaghe has quit IRC22:19
clarkband if he is happy with it I can go ahead and make and mount the volume22:19
clarkbnibalizer: you mean don't dynamically generate it just write it out upfront?22:20
*** mylu has quit IRC22:20
nibalizerclarkb: ya22:20
nibalizeri mean what would you dynamicall generate it from? the private data?22:20
*** IlyaG has quit IRC22:20
clarkbnibalizer: the same way we do it today22:20
*** markvoelker has quit IRC22:20
nibalizeri don't know what that means22:21
clarkbexcept that we have an extra dimension if writing it to real hiera files22:21
*** markvoelker has joined #openstack-infra22:21
clarkbnibalizer: in tools/apply-test.sh we modify every hiera line so that it does not do a hiera lookup, those modications use dfeault data or make up a default if there isn't one22:21
*** IlyaG has joined #openstack-infra22:21
nibalizeri thought you were going to make a file that had keys for everything22:22
clarkbso you could do that dynamically, the problem is there is an extra dimension when you go to real hiera files since individual hosts can have their own files22:22
nibalizerwith dummy data22:22
*** markvoelker_ has joined #openstack-infra22:22
clarkbno I was going to do the dynamic thing until I relaised its a pita22:22
clarkbthen I just updated my change to work around this problem wit hour testing22:22
nibalizerturned off some validation or so?22:22
clarkbbecause I could spend the rest of the day fixing the tests or the next 5 minutes getting my change to pass22:23
clarkbnibalizer: load the data from file instead of hiera22:23
devkulkarniclarkb: tried running reproduce.sh. it failed. this is the content of /opt/stack/logs/devstacklog.txt.gz http://paste.openstack.org/show/492369/22:23
pabelangerfungi: pleia2: getting pulled away from computer by family.  I'll pass the torch to pleia2 for a while to drive.22:24
pabelangerfungi: pleia2: continue with out me, and will come back shortly22:24
pleia2pabelanger: noted, I need to leave in 35 minutes to take a kitty to the vet22:25
pleia2fungi, pabelanger: I think the next step is restarting zuul, which should be pretty straight forward?22:25
*** markvoelker has quit IRC22:26
clarkbdevkulkarni: is your job doing something funny and cloning solum out of band?22:26
*** sdake has joined #openstack-infra22:26
clarkbdevkulkarni: that might actually be a bug in reproduce.sh22:27
clarkbdevkulkarni: you can likely work around that by adding the PROJECTS= declarations from your jenkins job22:27
devkulkarniclarkb: as in adding PROJECTS=solum in reproduce.sh?22:28
*** sdake_ has quit IRC22:28
EmilienMfungi: yesterday, I released the puppet modules and for some reasons (a stupid mistake in my script...), I failed for one module (puppet-aodh). How bad is it to delete an tag?22:28
devkulkarniclarkb: we are not doing out of band cloning .. but looking through our code to be sure22:28
*** austin81 has left #openstack-infra22:28
EmilienMif it's too bad I can create a second tag.22:29
fungipleia2: pabelanger: yep, it's just exporting the current list of changes from the check and gate pipelines (we have a convenience script for that) and then restarting zuul, waiting for less common node types to reregister in gearman, then reenqueuing everything for check/gate22:29
openstackgerritSteven Dake proposed openstack-infra/system-config: Move statusbot and eavesdrop bot for kolla  https://review.openstack.org/29900922:29
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Add PROJECTS var to reproduce.sh  https://review.openstack.org/29901022:29
clarkbdevkulkarni: ^ I think for the future that will fix it22:30
fungiEmilienM: consider tags mostly undeletable (and certainly not reusable)22:30
clarkbdevkulkarni: no, the issue is that PROJECTS was not ending up in the reproduce.sh script see 29901022:30
clarkbjeblair: 299010 will probably interest you22:30
devkulkarniclarkb: looking22:30
EmilienMfungi: ok; i'll push another tag then.22:30
fungiEmilienM: tag deletion doesn't propagate on remote update, nor does replacement by a new tag22:30
fungiso it can lead to some ugly side effects22:30
openstackgerritJohn L. Villalovos proposed openstack-infra/project-config: Ironic uses grenade plugin. Setup GRENADE_PLUGINRC  https://review.openstack.org/29901122:31
EmilienMfungi: ok... my mistake...22:31
clarkbpabelanger: did your docs update for the afs rw path get pushed?22:31
*** mylu has joined #openstack-infra22:33
*** tonytan4ever has quit IRC22:33
jeblairclarkb: yes!  though the conflict-with has pointed me at https://review.openstack.org/280027  ; see the comment i left on yours22:34
*** mylu has quit IRC22:34
EmilienMfungi: I pushed 8.0.1 ...22:34
*** annegentle has quit IRC22:35
clarkbjeblair: I don't think that is necessary we already dedup22:35
clarkbbut this is a little cleaner22:36
clarkbI will go ahead and abandon mine22:36
jeblairclarkb: oh, we dedup $PROJECTS?22:36
clarkbjeblair: yup22:37
jeblairthe only thing i don't like about tonyb's change is that it treats $PROJECTS very specially22:37
clarkbwas the only way to sanely start removing entries from PROJECTS22:37
jeblairclarkb: ok22:37
pabelangerpleia2: fungi back22:37
pabelangerclarkb: no, I can push it up in a few22:37
pabelangerclarkb: unless you want to22:37
tonybjeblair: I assume were talkign aboiut my save PROJECTS thing in devstack-gate?22:38
clarkbpabelanger: no you should, just waiting for it :)22:38
tonybjeblair: I agree we're treating is specially but AFIOCT it's the only option22:38
tonybjeblair: there are lots of things the reproduce misses which makes it annoying22:39
clarkbdoes that var need to be declared like the others?22:39
tonybjeblair: do you have an alternate idea I can look at implementing?22:39
jeblairtonyb: well, it would be nice if it were easy to add the missing vars22:39
*** markvoelker_ has quit IRC22:39
jeblairtonyb: if every one is this complicated, that's :(22:39
openstackgerritPaul Belanger proposed openstack-infra/system-config: Document read-write afs volumes  https://review.openstack.org/29901322:40
pabelangerclarkb: ^22:40
jeblair(also, maybe once we remove projects from $PROJECTS we can rename it to DEVSTACK_GATE_PROJECTS)22:40
jeblairthen we can drop the specialness22:40
clarkbjeblair: do you remember why declare -x was used over straight up export?22:40
tonybjeblair: I agree.  I guess I was doing the simple things first and then trying to abstract it later once we can see a pattern22:41
jeblairtonyb: wfm for now22:41
jeblairclarkb: nope22:41
jeblairclarkb: (probably because i started by typing "export" and saving the output)22:41
*** mylu has joined #openstack-infra22:41
anteayasorry for disappearing, meetings, then phone call then dinner, now I want to take a walk while I can, back later22:42
mgagnezaro: are there any plan to publish latest scp plugin to update center?22:42
clarkbmgagne: I think we did22:42
tonybjeblair: renaming PROJECTS could work.22:42
mgagneclarkb: I unfortunately think not, still 1.822:42
clarkbmgagne: ok well we did their stupid rule stuff and pushed a new tag  Ithought22:43
clarkboh that was for a differnet plugin22:43
clarkbzmq plugin is the one we did22:43
mgagneI don't see any tag https://github.com/jenkinsci/scp-plugin/22:43
clarkbtonyb: does that work without exporting like the other vars?22:44
*** annegentle has joined #openstack-infra22:44
*** fawadkhaliq has quit IRC22:44
*** fawadkhaliq has joined #openstack-infra22:44
*** annegentle has quit IRC22:45
tonybclarkb: it'd still be a special case.22:45
clarkbtonyb: yes I know22:45
clarkbalso I think you may need quotes22:46
*** fawadkhaliq has quit IRC22:46
clarkbsince there will be spaces in that list22:46
clarkbtonyb: commented22:47
tonybclarkb: thanks.  I'll fix them.22:48
*** esker has quit IRC22:48
*** baoli has quit IRC22:51
*** hashar has quit IRC22:51
*** gordc has quit IRC22:51
*** Sukhdev has quit IRC22:51
*** nadya has joined #openstack-infra22:51
pabelangerfungi: look like the job just failed and was kicked out of the gate22:52
*** fawadkhaliq has joined #openstack-infra22:52
openstackgerritMerged openstack-infra/project-config: Run the v3 only job in glance  https://review.openstack.org/29319922:52
fungipabelanger: awesome, yeah, it reported the timeout to the change22:52
pleia2poor changes, but yay for us22:52
pleia2(kind of, timeout is not so yay)22:53
pabelangerbut yay for us22:53
*** sdague has quit IRC22:53
pleia2looks like fungi queued up the script?22:53
fungiyep, so let's move on with the restart steps. i didn't expect to wait anywhere near that long, but i'll make good use of the logs since the job reported22:53
fungithat script queued up there will save a copy of the check pipeline as a script that can be used to reenqueue it all22:53
openstackgerritTony Breeds proposed openstack-infra/devstack-gate: Save PROJECTS form the job defintion for use in reproduce.sh  https://review.openstack.org/28002722:53
clarkbif it isn't one thing it is another :)22:54
clarkbdowncase is used in our puppet conf erb22:54
clarkbbut apparently is not available on our internap trusty nodes?22:54
fungipabelanger: pleia2: we'll in rapid succession do that and the gate pipeline similarly, then ungracefully stop and start zuul22:54
clarkbnibalizer: ^ any idea what is going on with that?22:54
*** jcooley_ has quit IRC22:54
pabelangerfungi: pleia2: sorry, dropped22:54
clarkbnibalizer: http://logs.openstack.org/55/293155/8/check/gate-infra-puppet-apply-ubuntu-trusty/372e007/console.html22:55
nibalizerjust like downcase($foo)22:55
fungipabelanger: pleia2: i'll let one of you punch enter on that script and then do similarly for gate > gate.sh22:55
clarkboh its because fqdn is undefined22:55
pabelangerfungi: pleia2 go for it22:55
nibalizeri think you called downcase on a variable that isn't defined22:55
clarkbso it tries to downcase a nil var22:55
*** nadya has quit IRC22:56
fungipabelanger: pleia2: and then service zuul stop and check the process list to make sure it's dead dead and the service zuul start again22:56
clarkbnibalizer: ya its because fqdn fact isn't working22:56
pleia2fungi, pabelanger, ok, on it22:56
* clarkb wonders if glean changed again22:56
pleia2fungi: right?22:56
fungipleia2: yep22:56
fungiprobably need to give that zombie a few secs22:57
pleia2yep, there it goes22:57
fungiyeah, looks good22:57
fungino need to sudo since we're already in an interactive root shell, but that's fine22:57
pleia2muscle memory :)22:57
EmilienMwe have a new funny error in Puppet CI: http://logs.openstack.org/03/298903/1/gate/gate-puppet-ceilometer-puppet-beaker-rspec-ubuntu-trusty/b37eea0/console.html#_2016-03-29_22_05_42_19122:57
*** esker has joined #openstack-infra22:57
EmilienME: Unable to locate package hiera (ubuntu)22:58
pleia2fungi: anything to do before we run our scripts?22:58
*** kzaitsev_mb has joined #openstack-infra22:58
clarkbEmilienM: I think that is because hiera is built into puppet now/22:58
clarkbEmilienM: maybe they killed the package I dunno22:58
EmilienMclarkb: in puppet4, not in 322:58
EmilienMand trusty ships 322:58
EmilienMwell, afik.22:58
pabelangerpleia2: fungi: all that is left is to enqueue at this point I take it?22:59
*** eil397 has quit IRC23:00
pleia2I think so23:00
clarkbEmilienM: apt-cache search hiera doesn't show a hiera23:00
fungipleia2: pabelanger: yeah, let's make sure we've got uncommon node types registered in gearman23:00
clarkbthere is a ruby-hiera23:00
*** dizquierdo has quit IRC23:00
pleia2fungi: oh right, where do we look for that?23:00
fungipleia2: pabelanger: otherwise a lot of these changes could get kicked back with NOT_REGISTERED for jobs needing them23:00
dansmithdid something just happen?23:01
dansmitheverything dumped out of the check queue23:01
pabelangerdansmith: yes23:01
EmilienMclarkb: It's weird, it worked a few hours ago, on the same cloud (ovh) http://logs.openstack.org/23/298923/1/gate/gate-puppet-oslo-puppet-beaker-rspec-ubuntu-trusty/0f02cac/console.html#_2016-03-29_20_27_02_18623:01
*** rbrndt_ has quit IRC23:01
pabelangerdansmith: we are reloading zuul23:01
fungidansmith: we're restarting zuul. will be reenquing it all in moments23:01
fungiwe waited for a gate reset to lessen the loss of throughput23:02
clarkbEmilienM: maybe they repos have been cleaned up? I am just using my local machine there is no hiera package in trusty23:02
clarkbEmilienM: the puppetlabs repos may have a hiera package but we have ripped those out for a long time t oavoid hitting them on every apt-get update23:02
EmilienMclarkb: can we tell to apt-get to not fail in jenkins jobs if the package is not present?23:02
clarkbEmilienM: I think its better to only list the things you intend to install23:02
*** kzaitsev_mb has quit IRC23:02
clarkbyum has this behavior and it makes me wnat to never use yum :)23:02
pabelangerclarkb: EmilienM: http://apt.puppetlabs.com/pool/trusty/main/h/hiera/ still exists23:02
clarkbpabelanger: ya I don't think we are using the puppetlabs repo in that job though at least not that early on23:03
EmilienMpabelanger, clarkb: the problem is not in puppetlabs repo23:03
fungipleia2: pabelanger: yeah, looks like we've got all the ones i can think of showing up, so we should be all set23:03
EmilienMthe problem is in the macro that tries to cleanup the node23:03
pabelangerclarkb: EmilienM: Ah23:03
pleia2fungi: great23:03
EmilienMwe run that: sudo apt-get remove -y --purge facter puppet puppet-common hiera23:03
EmilienMon every nodes23:03
EmilienMand now it starts to fail23:03
pleia2fungi: gate or check first?23:04
clarkbEmilienM: that probably means that with new images there is no longer a hiera package to clean up?23:04
fungipleia2: pabelanger: so we enqueue gate first then check, that way we don't have to wait for zuul to hand jobs to all the check pipeline before the reenqueued gate jobs get some23:04
* pleia2 big red button23:04
clarkbso the fqdn fact doesn't work if hostname -f doesnt return something with '.'s in it23:05
clarkbit is the dumbest thing in the world23:05
*** mescanef has quit IRC23:05
clarkbbut thats the situation on our test nodes now due to glean?23:05
fungibig red openstack-shaped button23:05
*** Sukhdev has joined #openstack-infra23:05
pleia2this is strangely satisfying23:05
EmilienMclarkb: http://git.openstack.org/cgit/openstack-infra/project-config/commit/?id=4ff660c4eee680c12e40b9c38216749fc3d46af423:05
EmilienMclarkb: I'm going to revert that.23:06
clarkbEmilienM: please dont23:06
clarkbEmilienM: at least not until we confirm that that is what we should do23:06
EmilienMclarkb: I ran a stupid recheck to see if it's consistent...23:06
clarkbEmilienM: we should figure out if hiera is actually no longer a package or just no longer getting installed or maybe something else is purging it23:07
*** cloudtrainme has joined #openstack-infra23:07
*** gokrokve has quit IRC23:07
devkulkarniclarkb: ran reproduce.sh. seeing same problem as gate. here is the list of log files http://paste.openstack.org/show/492375/ notice that the n-dhcp target (n-dhcp.log.2016-03-29-223710) does not exist23:07
clarkbdevkulkarni: neat! so ya thats definitely a devstack bug23:07
dansmiththe reason I noticed the dump was I was waiting to recheck something that hit the vexxhost hang. so, thanks @infra for doing it for me :P23:07
clarkbdevkulkarni: if the target doesn't exist it shouldn't create a symlink23:08
EmilienMclarkb: it seems like it's not hiera but ruby-hiera in trusty http://packages.ubuntu.com/trusty/ruby-hiera23:08
fungidansmith: yep, courtesy recheck without reporting a failure for you ;)23:08
dansmithfungi: awesome new feature23:08
clarkbEmilienM: yup, but our image builds should be using the puppetlabs repo that pabelanger linked to I think23:08
clarkbEmilienM: then at the end of the build we remove that repo23:08
pabelangerclarkb: EmilienM: hiera was installed as of yesterday dibs: http://nodepool.openstack.org/dib.ubuntu-trusty.log.2016-03-28_1723:08
clarkbbut even with the repo removed apt-get should know how to uninstall a package23:09
fungidansmith: i'm also looking at a grenade job that hit a timeout in vexxhost. looks like apt-get install was running crazily slow23:09
EmilienMpabelanger: oh, I wasn't aware puppet / hiera stuff was installed from puppetlabs repos.23:09
pabelangerclarkb: EmilienM missing from today: http://nodepool.openstack.org/dib.ubuntu-trusty.log23:09
dansmithfungi: yeah23:09
clarkbpabelanger: neat, I wonder if that was a change upstream?23:09
clarkblike maybe it is no longer a dep?23:09
EmilienMnice shot pabelanger23:09
EmilienMclarkb: /me digging changelogs in packaging.23:09
EmilienM_ody: fyi ^23:10
*** cloudtrainme has quit IRC23:10
devkulkarniclarkb: cool. so the fix will probably somewhere between 630-640 of https://github.com/openstack-dev/devstack/blob/master/lib/nova ?23:10
EmilienMclarkb: don't you have the error in infra jobs?23:10
clarkbdevkulkarni: possibly, I am not super familiar with devstack23:10
clarkbEmilienM: no23:10
clarkbEmilienM: we don't purge hiera23:10
devkulkarniclarkb: ok.. do you know which channel do the devstack folks hang out generally?23:11
clarkbdevkulkarni: #openstack-qa23:11
*** Qiming has joined #openstack-infra23:11
devkulkarniclarkb: thanks23:11
*** esker has quit IRC23:11
fungiclarkb: EmilienM: i've been out of the loop dealing with some other stuff. where's an example failure log for this package removal issue?23:12
EmilienMfungi: http://logs.openstack.org/03/298903/1/gate/gate-puppet-ceilometer-puppet-beaker-rspec-ubuntu-trusty/b37eea0/console.html#_2016-03-29_22_05_42_19123:12
EmilienMand it's consistent, I see the failure in every trusty jobs, since a few minutes23:13
clarkbEmilienM: its likely that our image uploads are completting and switching over to todays build which does not have that package23:13
fungiEmilienM: thanks23:13
clarkband may also explain why my job just started to fail on fqdn things23:13
EmilienMclarkb: at least we have an updated image :-)23:13
clarkbassuming things cut over and picked up new behavior23:13
*** EricGonczer_ has joined #openstack-infra23:14
fungiwe can pretty easily delete those images and fall back to the previous ones as a short-term relief while we figure out what needs fixing23:15
pleia2fungi, pabelanger: need to rush off to the vet with my little critter23:15
clarkbhostname -f on internap says localhost23:15
clarkband its /etc/hosts is different than ovh/rax23:15
fungipleia2: thanks! looks like we're just waiting for the check reenqueue to complete, so that was teh final step anyway23:15
mgagneclarkb: aren't you guys building your own image?23:15
pabelangerEmilienM: clarkb: fungi: we did land 297964 this morning. I wonder if this is a side affect23:16
pleia2fungi: yep, seems so23:16
pabelangerEmilienM: clarkb: fungi which adds xenial support23:16
*** achanda has joined #openstack-infra23:17
clarkbmgagne: yes its likely our image that is broken not anything in the cloud23:17
*** fawadkhaliq has quit IRC23:18
mgagneclarkb: what can I do to help you?23:18
pabelangerEmilienM: clarkb: fungi: Yup, something doesn't look right.  We are no longer downloading puppetlabs-release-trusty.deb23:18
fungipabelanger: yeah, i suspect a subtle logic error there rather than changes to the package repo (i just checked it, timestamps are old)23:18
mgagneclarkb: ok, I thought you mentioned my name ^^'23:19
clarkbpabelanger: does < work that way?23:19
fungiwe really should set -x that script... do we really not?23:19
*** fawadkhaliq has joined #openstack-infra23:20
pabelangerclarkb: I thought it did23:20
EmilienMpabelanger: yes, it seems like we don't install the deb anymore :(23:20
*** EricGonczer_ has quit IRC23:20
fungiokay, any objection to me rolling back nodepool images while we work on fixing?23:21
fungipabelanger: pleia2: jeblair: btw, the check changes finished reenqueuing, so other than keeping an eye on zuul status and cacti graphs, this restart is done23:21
EmilienMclarkb, pabelanger: I'm also wondering if the < works23:22
*** fawadkhaliq has quit IRC23:22
pabelangertypo in my script23:23
EmilienMwe should maybe use if (versioncmp($::operatingsystemmajrelease, '16') < 0) {23:24
clarkbon ovh-gra1 the hostname has novalocal appended to it but internap does not23:24
clarkbI wonder if that makes glean behave differently, it is the same glean version in both cases23:24
clarkboh wait, dhcp can assign the hostname info too23:24
openstackgerritPaul Belanger proposed openstack-infra/system-config: Fix typo with xenial logic  https://review.openstack.org/29903123:24
clarkbmaybe we are fighting dhcpd23:24
fungiyou can unconfigure that in dhclient23:24
pabelangerguess we need bashae23:25
*** jtomasek has quit IRC23:25
fungirather, you can configure dhclient to ignore it23:25
mgagneclarkb: we don't have DHCP enabled and override the dhcp_domain in Nova with an empty string.23:25
EmilienMpabelanger: nice shot !23:25
clarkbman command not found23:25
* clarkb is ready to kill these "minimal" images23:25
EmilienMpabelanger: I was also wondering about the fact test for operatingsystem23:25
EmilienMbut that's fine23:25
clarkbmgagne: ah, so we are configuring with glean in internap as well?23:26
clarkbI can confirm in /tec/network/23:26
fungipabelanger: i'm back to wondering why that script isn't set -x so we can see what it's doing in the image builds23:26
mgagneclarkb: don't know the details23:26
clarkbah yup its static23:26
pabelangerfungi: I can add it on follow up\23:26
pabelangerand -e too23:26
fungipabelanger: -e may be a lot more problematic23:27
EmilienMdid we roll back the ubuntu-trusty image yet?23:27
fungiEmilienM: i'm working on it now23:27
*** kzaitsev_mb has joined #openstack-infra23:28
clarkboption domain-name "local" is coming from dhcp on ovh23:28
*** ZZelle has quit IRC23:28
*** mylu has quit IRC23:28
clarkbI bet this is enough to make the differnet behaviors happen23:28
clarkbbut facter fqdn returns nothing in both cases so may not actually explain why puppet is unhappy23:29
EmilienMclarkb: cat /etc/hostname ?23:29
clarkbEmilienM: the hostname is the hostname in both cases, does not include a domain23:29
clarkband facter won't give you an fqdn of hostname,23:29
clarkber hostname.23:29
fungiokay, the ones which had uploaded so far today (internap-nyj01, osic-cloud1, ovh-gra1, rax-ord, vexxhost-ca-ymq-1) have been deleted23:30
fungii'll keep an eye out for any more popping into ready state this evening23:30
clarkbI wonder if that is related to the fqdn madness23:31
clarkblike maybe different versions of facter behaving differently23:31
*** ybathia has quit IRC23:31
*** eil397 has joined #openstack-infra23:32
clarkb1.7.5 v 2.4.623:32
clarkbwhat do you know23:32
clarkbI bet thats it23:32
fungii don't think there are any more ubuntu-trusty images that are going to try to upload (i don't see any more in a building state, and i also did a dib-image-delete of the original for them)23:32
*** EricGonczer_ has joined #openstack-infra23:32
clarkbmordred: rcarrillocruz greghaynes for the record glean appears to write different /etc/hosts in the directly static configured interface case vs the dhcp case. Guessing that dhcp races it to writing the file out23:32
*** gokrokve has joined #openstack-infra23:33
*** kzaitsev_mb has quit IRC23:33
clarkblets see if I can get my change to work on a recheck23:33
*** angdraug has quit IRC23:34
*** gokrokve has quit IRC23:37
fungiclarkb: in dhclient.conf i suspect we want: supersede hostname "$OUR_THING";23:37
fungier, host-name23:38
openstackgerritPaul Belanger proposed openstack-infra/system-config: Activate debugging for install_puppet.sh  https://review.openstack.org/29903423:38
pabelangerfungi: okay, just -x for now ^23:38
clarkbfungi: Iam going to leave that up to glean to sort out, imo it shouldn't be doing that and instead it should make /etc/hosts look like what dhcpds will write23:38
fungiclarkb: should probably also do the same for domain-name too23:38
*** jtomasek has joined #openstack-infra23:39
fungiclarkb: i would assert that we don't actually want dhclient modifying our host-name and domain-name on these machines, but i could be wrong23:39
clarkbthere is definitely a tug of war going on between how much glean should be involved in23:39
clarkbI personally don't like glean being involved in much because it constantly breaks23:40
fungithough instructing dhclient to set them to what we want seems like a reasonable solution23:40
clarkbwhereas dhclient and dhcpcd have both worked for many years for me23:40
fungiunfortunately, i don't know how we'd rely on dhclient to set these values on the system in environments where there's no dhcpd to query23:41
*** fawadkhaliq has joined #openstack-infra23:41
*** ybathia has joined #openstack-infra23:41
*** gokrokve has joined #openstack-infra23:41
clarkbright, it is an unfortunate bit of design for some clouds23:42
clarkband glean exists for a raisin. But we still don't have a good answer for rabbitmq + glean I don't think23:42
fungirabbit stew23:42
JayFWe just saw a host of downstream failures around this change: https://github.com/bbangert/routes/pull/65 in a neutron dependency. Thought I'd let you all know in case you see some craziness there. cc+thanks morgabra + clif_h23:43
* fungi should have said hasenpfeffer23:43
*** sdake has quit IRC23:43
JayFAlthough I guess with upper-constraints, that should shelter openstack ci from the change, assuming we get the upstream author to restore his compatibility :)23:44
*** sdake has joined #openstack-infra23:44
clarkbJayF: and assuming contraints is tested against something that covers that23:44
clarkbJayF: but generally it works23:44
*** sdake has quit IRC23:44
*** sdake has joined #openstack-infra23:44
JayFclarkb: well figured I'd put that link in some IRC logs and minds in case something blows up related to it here :)23:44
morgabraalso https://review.openstack.org/#/c/298855/23:45
fungiclarkb: well, changes have also been submitted to global requirements to skip 2.323:45
clarkbdevstack-centos7 nodes unregistered23:46
clarkbpresumably after zuul restart23:46
*** camunoz has joined #openstack-infra23:49
pabelangerJust noticed that too23:49
* clarkb thinks it is a sign23:50
*** torandu has quit IRC23:50
clarkbfirst gpg problems, then fqdn problems, now gearman problems23:50
ianwi was just running some experimental tests to ensure centos-7 == devstack-centos7 ... but yeah ... then everything exploded23:51
pabelangerjust cleaned up 18 leaked ips from bluebox23:52
pabelangerchecking osic now23:52
*** torandu has joined #openstack-infra23:52
clarkbcrinkle: fwiw I thought the mellanox cards were capable of running infiniband or 10Gbe and we had 10Gbe gbic things in them23:52
clarkbcrinkle: which is why tripleo suffered through using mellanox over the non mellanox options (to get better throughput over the available 1gig interfaces)23:53
clarkbgreghaynes: SpamapS lifeless ^ you may be able to correct me though23:53
crinkleyeah they would know better than me23:53
clarkbthankfully the kernel issues on mellanox cards seem to be mostly sorted out now but for a long time there were issues aiui23:53
crinklethey're still a big pain23:54
clarkband I think 1x10Gbe is > 2x1gig even if we hvae to jump through hoops for neutron23:54
crinklenot kernel issues but they make ironic booting take forever23:54
*** yamada-h has joined #openstack-infra23:54
*** mriedem has joined #openstack-infra23:55
*** gokrokve has quit IRC23:55
*** achanda has quit IRC23:56
*** mylu has joined #openstack-infra23:58
*** yamada-h has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!