Monday, 2016-10-17

*** ashw has joined #rdo00:09
*** gildub has quit IRC00:19
*** limao has joined #rdo00:22
*** limao_ has joined #rdo00:26
rdogerritEmilien Macchi created rdoinfo: pin puppetlabs-ntp to 4.2.x branch
*** limao has quit IRC00:29
*** gildub has joined #rdo00:29
*** gildub_ has joined #rdo00:32
rdogerritrdo-trunk proposed rdoinfo: Bump rdoinfo tags to latest upper-constraints
*** gildub has quit IRC00:35
*** gdubreui has joined #rdo00:36
*** dhill_ has quit IRC00:37
*** gildub_ has quit IRC00:37
*** akshai has quit IRC00:39
*** ashw has quit IRC00:39
*** noslzzp has joined #rdo01:17
rdogerritMerged rdoinfo: pin puppetlabs-ntp to 4.2.x branch
rdogerritEmilien Macchi created rdoinfo: pin puppet-ntp also for Newton
*** eaguilar has joined #rdo01:28
*** tiswanso has joined #rdo01:30
*** fzdarsky__ has joined #rdo01:32
*** gildub_ has joined #rdo01:34
*** gdubreui has quit IRC01:35
*** fzdarsky_ has quit IRC01:36
*** dmacpher has joined #rdo01:36
*** gdubreui has joined #rdo01:36
*** jcoufal has quit IRC01:38
*** rhallisey has quit IRC01:38
*** gildub_ has quit IRC01:39
*** vagarwal has quit IRC01:44
*** noslzzp has quit IRC01:50
*** noslzzp has joined #rdo01:51
*** gdubreui has quit IRC01:53
*** gildub has joined #rdo01:53
*** Duck has joined #rdo02:04
*** zshi_ is now known as zshi02:08
*** aortega has quit IRC02:25
*** tiswanso has quit IRC02:35
*** ashw has joined #rdo02:41
*** crossbuilder has joined #rdo02:44
*** crossbuilder_ has quit IRC02:45
*** tiswanso has joined #rdo02:55
*** tiswanso has quit IRC03:10
*** tiswanso has joined #rdo03:10
*** paragan has joined #rdo03:13
*** paragan has joined #rdo03:13
*** limao_ has quit IRC03:13
*** chlong has joined #rdo03:30
*** tiswanso has quit IRC03:32
*** ashw has quit IRC03:39
*** eaguilar has quit IRC03:48
*** limao has joined #rdo03:54
*** noslzzp has quit IRC03:58
*** suresh12 has joined #rdo04:03
*** arxcruz has quit IRC04:06
*** kashyap` has joined #rdo04:20
*** kashyap has quit IRC04:20
*** paragan has quit IRC04:20
*** snarwade has quit IRC04:20
*** snarwade has joined #rdo04:20
*** Eywa has quit IRC04:23
*** paragn has quit IRC04:24
*** kashyap` has quit IRC04:24
*** apetrich has quit IRC04:26
*** kashyap has joined #rdo04:29
*** apetrich has joined #rdo04:33
*** numans has joined #rdo04:42
*** suresh12 has quit IRC04:53
*** suresh12 has joined #rdo04:53
*** suresh12 has quit IRC05:03
*** arxcruz has joined #rdo05:17
*** chandankumar has joined #rdo05:22
*** apetrich has quit IRC05:27
*** apetrich has joined #rdo05:28
*** Son_Goku has joined #rdo05:28
*** Son_Goku has quit IRC05:37
*** Son_Goku has joined #rdo05:42
*** sudipto has joined #rdo05:44
*** dgurtner has joined #rdo05:44
*** sudipto_ has joined #rdo05:44
*** satya4ever has joined #rdo05:45
*** jaosorior has joined #rdo05:48
*** mcornea has joined #rdo05:51
*** paragan has joined #rdo05:52
*** pbandark has joined #rdo05:54
*** bhavik has quit IRC05:54
*** Son_Goku has quit IRC05:55
*** rdas has joined #rdo05:57
*** dgurtner has quit IRC05:57
*** pgadiya has joined #rdo05:58
*** florianf has joined #rdo05:59
*** rcernin has joined #rdo06:06
*** Poornima has joined #rdo06:13
*** Poornima has quit IRC06:13
*** ealcaniz has joined #rdo06:17
*** coolsvap has joined #rdo06:20
*** jbadiapa_ has joined #rdo06:20
*** Son_Goku has joined #rdo06:22
*** shadower has joined #rdo06:26
*** rasca has joined #rdo06:29
*** jprovazn has joined #rdo06:29
*** paragan has quit IRC06:29
*** jtomasek_ has joined #rdo06:31
*** smeyer has joined #rdo06:31
*** King_InuYasha has joined #rdo06:36
*** Son_Goku has quit IRC06:38
*** King_InuYasha is now known as Son_Goku06:39
*** Jeffrey4l has quit IRC06:45
*** paragan has joined #rdo06:45
*** paragan has joined #rdo06:45
*** jtomasek_ has quit IRC06:47
*** gildub has quit IRC06:47
*** dgurtner has joined #rdo06:50
*** dtrainor has quit IRC06:52
*** oshvartz has joined #rdo06:53
*** saneax-_-|AFK is now known as saneax06:53
*** iranzo has joined #rdo06:57
*** Jeffrey4l has joined #rdo07:00
*** tesseract has joined #rdo07:03
*** tesseract is now known as Guest2107707:04
*** emalikov has joined #rdo07:09
*** dhoppe_ has quit IRC07:11
*** dhoppe has joined #rdo07:11
*** imcsk8 is now known as imcsk8_PTO07:14
*** amoralej|off is now known as amoralej07:16
*** ihrachys has joined #rdo07:18
*** vimal has joined #rdo07:19
*** jlibosva has joined #rdo07:21
*** apevec has joined #rdo07:23
*** apevec has quit IRC07:23
*** apevec has joined #rdo07:23
imcsk8_PTOapevec: hello i'm on your timezone! :P07:25
*** k4n0 has joined #rdo07:25
imcsk8_PTOapevec: i found this problem on saturday:, mitaka needs rdo testing to be enabled in order to work...07:25 bug 1385292 in openstack-packstack "Packstack mitaka does not work if testing repo is not enabled" [Urgent,New] - Assigned to mmagr07:25
*** Son_Goku has quit IRC07:26
*** oshvartz has quit IRC07:26
*** gchamoul|afk is now known as gchamoul07:30
*** jpena|off is now known as jpena07:31
jpenagood morning!07:33
*** openstackgerrit has quit IRC07:34
*** openstackgerrit has joined #rdo07:35
*** ChanServ sets mode: +v openstackgerrit07:35
*** jpich has joined #rdo07:35
*** aufi has joined #rdo07:36
*** flepied1 has quit IRC07:36
rdogerritJavier Peña proposed rdoinfo: pin puppet-ntp also for Newton
imcsk8_PTOjpena: good morning, do you know interesting stuff to do in brussels?07:37
jpenaimcsk8_PTO, you can see the Atomium, the Maneken Pis (probably mispelled), and try some Belgian beer, they're excellent :)07:38
jpenaimcsk8_PTO, are you there?07:38
imcsk8_PTOjpena: yes, we drove from paris last night :)07:39
*** suresh12 has joined #rdo07:39
imcsk8_PTOi found a "belguian beer tour"07:40
*** zoli_gone-proxy is now known as zoliXXL07:41
rdogerritAlfredo Moralejo created rdoinfo: Fix pin of puppet-ntp
jpenaamoralej: re., there's already
imcsk8_PTOjpena: i think there's a packaging problem with mitaka: bug 1385292 in openstack-packstack "Packstack mitaka does not work if testing repo is not enabled" [Urgent,New] - Assigned to ichavero07:42
*** dtantsur|afk is now known as dtantsur07:42
amoralejright jpena, i didn't see it...07:43
*** dmacpher has quit IRC07:43
amoraleji like more my version, to be honest...07:43
jpenaimcsk8_PTO, that's because we need to have a more up-to-date package in the mitaka repo07:43
chandankumarjpena, amoralej number80 apevec \o/07:43
*** suresh12 has quit IRC07:44
jpenathere's also that the erlang package requires all the wx stuff (and it's not built), but we can fix that in packstack by having the latest version07:44
zoliXXLgood morning07:44
amoralejis review.r.o down?07:45
imcsk8_PTOjpena: i thought that was it, because when i test packstack mitaka from source i don't have that problem07:46
*** shardy has joined #rdo07:46
*** Son_Goku has joined #rdo07:47
amoralejjpena, can you access
jpenaamoralej, nope07:47
*** Poornima has joined #rdo07:47
*** rdogerrit has quit IRC07:48
*** gtrxcb has quit IRC07:50
jaosorior seems to be down. TripleO CI started to fail trying to access it07:50
*** ade_b has joined #rdo07:50
*** Poornima has quit IRC07:52
amoralejyes jaosorior07:52
amoralejwe are working on it07:52
jaosorioramoralej: ah, alright, just noticed and wrote it here. Anyway, thanks07:52
*** dalvarez has joined #rdo07:53
amoralejbtw, jaosorior, what is using tripleo-ci review.r.o for?07:54
amoralejdownload ansible roles?07:54
*** ohamada has joined #rdo07:55
*** Amita has joined #rdo07:56
jaosorioramoralej: for building packages with it apparently. for instance, it failed with this puppet-neutron change. It checks out the change from OpenStack's gerrit, then tries to fetch the distgit repo to build the package which comes rfom rdo's gerrit.
*** Amita has joined #rdo07:56
*** gema_ is now known as gema07:58
*** dbecker has joined #rdo08:01
*** limao has quit IRC08:02
*** limao has joined #rdo08:03
jpenaabout the review.rdo outage: today is the rcip-dev migration (moving to another datacenter). So review.rdo will be down until the migration is done08:04
amoralejjaosorior, it's strange that it's trying to build puppet-neutron in the
amoralejah, i got it, it builds the tested change08:07
*** Guest69614 is now known as assassin08:12
*** assassin has quit IRC08:12
*** assassin has joined #rdo08:12
*** permalac has quit IRC08:15
*** King_InuYasha has joined #rdo08:19
*** gszasz has joined #rdo08:19
*** Son_Goku has quit IRC08:22
*** racedo has joined #rdo08:22
*** athomas has joined #rdo08:25
*** pnavarro has joined #rdo08:30
*** King_InuYasha has quit IRC08:32
*** Son_Goku has joined #rdo08:33
*** hlkv6 has left #rdo08:35
*** Son_Goku has quit IRC08:36
apevecjpena, hmm, we could force rdoinfo change to switch to github/rdo-packages during outage?08:37
apevecdirect push to github08:37
jpenaapevec: that won't be of help, since DLRN will try to use review.rdo when building08:37
apevecwhere? it uses rdoinfo doesn't it?08:38
*** Son_Goku has joined #rdo08:38
jpenayes, but all distgit sources are set to review.rdo08:38
apevecthat's what I'm proposing to switch to github08:38
apevecduring outage08:38
jpenaah ok, I thought you were referring to the rdoinfo change to pin puppet-ntp08:39
apevecoh that did not merge?08:39
jpenathe first one did, but it only set a proper pin for ocata (not ocata-uc or newton)08:40
apevecwe need to add "how to pin" in docs08:40
jpenaok, I can do it manually on the dlrn instance, then revert back once review.rdo is online08:41
apevecjpena, that won't help for tripleoci08:41
apevecthey use dlrn directly08:41
apevecbut yeah, let's do that first08:41
apevecso first we need to merge puppet-ntp fixup08:42
apevecjpena, can you push that fixup to your github?08:44
jpenaapevec: sure, give me a sec08:44
apevecI didn't fetch it from review.rdo in time08:44
*** KarlchenK has joined #rdo08:46
*** anton- has quit IRC08:46
*** rdas has quit IRC08:46
jpenathere is no mirror in rdo-packages for the puppet distgits :-/08:46
jpenaso changing rdoinfo won't help :(08:47
apevecdamn, then we're stuck08:47
apevecwhy don't we mirror them?08:48
jpenano idea. I'll check that once the server is back08:48
apevecmaybe we wanted them in separate org?08:49
jpenacould be. gchamoul do you know by chance? ^^08:49
*** karimb has joined #rdo08:51
*** ccamacho has joined #rdo08:51
apevecforefast for BCN week of Oct 24: rain and showers... any more bad news this Monday?08:53
*** Son_Goku has quit IRC08:53
*** anton- has joined #rdo08:55
*** paramite has joined #rdo08:57
*** Son_Goku has joined #rdo08:59
*** Son_Goku has quit IRC08:59
*** derekh has joined #rdo08:59
*** jschwarz is now known as jschwarz|ooo08:59
*** milan has joined #rdo08:59
*** ade_b has quit IRC09:00
* gchamoul is reading ...09:02
*** Son_Goku has joined #rdo09:02
*** rdem_ is now known as rdem09:02
*** limao_ has joined #rdo09:03
*** limao has quit IRC09:04
gchamouljpena: we've made the migration into SF from but If I remember well, it was expected to use rdo-packages org as a mirror!09:05
jpenagchamoul: I see the last updates are from March 18... so it looks like there was a missing step and we never switched to rdo-packages09:06
gchamoulthat's actually what I understood 10 monthes ago09:06
gchamouljpena: it seems yes :(09:06
remix_tjtrown: hi, i'm still looking at the redis issue. The ocf resource *should* create the required directory in /var/run/, don't know what's wrong09:06
*** pcaruana has joined #rdo09:08
*** karimb has quit IRC09:09
*** KarlchenK has quit IRC09:11
*** flepied has joined #rdo09:12
amoralejremix_tj, i found it was actually creating when pacemaker is enabled09:12
amoralejyou found it's not creating it?09:13
*** anshul has joined #rdo09:13
jlibosvaIs down or it's just me?09:13
amoralejit's down09:14
apevecjlibosva, planned outage, should be back in p.m.09:15
jlibosvaok, thx09:15
*** anshul has quit IRC09:17
*** Alex_Stef has joined #rdo09:18
*** hewbrocca-afk is now known as hewbrocca09:21
*** gfidente has joined #rdo09:23
*** gfidente has quit IRC09:23
*** gfidente has joined #rdo09:23
*** karimb has joined #rdo09:23
*** Alex_Stef has quit IRC09:23
apevecjpena, in the meantime, could you force puppet-ntp rebuild with your fixup?09:24
jpenaapevec: puppet-ntp is specifically affected by the review.rdo outage (distgit is only there)09:25
apevecjpena, but there is local checkout on dlrn ?09:25
jpenalet me see if I can trick dlrn into not trying to refresh it09:25
apevecyeah that09:25
apevecbut looks like we also have ftbfs... consistent is rather old09:25
*** chlong has quit IRC09:26
*** limao_ has quit IRC09:26
apevechrm app-catalog-ui09:26
apevecjpena, ^ let's drop it temp from local rdoinfo copy09:26
apevecto get fresh consistent09:26
*** limao has joined #rdo09:27
apevecit's been failing for 10days, time for unresponsive maintainer action09:27
jpenaapevec: it's 10 days since last success, actually09:27
jpenaI proposed a fix for the ftbfs last Friday09:27
*** Son_Goku has quit IRC09:29
*** limao has quit IRC09:29
*** Son_Goku has joined #rdo09:30
*** paragan has quit IRC09:31
*** Alex_Stef has joined #rdo09:32
amoralejbtw, jpena the issue with app-catalog-ui is affecting centos-newton also09:34
jpenaamoralej: makes sense, they're branchless09:35
jpenaso we'll need a cheryr-pick09:36
amoralejand maybe use a snapshot to make it builds in cbs09:38
jpenaapevec: ok, it took some hacks but we've got a new
jpenawith puppet-ntp based on branch 4.2.x09:39
apevecexcellent, thanks!09:39
apevecok that should unblock at least part of tripleoci09:40
apevecEmilienM, ^09:40
apevecpatches to puppet-tripleo will be still failing b/c of distgit09:40
*** jaosorior has quit IRC09:41
remix_tjamoralej: i found that is created, but resource is not starting09:41
*** jaosorior has joined #rdo09:42
amoralejmmmm, i saw it working fine with pacemaker profile in rdo-ci09:42
amoralejdo you have any log remix_tj?09:42
*** permalac has joined #rdo09:42
remix_tjamoralej: i have only outputs from pcs resource show reporting redis as failed09:42
amoralejand in the redis.log , it complains about the socket?09:43
openstackgerritJavier Peña proposed openstack/packstack: Change default Identity API from v2.0 to v3
remix_tjamoralej: this is the output and yes, on redis.log i have
*** tosky has joined #rdo09:47
chandankumarpackage review python-ruamel-ordereddict : bug 1385534 in Package Review "Review Request: python-ruamel-ordereddict - Ordered dictionary with Key Insertion Order" [Medium,New] - Assigned to nobody09:47
*** vagarwal has joined #rdo09:49
remix_tjamoralej: i initially thought was an issue about selinux set to enforcing, but now is permissive and still not ok09:49
amoralejremix_tj, what repo are you using?09:49
remix_tjamoralej: newton from cloud sig (rdo-release.rpm installed one)09:51
amoralejthis is from a pacemaker instalation with the same repos
*** toanju has joined #rdo09:55
remix_tjvery strange09:56
remix_tji'm running RHEL709:56
remix_tjmaybe that's the difference09:56
amoralejcheck the resource-agents version09:57
remix_tjuhm, you're right09:57
remix_tji'll take a look09:57
remix_tjhow do i find in ci which package version is installed?09:58
*** rgogunskiy has joined #rdo10:00
*** dgurtner has quit IRC10:01
remix_tji have resource-agents-3.9.5-54.el7_2.10.x86_64, maybe that's the difference10:02
remix_tji'll check10:02
remix_tjuhm, latest available in my repository now is .1610:04
*** sudipto has quit IRC10:04
remix_tjnow running a refresh to check if something new is available10:04
*** sudipto_ has quit IRC10:04
amoralejaccording to changelog there has been no changes in redis from 10 to 1710:05
*** dalvarez has quit IRC10:06
remix_tjamoralej: since i'm using older cluster packages, maybe recreating images with more up to date setup would be a nice option.10:07
*** flepied has quit IRC10:07
*** kashyap has left #rdo10:08
amoralejremix_tj, there are images built yesterday
*** ushkalim has joined #rdo10:09
*** panda_ is now known as panda10:10
remix_tjamoralej: i'm building my own images with RHEL7, i suppose the ones you're linking are using centos10:11
*** dries_ has quit IRC10:14
amoralejcould you do a pcs resource show redis?10:14
*** dries has joined #rdo10:15
apevecjpena, oh, review.rdo is back?10:20
jpenais it?10:20
jpenanot for me10:21
apevecit was review.o.o I've hit :)10:21
apevecwe need different background theme10:21
*** flepied has joined #rdo10:22
*** fzdarsky__ has quit IRC10:25
dmelladoapevec: jpena lol10:25
apevecjpena, amoralej - which ones would be rc-candidates?
apevecI proposed
amoralejalso i think10:28
jpenaapevec: I'd say and
*** fzdarsky has joined #rdo10:28
apevecok, let me add topic10:28
*** Son_Goku has quit IRC10:28
EmilienMapevec, jpena: thanks10:28
*** zoliXXL is now known as zoli|lunch10:28
EmilienMgood morning here10:28
apevecEmilienM, I've added info and question in
EmilienMapevec: so we fixed ocata, but also newton?10:29
apevec" how will tripleoci pick that up, does it require "current-tripleo" promotion? "10:29
apeveclet's check that fix works for master first10:30
EmilienMapevec: no, afik for tripleo & puppet projects, tripleo-ci takes latest consistent10:30
apevecwhere is that package list?10:30
apevecit might be "current" iirc10:30
EmilienMapevec: looks like it worked here:
EmilienMlet me check ntp version10:31
apevecjpena, ^ we'll need same hacks (removing app-catalog-ui and puppet-ntp fixup) on newton dlrn too10:32
jpenaapevec: ack, I'll do it there, too10:32
EmilienMmhh no10:33
EmilienMi'm not sure it's fixing master yet10:33
EmilienMI did a recheck, let's see10:33
*** sudipto has joined #rdo10:34
*** sudipto_ has joined #rdo10:34
apevecEmilienM, it's "current"
*** noslzzp has joined #rdo10:36
*** gfidente has quit IRC10:36
jpenaapevec, EmilienM done!10:37
*** sudswas__ has joined #rdo10:38
*** noslzzp has quit IRC10:38
*** sudipto_ has quit IRC10:39
*** sudipto has quit IRC10:39
*** sudipto_ has joined #rdo10:39
*** noslzzp has joined #rdo10:39
afazekasapevec, Can u have look on this:
EmilienMapevec: wdyt if I implement something like and the automatic updates at each new release?10:43
EmilienMat each new release of a puppet module10:43
EmilienMjpena: any ETA before review.rdoproject comes back?10:44
jpenaEmilienM: 15:00 UTC10:45
EmilienMjpena: was it scheduled?10:45
EmilienMall tripleo CI jobs are red10:45
EmilienMand we were about to release Newton RC310:45
apevecEmilienM, that's what I asked for, we obviously need "upper-constraints" for puppet-*10:45
apevecshould we start using poi Puppetfile ?10:46
EmilienMjpena: it's down for 4.15h more?10:46
apevecor tripleo needs more specific versions?10:46
jpenaEmilienM, they're moving to another datacenter, so that's the expected downtime10:46
EmilienMshardy: ^10:46
EmilienMjpena: that's terrible for us10:46
apevecEmilienM, we have workaround, dlrn repos are available10:46
EmilienMwait what is workaround10:47
apevecEmilienM, we postponed it as much as we could, and I didn't know clear RC schedule when I OKed the date10:47
EmilienMit was on mailing list10:47
EmilienMwhat is the workaround?10:47
*** rhallisey has joined #rdo10:47
apevecEmilienM, jpena can push critical changes locally on dlrn10:47
*** Alex_Stef has quit IRC10:47
EmilienMit's failing everywhere :(10:47
apevecjpena, ^ ok so we could change distgit in rdoinfo for what we have on github/rdo-packages ?10:48
apeveclet's do that, I'll push the change10:48
EmilienMGetting to ./data/openstack-tripleo-heat-templates_distro (rpm-master)10:49
EmilienMit fails^10:49
jpenaapevec: yes, for everything we have in github/rdo-packages (all except puppet-* )10:49
EmilienMhow can we workaround that?10:49
apevecas I said above, will workaround it in rdoinfo10:49
*** gfidente has joined #rdo10:50
*** gfidente has quit IRC10:50
*** gfidente has joined #rdo10:50
apevecre. schedule - outage was planned before RC schedule details were posted10:50
apevecalso was topic on meeting last week10:50
apevecwe missed tripleoci impact10:50
*** rhallisey has quit IRC10:51
jschlueterEmilienM: what all is planned for part of RC3 tag for tripleo? ... have a link to the tag review?10:53
*** Alex_Stef has joined #rdo10:53
* jschlueter wondering if openstack-tripleo-common is in there10:53
*** rhallisey has joined #rdo10:53
apevecjpena, please review before I push it10:56
apevecto real rdoinfo repo10:57
apevecjpena, also please push your followup fix so I can include it10:57
apevecpush to your github10:57
jpenait looks ok to me10:57
*** dgurtner has joined #rdo10:57
jpenalet me push my fix10:57
*** Alex_Stef has quit IRC10:58
EmilienMjschlueter: no link10:59
jschlueterEmilienM: ack just wondering if will be in a tag soon for openstack-tripleo-common10:59
*** pkovar has joined #rdo11:01
EmilienMjschlueter: it will be in a tag when we'll be able to release something11:01
*** florianf has quit IRC11:01
EmilienMapevec: "puppet-* are NOT mirrored so those will still fail"11:03
EmilienMapevec: do you mean tripleo CI will remain down?11:03
apevecyeah unfortunately11:03
apevecreviews on puppet-* will be failing11:03
apevecwhich trigger package rebuild11:04
apevecbut tht should work now, I've pushed rdoinfo change11:04
EmilienMshardy: ^11:04
EmilienMwe need puppet-tripleo11:04
EmilienMso we need to wait 15UTC I guess...11:04
apevecEmilienM, ok, let's fix puppet-tripleo11:04
apevecdo you have a copy of latest distgit ?11:04
apevecwe'll create it manually on rdo-packages11:04
apevecjpena, ^ can you grab latest copy from dlrn machine?11:05
apevecand push to your github11:05
shardyThanks for looking into workarounds folks11:05
apevecI'll create empty repo11:05
EmilienMyeah thanks apevec and jpena11:05
shardyout of interest, where was this planned outage communicated?11:05
shardy(it may have been on openstack-dev, but evidently I missed it)11:06
jpenaapevec: you want the latest rdoinfo copy?11:06
apevecon rdo-list, on meeting11:06
apeveclast week11:06
apevecjpena, puppet-tripleo-distgit11:06
jpenaah, ok11:06
apevecjpena, I've pushed rdoinfo, see commit link above11:06
shardyapevec: Ok, sounds like perhaps looping in openstack-dev would have been good given the heavy dependence of openstack CI on these repos11:06
EmilienMon the list?11:06
EmilienMI haven't seen it11:06
EmilienMrdo-list? what thread?11:07
apevecit was at least in meeting minutes but nm digging into this now, we'll improve next time11:07
EmilienMlast email I saw about downtime was [rdo-list] Scheduled maintenance of 2016-10-10 13:00 UTC11:07
apevecthat was SF upgrade11:08
EmilienMI haven't seen anything else11:08
EmilienMit is not obvious for people if it's not in the list11:08
*** zoli|lunch is now known as zoli11:08
apevecflepied, ^ did we miss to send up separate outage announcement for datacenter move?11:08
remix_tjamoralej: this is pcs resource show redis11:09
*** sudipto_ has quit IRC11:11
*** sudswas__ has quit IRC11:11
*** zoli is now known as zoliXXL11:11
amoralejit should use default values remix_tj, so it should work...11:11
EmilienMapevec: where is the script that runs uc updates?11:11
*** dgurtner has quit IRC11:11
apevecthere's job now, jpena ^ do you have a link?11:12
*** gfidente has quit IRC11:12
apevecEmilienM, there is in rdoinfo repo11:12
EmilienMI'll try to make the same thing for puppet modules11:12
apevecbut that works with upper-constraints.txt only11:12
apevecEmilienM, ack11:12
*** florianf has joined #rdo11:12
apevecwe could add that into the same script11:12
jpenaapevec: the bot is in review.rdo, so... :)11:12
apevecahh :(11:13
apevecdo you have tripleo-puppet-distgit ready?11:13
remix_tjamoralej: i'm not very expert of pacemaker, but in case of broken service i simply do pcs resource cleanup redis, pcs resource enable redis11:14
jpenajust finished it11:14
*** paragan has joined #rdo11:14
apevecjpena, thanks - please double-check
remix_tjbut now i try with newer pacemaker packages, since i'm not up to date with this packages11:14
jpenaapevec: are you going to sync from my repo? It's empty now11:14
jpena(the one in rdo-packageS)11:14
apevecyeah, pushing your copy there now11:15
amoralejyes, remix_tj those seems the right commands to re-enable it11:15
jpenaapevec: ok, then it looks good to me11:15
apevecjpena, oh you didn't keep git history?11:17
jpenaapevec: nope11:18
apevechmm, that might make a mess11:18
*** dgurtner has joined #rdo11:18
*** dgurtner has joined #rdo11:18
apeveccan you copy .git from dlrn ?11:18
jpenaapevec: true, let me give it a try11:18
apevecor wait, I've ssh there :)11:18
*** jkilpatr has joined #rdo11:20
jpenaapevec: fixed in my repo11:20
*** _degorenko|afk is now known as degorenko11:22
apevecEmilienM, shardy - please recheck one puppet-tripleo review11:23
EmilienMlet's see how it goes in zuul11:23
apevecjpena, amoralej - - 4 total, anything else?11:25
jpenathat's it for me11:25
*** alexismonville has joined #rdo11:26
amoralejyes, i think thats it11:26
openstackgerritJavier Peña proposed openstack/packstack: Change default Identity API from v2.0 to v3
EmilienMjpena: where are we with puppet4?11:30
jpenaEmilienM, now works fine for packstack with puppet3 and puppet4, so once it's merged we should be good11:30
EmilienMjpena: nice11:31
*** dpeacock has joined #rdo11:31
*** matbu is now known as matbu|brb11:31
EmilienMapevec: ok it works11:33
EmilienMdlrn:Getting to ./data/puppet-tripleo_distro (rpm-master)11:34
*** karimb has quit IRC11:35
EmilienMapevec: I still see an error during delorean commands11:36
EmilienMI'm waiting for log files11:36
EmilienMReceived exception Error in build_rpm_wrapper for puppet-tripleo11:36
apevechmm, anymore?11:37
EmilienMI need to wait the job to finish so I have all logs11:37
*** alexismonville has quit IRC11:37
EmilienMwill let you know in  very show time11:37
*** aortega has joined #rdo11:38
*** dgurtner has quit IRC11:43
*** dprince has joined #rdo11:43
amoralejthat spec is wrong, %setup should point to openstack-tripleo-5.2.011:44
*** zoliXXL is now known as zoli|brb11:44
apevecjpena, ^ hmm, was that latest version?11:44
EmilienMit sounds like the distgit is old?11:45
* jpena checks11:45
amoralejEmilienM, where is the full log?11:45
amoralejjpena, last is in /home/centos-ocata/data/puppet-tripleo_distro11:46
jpenayep, I think I got an older version11:47
*** gfidente has joined #rdo11:47
jpenait should be ok now, sorry11:48
jpenaapevec: can you re-import?11:48
*** dmacpher has joined #rdo11:48
EmilienMjpena: np, thanks for updating it11:48
apevecjpena, ack11:48
apevecjpena, only rpm-master, newto-rdo is good?11:49
jpenaI re-pushed both11:50
apevecok, no changes in newto-rdo so it was good11:50
*** aortega has quit IRC11:50
apevecEmilienM, recheck please11:51
*** iclebyte has joined #rdo11:52
*** k4n0 has quit IRC11:53
EmilienMapevec: do you run p-o-i and tripleo jobs when updating uc?11:54
apevecjpena ^ has job in the making11:55
EmilienMI'm thinking how to make it for tripleo11:55
apevecbut I think it's running only weirdo jobs ?11:55
*** jayg|g0n3 is now known as jayg11:56
*** steveg_afk has joined #rdo11:58
*** panda is now known as panda|afk11:58
*** steveg_afk has quit IRC11:58
*** amuller has joined #rdo12:00
*** dgurtner has joined #rdo12:03
*** alexismonville has joined #rdo12:03
*** k4n0 has joined #rdo12:05
*** sreichar has joined #rdo12:05
*** jcoufal has joined #rdo12:08
*** hewbrocca is now known as hewbrocca-afk12:09
EmilienMapevec: all sounds fine now12:09
*** pgadiya has quit IRC12:10
*** amoralej is now known as amoralej|lunch12:11
*** Son_Goku has joined #rdo12:11
*** karimb has joined #rdo12:12
EmilienMapevec: why don't we have puppet-* in ?12:12
EmilienMapevec: nevermind. I have an idea how to make this puppetfile thing12:14
EmilienMwithout hardcoding list of modules12:14
*** kgiusti has joined #rdo12:15
*** fultonj has quit IRC12:17
*** fultonj has joined #rdo12:17
*** coolsvap has quit IRC12:18
*** Poornima has joined #rdo12:19
*** wfoster has quit IRC12:19
EmilienMapevec: newton is also fixed right?12:19
*** sai has quit IRC12:19
*** kgiusti has quit IRC12:20
*** rdas has joined #rdo12:20
jpenaEmilienM, yes, the puppet-ntp package is updated too12:20
*** rook_ has quit IRC12:20
EmilienMthx a lot12:20
*** kgiusti has joined #rdo12:20
*** v1k0d3n has quit IRC12:20
*** maeca has joined #rdo12:20
*** Guest23990 is now known as zigo12:21
weshayis the rcip cloud down?12:22
weshayflepied, ^12:22
flepiedweshay: yes we are moving the machines12:22
EmilienMflepied: where was announcement?12:23
weshayflepied, k.. thanks fred12:23
flepiedEmilienM: in RDO meetings and mailing list iirc12:23
EmilienMflepied: what thread?12:23
EmilienMTBH I don't think everyone read RDO meetings notes12:24
*** rbowen has joined #rdo12:25
*** ChanServ sets mode: +o rbowen12:25
*** wfoster has joined #rdo12:25
*** rook has joined #rdo12:26
*** rook is now known as Guest4069312:26
*** sai has joined #rdo12:27
*** aortega has joined #rdo12:27
*** dgurtner has quit IRC12:29
*** morazi has joined #rdo12:29
*** rgogunskiy has quit IRC12:29
*** dtantsur is now known as dtantsur|mtg12:30
trownit is probably a good idea to send outage information in separate email from meeting notes12:31
*** rgogunskiy has joined #rdo12:31
*** unclemarc has joined #rdo12:32
*** zoli|brb is now known as zoli12:32
*** zoli is now known as zoliXXL12:32
*** shaunm has quit IRC12:34
EmilienMtrown: definitly. IIUC nobody outside rdo was aware about this outage12:36
trownEmilienM: ya, ideally that outage does not impact anyone outside of RDO too12:38
EmilienM"ideally" :)12:38
*** gfidente has quit IRC12:39
*** akshai has joined #rdo12:40
dmsimardI dropped the ball on that, the action was mine to send a reminder to rdo-list12:40
dmsimardjruzicka: did you send meeting minutes last week? Not finding them12:42
*** sileht has quit IRC12:42
*** akshai_ has joined #rdo12:42
*** trown is now known as trown|brb12:43
*** eaguilar has joined #rdo12:43
*** matbu|brb is now known as matbu12:43
leifmadsenmorning all!  question around packaging -- EPEL is disabled when doing builds right?12:44
dmsimardleifmadsen: EPEL is not enabled unless explicitely enabled, yes12:44
leifmadsendmsimard: is that a valid scenario though? I ask because I'm helping someone build a package that I'd like to submit back into RDO, but I haven't done this before :) One of the deps they have I think is only available in EPEL.12:45
apevecEmilienM, re.  puppet-* in - will be fixed after outage12:45
leifmadsenso I wasn't sure if I could pass that into dlrn while building, and if so, if that was allowed to be submitted12:45
apevecleifmadsen, which dep?12:45
leifmadsenapevec: SymPy12:45
dmsimardleifmadsen: nothing in RDO currently requires EPEL and we'd like to keep it that way -- depending on what the requirement is we can maybe pull it12:45
apevecwe need to add it in RDO / CBS12:45
apevecleifmadsen, and which package requires that?12:46
*** akshai has quit IRC12:46
leifmadsenok, so the preference would be to package any deps a package needs to be built for RDO as well12:46
leifmadsenapevec: vitrage12:46
*** jprovazn has quit IRC12:46
apevecleifmadsen, s/preference/requirement/12:46
apevecEPEL is evil :)12:47
leifmadsenpedantic mode: enabled12:47
*** zaneb has quit IRC12:47
EmilienMapevec: ack12:47
*** zaneb has joined #rdo12:48
*** jpena is now known as jpena|lunch12:50
*** sileht has joined #rdo12:50
*** abehl has joined #rdo12:50
number80leifmadsen: ha, I'm supposed to meet them next week12:51
dmsimardapevec, number80: when are we going to push for a mitaka stable update ?12:53
leifmadsennumber80: yea same here :)12:54
leifmadsenI was mostly looking at it as a thing to learn how to package stuff12:54
leifmadsenso I got as far as making everything work in dlrn before I ran into a dep issue12:54
dmsimardWould be nice to do it this week if we can.. otherwise we're looking at 2+ weeks again without updates12:54
*** limao has joined #rdo12:54
dmsimardleifmadsen: oh, you'll be at BCN ?12:54
leifmadsendmsimard: ummm... OSS? :)12:54
dmsimardleifmadsen: yeah12:54
dmsimardopen source software, right? (jk)12:54
leifmadsenindeed I will -- got the +1 just a couple weeks ago :)12:55
leifmadsendmsimard: lolz... yea... acronym overload12:55
dmsimardleifmadsen: neat, would love to pick your brain about your ara usage12:55
*** satya4ever has quit IRC12:55
number80dmsimard: lemme check the queue first12:55
leifmadsendmsimard: oh for sure! Yolanda will be there too12:55
number80leifmadsen: actually, vitrage is packaged12:55
leifmadsensince when?12:55
*** strigazi1AFK has joined #rdo12:55
number80 bug 1342987 in Package Review "Review Request: openstack-vitrage - OpenStack RCA (Root Cause Analysis) Engine" [Unspecified,Assigned] - Assigned to hguemar12:55
leifmadsennumber80: oh yea, I've been looking at that -- I don't really like the spec as is though12:56
leifmadsenand it fails under dlrn as is12:56
leifmadsenthat's what I've been looking at12:56
number80leifmadsen: then, you need to work with them :)12:56
leifmadsennumber80: I know, I'm meeting with them next week12:56
leifmadsenthat's why I was asking about deps and such12:56
*** karimb has quit IRC12:57
*** egallen has joined #rdo12:57
*** limao_ has joined #rdo12:57
*** strigazi_AFK has quit IRC12:58
* number80 is depiling mails ...12:59
*** tiswanso has joined #rdo12:59
*** tiswanso has quit IRC12:59
*** tiswanso has joined #rdo13:00
*** limao has quit IRC13:00
dmsimardtrown|brb: fyi the mitaka stable packstack failures are known13:00
*** ohochman has joined #rdo13:01
*** emalikov has quit IRC13:01
dmsimardtrown|brb: backported testing improvements (nested kvm virt) that haven't yet made it to stable13:01
*** Goneri has joined #rdo13:02
*** Son_Goku has quit IRC13:02
number80dmsimard: tagging launched13:04
*** jmelvin has joined #rdo13:04
dmsimardnumber80: tagging of what to what13:04
*** elmiko has joined #rdo13:04
number80dmsimard: mitaka-testing -> release13:04
dmsimardnumber80: ok, fwiw everything was green yesterday:
number80yep,I checked that13:05
*** ohochman has left #rdo13:05
dmsimardAnyone else hwere ever used git-crypt ?
dmsimardJust looking for feedback on whether or not it's a sane option to version sensitive files in public repos13:07
*** eaguilar_ has joined #rdo13:08
*** mlammon1 has joined #rdo13:08
dmsimardI'm not a crypto maniac and we're not talking about storing state government secrets but still :)13:08
*** eaguilar has quit IRC13:09
dmsimardIt's fairly easy to use, too, ex:
adarazsdmsimard: interesting solution.13:10
miscdmsimard: I think arrfab use it (centos sysadmin)13:10
adarazsdmsimard: I remember something about an RCIP maintenance/migration. is that happening now?13:10
dmsimardmisc: yeah arrfab is the one who recommended to me13:10
dmsimardadarazs: yes13:10
adarazsweshay: ^ so that's the reason the logserver is down :)13:10
dmsimardadarazs: I sent a (late) reminder to rdo-list13:10
dmsimardadarazs: I pinged both you and weshay about it at the last meeting13:10
adarazsdmsimard: yeah, I was having a hazy memory of it :)13:11
weshaydmsimard, yes.. agree. thanks13:11
dmsimardadarazs, weshay: sorry :(13:11
weshaydmsimard, ur fine13:12
*** bhavik has joined #rdo13:13
*** trown|brb is now known as trown13:14
*** dneary has quit IRC13:15
*** dneary has joined #rdo13:17
number80interesting topic, what to do if we need to add a patch in stable branch13:17
*** dgurtner has joined #rdo13:17
number80since same spec is used in DLRN and stable packages *and* patch got merged but not released13:17
* number80 thinks about using %dlrn macro and %autosetup13:18
*** pcaruana has quit IRC13:18
chandankumardmsimard, hello13:19
dmsimardchandankumar: hey13:19
dmsimardnumber80: I'm not sure what you mean13:19
*** gfidente has joined #rdo13:20
chandankumardmellado, regarding this , i was saying to send a patch upstream <>, to include readme and license file to get included in the source tarball.13:20 bug 1385105 in Package Review "Review Request: python-marathon - Python client library/interface to the Mesos Marathon REST API" [Unspecified,Assigned] - Assigned to chkumar24613:20
dmsimardnumber80: I think it's more about what do we use as a trigger to release to stable beyond dot releases13:20
chandankumarmight be adding a manifest file will work!13:20
*** flepied has quit IRC13:20
dmsimardchandankumar: yeah, they also exclude the docs and the tests from the release tarball -- I wanted to package those too but couldn't.13:21
dmsimardchandankumar: This is sort of time sensitive (missing runtime dependency in trunk) so I think we can start with this and change the spec if they release of 0.8.713:21
*** Son_Goku has joined #rdo13:21
*** dustins has joined #rdo13:22
dmsimardI see it as an iterative improvement opportunity rather than a blocker :)13:22
chandankumardmellado, yes!, sure, i will update the final review very soon!13:22
dmsimarddmellado: thanks for packaging marathon <313:22
jruzickadmsimard, sorry, I always forget about that part because I'd assume it's automated.13:22
jruzickadmsimard, minutes sent13:23
dmsimardjruzicka: it's okay I did some url crafting shinanigans to get what I wanted :p13:23
jruzickadmsimard, yup, so did I :-p13:23
jruzickacould be done by meetbot13:23
*** dneary has quit IRC13:25
dmsimardjruzicka: make an errbot plugin :p13:25
jruzickayup a bot to command meetbot13:25
*** Poornima has quit IRC13:26
jruzickamight be fun13:26
dmsimardjruzicka: did you know they were working on changing all the openstack bots to errbot? ;)13:26
dmsimardfrom supybot/meetbot13:26
*** flepied has joined #rdo13:26
dmelladochandankumar: dmsimard13:26
jruzickadmsimard, I've seen the proposal, so it was agreed on? :)13:26
dmelladowhat's up xD13:26
*** jprovazn has joined #rdo13:27
jruzickadmsimard, that is an upgrade indeed13:27
dmsimardjruzicka: ah, don't know, haven,t looked recently13:27
*** Poornima has joined #rdo13:27
dmsimarddmellado: it's okay, the typical dm typo13:27
dmelladodmsimard: but even you? xD13:27
jruzickajust dm* problems13:27
*** myoung|gone is now known as myoung13:28
number80dmsimard: we can't really start pushing packages in stable repositories from random commit13:28
*** imcleod has joined #rdo13:28
*** panda|afk is now known as panda13:28
number80let's say commit X fix a critical bug, but it is preceded by 20 other unrelated commits13:28
*** eharney has joined #rdo13:28
*** jeckersb_gone is now known as jeckersb13:29
dmsimardnumber80: right, that's what I'm syaing13:29
dmsimardnumber80: I don't think upstream does dot releases on stable releases that often anymore13:29
dmsimardnumber80: maybe we can discuss this in BCN13:30
dmsimardrelease/update cadence13:30
dmsimardwhile not infra-specific, I'll add it to rdo infra meetup topics if we have some extra time13:30
number80dmsimard: we can't release at random commits, but it doesn't handle critical bugfix13:30
dmsimardoh, etherpad is down >_<13:30
number80but yeah, next week :)13:31
jruzickamany things are down.13:32
*** ohochman has joined #rdo13:36
*** dneary has joined #rdo13:37
*** ohochman has left #rdo13:37
*** Poornima has quit IRC13:40
*** pilasguru has joined #rdo13:41
*** egallen has quit IRC13:41
*** trown is now known as trown|brb13:41
*** egallen has joined #rdo13:41
*** dgurtner has quit IRC13:41
EmilienMapevec: ok I have a PoC13:42
*** egallen has quit IRC13:42
*** egallen has joined #rdo13:42
*** egallen has quit IRC13:42
EmilienMapevec: it's a first iteration bear with me :)13:43
*** egallen has joined #rdo13:43
*** egallen has quit IRC13:43
*** egallen has joined #rdo13:44
*** egallen has quit IRC13:44
*** egallen has joined #rdo13:44
*** egallen has quit IRC13:45
*** ealcaniz has quit IRC13:45
chandankumarrbowen, ping pm?13:45
rbowenSure. :-)13:45
rbowenAlso, good morning #rdo!13:46
Humbedoohgood moaning, have you come on your bøhcycle?13:46
*** pblaho has joined #rdo13:47
EmilienMapevec: I propose we run this script every day in RDO and it would also run quickstart job13:50
*** shaunm has joined #rdo13:52
*** shaunm has quit IRC13:52
*** zaneb has quit IRC13:54
*** mengxd has joined #rdo13:55
socialEmilienM: shouldn't we pin puppetlabs-ntp for newton?13:55
EmilienMsocial: it's done AFIK13:55
*** zaneb has joined #rdo13:56
socialEmilienM: ah and also in master13:56
socialEmilienM: because of this13:57
EmilienMsocial: ?13:58
*** sasha2 has quit IRC13:58
*** sasha2 has joined #rdo13:58
socialEmilienM: #033[1;31mError: Syntax error at 'Any'; expected ')' at /etc/puppet/modules/ntp/manifests/init.pp:2 on node overcloud-novacompute-0.localdomain#033[0m is from puppetlabs-ntp commit above 4.2.0 and it's error here
*** egallen has joined #rdo14:00
*** eaguilar_ has quit IRC14:00
*** eaguilar has joined #rdo14:01
EmilienMsocial: have you tried recheck?14:01
*** dtantsur|mtg is now known as dtantsur14:01
EmilienMI saw it working on stable/newton CI jobs14:01
*** jpena|lunch is now known as jpena14:03
*** amoralej|lunch is now known as amoralej14:03
*** hogepodge has quit IRC14:04
*** karimb has joined #rdo14:04
amoralejjpena, master promotion is blocked by
openstackLaunchpad bug 1633864 in puppet-openstack-integration "scenario002 fails after adding support to ec2api" [Undecided,New]14:07
amoralejany hint about where should it be fixed?14:07
amoraleji have doubts14:07
dmsimardamoralej: can try pinging the guy who added ec214:08
dmsimardthat's fairly recent14:08
amoralejyes, last week14:08
*** saneax is now known as saneax-_-|AFK14:10
*** trown|brb is now known as trown14:11
*** Goneri has quit IRC14:14
*** Goneri has joined #rdo14:15
*** shaunm has joined #rdo14:15
jpenaamoralej: based on your bug description, it's either something to be fixed in puppet-tempest or p-o-i14:15
*** ashw has joined #rdo14:16
dmsimardjpena: been discussing in #puppet-openstack14:17
*** lucasagomes is now known as lucas-hungry14:18
dmsimardtrown, weshay: I'm connecting the new cloud slaves to jenkins now, I'm going to tag them rdo-dev and run some jobs on them to see if I missed anything14:18
weshaydmsimard, thanks man14:18
trowndmsimard: awesome thanks14:18
dmsimardweshay, trown: these are 2 core/4gb machines, three of them. I'll start at 4 threads each and should be plenty.14:19
*** ealcaniz has joined #rdo14:19
dmsimardI'll slightly lower the amount of threads on the permanent slave because it's pretty loaded in general.14:19
*** ohamada_ has joined #rdo14:20
*** ohamada has quit IRC14:20
*** artom has quit IRC14:24
*** jkilpatr has quit IRC14:24
dmsimardadarazs: is that you? "0 2 * * * find ~/.ansible/tmp/ -maxdepth 1 -cmin +300 -exec rm -rf {} +;"14:25
*** coolsvap has joined #rdo14:25
dmsimardadarazs: cause it's probably /tmp/ that you're interested in14:26
dmsimard# ls -al /tmp/ |wc -l14:26
adarazsdmsimard: yes. ansible was leaking and couldn't find the solution :)14:26
adarazsdmsimard: when I did this hack it was in that directory.14:26
adarazsdmsimard: a lot has changed since I made this fix.14:26
*** mosulica has joined #rdo14:27
*** dhill_ has joined #rdo14:27
*** eaguilar_ has joined #rdo14:28
*** vimal has quit IRC14:28
*** eaguilar has quit IRC14:29
*** richm has joined #rdo14:31
*** aufi has quit IRC14:32
*** eharney has quit IRC14:33
adarazsdmsimard: you can try to comment it out/remove it and monitor the space on the node.14:38
*** abehl has quit IRC14:41
*** dgurtner has joined #rdo14:42
dmsimardtrown: what oooq job can I copy/tag rdo-dev for testing slaves ?14:42
dmsimardso far so good on
trowndmsimard: tagged and kicked
dmsimardjruzicka: were you the one that came up with the owl ascii ? I forget14:45
trowndmsimard: it was ccamacho14:45
EmilienMdmsimard: I guess you already know why puppet & tripleo CI hasn't got promotion since 4/5 days?14:45
dmsimardtrown: I was looking at and it looks like the OoO should be eyes+beak O o O or something like that14:46
dmsimardtrown: I'm sick, tired and no caffeine though so yeah14:46
jruzickadmsimard, I do like ascii art, but which owl? :D14:46
*** Son_Goku has quit IRC14:47
ccamachodmsimard hey!, yeahp long time ago :)14:47
dmsimardEmilienM: not particularly aware of anything specific right now, I remember a sahara test failure that was hard to troubleshoot without logs14:47
dmsimardEmilienM: and the log fix from weirdo was blocked due to master/trunk being broken.. I was going to force push them today but is down14:48
*** Amita has quit IRC14:48
EmilienMdmsimard: :(14:49
dmsimardneed to afk a bit, brb14:49
*** dprince has quit IRC14:49
*** dmsimard is now known as dmsimard|afk14:49
*** dneary has quit IRC14:51
*** vaneldik has joined #rdo14:53
*** artom has joined #rdo14:53
*** dneary has joined #rdo14:53
*** vimal has joined #rdo14:54
*** vimal has quit IRC14:55
dmsimard|afkEmilienM: there's also ec2api, see #puppet-openstack recent discussion14:55
* dmsimard|afk back to afk14:55
*** Son_Goku has joined #rdo14:56
*** artom has quit IRC14:58
*** lucas-hungry is now known as lucasagomes14:58
*** artom has joined #rdo15:00
*** Son_Goku has quit IRC15:00
*** artom_ has joined #rdo15:02
*** jkilpatr has joined #rdo15:02
*** pcaruana has joined #rdo15:03
*** marrusl has joined #rdo15:03
*** Son_Goku has joined #rdo15:03
*** ealcaniz has quit IRC15:04
*** artom has quit IRC15:05
*** mvk has quit IRC15:09
*** dgurtner has quit IRC15:11
*** limao_ has quit IRC15:12
*** dgurtner has joined #rdo15:13
*** dgurtner has quit IRC15:13
*** dgurtner has joined #rdo15:13
chandankumarjpena, amoralej
*** rdobot has joined #rdo15:15
*** ChanServ sets mode: +v rdobot15:15
rdobot[sensu] RESOLVED: - check-delorean-newton-current @ |#| No build failures detected:
*** alexismonville has quit IRC15:17
rdobot[sensu] NEW: - check-delorean-master-head-current @ |#| Build failure on centos7-master-head/current: sahara, python-saharaclient, oslo.log, oslo.versionedobjects, app-catalog-ui:
rdobot[sensu] NEW: - check-delorean-liberty-current @ |#| Build failure on centos7-liberty/current: app-catalog-ui:
rdobot[sensu] NEW: - check-delorean-mitaka-current @ |#| Build failure on centos7-mitaka/current: app-catalog-ui:
chandankumarnumber80, jpena amoralej apevec rbowen dmellado Good night, see you tomorrow :-)15:20
amoralejGood night chandan15:21
number80good night15:21
*** chandankumar has quit IRC15:21
*** dgurtner has quit IRC15:21
*** alexismonville has joined #rdo15:23
openstackgerritJavier Peña proposed openstack/packstack: Change default Identity API from v2.0 to v3
apevecsensu is back, that means rcip outage is over?15:24
apevecbut review.rdo still down15:24
*** rgogunskiy has quit IRC15:24
*** mosulica has quit IRC15:25
*** egallen has quit IRC15:25
EmilienMapevec: so it's not clear to me if we run any tests outside unit tests when bumping uc15:26
apevecthere's experimental job WIP15:26
*** rcernin has quit IRC15:26
apevecbut otherwise theory is: if it is good for upstream, it must be for us15:27
apevecso normally we should just merge it15:27
EmilienMapevec: mhh, ok15:27
EmilienMapevec: so if puppet wants to follow this model, we should not run jobs in RDO CI when bumping puppet modules versions15:28
EmilienMie: puppetlabs-mysql gets a new tag, our CI job will bump rdo.yml and that's it. No CI job. right?15:28 is back!15:29
*** egallen has joined #rdo15:29
apevecEmilienM, "this model" assumes there's equivalent of upper-constraints.txt for puppet modules, i.e. something which passed some CI somewhere15:29
*** alexismonville has quit IRC15:29
*** egallen has quit IRC15:30
apevecit is not just blindly bumping to the latest upstream release15:30
*** alexismonville has joined #rdo15:30
apevecjpena, ok, now I need to sync rdoinfo back to review.rdo15:30
EmilienMapevec: the problem is p-o-i has less modules than tripleo15:31
jpenaapevec: yep. For now, the rdoinfo update cron job is stopped at the DLRN server15:31
EmilienMapevec: we can't take p-o-i puppetfile as reference15:31
apevecahh, ok so we need to come up with some job somewhere15:33
apevecjpena, dmsimard|afk ^ experimental rdoinfo job is weirdo only?15:33
jpenaapevec: for now, it's just create updated packages with DLRN15:34
EmilienMapevec: i could investigate how to do it in tripleo CI15:34
apevecjpena, ah no child jobs using those packages?15:34
jpenanot yet15:34
EmilienMapevec: having the same mechanism that we have in poi15:34
apevecEmilienM, is there complete Puppetfile for tripleo ?15:34
EmilienMapevec: and use uc sync in rdo like we do for other libs15:35
apevecEmilienM, that would be good15:35
jpenawe need to get the packages built first (still having some small issues)15:35
EmilienMapevec: no we don't but I could do it.15:35
*** mengxd has quit IRC15:35
*** egallen has joined #rdo15:36
*** Son_Goku has quit IRC15:37
*** pilasguru1 has joined #rdo15:37
*** pilasguru has quit IRC15:37
*** pilasguru1 is now known as pilasguru15:37
apevecjpena, dang, review.rdo has already pushed his start to github mirror15:39
*** maeca has quit IRC15:39
*** jkilpatr has quit IRC15:40
apevecok, restored15:40
*** egallen has quit IRC15:41
apevecEmilienM, ^ there might have been failures from review.rdo restart until now, recheck if you see it15:41
EmilienMapevec: ack15:41
*** Son_Goku has joined #rdo15:42
apevecamoralej, jpena - has nofiles hotfix survived review.rdo reboot?15:42
amoralejlet me check...15:43
*** smeyer has quit IRC15:43
amoralejnot, it didn't apevec15:43
apevecfbo, ^ FYI15:44
amoraleji changed online again, but it requires change in systemd unit file15:44
apevecthat was right?15:44
amoralejyes, it is15:45
*** jistr is now known as jistr|biab15:46
*** choirboy has joined #rdo15:46
*** aufi has joined #rdo15:49
*** athomas has quit IRC15:51
*** hogepodge has joined #rdo15:53
*** eaguilar_ has quit IRC15:54
*** jkilpatr has joined #rdo15:54
*** akshai_ has quit IRC15:56
*** eaguilar has joined #rdo15:56
*** athomas has joined #rdo15:56
*** artom__ has joined #rdo15:56
*** egallen has joined #rdo15:57
jpenaapevec: I see rdoinfo is now ok, can I re-enable the cron jobs in the dlrn instance?15:58
apevecjpena, yes15:58
amoralejwe'll be inconsistent with app-catalog-ui FTBFS again15:58
*** iranzo has quit IRC15:58
apevecah yes,15:59
apeveclet's merge that fix15:59
*** mandre has quit IRC15:59
apevecI wanted actual maintainers to chime in15:59
amoralejyeah, i didn't +W it15:59
amoralejwaiting for them...15:59
*** egallen has quit IRC15:59
*** artom_ has quit IRC16:00
*** weshay is now known as weshay_brb16:00
apevecmerging, next time we will remove project from rdoinfo if maintainers are unresponsive, but need to document that procedure first16:00
* apevec puts it as topic for Wed meeting16:01
*** vaneldik has quit IRC16:02
*** egallen has joined #rdo16:06
apevecamoralej, it will need backport to newton-rdo16:06
*** egallen has quit IRC16:06
amoralejand pining a commit to pass cbs16:06
amoralejunless we prefer ?dlrn macro16:06
jschlueterEmilienM: myoung found a HA failure on centos using RDO newton/current-passed-ci16:06
jschlueterhow do we go about digging for what caused this error16:07
amoralejjschlueter, where are the logs?16:07
jschlueteramoralej: ^^16:07
*** akshai has joined #rdo16:08
jschluetercase is that overcloud deploy completes but heat stack-list returns nothing16:08
jschlueterhow can we figure out what failed?16:08
apevecamoralej, let's worry about that when they'll have next release, looks like current 1.0.0 is for >= mitaka
apevecoh wait, it won't pass cbs job16:09
jschlueteris the script that ran16:09
amoralejyeap apevec, it won't pass validate16:09
*** eharney has joined #rdo16:10
apevecamoralej, but something is not right, 1.0.0 cbs build is Apr 1st and is Oct 16 ?!16:10
*** rcernin has joined #rdo16:10
apevecis upstream overwriting release tags?16:10
amoralejoct 16, 201516:10
apevechmm, which tarball is that??
*** ealcaniz has joined #rdo16:11
apevecsomething went wrong here16:11
*** ealcaniz has quit IRC16:11
*** dtrainor has joined #rdo16:12
*** eaguilar has quit IRC16:12
*** abehl has joined #rdo16:13
*** zaneb has quit IRC16:13
*** dprince has joined #rdo16:14
*** mandre has joined #rdo16:15
*** akshai has quit IRC16:16
*** newmember has quit IRC16:17
*** jpich has quit IRC16:18
amoralejmyoung, mistral couldn't create a plan, problems with swift in
amoraleji'm digging on it16:20
*** zoliXXL is now known as zoli|gone16:20
myoungamoralej: thanks!  please leave a breadcrumb trail, while fish is tasty, I would love to learn here :)16:21
trownjschlueter: "Unable to create plan. A container with the name overcloud already exists."16:21
myoungamoralej: to fish16:21
*** zoli|gone is now known as zoli_gone-proxy16:21
jschluetertrown: where did you see that16:21
*** dtrainor has quit IRC16:21
jschlueterwe are wanting to learn to fish16:21
trownjschlueter: myoung, so there is already an overcloud that was deployed to that environment16:21
*** dtrainor has joined #rdo16:22
*** jistr|biab is now known as jistr16:22
trown6 lines from the bottom of
apevecflepied, need your script on - last job #77 failed
flepiedapevec: ok running it16:22
*** egallen has joined #rdo16:22
trownin general, if that deploy command returns that quickly with that little output something failed with pre-deploy validation16:22
jschluetertrown: thanks! found that16:23
amoralejmyoung: "Unable to create plan. A container with the name overcloud already exists."16:24
myoungtrown, amoralej: where would the logs for the pre-validate live, or is there a way to enable more verbose logging from the deploy command?16:24
trownmyoung: I think the log message is pretty clear no?16:25
*** suresh12 has joined #rdo16:25
amoraleji think problem seems to be in
apevecamoralej, trown is the newton issue you're looking at related to ?16:25
* jschlueter not following what is telling us that and how to follow the crumbs16:25
jschlueternot obvious to me a newbie to debugging this16:25
*** abehl has quit IRC16:25
*** numans has quit IRC16:26
d0ugalamoralej: Yeah, apetrich showed me a similar error to that.16:26
d0ugalamoralej: it looks like it is an issue when mistral speaks to swift16:26
amoralejapevec, seems diferent16:26
trownapevec: nope... I think that might be some transient issue, saw a different job fail that way and then succeed on the following run16:26
amoralejyes, i'm trying to find out some more info from the swift side16:26
trownapevec: definitely worth investigating, but I think it is likely a false negative16:27
myoungthe RHEL version of this job doens't have the container message, it just fails with no messaging...looking at the executor log for that one as well16:27
*** rbowen has quit IRC16:27
*** rbowen has joined #rdo16:27
*** ChanServ sets mode: +o rbowen16:27
d0ugaltrown: I think we probably need to add some retrying to that code.16:27
apevec just has /home/stack/overcloudrc: No such file or directory16:28
*** jtomasek has quit IRC16:28
*** akshai has joined #rdo16:28
amoraleji saw that overcloudrc non existing once16:28
*** dprince has quit IRC16:28
myoungapevec that message is because the deploy fails, but iin the script we check for stacks with CREATE_FAILED16:28
*** dmsimard|afk is now known as dmsimard16:28
apevecthere must be error in deploy log but I don't see it16:28
amoralejyes, right...16:29
*** dprince has joined #rdo16:29
vkmchey all, I'm trying to run tempest on a tripleo deployment... working on the tempest.conf file right now... is using a cirros image for this purpose good enough?16:29
myoungso the script is assuming the overcloudrc file is there but it's not (no overcloud actually deployed)16:29
*** zaneb has joined #rdo16:29
*** jbrooks has joined #rdo16:29
apevecah there's interleaved nable to establish connection to in deploy log16:29
apeveclike this: 2016-10-17 14:19:55Z [overcloud.Compute.0.UpdateConfig]: CREATE_IN_PUnable to establish connection to
apevecROGRESS  state changed16:30
apevecwhy is that not producing error message?16:30
amoralejmyoung, it's strange we don't have more messages in
trownya that is at least a tripleo bug that it does not 1) exit non-zero and 2) end up in failed deploy16:30
vkmcEmilienM, ^16:31
*** suresh12 has quit IRC16:32
vkmcall right, I'll assume yes16:33
apevectrown, this was filed by Wes:
openstackLaunchpad bug 1631177 in tripleo-quickstart "overcloud deployment fails, but oooq does not detect the failure" [Undecided,New]16:33
apevectrown, should that be moved somewhere else?16:33
*** paramite has quit IRC16:34
*** READ10 has joined #rdo16:34
trownI dont think it can be a CI tools fault to not detect a process exiting with 0 on error16:34
*** abregman has joined #rdo16:34
apevecI'd agree16:34
trownit is a tripleoclient bug16:34
apevecmove it! :)16:35
dmsimardtrown: looks like the weirdo and oooq job passed16:35
trownroot of that issue is likely related to this in heat-engine.log 2016-10-17 14:19:46.934 20482 ERROR oslo_db.sqlalchemy.exc_filters DataError: (1406, u"Data too long for column 'resource_properties' at row 1")16:35
*** racedo_ has joined #rdo16:35
dmsimardtrown: no errors in log collection or cicoclient, lgtm16:36
dmsimardtrown: enabling them now16:36
*** racedo has quit IRC16:36
trowndmsimard: cool, how do we keep promote from getting scheduled there? there is another label?16:36
apevectrown, hmm, what is it inserting there?16:37
apevecand why does it vary16:37
trownit looks like hiera16:38
trownseeing if I can reproduce it locally to better poke at what is happening there16:39
dmsimardtrown: I think promotion jobs are laballed "rdo-promote" or something16:39
*** jkilpatr has quit IRC16:39
dmsimardtrown: the cloud slaves only have the label "rdo"16:39
*** ihrachys has quit IRC16:40
dmsimardthat said, the weirdo generic jobs can run anywhere (stable/cbs jobs)16:40
apevectrown, ok, that smells like blocker16:40
*** toanju has quit IRC16:42
*** jlibosva has quit IRC16:42
trowngetting some food while that reproducer sets up16:43
*** trown is now known as trown|lunch16:43
*** abregman is now known as abregman|afk16:43
*** maeca has joined #rdo16:44
*** derekh has quit IRC16:44
apevectrown|lunch, ack - added it as 4. in
*** weshay_brb is now known as weshay16:47
jpenaapevec: re. replicating the puppet-* modules to, I've been talking to fbo and we need to pre-create the repos in the organization. For this, we need to be members of the group16:48
*** Goneri has quit IRC16:49
*** aufi has quit IRC16:50
*** myoung is now known as myoung|lunch16:50
*** pkovar has quit IRC16:52
*** eaguilar has joined #rdo16:52
*** jkilpatr has joined #rdo16:53
*** jeckersb is now known as jeckersb_gone16:53
*** sreichar has quit IRC16:53
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: app-catalog-ui:
openstackgerritJavier Peña proposed openstack/packstack: Change default Identity API from v2.0 to v3
*** Son_Goku has quit IRC16:56
apevecjpena, isn't there a script to create repos in github ?16:57
*** jaosorior has quit IRC16:57
jpenaapevec: yes, but we need an oauth key to run it. And to be able to create repos in an organization, you need to be a member16:57
apevecok, adding you16:57
*** amoralej is now known as amoralej|off16:57
amoralej|offi need to leave now, doctor appointment16:58
apevecjpena, invite sent16:58
*** rpioso has joined #rdo16:59
*** egallen_ has joined #rdo16:59
*** egallen has quit IRC16:59
*** egallen_ is now known as egallen16:59
*** flepied has quit IRC16:59
rdobot[sensu] NEW: - check-delorean-master-head-consistent @ |#| Could not successfully retrieve repository info16:59
jpenaapevec: ok, repo creation via api works. Preparing the mass-creation17:00
*** k4n0 has quit IRC17:00
apevecsome rdoinfo jobs failed with Could not resolve host: review.rdoproject.org17:00
*** kgiusti has quit IRC17:00
apevecwas that before nofiles fix?17:00
apevecdo we have only 1 node for check?
apevec2 queued 1 running17:02
*** beekneemech has quit IRC17:02
*** karimb has quit IRC17:02
*** jtomasek has joined #rdo17:04
iclebytewhat kind of storage are most people running with RDO ?17:04
*** jaosorior has joined #rdo17:06
*** jtomasek has quit IRC17:06
*** jtomasek has joined #rdo17:06
*** sreichar has joined #rdo17:07
*** bnemec has joined #rdo17:07
*** newmember has joined #rdo17:08
*** jlibosva has joined #rdo17:09
jpenaiclebyte, it usually depends on the installation size. Most of the time I see either local storage (small installations) and ceph (large ones)17:09
*** ohamada_ has quit IRC17:09
iclebytejpena, we are looking at a new installation (previously a vmware house) and interested in trying openstack. I would like to build a redundant network raid1 type storage solution to back it. was looking at gluster. is ceph more in line with what we want?17:12
*** tiswanso has quit IRC17:12
*** tiswanso has joined #rdo17:13
jpenaiclebyte, if you want a shared storage for the VM images and disks, ceph is usually what you'll need17:13
*** tosky has quit IRC17:15
rdobot[sensu] NEW: - check-delorean-master-head-current @ |#| Build failure on centos7-master-head/current: sahara, python-saharaclient, oslo.log, oslo.versionedobjects:
rdobot[sensu] NEW: - check-delorean-liberty-current @ |#| Build failure on centos7-liberty/current: app-catalog-ui:
rdobot[sensu] NEW: - check-delorean-mitaka-current @ |#| Build failure on centos7-mitaka/current: app-catalog-ui:
iclebytejpena, okay i will take a closer look. is there any particular configuration of ceph which aligns with what i'm trying to do?17:16
jpenaiclebyte: I'm not a ceph expert, but I guess there are others in this channel who can provide guidance17:16
*** JuanDRay has joined #rdo17:18
*** dustins has quit IRC17:19
*** pbandark has quit IRC17:19
openstackgerritJavier Peña proposed openstack/packstack: Change default Identity API from v2.0 to v3
*** k4n0 has joined #rdo17:21
*** jeckersb_gone is now known as jeckersb17:21
*** jpena is now known as jpena|off17:23
*** newmember has quit IRC17:24
*** suresh12 has joined #rdo17:25
*** paramite has joined #rdo17:26
*** suresh12 has quit IRC17:28
*** suresh12 has joined #rdo17:29
*** tiswanso has quit IRC17:31
*** trown|lunch is now known as trown17:31
*** tiswanso has joined #rdo17:31
*** alexismonville has quit IRC17:32
dmsimardthe guy that managed pypi was at hpe and they laid him off17:33
*** alexismonville has joined #rdo17:35
pabelangerthey laid off a lot of people17:37
*** gszasz has quit IRC17:37
*** JuanDRay has quit IRC17:37
dmsimardyeah just saw for pleia2 as well17:38
dmsimardsaw an article that mentioned >3k jobs17:38
weshaythings seem to be back17:38
dmsimardweshay: define things17:39
weshayin rdo infra17:39
weshaylogging, etherpads etc17:39
trownaccelerating next17:39
dmsimardweshay: oh, I take no credit, it was all rcip-dev's team work :)17:39
weshaykidding :)17:40
*** flepied has joined #rdo17:40
*** jaosorior has quit IRC17:41
*** jaosorior has joined #rdo17:42
*** dprince has quit IRC17:42
*** dprince has joined #rdo17:43
miscso between hpe and mirantis, there isn't more big player besides RH ?17:44
dmsimardRH is probably biggest, Mirantis second17:44
*** ayoung has joined #rdo17:45
dmsimardthen RAX, I guess17:45
dmsimardSUSE happens to be in top 5 but that should be scratched off and named ajaeger instead :p17:45
*** Goneri has joined #rdo17:46
dmsimardah, nevermind, it's in top 5 for ocata which is not finished yet. It's not top 10 in newton.17:46
*** degorenko is now known as degorenko|afk17:46
miscbut didn't Mirantis also decided to "fir^W reduce their work force" ?"17:47
dmsimardI heard rumors about that but never read news about it17:47
dmsimardIt's not a public company so I guess they don't necessarily have to disclose too much17:47
miscthere is 5% of independant working on openstac k?17:48
misclike people doing it for fun ?17:48
dmsimardmisc: independant = contributing as unaffiliated to a company17:48
trowncould be for fun, or students17:48
dmsimardI think the criteria is <30k$/yr to be considered independant17:48
*** Alex_Stef has joined #rdo17:49
trownapevec: kicked newton promote again, my reproducer did not reproduce the failure in CI17:49
*** imcleod has quit IRC17:50
*** jaosorior has quit IRC17:51
*** pnavarro has quit IRC17:51
*** paramite has quit IRC17:51
openstackgerritMerged openstack/packstack: Use neutron::keystone::authtoken for Neutron manifests
*** imcleod has joined #rdo18:00
*** apevec has quit IRC18:00
*** sambetts is now known as sambetts|afk18:01
*** Alex_Stef has quit IRC18:01
*** dtantsur is now known as dtantsur|afk18:02
rbowendmsimard: Thanks for the short maintenance window. Everything's happy now?18:02
dmsimardrbowen: looks okay to me18:02
*** akshai has quit IRC18:02
*** akshai has joined #rdo18:03
*** artom__ is now known as artom18:07
*** apevec has joined #rdo18:13
*** apevec has quit IRC18:13
*** apevec has joined #rdo18:13
*** suresh12 has quit IRC18:14
*** rhallisey has quit IRC18:16
*** shardy is now known as shardy_afk18:19
*** suresh12 has joined #rdo18:20
*** shaunm has quit IRC18:21
*** jcoufal has quit IRC18:23
*** cschwede|mtg is now known as cschwede18:23
*** dustins has joined #rdo18:28
*** Guest21077 has quit IRC18:29
*** dprince has quit IRC18:32
*** dprince has joined #rdo18:32
*** jmelvin has quit IRC18:40
*** jmelvin has joined #rdo18:41
*** jkilpatr has quit IRC18:43
*** nmagnezi has joined #rdo18:44
*** ihrachys has joined #rdo18:48
*** rhallisey has joined #rdo18:49
*** jmelvin has quit IRC18:49
*** jmelvin has joined #rdo18:49
*** maeca has quit IRC18:50
*** alexismonville has quit IRC18:54
*** jkilpatr has joined #rdo18:56
*** cwolferh has joined #rdo18:59
*** morazi has quit IRC18:59
*** pblaho has quit IRC19:00
*** dpeacock has quit IRC19:01
*** dyasny_ is now known as dyasny19:03
*** akshai has quit IRC19:03
*** akshai has joined #rdo19:04
apevectrown, so we have a race?19:08
*** ihrachys has quit IRC19:09
apevecci.centos is good at exposing them I guess19:09
*** tiswanso has quit IRC19:11
*** jkilpatr has quit IRC19:11
*** alexismonville has joined #rdo19:11
trownapevec: I guess... it is pretty odd error for a race19:11
*** jkilpatr has joined #rdo19:12
*** tiswanso has joined #rdo19:12
*** suresh12 has quit IRC19:12
apevecmaybe just overloaded heat engine?19:13
trowndmsimard: not sure we needed to reduce the number of executors for the main jenkins slave... unless we explicitly make it only for promote19:13
trownya, but why would overloaded heat engine lead to a DB column overflow?19:13
trowndmsimard: mind if I bump back up to get our queue cleared?19:14
trowndmsimard: specifically interested in the newton promote jobs that are stuck waiting on an executor19:15
apevectrown, yeah, ignore me ... I guess comparing hiera which worked vs failed one might give more clues19:17
trownya, I am interested if it consistently reproduces in CI19:17
*** ihrachys has joined #rdo19:19
apevecanyone has decoder for json string with19:19
apevecoops, copy/paste killed vim19:21
*** Guest40693 is now known as rook19:25
*** mcornea has quit IRC19:26
*** artom has quit IRC19:28
trownapevec: oddly I have this on my reproducer env... but no error in heat-engine logs
*** ihrachys has quit IRC19:29
apevectrown, so resource_properties is sqlalchemy.PickleType19:29
apevecwe'll need to escalate to Heat folks I guess19:29
apeveczaneb, ^ have you ever seen Data too long on event.resource_properties ?19:30
*** egallen has quit IRC19:30
remix_tjamoralej: i did a new setup updating cluster packages to latest available in RHEL, but i have still redis not starting19:30
apeveczaneb, issue 4. in
apevecwhat could be making resource_properties grow ?19:30
zanebapevec: I have never *not* seen that. it occurs on every deployment19:31
apevecoh so red herring?19:32
*** egallen has joined #rdo19:32
apevectrown, ^ fyi :)19:32
apeveczaneb, so why is it logged then? :)19:32
*** abregman|afk is now known as abregman19:32
apevecthere must be some background story19:33
openstackgerritMerged openstack/packstack: Remove unix socket from redis configuration
dmsimardtrown: even the ssh prompt lags when there are too many jobs running.19:33
zanebapevec: oslo_db logs it before we have a chance to handle the exception19:33
trownya, odd that in CI it is getting logged as ERROR, but in my reproducer I do not get any ERROR19:33
trowndmsimard: ya but with 16 last week ssh prompt was fine19:33
zanebwe do plan to fix the log too btw:
openstackLaunchpad bug 1524013 in heat "Arbitrarily-large resource properties should be allowed in events" [Medium,In progress] - Assigned to Crag Wolfe (cwolfe)19:33
zanebbut a good fix is complicated so it's taking a while19:34
dmsimardtrown: was never fine to me at 16 and load was above 1219:34
*** ccamacho has quit IRC19:34
*** jcoufal has joined #rdo19:34
dmsimardCam make it higher but we have to find a solution to unpin jobs.19:34
*** myoung|lunch is now known as myoung19:34
*** rasca has quit IRC19:35
trowndmsimard: we could also just remove "rdo" label from the non-cloud slave so it is ONLY for promote19:35
*** bdeschenes has joined #rdo19:35
trownproblem is that non-promote jobs can get scheduled anywhere but there is no scheduler logic to make them prefer the cloud slaves19:36
*** egallen has quit IRC19:37
trownk, went with that solution for now then19:37
trownrdo-ci-slave01 only has rdo-promote label19:37
*** karimb has joined #rdo19:38
dmsimardI think 12 threads for non promote jobs is enough.. if it's not we'll get slapped on the wrists for using too many nodes anyway so let that be artificial rate limiting19:39
trownsure, I think we are fine if we schedule better19:39
*** nmagnezi has quit IRC19:43
*** lucasagomes is now known as lucas-afk19:43
trownapevec: so we are still left with the deploy losing connection to heat at the start of the deploy 2016-10-17 14:19:55Z [overcloud.Compute.0.UpdateConfig]: CREATE_IN_PUnable to establish connection to
trownROGRESS  state changed19:43
trownwhich at least makes more sense as possibly resource related19:43
*** egallen has joined #rdo19:44
trownload average does spike to >50 around then
apevecso it could be overloaded heat engine after all19:45
trown might help19:46
*** jkilpatr_ has joined #rdo19:47
*** maeca has joined #rdo19:47
*** krabador has joined #rdo19:47
apevectrown, is that on tripleo-rc3 list?19:48
apevecah it is19:48
trownapevec: ya, and that is the backport19:48
* apevec reads topic19:48
*** READ10 has quit IRC19:48
trownit already landed on master19:48
apevecthat explains we see it only in newton job19:48
*** milan is now known as milan-afk19:48
*** jkilpatr has quit IRC19:49
apevecci.centos lottery is really good at exposing such bugs19:50
*** ccamacho has joined #rdo19:51
apevecdmsimard, was review.rdo rebooted? Looks like Alfredo's nofile fix vanished again:
apeveche applied it earlier19:53
apevecor is it just flaky networking19:53
dmsimardapevec: rebooted beyond the migration you mean ?19:53
dmsimardLet me see19:54
*** jkilpatr_ has quit IRC19:54
trownapevec: ya it is like reverse performance testing19:54
dmsimardapevec: I don't know about networking or nofiles19:55
dmsimardbut there is a *LOT* of selinux avcs19:55
apevecyeah, it's weird, dlrn failed when installing rpms
dmsimardapevec: a short excerpt:
apevecand CI job on review failed with Cloud not resolve host
dmsimardapevec: what job is that from ? did that run on a rcip node or trystack ?19:57
*** jayg is now known as jayg|g0n319:57
apevecdmsimard, how do I tell?19:57
apevecBuilding remotely on bare-centos-7-rcip-dev-70949 (bare-centos-7) in workspace /home/jenkins/workspace/DLRN-rpmbuild19:57
dmsimardapevec: rcip :)19:58
*** brad[] has quit IRC20:00
*** Humbedooh has quit IRC20:00
*** Humbedooh has joined #rdo20:00
apevecok, I've rechecked it20:00
*** Humbedooh has quit IRC20:01
*** Humbedooh has joined #rdo20:01
*** paragan has quit IRC20:02
dmsimardserver seems healthy at first glance ..20:02
trownback in a bit, catching bus20:03
*** trown is now known as trown|afk20:03
apevecdmsimard, how does it work w/ all those avcs, is it permissive?20:04
*** paragan has joined #rdo20:04
dmsimardyeah, it's permissive20:04
dmsimardI sent an email about it still though20:04
apevecnot cool20:04
dmsimardapevec: how did amoralej bump the nofile ?20:05
dmsimardulimit -n returning 1024, don't see anything particular in sysctl20:05
dmsimarddoesn't seem to be anything overwriting defaults in /etc/security20:06
*** dsneddon is now known as dsneddon_20:07
openstackgerritAlfredo Moralejo proposed openstack/packstack: Remove unix socket from redis configuration
*** jkilpatr_ has joined #rdo20:08
dmsimardoh hey, trystack issue again20:09
*** pmyers has quit IRC20:10
EmilienMdmsimard: have you tried to trigger promotion jobs for puppet?20:10
*** pmyers has joined #rdo20:10
dmsimardEmilienM: I don't trigger anything, it's automatic20:11
openstackgerritMerged openstack/packstack: Specify auth_url in aodh::auth
*** paragan has quit IRC20:11
dmsimardapevec: trystack fixed.. the router gateway keeps getting knocked out20:12
EmilienMdmsimard: ok, I'll look at why its red then20:13
dmsimardthe rdogerrit bot is down, I'll restart it20:14
dmsimardEmilienM: ^ but you should have seen that I merged the log fix20:14
*** rdogerrit has joined #rdo20:14
EmilienMdmsimard: okI'll look20:14
apevecdmsimard, see
*** dgurtner has joined #rdo20:15
dmsimardEmilienM: it'll be effective for the next run ..20:15
apevecAlfredo used prlimit20:15
dmsimardapevec: ah, I see .. nothing in the error talks about max number of files though, it's a DNS resolution error20:15
dmsimardwhich is probably even weirder because review.r.o is able to resolve itself ..20:16
dmsimardfrom /etc/hosts20:16
apevecthat is weird indeed20:16
*** eaguilar_ has joined #rdo20:19
remix_tjsorry tdasilva, wrong nick20:19
apevecleifmadsen, that sympy dep pulled-in chain: sympy -> python-fastcache -> pycmd20:19
tdasilvaremix_tj: :)20:19
leifmadsenapevec: oh?20:19
leifmadsenthere were a bunch more deps he identified in the bugzilla too that seem to be EPEL only as well20:19
remix_tja question for who can reply: is there a map with timezones of all rdo? so i know where i can find someone20:20
leifmadsenso my real question is, how does that typically get dealt with in package building? Should I port a spec from EPEL and have it built in RDO as a dep for the package?20:20
rbowenI'm not aware of one. We could make one, but it'll take a while.20:20
remix_tj*where, wen20:20
*** eaguilar has quit IRC20:20
apevecleifmadsen, deps are rebuilt as-is from Fedora or EPEL distgit if possible20:20
dmsimardremix_tj: there is probably an internal Red Hat one but that doesn't include all of the non-redhat community members20:21
EmilienMdmsimard: ok20:21
remix_tj(it's late here, i start being tired)20:21
leifmadsenapevec: ok cool, that makes sense20:21
*** dsneddon has joined #rdo20:21
remix_tjbecause someone i start talking with someone in afternoon (here CET)20:21
*** brad[] has joined #rdo20:21
apevecleifmadsen,  what'20:21
apevecs the review bz again?20:21
*** rdas has quit IRC20:21
remix_tjand if i return home and i would like to continue chatting but the other user is away20:22
leifmadsen bug 1342987 in Package Review "Review Request: openstack-vitrage - OpenStack RCA (Root Cause Analysis) Engine" [Unspecified,Assigned] - Assigned to hguemar20:22
*** maeca has quit IRC20:22
apevecwhere do they get those funny project names20:22
leifmadsenwho knows :)20:23
remix_tjapevec: lol20:23
leifmadsenI type it wrong all the time20:23
leifmadsenapevec: yea I noticed that python-fastcache was also in EPEL (hadn't looked if it was in delorean repo or not)20:23
dmsimardapevec: looks like your recheck passed.. let me know if you see anything else, I'll look20:24
apevecdmsimard, ok, booking it under "network gremlins"20:25
*** pcaruana has quit IRC20:25
apevecleifmadsen, right so gmpy2 is next
apevecwth they need sympy ...20:26
leifmadsenapevec: oh did you just add python-fastcache as a dep instead of SymPy directly?20:26
apevecA Python library for symbolic mathematics20:26
apevecis that Mathematica clone?20:26
dmsimardapevec: network gremlins... That explains a lot of things.20:26
leifmadsenapevec: great question :)  I'm only here to help package heh20:26
apevecleifmadsen, I need to resolve dep tree and start from leaves20:27
apevecso one branch was sympy -> python-fastcache -> pycmd20:27
*** trown|afk is now known as trown20:28
leifmadsengotcha... yea I'm still so new to building RDO I don't know where everything lives and where deps are all pulled from20:28
leifmadsendone lots of rpm builds, but always just standard builds with mock on epel directly20:28
apevecleifmadsen, this is the same, modulo epel repo20:28
leifmadsenbasically just a cherry picked subset I guess?20:29
apevecleifmadsen, ok, next branches are gmpy2, matplotlib and mpmath20:29
apevecwhich might pull more20:29
leifmadsenrpm deps are a helluva drug20:29
*** iranzo has joined #rdo20:30
*** iranzo has quit IRC20:30
*** iranzo has joined #rdo20:30
apevecbut really, sympy for what, RCA tools ?!20:30
apevecis that research project funded by EU ?20:31
leifmadsenhaven't really tested the project, just had someone interested in getting it packaged, and took a quick stab at it with dlrn to learn some packaging commands :)20:31
*** jkilpatr_ has quit IRC20:34
*** artom has joined #rdo20:42
remix_tjtrown: i've rebuilt image with selinux disabled using selinux-permissive, redis wasn't still starting. Then i noticed something strange: rebuilding overcloud image, only qcow2 file was created and no vmlinuz and initramfs were created20:42
remix_tji rebuilt so the image without that option and then disabled selinux on the fly with setenforce 0. Now the service has started.20:43
*** toanju has joined #rdo20:43
remix_tji think that something is not ok with selinux-permissive element20:44
remix_tj(or with the way that i use to include it)20:45
trownremix_tj: hmm, quite possibly on RHEL, since that is not tested anywhere20:45
trownwe definitely use it to build the CentOS images, but official RHEL images have SELinux enforcing20:46
*** eharney has quit IRC20:46
*** jprovazn has quit IRC20:47
remix_tji check now the log of builiding without selinux-permissive and check which elements are included20:47
remix_tjthen test with selinux-permissive included through OVERCLOUD_FULL_DIB_EXTRA_ARGS=selinux-permissive and see what is chaging20:47
trownremix_tj: hard to think that only the selinux-permissive element would be the problem... it is pretty basic
trownthat script is the only thing it does20:49
remix_tjtrown: sure, but maybe is the way i'm using to include it in the image that is buggy20:50
trownya, maybe...20:50
remix_tji'm checking now the logs20:50
dhill_I can no longer introspect starting at newton...20:52
dhill_something broke20:53
dhill_it was working with internal RHOSP poodles and since the last poodle , RHOSP 10 can no longer introspect.   I can reproduce this with RDO newton and ocata too20:54
dhill_I'm wondering if it's because I don't wait long enough between the importation of the json file and the introspection20:55
*** trown is now known as trown|outtypewww20:55
dhill_oh it's not introspection that's failing it's the importation of the instackenv.json file20:55
*** jkilpatr has joined #rdo20:56
dhill_anyone has this issue?20:56
*** dgurtner has quit IRC20:56
*** iranzo has quit IRC20:57
apevecdmsimard, puppet-tripleo built after --package-name puppet-tripleo --recheck20:59
apevecwe have fresh centos7-newton/consistent/21:00
*** suresh12 has joined #rdo21:00
*** unclemarc has quit IRC21:00
*** suresh12 has quit IRC21:02
*** iranzo has joined #rdo21:02
*** iranzo has quit IRC21:02
*** iranzo has joined #rdo21:02
*** mvk has joined #rdo21:05
*** florianf has quit IRC21:06
*** karimb has quit IRC21:06
*** marst has joined #rdo21:08
marstHello! Can anyone confirm if RDO8 (liberty) has support for neutron RBAC? Release notes ( ) says it's supported, but some people argue that it's not.21:09
*** jcoufal has quit IRC21:09
*** akshai has quit IRC21:10
*** dgurtner has joined #rdo21:13
*** dgurtner has joined #rdo21:13
*** eaguilar_ has quit IRC21:17
*** dprince has quit IRC21:17
*** pilasguru has quit IRC21:19
apevecRDO doesn't use numbers for versions, for OSP you should ask Red Hat support21:22
apevecRDO is pure upstream in RPM, so it has what upstream has21:22
*** iranzo has quit IRC21:23
apevecif using RDO, use latest RDO Newton please21:23
*** eaguilar has joined #rdo21:23
*** athomas has quit IRC21:23
*** amuller has quit IRC21:23
apevecLiberty is going EOL next month, following upstream lifecycle
*** karimb has joined #rdo21:24
*** toanju has quit IRC21:24
*** newmember has joined #rdo21:25
*** x220_ has joined #rdo21:26
*** rhefner has quit IRC21:27
*** aortega has quit IRC21:27
openstackgerritAlan Pevec proposed openstack/packstack: swift formpost earlier in the pipeline
*** mlammon1 is now known as mlammon|afk21:27
openstackgerritAlfredo Moralejo proposed openstack/packstack: Specify auth_url in aodh::auth
*** eaguilar has quit IRC21:27
openstackgerritAlan Pevec proposed openstack/packstack: Use neutron::keystone::authtoken for Neutron manifests
*** rhefner has joined #rdo21:29
*** newmember has quit IRC21:29
*** eaguilar has joined #rdo21:31
*** iranzo has joined #rdo21:31
*** suresh12 has joined #rdo21:32
marstapevec: thanks a lot! Will do.21:33
remix_tjtrown|outtypewww: check this: seems that several build elements are missing when specifying that option. I'll make a deeper check tomorrow morning (if someone doesn't give me other "important" things to make)21:33
*** jlibosva has quit IRC21:37
*** suresh12 has quit IRC21:38
*** akshai has joined #rdo21:40
*** abregman is now known as abregman|afl21:42
*** abregman|afl is now known as abregman|afk21:42
*** iranzo has quit IRC21:43
*** nyechiel has joined #rdo21:44
*** shardy_afk has quit IRC21:44
*** karimb has quit IRC21:45
*** pilasguru has joined #rdo21:46
*** suresh12 has joined #rdo21:47
*** pilasguru has quit IRC21:51
openstackgerritMerged openstack/packstack: Remove unix socket from redis configuration
*** ohochman has joined #rdo21:56
rdogerritFrederic Lepied created openstack/glance-distgit: Add a dependency on os-client-config
*** x220_ has quit IRC21:57
*** dustins has quit IRC22:00
*** jmelvin has quit IRC22:01
weshayapevec, trown|outtypewww can we enable master-current-tripleo?22:03
*** bdeschenes has quit IRC22:04
*** marst has quit IRC22:05
*** rcernin has quit IRC22:06
*** weshay is now known as weshay_bbiab22:09
*** nyechiel has quit IRC22:10
*** elmiko is now known as _elmiko22:16
*** social has quit IRC22:17
*** ashw has quit IRC22:18
*** dneary has quit IRC22:23
*** marst has joined #rdo22:24
*** marst has quit IRC22:38
*** aortega has joined #rdo22:42
*** kaminohana has joined #rdo22:43
*** panda is now known as panda|Zz22:43
rdogerritFrederic Lepied created DLRN: Allow to fetch an upstream review during a build on the gate pipeline
rdogerritFrederic Lepied proposed openstack/glance-distgit: Add a dependency on os-client-config
*** marst has joined #rdo22:46
*** bdeschenes has joined #rdo22:46
*** pilasguru has joined #rdo22:47
*** marst has quit IRC22:53
*** Humbedooh has quit IRC22:54
*** gildub has joined #rdo22:55
*** Humbedooh has joined #rdo22:59
*** Humbedooh has quit IRC23:00
*** Humbedooh has joined #rdo23:00
*** eaguilar has quit IRC23:02
*** dsneddo__ has joined #rdo23:02
*** dsneddon has quit IRC23:04
*** ushkalim has quit IRC23:04
*** ushkalim has joined #rdo23:05
*** egallen has quit IRC23:10
*** kgiusti has joined #rdo23:13
*** kgiusti has left #rdo23:14
*** kgiusti has quit IRC23:14
*** pilasguru has quit IRC23:18
*** marrusl has quit IRC23:25
*** dgurtner has quit IRC23:32
*** sreichar has quit IRC23:34
*** chlong has joined #rdo23:40
*** ashw has joined #rdo23:52
*** Goneri has quit IRC23:53
*** tiswanso has quit IRC23:53
*** Pavo has quit IRC23:57

Generated by 2.14.0 by Marius Gedminas - find it at!