Thursday, 2017-08-31

pabelangerdmsimard: is somebody doing ARM images for RDO and docker?00:06
rdogerritAlan Pevec proposed rdo-infra/centos-release-openstack pike-rdo: Update the spec for centos-release-openstack-pike
pabelangeraarch64 and armhf for example00:06
rdogerritAlan Pevec proposed rdo-infra/centos-release-openstack pike-rdo: Update the spec for centos-release-openstack-pike
rdogerritrdo-trunk created rdoinfo master: Promote CBS tags update for newton-testing
rdogerritrdo-trunk created rdoinfo master: Promote CBS tags update for pike-testing
rdogerritrdo-trunk proposed rdoinfo master: Promote CBS tags update for ocata-testing
dmsimardpabelanger: not a good person to ask. number80 or tonyb might know. We don't have any of these right now, that's for sure but it might be planned. Pike is officially the first release with hopefully full ppc64le and aarch64 support.00:08
pabelangerdmsimard: cool00:08
* tonyb reads scrool back00:08
pabelangerjust at docker meetup here in town and somebody was asking for docker images on arm00:08
tonybpabelanger: I don't think so but I'd like to think that ppc64le and aarch64 get rolled into whatever workflow is decided upon for the other trippleo containers00:09
pabelangertonyb: okay, cool.00:09
pabelangerfigured I'd ask00:10
tonybpabelanger: I'm *sure* anyone dowing a product would have them interanlly00:10
pabelangerYa, persons was just asking is anybody was doing docker and aarch64, wasn't sure if RDO was or not00:11
*** thegreenhundred has joined #rdo00:14
tonybpabelanger: fair enough.00:14
apevecdmsimard, I've updated to enable testing repo by default, wdyt00:16
*** sasha2 has quit IRC00:16
apevecjust to have something to run after pushing to testing00:16
dmsimardapevec: on my phone right now00:17
apevecok, review when you can00:17
dmsimardBut I vaguely remember us doing something similar in ocata perhaps00:17
apevecI thought new rdoinfo jobs would save us of that00:18
apevecbut alas00:18
number80pabelanger: ARM docker images for RDO, not that I heard of. But ping radez, he may have some info on that.00:20
*** sasha2 has joined #rdo00:20
radeznope, I've only got qcow images, no docker images00:21
number80radez: ok, then nobody is working on docker images for aarch64 that I heard of :)00:22
*** Son_Goku has joined #rdo00:31
pabelangeractually, guy I was talking to in Ottawa, knows radez. So pointed him your way00:33
pabelangerfrom Dell00:33
radezI imagine I'll get to the containers eventually,00:33
radezbut I'm tryign to finish up the tripleo work00:33
pabelangerYa, thanks what I basically said00:33
pabelangersomebody likely working on something00:34
radezand it's been slow going functioning on centos 7.300:34
radezthe 7.4 alt suppoort should help out a ton00:34
*** sasha2 has quit IRC00:37
*** Son_Goku has quit IRC00:37
*** morazi has joined #rdo00:41
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova:
dmsimardapevec: +200:45
dmsimardapevec: but we still need to get it built and shipped to -extras before being able to properly test it00:45
dmsimardI think at least the weirdo jobs are looking good00:48
dmsimardbut we need the oooq coverage..00:48
*** MrAbaddon has quit IRC00:54
*** spotz has quit IRC01:00
rdogerritMerged rdo-infra/centos-release-openstack pike-rdo: Update the spec for centos-release-openstack-pike
number80And it failed again01:05
*** MrAbaddon has joined #rdo01:07
number80Automation needs to be driven by AI to be super duper efficient01:10
apevecso packstack all pass01:11
*** Son_Goku has joined #rdo01:11
apevecpoi 2 and 3 failed01:11
number80failure are unrelated for poi 201:13
apevecit's tempest failures01:14
apevec1 object storage and 7 ec2api01:16
number80yeah, but it shouldn;t happen01:16
apeveccannot tell until you root cause those failures01:16
apevecnumber80, ec2api errors must be incompatiblity between cross-tagged old ec2api release and pike clients:      nova_floating_ip = nova.floating_ips.create()01:21
apevec2017-08-31 00:42:06.186 30932 ERROR ec2api.api AttributeError: 'Client' object has no attribute 'floating_ips'01:21
apevecwe'll need snapshot, they still don't have release from stable/pike01:22
number80Latest commit that successfully passed CI is 5061c009cfc83d0efc51a9ad4d2f0560046e0d8001:24
number80Erm no, it's 74906017f7bb7e273600b57e9f6131afa7ac5f7d01:25
*** imcleod has joined #rdo01:25
apevecnumber80, ack, that's stable/pike head01:27
apevecbut again, how did they create stable/pike w/o rc1 tag... again self-managed?!01:28
apevecfor such critical project01:28
apevecmaybe poi should not consider ec2-api is worth including in scenario002 :(01:29
apevecis NOT worth*01:29
apevecanyway, snap it is01:29
apevecnumber80, can you do it?01:29
apevecI'm digging into other failures01:29
rdogerrithguemar created openstack/ec2-api-distgit rpm-master: Requirement sync for Pike
number80I was already preparing that ^01:33
*** dmacpher has joined #rdo01:33
*** jhershbe has joined #rdo01:35
*** spotz has joined #rdo01:35
apevecobject storage failure is:01:38
apevectesttools.matchers._impl.MismatchError: 'tempest-d--67488436' not in ['tempest-a--852248077', 'tempest-b--1660248112', 'tempest-c--1046594035', 'tempest-e--2080095498', 'tempest-f--81986837']01:38
apevecmight be timing issue, since other swift test cases passed01:38
number80CI passed after reqcheck, I can branch now01:39
rdogerrithguemar created config master: Create pike-rdo branch for openstack-ec2api
apevecand in poi 003 we have Puppet (err): Failed to apply catalog: undefined method `[]' for nil:NilClass01:42
apeveclooks like in puppet-murano
number80I haven't looked at poi301:42
apevecdmsimard, ^01:42
apevecany ideas?01:42
*** liujiong has joined #rdo01:42
dmsimardLemme get my laptop01:42
apevecpuppet-murano is new in Pike01:43
rdogerritMerged openstack/ec2-api-distgit rpm-master: Requirement sync for Pike
number80at least, puppet-murano packaging is sane and it's the latest in pike branch (and passed CI), so it does not come from here01:44
number80here = packaging01:45
apevecthen it might be change in poi01:45
* dmsimard has a laptop01:46
rdogerritMerged config master: Create pike-rdo branch for openstack-ec2api
apevec/usr/share/openstack-puppet/modules/murano/lib/puppet/provider/murano_application/murano.rb:40:in `block in instances'01:46
dmsimardyeah looking now01:47
apevecso maybe instances creation failed, but still bad error reporting01:47
*** jhershbe has quit IRC01:49
number80here we go01:50
rdogerrithguemar created openstack/ec2-api-distgit pike-rdo: Pike update (74906017f7bb7e273600b57e9f6131afa7ac5f7d)
number80I still need to figure out a proper version number not to conflict with ocata or future pike release01:53
number80Maybe I'll go with 4.1.0 and pre-release style NVR01:54
dmsimardapevec, number80: unrelated but wtf ?
dmsimardlooks like the horizon cache thing on restart that's going crazy01:54
apevecmight be some xstatic dep missing?01:54
dmsimarddunno, worth looking into. Adding to todos01:55
number80we have very pyscss =>
apevecwhat's in u-c ?01:55
dmsimardI lost the etherpad, where is it ?01:55
number80same version01:56
rdogerrithguemar proposed openstack/ec2-api-distgit pike-rdo: Pike update (74906017f7bb7e273600b57e9f6131afa7ac5f7d)
number80Similar ticket for mitaka => bug 1332457 in openstack-horizon "Unable to start horizon (mitaka)" [High,Closed: notabug] - Assigned to iserrano02:00
*** jmlowe has quit IRC02:00
* number80 notes pinging mrunge during CET morning to see if he has any hints02:01
*** jmlowe has joined #rdo02:01
dmsimardgot nothing so far on the murano thing, still looking02:01
number80 <= found the fix for the mitaka issue02:02
number80Not worthy to be called a fix02:02
* number80 spit on soil when we increase timeouts to solve issues02:03
* apevec gives up for tonight02:05
apevecsee you in few h02:05
*** apevec has quit IRC02:05
* number80 finishes current task02:07
number80otherwise I won't be able to sleep anyway02:07
rdogerrithguemar proposed openstack/ec2-api-distgit pike-rdo: Pike update (74906017f7bb7e273600b57e9f6131afa7ac5f7d)
*** sasha2 has joined #rdo02:18
rdogerritMerged openstack/ec2-api-distgit pike-rdo: Pike update (74906017f7bb7e273600b57e9f6131afa7ac5f7d)
rdogerrithguemar proposed rdoinfo master: Push all pike-candidate to pike-testing
* number80 off to sleep02:37
*** ykarel has joined #rdo02:44
*** imcleod has quit IRC02:47
*** MrAbaddon has quit IRC02:47
*** gtrxcb has joined #rdo02:54
*** cliles has quit IRC02:54
*** dmacpher has quit IRC03:00
*** MrAbaddon has joined #rdo03:02
*** scorcoran has joined #rdo03:04
*** ykarel has quit IRC03:13
*** jmlowe has quit IRC03:13
*** jmlowe has joined #rdo03:14
*** dmacpher has joined #rdo03:17
*** udesale has joined #rdo03:37
*** links has joined #rdo03:39
*** gkadam_ has joined #rdo03:40
*** scorcoran has quit IRC03:40
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova:
*** dmacpher has quit IRC04:01
*** dmacpher has joined #rdo04:02
*** ykarel has joined #rdo04:07
*** janki has joined #rdo04:10
*** shreshtha has joined #rdo04:15
*** aludwar has quit IRC04:21
*** vijay3093 has joined #rdo04:21
*** vijay3093 has quit IRC04:22
*** anilvenkata|AFK is now known as anilvenkata04:23
*** jhershbe has joined #rdo04:23
ykarelnumber80, hi04:23
ykareli saw jobs are failing for 916104:23
ykarelpackstack and poi04:24
ykarelError is in launching VMs for jobs: Server Error for url: Gateway Time-out04:24
ykarel3 poi and 3 packstack jobs failed with same reason04:26
ykarelIs someone looking into it04:26
ykarelin my tenant i am able to create VM from volume, so may be the issue resolved, recheck might help04:32
*** masonjoyers__ has quit IRC04:39
*** masonjoyers__ has joined #rdo04:39
*** jlabarre has quit IRC04:41
*** sasha2 has quit IRC04:44
*** anshul has joined #rdo04:49
*** dpawar has joined #rdo04:51
ykarelnumber80, posted recheck to see if issue resolved04:57
ykarelinfo: 1.5 hours ago newton promotion jobs also failed with same reason05:02
*** rbartal has joined #rdo05:07
*** iranzo has joined #rdo05:15
*** ratailor has joined #rdo05:18
*** aditya_r has joined #rdo05:18
*** rbartal has quit IRC05:31
*** abregman has joined #rdo05:34
*** richm has quit IRC05:36
*** aditya_r has quit IRC05:40
*** aditya_r has joined #rdo05:40
*** mosulica has joined #rdo05:49
*** agurenko has joined #rdo05:50
*** pgadiya has joined #rdo05:52
*** sbonazzo has joined #rdo05:52
*** fzdarsky|afk has joined #rdo05:55
dpawarapevec: Not sure is this correct place to ask this question, does RedHat support not tripleo based (manual) deployment officially for Pike ? for OSP11 it seems like it is supported Any idea, where should i post this question if this is not appropriate palce?05:55
dpawarHi, Not sure is this correct place to ask this question, does RedHat support not tripleo based (manual) deployment officially for Pike ? for OSP11 it seems like it is supported Any idea, where should i post this question if this is not appropriate palce?05:56
*** sbonazzo has quit IRC06:01
dpawardmsimard: ^^^ any idea ?06:02
ykarelnumber80, this time 5 vms created, one failed with error: 'message': u'Build of instance 83e377b0-a35e-4794-b6f3-702e0c2b1d34 aborted: Volume resource quota exceeded06:08
ykareldmsimard, jpena|off apevec ^^ who can fix this06:08
*** lkuchlan has joined #rdo06:10
*** lkuchlan is now known as lkuchlan_wfh06:11
*** sbonazzo has joined #rdo06:13
*** aditya_r has quit IRC06:15
*** links has quit IRC06:15
*** aditya_r has joined #rdo06:15
*** emalikov has joined #rdo06:19
*** pcaruana has joined #rdo06:22
*** mpeterson has joined #rdo06:26
*** dmacpher has quit IRC06:27
rdogerritchkumar246 created openstack/watcher-distgit pike-rdo: Remove tempest plugin reference from watcher project
rdogerritchkumar246 created openstack/watcher-distgit pike-rdo: Removed reference of intree tempest plugin for pike
*** jtomasek has quit IRC06:36
*** jtomasek has joined #rdo06:36
*** yfried has joined #rdo06:37
rdogerritchkumar246 created rdoinfo master: Switch pike source for tempest to tag 17.0.0
*** jtomasek has quit IRC06:39
*** dpawar has quit IRC06:39
*** jtomasek has joined #rdo06:40
*** rcernin has joined #rdo06:40
*** nyechiel has joined #rdo06:42
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova:
*** sc has joined #rdo06:42
*** snarwade has joined #rdo06:49
*** oidgar has joined #rdo06:49
*** jprovazn has joined #rdo06:49
*** dpawlik has quit IRC06:53
*** tvignaud has joined #rdo06:54
*** danpawlik has joined #rdo06:57
*** apevec has joined #rdo06:57
*** apevec has quit IRC06:57
*** apevec has joined #rdo06:57
apevecykarel, for infra issues in rdocloud we need jpena|off07:00
*** tshefi has joined #rdo07:00
*** dpawar has joined #rdo07:00
apevecdpawar, re. OSP support: not right channel, and OSP12 based on Pike is not released yet, in general Red Hat will support issues with end-result cloud, regardless of deployment method but we recommend OSP Director i.e. productized TripleO.07:03
dpawarapevec: Thanks !07:05
*** dgurtner has joined #rdo07:07
*** dgurtner has joined #rdo07:07
*** fzdarsky|afk is now known as fzdarsky07:13
*** apetrich has quit IRC07:14
rdogerritchkumar246 proposed openstack/watcher-distgit pike-rdo: Remove tempest plugin reference from watcher project
*** apetrich has joined #rdo07:15
*** larizmen has joined #rdo07:17
*** iranzo has quit IRC07:20
remix_tjsorry hamzy i was AFK. Since our timezone are shifted, when you went out for lunch, i was going out to dinner plus rock festival :-D07:22
*** gtrxcb has quit IRC07:22
remix_tjhamzy: anyway i see resources.NovaCompute: Went to status ERROR due to \"Message: No valid host was found. There are not enough hosts available. So i suspect that previous stack cleanup hasn't been done very well.07:22
*** iranzo has joined #rdo07:23
*** iranzo has quit IRC07:23
*** iranzo has joined #rdo07:23
*** sbulage has joined #rdo07:23
*** rwmjones has joined #rdo07:23
*** racedo has joined #rdo07:28
*** ragiman has joined #rdo07:28
*** links has joined #rdo07:28
*** egallen has joined #rdo07:28
*** jlibosva has joined #rdo07:31
*** tesseract has joined #rdo07:32
*** hjensas has joined #rdo07:32
*** apetrich has quit IRC07:35
rdogerritYatin Karel created puppet/puppet-murano-distgit pike-rdo: Pike update (75837136f0c8a9c7b334d6a3d757425382058934)
*** abregman is now known as abregman|mtg07:36
ykarelapevec, poi scenario 003 failing in pike-testing for rdoinfo, hope this fixes it ^^07:37
*** rbartal has joined #rdo07:38
*** hjensas has quit IRC07:38
*** jpich has joined #rdo07:39
*** cylopez has joined #rdo07:41
*** apetrich has joined #rdo07:42
*** funzo_ has joined #rdo07:43
*** jpena|off is now known as jpena07:43
jpenagood morning07:43
jpenaykarel: are we having issues with RDO Cloud? Let me check07:44
ykareljpena, Ok07:44
ykareljpena, current issue: aborted: Volume resource quota exceede07:44
*** egonzalez has joined #rdo07:45
jpenawe seem to be ok in terms of volumes now07:45
jpenaoh wait07:45
jpenamaybe it was another tenant07:45
*** funzo has quit IRC07:46
chandankumarnumber80: jpena ykarel apevec \o/07:47
jpenaykarel: which job was having trouble?07:47
chandankumarjpena: number80
chandankumarto fix watcher project07:47
* chandankumar is dreaming whole night about tempest plugin packaging during sleep07:48
jpenachandankumar: well, the tempest plugin is still there in stable/pike, shouldn't we do it the other way around? I mean, only have the new watcher-tests-tempest plugin in queens and later07:48
*** dpawar has quit IRC07:49
jpenaykarel: ok, so it was one of the ci.centos jobs, and those are running on the rdo-ci tenant07:49
ykareljpena, ^^ one07:50
chandankumarjpena: that is also doable07:50
chandankumarjpena: then better to untag watcher tempest plugin07:50
*** aufi has joined #rdo07:50
jpenachandankumar: right, that was the plan. We untagged it from CBS yesterday, and my plan was to remove it from rdoinfo for dlrn today07:50
chandankumaror i can backport tempest plugin removal patch for pike07:50
jpenaykarel: now we should be ok, we have enough volume quota for at least 6 instances07:51
jpenahowever, I need to understand why some instances have a 80 GB volume and some others use 10007:51
chandankumarbecause if any tests break, backporting to watcher project for pike is going to be hard07:51
ykareljpena, Ok,07:52
ykareljpena there is one more failure poi scenario 003 in murano07:52
ykareli have pushed,07:52
apevecykarel, have you figured out what's going on with murano in poi 003 ?07:53
apevecoh I see07:53
jpenaykarel: so the murano issue requires a later puppet-murano commit?07:54
apevecykarel, do you have specific commit which fixes the issue?07:54
chandankumarjpena: number80 yesterday was a good learning for me for handling tempest plugin release07:54
ykarelapevec, to me it looks this:
*** hewbrocca_afk is now known as hewbrocca07:55
number80ykarel: I don't any relevant changed compared to 11.3.007:57
ykarelnumber80, is there07:57
*** florianf has joined #rdo07:58
jpenaa change in the gemspec shouldn't justify the error ;./07:59
ykareli could see the difference in passing pike promotion jobs and failing jobs, so i thought this could be the reason08:00
*** dpawar has joined #rdo08:00
*** egallen has quit IRC08:00
ykarelthere we have: puppet-murano-11.3.1-0.20170825180829.7583713.el7.centos.noarch08:00
apevecmust be some other change08:01
number80Well, I am myself skeptic but it could be some puppet wizardry08:01
number80yeah, maybe in murano or muranoclient08:01
apevecgem definition should not matter with rpm packaged puppet module?08:01
apevecat least I hope, otherwise that means we are pulling from git with puppet-* RPMs ?!08:02
*** jhershbe has quit IRC08:02
ykarelapevec, Ok08:04
*** egallen has joined #rdo08:04
apevecykarel, it'd be rather change in murano itself?08:04
apevecs murano commit  in passed-ci  ?08:05
ykarel^^ same with 4.0.008:07
jpenaapevec: the puppet-murano commit we have in passed-ci is the latest in stable/pike08:08
rdogerritJavier Peña created rdo-infra/ci-config master: Use the same volume size in all VM creation attempts
jpena^^ will fix the 80 vs 100 GB discrepancy08:13
jpenanow for murano08:13
apevecykarel, right, 94ba1de == tag 4.0.0 == stable/pike head08:13
apevecykarel, but what is the actual failure, anything in murano logs?08:14
apevecand who owns murano? let's reach out to them08:14
jpenaapevec: the error is at the end of
jpenabut it doesn't make a lot of sense to me08:14
apevec- mathieu.velten@cern.ch08:14
apevec- asteroid566@gmail.com08:14
apevecjpena, yeah, that's on puppet side08:15
jpenais there another error on murano?08:15
apevecnumber80, ^ know anyone of maintaners?08:15
ykareljpena, i couldn't see any08:15
apevecsince we don't have murano expertise, I would drop it from release08:16
apevecunless maintainers show up08:16
number80apevec: yeah drop it from GA, and I'll ping Igor08:16
apevecykarel, is murano unconditional included in poi ?08:16
jpenaI'm going to try a local VM install with this repo, so I can play around08:17
*** openstackgerrit has quit IRC08:17
rdogerritMerged rdo-infra/ci-config master: Use the same volume size in all VM creation attempts
*** mcornea has joined #rdo08:18
*** abregman|mtg is now known as abregman08:18
apevecykarel, jpena - so we need to override scenario003.pp - is there a way in weirdo to do that?08:18
*** jhershbe has joined #rdo08:20
ykarelapevec, don't think so, checking08:20
jpenaapevec: not that I'm aware of. We could propose a patch to, but that would require acceptance from the puppet people08:21
apevecyeah, and backport to stable/pike so unlikely08:21
apevechey, why don't we package poi ?08:22
apevecso it's part of promoted set08:22
apevecnow it's always from git08:22
apevecso poi changes could break us08:22
apevecwith poi rpm, we could have patches like this08:22
*** dpawar has quit IRC08:23
apevecbut also, having a place in weirdo to apply workaround would be good08:23
apevecotherwise, like in this case RDO cannot define what we want to ship08:24
*** gfidente has joined #rdo08:25
ykarelapevec, looks good08:29
*** lucas-afk is now known as lucasagomes08:30
*** jhershbe has quit IRC08:30
apevecykarel, what looks good, there's a way to patch poi already?08:31
apevecI was looking at ansible-role-weirdo* but didn't find it yet08:32
ykarelapevec, to have rpm for poi08:32
ykarelapevec, i too didn't find08:32
ykarelapevec, apevec, to have rpm for poi is good to apply patches.08:32
jehanehi, I'm having a quick (and probably stupid) question08:33
jehanewe have installed RDO Mitaka with ceph hammer08:33
jehanewe are planning to upgrade ceph to jewel but we can't upgrade OpenStack yet. Is this a problem ?08:34
jehaneis an openstack version locked on a ceph version ?08:35
apevecjehane, it's not tested in CI but iirc somebody reported running Mitaka w/ jewel08:35
jehaneapevec: ok08:35
jehanedo you remember if they were using ceph for nova and glance ?08:35
number80jehane: you could try hitting the mailing-list08:37
jehanenumber80: thanks, I will try08:37
number80what would be interesting is that people fill a page with their setup so we know what are the combination that have been known to work08:38
apevecjehane, found it, it was in one meeting minutes:
apevec"Jewel is known to work with Mitaka but we are not testing it anywhere"08:39
*** sfbender has quit IRC08:40
*** snarwade has quit IRC08:40
apeveciirc it was mnaser running jewel with mitaka08:40
apevecjehane, so yes, ask on rdo-list, people will chime in08:40
apevecykarel, jpena - we could to insert patched poi,08:42
apevecwhere is refspec coming from, and could we override it in the job ?08:43
apevecit is normally via gerrit env vars right?08:43
apevecwhat if just set them and point to a poi fork?08:43
*** tosky has joined #rdo08:44
*** karimb has joined #rdo08:45
*** yfried has quit IRC08:45
jpenaI'm trying to see where that comes from08:45
jpenaI can't find that option in
*** jhershbe has joined #rdo08:46
*** nyechiel has quit IRC08:48
*** liujiong has quit IRC08:53
*** nyechiel has joined #rdo08:56
jpenaapevec: for a very quick hack, we could try and change to point to a fork disabling murano... but I'm not so sure about that08:57
*** MrAbaddon has quit IRC08:58
*** vnogin has joined #rdo09:08
* chandankumar is reformatting my laptop, too much hanging problem09:08
chandankumar*me will be not available for few moments09:08
*** MrAbaddon has joined #rdo09:11
jpenaI have reproduced the p-o-i scenario003 issue locally, now investigating09:11
ykareljpena, Great09:14
*** abregman is now known as abregman|mtg09:15
jpenaapevec: have we ever tested the new python-urllib3 package?09:16
number80jpena: do you have any hints of failure?09:17
number80I remember that python-requests broke openstack client builds in Fedora but not affecting runtime09:17
*** milan has joined #rdo09:18
*** ushkalim has joined #rdo09:18
*** tosky has quit IRC09:18
jpenanumber80: the p-o-i run fails when it tries to run a package prefetch for murano. When I ran "murano package-list" manually, urllib3 spit out some warning messages before showing its output, which *might* confuse the puppet provider (I saw that in the past)09:18
*** snarwade has joined #rdo09:19
jpenaan earlier urllib3 version doesn't show this behavior09:19
number80let's try remove urllib3/requests from GA review09:19
number80apevec: ^09:19
jpena <- this is the message I get09:20
*** tosky has joined #rdo09:20
jpenaI'll check if it's possible to ignore the warnings in puppet-murano, I remember seeing something like this in the past09:21
*** apetrich has quit IRC09:21
apevecnumber80, no, requests/urllib3 are g-r !09:21
apevecwe need those09:21
apevecif they don't work, then min g-r is wrong09:21
apevecand we need to update to u-c09:22
number80ack, but I wouldn't be surprised that they behave differently in our environment and spew araning09:22
number80especially with certificates handling which is patched09:22
number80it can be a separate review that we won't merge just to run CI jobs09:23
number80(and compare results to acertain that is the root cause)09:23
apevecjpena, warning msg affecting puppet provider is a bug in puppet module09:23
apevecnumber80, ok, let's drop requests urllib3 from GA09:24
apevecwe'll push it separately post GA09:24
apevecafter puppet module(s) are fixed09:24
rdogerrithguemar proposed rdoinfo master: Push all pike-testing to pike-release
number80^ unrelated to the topic we just discussed09:24
* apevec checks upstream u-c09:25
apevecpike u-c: requests===2.18.2 urllib3===1.2209:26
apevecI'll get those in cbs and prepare pike-update09:26
apevecbut let's just get this pike-testing push out09:26
apevecwe already missed regular 9am centos sign&push :(09:27
*** vnogin has quit IRC09:32
*** vnogin has joined #rdo09:33
number80ok, so if we get new update, I can just submit reviews09:33
* number80 was trying to split it up09:33
rdogerrithguemar proposed rdoinfo master: Push all pike-candidate to pike-testing
rdogerrithguemar proposed rdoinfo master: Push all pike-testing to pike-release
apevecnumber80, yeah, not going to push this into GA !09:36
jpenanumber80: the latest patchset for 9161 is re-adding some previous errors (puppet-stdlib and watcher-tests-tempest)09:37
apevecone rdoinfo CI run takes 2-3h09:37
apevecnumber80, you have out of date local review, git-review -d 916109:38
apevecthen edit09:38
apevechm, but now you need to restore ps5 first09:39
apevecjpena, go ahead and push the edit, since you did ps509:39
rdogerritJavier Peña proposed rdoinfo master: Push all pike-candidate to pike-testing
*** zz9pzza has quit IRC09:40
apevecjpena, ec2-api was reverted too
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova:
rdogerritJavier Peña proposed rdoinfo master: Push all pike-candidate to pike-testing
apevecok ps9 is good09:43
*** dpawar has joined #rdo09:47
*** MrAbaddon has quit IRC09:49
*** tumble has quit IRC09:51
apevecnumber80, your comment about watcher-tests-tempest under TODO - that's resolved now, right?09:52
jpenaapevec: I'm running some tests, can confirm the murano issue is due to the urllib3 warning confusing puppet-murano09:53
jpenaI'll work on a patch09:53
apevecbut, this will be post GA09:53
apevecI also wonder why it doesn't show upstream?09:53
jpenathey don't have the updated urllib3 maybe?09:54
apevecwe'll see if u-c version is better09:54
apevecjpena, upstream should run with even higher version from u-c09:54
*** wolverineav has joined #rdo09:54
jpenaapevec: but not p-o-i, that runs with rpms09:55
number80yeah, it's solved now09:55
apevecok, then murano gate09:55
apevecah nm, that's not testing puppet...09:55
apevecso yes, puppet was never tested with updated requests/urllib309:56
apevecjpena, and I guess debian has older versions too?09:56
apeveccorrection, ubuntu09:56
jpenaapevec: even better, they don't run murano tests on ubuntu09:57
apevecoh well, so we don't care that much about murano but will end up fixing it09:58
*** sbonazzo is now known as sbonazzo|afk09:59
* apevec sees opportunity for a motivational blog targeting package mainters: "80% of life is showing up"10:00
apeveccorrection, 80% of _success_ :)
*** tumble has joined #rdo10:05
*** egallen has quit IRC10:05
*** abregman|mtg is now known as abregman10:09
jpenaapevec: should help with the puppet-murano issue, let's see10:15
*** apetrich has joined #rdo10:16
rdogerritJavier Peña proposed rdoinfo master: Push all pike-candidate to pike-testing
*** udesale has quit IRC10:27
*** karimb has quit IRC10:28
*** richm has joined #rdo10:29
* chandankumar is back10:33
chandankumarnumber80: apevec Anything blocker on tempest plugin side for RDO-PIke release?10:33
number80chandankumar: I'm trying to figure if there aren't any hidden issue with them like the one we had yesterday10:35
number80It may be faster if you check if they have no upgrade path issue10:36
chandankumarnumber80: sure10:36
*** karimb has joined #rdo10:39
*** jkilpatr has quit IRC10:41
*** apetrich has quit IRC10:42
*** MrAbaddon has joined #rdo10:46
*** aditya_r has quit IRC10:48
*** rhallisey has joined #rdo10:50
*** raissa has quit IRC10:51
*** vnogin has quit IRC10:52
*** raissa has joined #rdo10:52
*** jhershbe has quit IRC10:52
chandankumarnumber80: keystone barbican safe, i found no other issues with tempest plugins10:53
number80excellent so we're good10:53
* number80 now looking at CI "realtime" logs10:54
chandankumarnumber80: one more thing we untagged watcher tempest plugin ? just to confirm na.10:54
*** agurenko has quit IRC10:54
number80chandankumar: I haven't untagged it from -candidate, but it's safe10:54
chandankumar i am not seeing here10:55
chandankumarso we are safe10:55
chandankumarnumber80: let's do the final push then we are good to go10:55
apevecnumber80, let's untagg it in CBS, otherwise bot will re-propose it10:56
*** gszasz has joined #rdo10:56
*** agurenko has joined #rdo10:58
*** vnogin has joined #rdo10:59
*** vnogin has quit IRC10:59
*** Caterpillar has joined #rdo11:00
chandankumartime to grab some food11:00
*** jkilpatr has joined #rdo11:01
*** vnogin has joined #rdo11:01
*** gcerami has joined #rdo11:01
number80excellent idea11:02
apevecyes! food++11:02
*** sbonazzo|afk has quit IRC11:03
*** abregman is now known as abregman|mtg11:03
*** sbonazzo has joined #rdo11:04
apevecjpena, is running 44min already and still in setup phase...11:06
jpenaapevec: it shouldn't take long11:06
*** dpawar has quit IRC11:07
* apevec looks at timestamps in console11:07
apevecstarted :2111:07
*** pkovar has joined #rdo11:07
apevecdownloading rpms from cbs until :5711:08
apevecnow it's doing test install all of them in chroot11:09
*** Caterpillar has quit IRC11:09
*** cylopez has quit IRC11:10
*** jistr is now known as jistr|biab11:12
*** Caterpillar has joined #rdo11:12
ykareljpena, also check if there is enough volume quota for running weirdo jobs11:13
ykarelotherwise we have to wait longer11:14
jpenayep, the rdo-ci tenant is empty now11:14
ykareljpena, ack11:14
jpenavalidate-buildsys-tags is done!11:15
jpenanow for weirdo jobs11:15
*** Caterpillar has joined #rdo11:16
*** Caterpillar has quit IRC11:16
*** morazi has quit IRC11:17
*** Caterpillar has joined #rdo11:22
jpenadamn, p-o-i scenario002 has failed due to a network issue ->
*** jhershbe has joined #rdo11:25
*** eaguilar has quit IRC11:34
*** egallen has joined #rdo11:35
*** thrash|g0ne is now known as thrash11:41
*** amuller has joined #rdo11:41
chandankumarrbowen: hello11:43
chandankumarrbowen: we need some help in advertizing about from RDO side.11:43
chandankumarrbowen: once i get the content, i will pass it to you.11:44
apevecjpena, again socket.error: [Errno 104] Connection reset by peer in wait_for :(11:44
*** lucasagomes is now known as lucas-hungry11:44
jpenaapevec: let's increase the delay in ?11:46
jpenato 30 seconds, for example11:46
jpenathat would give us some breathing room11:46
*** shreshtha has quit IRC11:51
*** abregman|mtg is now known as abregman11:52
*** adarazs is now known as adarazs_brb11:53
*** apetrich has joined #rdo11:54
*** thegreenhundred has quit IRC11:55
jpenadmsimard: around?11:55
dmsimardjpena: half awake11:55
*** eaguilar has joined #rdo11:55
dmsimardSure, go for 3011:55
jpenadmsimard: quick question, then. Why are we using search_regex in ?11:55
number80jpena: that's a good idea11:56
jpenaI understand we're trying to make sure openssh is avaliable, but... Wouldn't it be enough just to check for the port?11:56
number80(the 30s thing)11:56
dmsimardBecause that's how Ansible documents it should be done11:56
dmsimardThte fact that the port is open is probably not enough11:57
sshnaidmhi, all, I wonder why hash of "consistent" repo on tripleo ( doesn't change during last 5 days, does somebody know?11:57
dmsimardI dunno.11:57
dmsimardsshnaidm: build failures and everyone is busy shipping pike11:57
dmsimardConsistent only moves forward when there are no build failures11:58
jpenaok, let's start with the 30 seconds delay, then I'll play with search_regex11:58
sshnaidmdmsimard, ok..11:58
rdogerritJavier Peña created rdo-infra/ci-config master: Increate delay when waiting for a vm to boot to 30
dmsimardsshnaidm: rdo/rdo11:59
dmsimardFeel free to contribute to fixing the builds :)11:59
sshnaidmdmsimard, and what to look there? CHECK-DELOREAN-MASTER-CURRENT ?12:00
sshnaidmdmsimard, I have barely idea how this works..12:00
dmsimardsshnaidm: yeah:
*** nijaba has quit IRC12:01
sshnaidmdmsimard, ok, I see nova fails. And where all logs, builds, etc..?12:01
dmsimardSee failures12:01
dmsimardI need to afk for a while, be back later12:02
* number80 thinks it'd be nice to have a log server with "realtime" visualisation12:02
*** vnogin has quit IRC12:02
*** nyechiel has quit IRC12:02
number80Jenkins UI sucks big time for that purpose12:02
dmsimardnumber80: ELK coming to review.rdo12:03
*** mvk has joined #rdo12:03
sshnaidmoook.. so we have "ImportError: No module named blockdiag.imagedraw", seems like we need dependency on python-blockdiag12:03
number80dmsimard: will it be public accessible?12:03
rdogerritMerged rdo-infra/ci-config master: Increate delay when waiting for a vm to boot to 30
number80sshnaidm: which package?12:03
*** jkilpatr has quit IRC12:03
*** morazi has joined #rdo12:04
sshnaidmnumber80, openstack-nova12:04
sshnaidmnumber80, ?12:04
number80I don't see where it's importing blockdiag and it was never detected by trunk CI12:05
* number80 grepped nova code and no reference to blockdiag12:05
sshnaidmnumber80, hmm.. I look here
* number80 will look12:05
sshnaidmnumber80, oh, maybe you're right..12:06
sshnaidmnumber80, it happens when /usr/bin/python2 build_sphinx12:06
number80likely sphinx or a sphinx extension issue12:07
*** karimb has quit IRC12:07
sshnaidmso it's actually sphinx problem, yeah12:07
rdogerritchkumar246 created openstack/tempest-distgit pike-rdo: Switch to tempest-17.0.0
sshnaidmnumber80, shouldn't we still add this as dep..? I don't know what is the process..12:08
number80it's not oslo.sphinx12:08
jpenap-o-i scenario003 ended up successfully, so the lower urllib3 version did the trick for murano12:08
chandankumarmyoung: tosky tbarron jschlueter and tempest-17.0.0 let's make manila happy12:08
dmsimardnumber80: sure12:08
number80sshnaidm: first investigating which package actually needs it, if it is sphinx or an extension, it might come back in another package build later12:09
*** jkilpatr has joined #rdo12:09
*** jbadiapa has quit IRC12:09
*** vnogin has joined #rdo12:10
*** vnogin has quit IRC12:11
*** adarazs_brb is now known as adarazs12:11
*** jkilpatr has quit IRC12:12
sshnaidmnumber80, it's sphinxcontrib-blockdiag extension for sphinx12:12
*** jkilpatr has joined #rdo12:12
sshnaidmnumber80, not sure I understood " it might come back in another package build later"12:12
*** jlabarre has joined #rdo12:12
*** vnogin has joined #rdo12:12
ykarelnumber80, sshnaidm introduced blockdiag12:12
*** scorcoran has joined #rdo12:13
*** vnogin has quit IRC12:14
sshnaidmykarel, hmm.. shouldn't it be added to extensions in beginning of file (line 36)? extensions = ['sphinx.ext.autodoc' ...12:14
number80ykarel: so, it's not pike12:15
* number80 pushes it to the background of the queue12:15
number80Pike GA first, then post-GA, and the rest later :)12:15
ykarelnumber80, yes it's not in pike12:15
ykarelnumber80, yeah12:16
jpenaI'm going to grab lunch. Someone please recheck once it finishes the current run12:16
*** jpena is now known as jpena|lunch12:16
ykarelnumber80, can we disable doc and man building for nova to move forward12:16
ykarelpromote consitent12:16
*** jkilpatr has quit IRC12:17
number80ykarel: no add the dependency in nova package12:17
ykarelnumber80, i think it's not there yet12:18
ykareli think we need to update deps.yml12:18
ykarelisn't it12:18
*** jayg|g0n3 is now known as jayg12:18
*** vnogin has joined #rdo12:19
ykarellooks like it's there:
number80yeah, but it's a question of hours12:19
number80it will be included in pike GA push12:19
*** vnogin has quit IRC12:19
number80I don't think that a lack of Queens promotions at this moment will be blocking anyone12:20
*** jistr|biab is now known as jistr12:20
dmsimardYeah, let's focus on getting Pike out12:20
dmsimardIf we miss today's opportunity to push and sign, we're delaying to monday and that would be awful12:20
dmsimardAnd also the first time in years that we've been delayed so much12:21
*** vnogin has joined #rdo12:21
dmsimardUsually RDO is out within hours of upstream release, we're already the day before12:21
dmsimardQueens is not important right now, sorry.12:21
number80damn job failed after 2 hours12:21
number80it's poi 212:22
*** vnogin has quit IRC12:22
dmsimardDropping kid at school and I'll try to help even though my allergies are killing me12:22
*** imcleod has joined #rdo12:22
*** vnogin has joined #rdo12:23
*** jprovazn has quit IRC12:24
*** pchavva has joined #rdo12:24
ykarelnumber80, i think jpena's fix will work this time12:24
*** jhershbe has quit IRC12:24
dmsimardBe back in ~30 mins or so12:24
*** vnogin has quit IRC12:25
*** jbadiapa has joined #rdo12:26
*** dprince has joined #rdo12:26
*** jprovazn has joined #rdo12:27
*** jkilpatr has joined #rdo12:30
*** noslzzp has joined #rdo12:30
*** imcleod has quit IRC12:31
*** vnogin has joined #rdo12:33
*** vnogin has quit IRC12:34
*** eharney has joined #rdo12:34
*** vnogin has joined #rdo12:35
*** rlandy has joined #rdo12:35
*** vnogin has quit IRC12:36
*** jaosorior has quit IRC12:37
*** pgadiya has quit IRC12:37
*** vnogin has joined #rdo12:38
*** thegreenhundred has joined #rdo12:38
*** jaosorior has joined #rdo12:38
*** vnogin has quit IRC12:39
*** vnogin has joined #rdo12:39
*** gkadam_ has quit IRC12:40
*** vnogin has quit IRC12:40
*** nyechiel has joined #rdo12:41
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova:
rbowenWhat's the status this morning. I gather it's not going as well as last time, hmm?12:42
*** vnogin has joined #rdo12:42
apevecrbowen, still various CI issues12:42
apevecjpena|lunch, have you increased timeout i.e. can I recheck 9161 ?12:43
ykarelapevec, yes12:43
ykarelhere ^^ increase12:43
*** jaosorior has quit IRC12:44
*** jaosorior has joined #rdo12:45
*** jaosorior has quit IRC12:45
*** jaosorior has joined #rdo12:46
apevecok, /me rechecked12:46
apevecdamn timeouts12:47
*** rhallisey has quit IRC12:47
*** noslzzp has quit IRC12:47
apevecimho such atomic timeouts should be ad-inf, then have external watchdog for the whole job12:48
*** noslzzp has joined #rdo12:49
rdogerritAttila Darazs created config master: tripleo: rename images-build to containers build
*** jmelvin has joined #rdo12:50
*** lucas-hungry is now known as lucasagomes12:51
*** raissa has quit IRC12:52
*** abregman is now known as abregman|mtg12:53
dmsimardapevec: hey, I mean, the cloud infrastructure we're using could be stable and reliable too12:54
dmsimardThat'd be great :)12:54
dmsimardAnyone want to give me a tl;dr of the situation ?12:54
dmsimardKB's deadline seems scary12:54
*** aditya_r has joined #rdo12:55
*** trown|outtypewww is now known as trown12:57
*** raissa has joined #rdo12:58
apevecdmsimard, what's the deadline?12:58
dmsimardapevec: see scrollback in #centos-devel12:59
apevecdmsimard, tl;dr just rechecked with increased timeout12:59
apevecdmsimard, don't see it12:59
apevecwhen was it?12:59
*** udesale has joined #rdo13:00
apevecI only see number80 and kbsingh discussing how little sleep they both had last night :)13:00
dmsimardthat was 2h30 ago13:01
apevecdmsimard, yeah, 4h is refering to the sleep length13:01
*** emalikov has quit IRC13:01
* dmsimard tired13:01
apevecbut realistical deadline is indeed in next few hours13:01
dmsimarddidn't get much sleep either, I'M going to die of allergies13:01
apevecso current recheck MUST succeed...13:01
*** sasha2 has joined #rdo13:02
dmsimardWhat is failing ? CI jobs ?13:02
*** sasha2 has quit IRC13:02
number80apevec: there will be no automated push for few days so we have to ping KB (that was the main topic of the sleepless folks discussion )13:02
apevecmurano was fixed by removing requests/urllib3 update13:02
*** sasha2 has joined #rdo13:02
apevecjpena has the fix in puppet-murano13:02
apevecwe'll push that after ga13:02
dmsimardI can hold nodes as required13:02
dmsimardso that we can troubleshoot13:02
dmsimardthat's what we did last night for a couple things13:03
dmsimardthe validate buildsys job isn't very good at telling us what's wrong, I added a todo13:03
apeveclast failure was wait_for issue which increased timeout should cover13:03
*** Goneri has joined #rdo13:03
apevecyeah, you need to look into each "child" weirdo job13:03
kbsinghapevec: my lack of sleep was an openshift problem :) but happy to keep an eye on things for you as well13:04
*** scorcoran is now known as scorcoran_biab13:05
dmsimardapevec: did oooq work ?13:05
dmsimarddid we ship a centos-release pkg ?13:05
*** imcleod has joined #rdo13:05
apevecI've tagged it for extras13:06
*** jprovazn has quit IRC13:06
apeveclemme check if it got pushed, not sure if that's automated13:06
*** jprovazn has joined #rdo13:06
*** lkuchlan_wfh has quit IRC13:06
apevecit's -0 w/ testing enabled, for now13:06
*** rhallisey has joined #rdo13:08
number80it is automated13:08
* number80 thinks it's checked every 4h or so13:09
dmsimardnumber80: we got told yesterday extras was managed by hughes13:09
number80Yep, Johnny13:09
dmsimardso not sure if he has to do something13:09
number80as long as it's tagged, it's ok13:10
apevecit is tagged
*** imcleod has quit IRC13:11
mnaserapevec we were indeed running jewel with mitaka13:11
apevecnumber80, but not in;O=D13:12
mnaserwe had to manually add the repos because centos-release-openstack-mitaka pulls in centos-release-ceph-hammer and causes a conflict when trying to install centos-release-ceph-jewle13:12
apevecmnaser, thanks for confirming, so my brain cells still somewhat work :)13:12
dmsimardmnaser: oi, btw 10.2.9 is coming soon, sometime next week hopefully13:13
dmsimardmnaser: and someone is working on luminous13:13
mnaserapevec np :)13:14
mnaserdmsimard nice!  im excited for bluestore13:14
* number80 holding breath as validate-buildsys-tags is almost finished weirdo-validate-buildsys-tags is on starting block13:14
*** udesale has quit IRC13:15
dmsimardI'll lock all the weirdo nodes13:15
dmsimardso that they don't get released13:15
*** artom has quit IRC13:15
*** pradk has joined #rdo13:15
*** dustins has joined #rdo13:16
mnaserare tagged releases out?  i mean we're running the trunk ones which are the same bit for bit but just having clean release names is nice :p13:16
apevecmnaser, running CI on them13:17
apevecbuilds are done in cbs13:17
mnaseroh sweet :)13:17
apevecmnaser, watch for results13:18
mnaserthank you i'll add myself to that review to get free email updates :p13:18
dmsimardmnaser: some of us are sleep deprived but we'll get it out13:18
mnaserno rush... the packages are all out there already and you guys have technically been building pike everyday anyways :p13:19
mnaserwe wouldn't have been able to upgrade so early / quickly if wasn't a thing13:19
*** jpena|lunch is now known as jpena13:20
apevec"We shall CI our release, whatever the cost may be!"13:20
jpenaso the previous run only had a failure in p-o-i scenario002, we're in good shape13:20
* jpena crosses fingers13:20
*** jprovazn has quit IRC13:20
* apevec knocks wood13:20
dmsimardmnaser: ah I was wondering how you upgraded so quickly13:21
dmsimardmnaser: be warned that upgrades from trunk to stable aren't tested, however, you're on your own for that13:21
mnaserdmsimard -- grab the hash of the release, get it from trunk, install pike deps repo and repo for that hash, install, voila13:22
jpenamnaser: while we wait for the CI job to finish, how's your Software Factory installation going? :)13:22
dmsimardmnaser: for example, you might end up rolling back packages because you're getting things that landed in stable/pike but were not released in the tarballs13:22
mnaserdmsimard i see.  shouldn't be too much of an issue (we'll test it out)13:22
dmsimardmnaser: the "deps" repo contains -testing which is not populated with openstack builds during the cycle but it is fully populated before we tag -release13:23
mnaserjpena good!  we took the zuul route so took the jenkins out of it and it "just works" .. took a little while for everyone to get used to it but its all good13:23
dmsimardmnaser: i.e, -testing is not self-contained and meant to work with trunk repos until ~release13:23
dmsimardmnaser: but when release draws near, you can use -testing by itself13:23
apevecdmsimard, not until we push 916113:24
*** kgiusti has joined #rdo13:24
apevecit's still only deps atm in pike-testing13:24
*** ratailor has quit IRC13:24
dmsimardoh hey I just got this email13:24
mnaserdmsimard i'm not too worried.  as long as the service itself is built from the same hash that we're running, libraries won't be that much of an issue13:24
dmsimardCongratulations David, We're celebrating 2 Years of your service and dedication to Red Hat and our mission.  Thanks for contributing to our success and helping to make Red Hat the best place to make a difference together!13:24
dmsimardmnaser: oh, yeah, it's just a disclaimer kind of statement13:25
jpenacongrats dmsimard ;)13:25
dmsimardmnaser: fine print if you will :)13:25
*** emalikov has joined #rdo13:25
mnaserdmsimard i thought my SLA covered full support for any RDO issues (ha-ha-ha)13:25
*** gszasz has quit IRC13:29
dmsimardmnaser: who's SLA, yours to your customers? :P13:29
*** scorcoran_biab is now known as scorcoran13:30
*** scorcoran is now known as scorcoran_mtg13:30
sshnaidmhow do I submit changed into I try via  but it show me a lot of commits to send.. I'm confused13:31
rbowendmsimard: 2 years!? Awesome. Congratulations.13:31
jpenasshnaidm: just git review, it should work13:32
* number80 considering to rewrite some jobs in rust13:32
number80more realistically, it would benefit them to be rewritten in python13:33
number80some stuff could easily be made concurrent13:33
*** jcoufal has joined #rdo13:34
dmsimardsshnaidm: you clone it, do a commit and do a git review ? shouldn't be any different than any other gerrit project13:35
*** jhershbe has joined #rdo13:35
*** abregman|mtg is now known as abregman13:36
sshnaidmdmsimard, seems I didn't submit changes yet to new gerrit..13:39
*** MrAbaddon has quit IRC13:39
jpenaweirdo jobs starting13:39
remix_tjrbowen: a quick question about the contributor list in pike release announcement: is list of people that specifically has worked on packaging for RDO?13:40
rbowenIt *should* be anybody that did anything on RDO that touched git in some way. However, we did note that some people were missing.13:40
rbowenNotably myself and chandankumar.13:40
dmsimardjpena: thanks, locking the VMs13:41
rbowenremix_tj: Is someone else missing?13:41
*** fultonj has joined #rdo13:42
rdogerritMerged config master: tripleo: rename images-build to containers build
jpenaall 6 VMs have been created without issue \o/13:43
*** cylopez has joined #rdo13:43
apevecjpena, did I imagine or there was a blog from you or Alfredo about deps management in RDO ?13:44
apevecor is it in docs somewhere?13:44
*** imcleod has joined #rdo13:44
jpenaapevec: I guess Alfredo did it13:44
* jpena checks13:44
remix_tjrbowen: i don't know. I was only looking at the list. Every time i see a contributor list on something i use, i grep for my name or nick to check if i forgot that i made some contribution :-P13:45
*** cylopez1 has joined #rdo13:45
*** shreshtha has joined #rdo13:45
*** gszasz has joined #rdo13:45
*** cylopez has quit IRC13:47
*** danpawlik has quit IRC13:50
*** ihrachys has joined #rdo13:50
rdogerritSagi Shnaidman created rdo-infra/ci-config master: Check URL of promotion every hour
chandankumarremix_tj: you can grep with your username, if it is missing, feel free to add.13:51
jpenaapevec: I can't find the blog post, maybe Alfredo didn't send it13:51
remix_tji think i haven't done any contribution directly to RDO, but only to tripleo13:52
apevecjpena, yeah, not sure where it was13:53
chandankumardmsimard: it will refresh your mind. :-)13:53
*** mpeterson has quit IRC13:53
rdogerritSagi Shnaidman proposed rdo-infra/ci-config master: Check URL of promotion every hour
*** gszasz has quit IRC13:53
*** gszasz has joined #rdo13:55
*** josecastroleon has quit IRC13:57
*** jaosorior has quit IRC13:57
*** jaosorior has joined #rdo13:59
*** vnogin has quit IRC13:59
*** josecastroleon has joined #rdo13:59
apevecsshnaidm, number80 - I would leave (nova master ftbfs) to Nova maintaers to act on14:03
apevecotherwise, they've never an opportunity when we fix their stuff14:03
apevecthey all got gerrit notification14:04
apevec(opportunity to learn about deps.yml etc.)14:05
*** snarwade has quit IRC14:05
*** MrAbaddon has joined #rdo14:05
*** jobewan has joined #rdo14:05
*** scorcoran_mtg is now known as scorcoran14:07
*** scorcoran is now known as scorcoran_biab14:07
sshnaidmapevec, is there somebody I can ping them..?14:08
dmsimardapevec: lol have you ever seen compute team fix a nova ftbfs ?14:10
*** vnogin has joined #rdo14:10
jpenaapevec: regarding the nova ftbfs, I think it's sphinxcontrib.actdiag, not blockdiag14:10
jpenaDEBUG: Could not import extension sphinxcontrib.actdiag (exception: No module named actdiag)14:11
*** marrusl has quit IRC14:11
dmsimardooo pike failed on missing release package again14:13
dmsimardI guess it's not out yet14:13
number80apevec: well, speaking for myself, I'm not even looking to master14:14
* number80 looked only to be sure if it wasn't affecting Pike14:14
jpena2/6 succeeded, 4 still going14:15
*** scorcoran_biab is now known as scorcoran14:16
*** abregman is now known as abregman|mtg14:16
*** jhershbe has quit IRC14:17
mnaserjpena i see im not the only one that has a bunch of tabs open :)14:18
dmsimardtrown: can we manually get some oooq coverage going for pike release testing ?14:18
*** imcleod has quit IRC14:18
dmsimardtrown: we're struggling to get it to work because with our current implementation it's expecting centos-release-openstack-pike to exist, which does not (yet)14:19
trowndmsimard: what do you mean by manual?14:19
dmsimardtrown: anything that isn't in the gate for
*** imcleod has joined #rdo14:20
*** Amita has joined #rdo14:22
trowndmsimard: I need more to go on... I should try to figure out how to get the pike pre-release bits on my local machine and test? or I need to setup some quick job? not exactly sure what the ask is14:22
dmsimardapevec, jpena, number80: I guess we could compromise on at least tagging -testing without ooo(q) coverage to at least get the bits in -testing and making it easier to test ooo(q) out of band14:23
jpena+1 for me14:23
jpenaall packstack jobs went fine, now waiting for p-o-i14:24
trown+1 here too14:24
dmsimardWe can have reasonable expectations that if weirdo is fully passing, ooo(q) will pass -- pike has been generally green in the pipeline for a while14:24
number80dmsimard: that's already the case14:24
dmsimardbut it's still required before we tag for release14:24
number80Yeah, but GA means we still have to tag them in -release14:24
hamzyremix_tj, FYI there is definitely an issue with ironic booting the nodes with ppc64le and petitboot, so I am investigating how to do it properly...14:26
mnaserso while we're waiting for p-o-i... magnum in pike relies on some stuff that far down the dependency chain require python-backports-ssl_match_hostname 3.5.014:27
mnaseror else it raises exceptions in the log all day long14:27
mnaseris there a way we can get that into rdo (el7 seems to ship 3.4.0)14:27
number80mnaser: is it breaking everything?14:27
number80if it's only cluttering logs, it can be a 0day update14:27
mnasernumber80 the feature of retrieving stats doesn't work that's all14:28
mnaserbut that's not even fully integrated14:28
mnaserso its not critical14:28
number80ok, let's document it and I'll prepare that update14:28
jpenaanyway, 3.5.0 is in fedora14:28
mnaserlet me get you more info14:28
*** imcleod has quit IRC14:29
* number80 is running a scratch build14:30
mnaseri have the exact patch that fixed the issue for us, let me get you info14:30
number80chandankumar, rbowen: can you point me to the release notes? Let's document known issues14:31
*** Son_Goku has quit IRC14:32
rbowennumber80, rather:
rbowenIt is currently "published: false"14:32
mnaserthat was the patch that applying locally resolved the issue14:32
rbowenWe just need to make that true to push it live.14:32
number80mnaser: thanks :)14:33
number80rbowen: well, we're still waiting for the CI to give us green light14:33
number80we still have another run after the current one if it passes14:33
*** sfbender has joined #rdo14:33
sfbenderJavier Peña created DLRN master: Complete Python 3 support
*** dgurtner has quit IRC14:34
dmsimardjpena, apevec, number80: only one job remaining, all green so far14:34
sfbenderJavier Peña proposed DLRN master: Complete Python 3 support
jpenago CI job, you can do it!14:36
*** artom has joined #rdo14:36
mnaserwould be nice if ansible streamed output back :p14:36
apevecdmsimard, re. push to -testing, w/o ooo: that's what I proposed last night, then run to get current DoD satisfied14:36
dmsimardmnaser: *cough* zuul v3 *cough*14:36
dmsimardapevec: ok, so we don't need trown to manually test things then14:36
apevecbut yes, we need that centos-release pushed14:37
apevecjust tagging for extras was not enough14:37
apevecI guess it's manual ?14:37
dmsimardWe might need to ping hughes14:37
dmsimardI think he has something to do14:37
number80jpena: poi scenario 2 passed and it was the one failing :)14:39
number80now remain poi 1 who passed previously14:40
jpenapoi 1 is now collecting logs, it's passed14:40
ykarelFinished: SUCCESS14:42
*** agurenko has quit IRC14:43
*** cylopez1 has quit IRC14:43
apevecnumber80, what is failing ?!14:43
apevecah _was_ failing14:43
number80now, we can stress again waiting it to get merged14:44
*** mpeterso1 has joined #rdo14:45
number80mnaser: can you open a bugzilla ticket for python-backports-ssl_match_hostname?14:46
jpenadmsimard: have you unlocked the VMs?14:46
number80we still need one to override base OS system14:46
dmsimardjpena: nope, I'll go ahead and unlock/delete them now14:46
* apevec piles up his +2 on 916114:46
dmsimardjpena: VMs are gone14:47
number80remember that true GA is =>
number80if it's in not in release repo, it ain't GA :)14:47
*** marrusl has joined #rdo14:47
*** iranzo has quit IRC14:47
apevecnumber80, right, we'll run ci.centos pipeline in parallel14:47
apevecwith merging 916214:48
*** josecastroleon has quit IRC14:48
apevecthen hold announcement until we have pass in rdo-promote-cloudsig-testing-pike14:48
*** iranzo has joined #rdo14:49
*** iranzo has quit IRC14:49
*** iranzo has joined #rdo14:49
mnasernumber80 i will do that14:49
number80thanks :)14:49
number80ah right, the bot is muted14:49
sfbenderJavier Peña proposed DLRN master: Complete Python 3 support
mnasernumber80 under which product/component ideally?14:51
*** josecastroleon has joined #rdo14:51
number80product is RDO14:51
*** aditya_r has quit IRC14:52
mnasernumber80 thank you, that was 99% of the work figuring that part out :p14:52
*** imcleod has joined #rdo14:52
number80Well, I like when I get bug reports that includes the fix :)14:52
*** josecastroleon has quit IRC14:53
*** strigazi has joined #rdo14:53
number80ah, I was wondering why it wasn't merged yet, but there's still a non-voting job running14:54
*** mpeterso1 has quit IRC14:54
*** jprovazn has joined #rdo14:55
*** janki has quit IRC14:55
mnaser done14:55 bug 1487296 in distribution "Old version of python-backports-ssl_match_hostname being shipped" [Unspecified,New] - Assigned to lars14:55
chandankumarwaiting for a job to finish14:55
dmsimardapevec: do we have an announcement draft somewhere ?14:56
dmsimardrbowen: ^14:56
dmsimardI just want to make sure there's some sort of note about RDO releasing *now* but how it will hit the CentOS mirrors in a while14:57
*** cylopez has joined #rdo14:57
dmsimardWe always get questions about that because we fire the announcement before it hits the mirrors14:57
rbowenThere isn't. Can you add a sentence/paragraph to that effect? Or give me one to add?14:57
dmsimardrbowen: looking14:57
*** mpeterson has joined #rdo14:58
*** jobewan is now known as jobewan_away14:58
*** vnogin has quit IRC14:59
*** rpioso has joined #rdo15:00
*** daidv has quit IRC15:03
*** aditya_r has joined #rdo15:03
*** daidv has joined #rdo15:04
*** anshul has quit IRC15:04
*** abregman|mtg is now known as abregman15:06
*** emalikov has quit IRC15:08
*** mpeterson has quit IRC15:09
*** nyechiel has quit IRC15:09
number80^ add a known issues section in release notes15:09
dmsimardnumber80, apevec, jpena: rbowen and I worked out an additional node in the header portion of the release notes "The release is making its way out to the CentOS mirror network, and should be on your favorite mirror site momentarily."15:11
dmsimardlooks okay to you ?15:11
*** dprince has quit IRC15:11
number80"is currently being replicated on CentOS mirrors, it will be shortly available on our favorite mirror:15:12
dmsimard"is currently" is sort of meh15:12
*** mosulica has quit IRC15:12
dmsimardif I read that announcement tomorrow15:12
dmsimardbut just bikeshedding15:12
*** thrash is now known as thrash|biab15:16
*** mpeterson has joined #rdo15:16
number80right is being replicated should be enough15:17
*** imcleod has quit IRC15:18
* number80 thinks that roman idioms speakers tend to like longer phrases15:18
dmsimardthat last rdoinfo job is collecting logs right now15:19
dmsimardnot entirely sure why it's even running, it's master, not pike15:19
dmsimarddid we bump master things in that rdoinfo patch ?15:19
*** ragiman has quit IRC15:20
sshnaidmapevec, do we have nova issue on ?15:20
*** mpeterson has quit IRC15:21
jpenait's now in the gate15:21
*** iranzo has quit IRC15:22
*** imcleod has joined #rdo15:23
*** vnogin has joined #rdo15:25
apevecjpena, 9162 will need rebase15:26
*** shreshtha has quit IRC15:27
jpenaapevec: yep, I'm waiting for the merge, it looks like a simple "rebase" on gerrit doesn't work15:27
*** ushkalim has quit IRC15:29
*** scorcoran is now known as scorcoran_food15:31
*** pcaruana has quit IRC15:32
*** ykarel has quit IRC15:32
*** sbonazzo has quit IRC15:33
*** abregman has quit IRC15:34
*** tshefi has quit IRC15:35
*** cylopez has quit IRC15:36
*** eaguilar has quit IRC15:38
*** eaguilar has joined #rdo15:39
*** jobewan_away is now known as jobewan15:40
*** jackNemrod has joined #rdo15:41
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova:
*** apetrich has quit IRC15:45
*** sreichar has joined #rdo15:47
*** thrash|biab is now known as thrash15:50
*** aditya_ra has joined #rdo15:53
*** aditya_r has quit IRC15:53
*** imcleod has quit IRC15:54
*** Amita has quit IRC15:57
*** mvk has quit IRC15:59
*** ykarel has joined #rdo16:00
*** egonzalez has quit IRC16:02
*** mpeterson has joined #rdo16:03
*** lucasagomes is now known as lucas-hungry16:04
*** mpeterson has quit IRC16:16
*** jpich has quit IRC16:17
rdogerrittrown created config master: Install ansible in container build job
*** jlibosva has quit IRC16:20
*** aufi has quit IRC16:21
dmsimardwe really sleeping 10 seconds between tags? :/16:21
* dmsimard following
jpenathat's what I thought16:22
dmsimardIt's too late to change for this run but we might want to sleep, I dunno, 5 seconds for release tagging ?16:22
dmsimardWe're tagging a /lot/ of packages16:22
jpenanumber80: do you know if there's any reason why we should wait? Like avoiding DoS?16:23
*** mpeterson has joined #rdo16:24
*** MrAbaddon has quit IRC16:24
dmsimardjpena: it sounds like we're making sure the tag went through16:24
dmsimardwe send the tag command16:24
*** jobewan has quit IRC16:24
dmsimardand then retry for as long as the package isn't showing up as tagged16:24
dmsimarda tag command is a koji task so it needs to run16:25
dmsimardand I /guess/ it could take a bit but is usually near instantaneous16:25
dmsimardI don't think retrying every 5 seconds is bad16:25
number80not sure why it's done like this16:25
rdogerritDavid Moreau Simard created config master: Sleep 5 seconds instead of 10
rdogerritDavid Moreau Simard proposed config master: Sleep 5 seconds instead of 10
number80When I was doing it manually, I did it in a single command line16:26
number80I am looking to rewrite it in python where I can optimize thing more easily16:27
number80like retrieving all those info in a single API call16:27
number80that's what graffiti does16:27
rdogerrittrown created config master: Reintroduce undercloud install to container image build
*** mcornea has quit IRC16:28
number80If a package is not registered in a tag, it can do it for you16:29
dmsimardOk but let's at least put that at 5 seconds for the time being16:29
*** dprince has joined #rdo16:29
dmsimardNow is not the time to rewrite the job :)16:29
*** masonjoyers__ has quit IRC16:30
*** mpeterso1 has joined #rdo16:30
rdogerritDavid Moreau Simard proposed config master: Sleep 5 seconds instead of 10
dmsimardjpena: fixed16:31
*** jistr is now known as jistr|afk16:31
*** larizmen has quit IRC16:31
dmsimardhang on a second16:31
dmsimardwe're sleeping 5 seconds16:31
dmsimardbefore checking if things have been tagged16:31
*** scorcoran_food is now known as scorcoran16:32
dmsimardwe should probably check first, and then sleep16:32
dmsimardwe might be sleeping for nothing16:32
*** mpeterson has quit IRC16:32
*** noslzzp has quit IRC16:33
*** aditya_ra has quit IRC16:33
*** noslzzp has joined #rdo16:33
*** hewbrocca is now known as hewbrocca_afk16:34
*** trown is now known as trown|lunch16:35
rdogerritDavid Moreau Simard proposed config master: Don't sleep for nothing and sleep for 5 seconds instead of 10
* dmsimard says we shouldn't rewrite the job16:36
* dmsimard rewrites the job16:37
dmsimardnumber80, jpena ^16:37
*** mpeterso1 has quit IRC16:39
number80dmsimard: shell is limiting us and many features are in graffiti already or can be fixed in graffiti16:42
dmsimardnumber80: that's fair, I just want to make sure we're not sleeping 10 seconds for absolutely nothing for each package we're tagging16:43
dmsimardwe could be nice and sleep 2 seconds if we get a successful tag back without retries16:44
rdogerritDavid Moreau Simard proposed config master: Don't sleep for nothing and sleep for 5 seconds instead of 10
dmsimardthere, sleep 2 added so we don't potentially spam cbs if everything was tagged16:44
dmsimardnumber80, jpena ^16:44
number80I guess Alfredo had issues with throttling16:45
dmsimardnumber80: the logic was reversed -- when you implement a retry/sleep16:46
dmsimardyou don't sleep before trying at least once16:46
*** morazi has quit IRC16:46
dmsimardthat's flawed16:46
dmsimardyou retry if you fail16:46
dmsimardyou don't wait 10 seconds before trying the first time16:46
*** milan has quit IRC16:47
dmsimard~280 packages at 10 second forced sleep is 45 minutes16:48
*** morazi has joined #rdo16:48
number80it's 5 min usually16:49
number80(in one call)16:49
dmsimardnumber80: look at the console
number80saw that16:50
dmsimardit's been going for far more than 5 minutes16:50
dmsimardover an hour now, so we should be done soon I think16:50
*** aditya_r has joined #rdo16:50
*** mpeterson has joined #rdo16:51
number80dmsimard: I have even more extreme optimization16:55
number80cbs buildinfo <list of nvr>16:55
number80then, split output16:55
dmsimardnumber80: I don't want to change the existing job too much16:55
dmsimardnumber80: we can optimize after release is out16:56
dmsimarduse graffiti or whatever else16:56
number80it's no big change, but it'll wait sync to release anyway16:56
dmsimardnumber80: need to merge so it's effective for release job16:57
rdogerritMerged config master: Don't sleep for nothing and sleep for 5 seconds instead of 10
*** anilvenkata has quit IRC17:03
*** mpeterson has quit IRC17:03
*** fzdarsky is now known as fzdarsky|afk17:04
*** Son_Goku has joined #rdo17:06
*** sshnaidm is now known as sshnaidm|off17:09
*** rcernin has quit IRC17:12
*** tesseract has quit IRC17:14
*** jkilpatr has quit IRC17:16
*** sbulage has quit IRC17:16
*** egallen has quit IRC17:18
*** cylopez has joined #rdo17:22
apevecdang (re. tagging one by one)17:22
apevecand sleep...17:23
apevecnumber80, I also assumed grafitti is used, but I guess we have not been reviewing enough all that :(17:23
*** jmelvin has quit IRC17:23
apevecit really should be one cbs call17:24
*** jackNemrod has quit IRC17:24
*** jmelvin has joined #rdo17:24
apevechow it's running over 2h17:25
rdogerritMerged rdoinfo master: Push all pike-candidate to pike-testing
jpenanow I'll update 916217:26
*** gfidente has quit IRC17:26
*** cylopez has quit IRC17:27
rdogerritJavier Peña proposed rdoinfo master: Push all pike-testing to pike-release
*** jkilpatr has joined #rdo17:29
apevecnow we need to wait for buildlogs sync17:30
chandankumarhow much buildlogs sync takes generally?17:30
apevecit's quick, but runs only every 2h17:31
apevecon even UTC hours17:31
apevecin ~45min17:31
*** trown|lunch is now known as trown17:33
apevecjpena, dmsimard - anything before enabling
rdogerritMerged config master: Reintroduce undercloud install to container image build
apevecdo we do it via ci-config or directly in jenkins UI ?17:34
jpenaI think it has to be done in the Jenkins UI, I can't find anything in the jjb description that allows us to enable/disable17:36
*** imcsk8 has joined #rdo17:37
dmsimardapevec: yeah I sent a patch to improve the tagging perf by several orders of magnitude, we can further improve later -- didn't want to risk introducing bugs17:39
dmsimardapevec: also, we should check with bstinson17:40
*** aditya_ra has joined #rdo17:40
dmsimardbuildlogs from inside cico != outside cico17:40
dmsimardSo the content might already be there (or not)17:40
apevecah that would be good!17:41
*** kambiz has quit IRC17:41
*** aditya_r has quit IRC17:41
*** vnogin has quit IRC17:42
*** vnogin has joined #rdo17:42
*** pkovar has quit IRC17:44
*** eaguilar has quit IRC17:44
*** kambiz has joined #rdo17:45
chandankumardmsimard: do we use openstack-health in rdo?
dmsimardchandankumar: no17:49
dmsimardThe stack to make it work is quite complicated17:50
*** jistr|afk is now known as jistr17:50
*** gcerami has quit IRC17:51
apevecdmsimard, I'm merging
*** aramteke has joined #rdo17:51
apevecmultiarch can be followup17:51
*** aramteke has quit IRC17:52
*** links has quit IRC17:52
*** aditya_ra has quit IRC17:53
*** eharney has quit IRC17:56
*** vnogin has quit IRC17:57
chandankumardmsimard: you mean npm stuffs in openstack-health or something else?17:59
*** ykarel_ has joined #rdo17:59
*** imcleod has joined #rdo17:59
*** gszasz has quit IRC18:00
*** rwsu has quit IRC18:00
*** Caterpillar has quit IRC18:00
number80Now OpenStack has a health service?18:01
*** Caterpillar has joined #rdo18:01
*** ykarel has quit IRC18:01
*** eaguilar has joined #rdo18:02
*** imcleod has quit IRC18:05
*** eze has joined #rdo18:06
EmilienMnumber80: it's old18:07
EmilienMit's to track gate jobs results18:07
EmilienMand know which tempest tests fail, etc18:07
*** eaguilar has quit IRC18:07
*** wolverineav has quit IRC18:08
EmilienMsee for example18:08
*** wolverineav has joined #rdo18:08
EmilienMit tracks periodic and gate (not check)18:08
EmilienMyou can even subscribe to RSS per project or per job18:08
number80meh, I wanted to joke about Everything as a Service18:08
number80I haven't done one for long :)18:09
*** jdennis has quit IRC18:09
jpena18:11:32 DEBUG  Error: Package: python2-pykafka-2.6.0-0.1.dev2.el7.x86_64 (temp-cloud7-openstack-common-release)18:11
jpena18:11:32 DEBUG             Requires: python-tabulate18:11
jpenanumber80: ^^18:11
jpenawe have an error in the common deps, it seems18:12
dmsimardchandankumar: openstack-health retrieves subunit files from jobs, it doesn't work on it's own and there's different components to make it work18:13
dmsimardchandankumar: there is a story on about it somewhere18:13
dmsimardjpena: hmmm, I think I remember python-tabulate showing up in one of the jobs yesterday18:14
* dmsimard checks logs18:14
jpenaapevec: ^^18:15
*** MrAbaddon has joined #rdo18:15
jpenadmsimard: I think I saw it in one of the upstream rpm-packaging reviews18:16
dmsimardiirc that was at the end of one of the buildsys tagging jobs18:16
*** rwsu has joined #rdo18:16
dmsimardI think we're good on public buildlogs;O=D18:17
dmsimardbut I'm probably late in the news18:17
apevecno, you are ahead :)18:18
apeveccdn edge node I'm hitting still has old content18:19
apevecbut it is usually refresh around :1718:19
dmsimardI hit a recheck on p-o-i
number80jpena: ack btw18:19
dmsimardchandankumar: openstack-health:!/story/20218:20
apevecok, what's with python-tabulate ?18:20
dmsimardapevec: jpena knows but I did see these warnings at the end of a run yesterday
*** jistr is now known as jistr|off18:21
dmsimardmight or might not be related18:21
chandankumardmsimard: thanks :-)18:21
apevechm, ovs 2.6.1 in pike18:21
jpenait looks like it is only required for monasca-common
jpenaand we're not shipping it in RDO, it's just there for rpm-packaging18:22
jpenaso I'll remove it from the review, ok?18:22
apevecbut it's blocking us18:22
apevecjpena, ack18:22
jpenathat's something we can fix with more time18:22
dmsimardjpena: could it be a transient dep ?18:23
dmsimarda dep of a dep18:23
dmsimardso, i.e, not in reqs.txt18:23
dmsimardor something18:23
number80jpena: drop it, have it as a post-GA thing18:23
* dmsimard thinking out loud18:23
* number80 is working on it18:23
rdogerritJavier Peña proposed rdoinfo master: Push all pike-testing to pike-release
* number80 is building python-tabulate18:25
*** jdennis has joined #rdo18:26
*** abregman has joined #rdo18:29
apevec looks good18:30
apevecok buildlogs cdn refreshed for me too18:32
*** jtomasek has quit IRC18:37
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova:
*** scorcoran is now known as scorcoran_biab18:51
*** imcleod has joined #rdo18:54
*** scorcoran_biab has quit IRC18:56
*** eharney has joined #rdo18:57
*** jkilpatr has quit IRC18:57
*** jkilpatr has joined #rdo18:57
*** scorcoran has joined #rdo18:58
*** jmelvin has quit IRC18:59
*** jmelvin has joined #rdo19:00
*** Son_Goku has quit IRC19:00
*** rcernin has joined #rdo19:01
*** ykarel_ has quit IRC19:02
*** imcleod has quit IRC19:04
*** iranzo has joined #rdo19:11
jschlueterapevec: just notice os-win has 2.2.0 tag coming off of stable/pike but master last tag is 2.1.019:14
apevecyes, known issue:  until first round of Queens releases... on master, PBR does not see newer tags pushed to stable/pike19:15
apevecI'll repropose workaround in dlrn19:15
apevectoo bad I didn't manage to work on it before this critical period19:16
dmsimardapevec: the build failed19:16
jschlueterapevec: also looks like the tarball vs branch is missing the ;sys_platform=win32 entries from requirements.txt19:16
dmsimardERROR testing new configuration for cloud7-openstack-pike-release19:16
dmsimardWe should had held the node ..19:17
apevecso what failed?19:18
dmsimardapevec, number80, jpena:
dmsimard19:04:09 DEBUG [Errno 14] HTTP Error 404 - Not Found19:18
dmsimardwtf ?19:18
apevec failure: repodata/repomd.xml from pike: [Errno 256] No more mirrors to try.19:19
apevec19:04:09 DEBUG [Errno 14] HTTP Error 404 - Not Found19:19
apevecyeah, something is not quite right19:19
apevecwe are trying to push there19:19
*** iranzo has quit IRC19:19
apevecI guess edge-case not covered, the initial push19:19
jpenaapevec number80: we have a few unresolved deps ->
*** eze is now known as eaguilar|afk19:19
dmsimardthose are the same I pasted earlier19:20
number80jpena: normal19:20
apevecyeah, repoclosure has few know issues, hence it's just warning19:20
jpenasame in newton19:20
*** eaguilar|afk is now known as eze19:20
number80are we trying to openstack-tripleo-ui-deps to -release?19:20
*** eze is now known as eaguilar|afk19:20
number80this is BR only19:20
jpenaalso "error testing new configuration for cloud7-openstack-pike-release"19:21
jpenathat's probably the key, I'll check19:21
dmsimardI will fire a recheck immediately and hold the node19:21
dmsimardworst case we kill the job19:21
apevecdmsimard, hold on19:21
number80seems not19:21
dmsimardapevec: ?19:22
apevecnumber80, ah yes we need to keep few BRs in -testing for dlrn19:22
number80btw I fixed matplotlib dependency but I can't remember if I submitted a build or not19:22
number80I did
apevecdmsimard, all clear, wanted to check tripleo-ui-deps is NOT pushed19:22
jpenaso the "repo not found" for openstack-pike seems to be the issue19:23
apevecnumber80, post-GA cleanup19:23
apevecjpena, right19:23
*** eaguilar|afk is now known as eze19:23
jpenaapevec: so what's the next step?19:23
apevecjpena, looks like initial push is not covered19:23
apevecmanual +2 Verified19:23
number80apevec: yeah, that was the plan, it's not even used in openstack19:23
number80(the -qt4 subpackage I mean)19:24
dmsimardI put a logs.tar.gz here
dmsimardwith the full logs19:24
dmsimard(for grep and things)(19:24
apevecthen run
*** pchavva has quit IRC19:25
apevecoh wait, manual +Verified will skip gate where cbs tagging happens19:25
dmsimardapevec, number80, jpena: ssh jenkins@
apevecnumber80, can you get grafitti yaml from job logs?19:25
number80apevec: it should be19:26
jschlueterapevec: dlrn tarball has the dropped requirements19:26
dmsimardapevec: this?
apeveccorrection: then run - bad paste earlier ...19:26
dmsimardapevec: we can forcefully get zuul to enqueue a gate job19:27
apevecjschlueter, aha, so we have too old setuptools there19:27
dmsimardapevec: regardless if it's passed check or not19:27
jschlueterapevec: ack19:27
apevecdmsimard, how does that work?19:27
apevecthat's admin action?19:28
dmsimarddirectly on the zuul server node yeah.19:28
apevecat least is promising, build-images passed19:29
dmsimardwait, so we don't actually care about the check gate for the release then19:29
dmsimardif we have cico19:29
apevecideally we'd care if it worked :)19:29
dmsimardoh, yeah we do19:29
dmsimardbecause it validates tags and stuff19:29
apevecyeah, but there's a problem b/c it expects released repo19:30
apevecit's designed for updates, not initial push19:30
apevecunless someone can debug that quickly19:30
apevecand add a special case when release repo is not there yet19:31
dmsimardthis is what the job runs
*** amuller has quit IRC19:31
dmsimardgating_scripts here;a=tree19:32
* dmsimard thinks all those bash scripts should be re-written to ansible for clarity19:34
dmsimardit probably has to do with this block ?;a=blob;f=buildsys-tags/;h=debebb5ff7813c72a7014e732dc687738c5e5e1c;hb=HEAD#l7919:35
dmsimardthe repo gets enabled but there's nothing in it19:35
dmsimardjpena, apevec, number80: do we want to hack it ^ ?19:38
dmsimardor ninja it19:38
dmsimardwe're probably going to have the same issue in the gate19:38
dmsimardso I think we have to hack it19:38
rdogerritJon Schlueter created openstack/os-win-distgit rpm-master: Let RPM handle requirements
*** jkilpatr has quit IRC19:40
rbowenSo, can someone give a quick status update?19:40
number80dmsimard: ninja but we'll have to tag them manually19:40
dmsimardrbowen: we've jumped the first big hurdle19:41
jpenaa *big* hack we could try is to change baseurl in;a=blob;f=buildsys-tags/mock-config.cfg;h=152c54557e1196ffcbd5c1f1f4168f6a8ef74c48;hb=HEAD19:41
dmsimardrbowen: now it's more of a tooling issue more than a packaging one19:41
jpenaline 5319:41
jpenaso when we enable the pike repo, we're actually using the buildlogs one instead of mirror.centos.org19:42
jpenado it once, get it tagged, undo19:42
dmsimardjpena: or just not enable it ?19:42
dmsimardbut yeah, I guess we could do that19:42
rbowenAre we likely to have Pike today?19:42
rdogerritrdo-trunk created openstack/tempest-distgit rpm-master: openstack-tempest: failed to build f1fbb3f
dmsimardapevec: I fired manually, it was disabled and didn't fire in the pipeline19:45
*** spanglywires has joined #rdo19:45
dmsimardapevec: oooq cbs testing jobs failed19:46
dmsimardfail on undercloud install19:46
dmsimardrbowen: things aren't looking particularly good right now19:46
rbowenok. Thanks for the update.19:46
dmsimardI know there's never a good time for PTO but this is an awful time for the guy who wrote a major portion of the release automation :/19:47
number80I can run graffiti to produce same result if we decide to ninja-merge for now19:50
dmsimardnumber80: CI jobs aren't passing19:50
number80just stating an option19:50
number80not saying we should do that19:50
number80if it's a tooling only issue19:51
*** jackNemrod has joined #rdo19:51
dmsimard2017-08-31 19:34:18 | 2017-08-31 19:34:18,170 INFO: [1;31mError: Execution of '/bin/yum -d 0 -e 0 -y install openstack-kolla' returned 1: Error: Nothing to do[0m19:51
dmsimard2017-08-31 19:34:18 | 2017-08-31 19:34:18,170 INFO: [1;31mError: /Stage[main]/Tripleo::Profile::Base::Docker_registry/Package[openstack-kolla]/ensure: change from purged to present failed: Execution of '/bin/yum -d 0 -e 0 -y install openstack-kolla' returned 1: Error: Nothing to do[0m19:52
dmsimard2017-08-31 19:45:16 | Reason: os-refresh-config failed. See log for details.19:52
jpenalet's think about the options19:54
*** egonzalez has joined #rdo19:54
jpenaif we ninja-merge and apply graffiti, we'll have a most-probably broken repo, but we can then do small incremental fixes19:54
dmsimardjpena: nothing we can't do with the existing -testing19:54
jpenammm right. Did this error happen with -testing?19:55
dmsimardwe could not run ooo in the rdoinfo gate19:55
dmsimardbefore tagging to -testing19:55
jpenayes, tes19:55
jpenaI mean19:56
jpenathe ooo error happened with the -testing repo19:56
dmsimardyes, those are from here19:56
jpenaso once we fix that one, we could ninja-merge to avoid the chicken&egg problem we have now19:56
dmsimardweirdo is running separately in
*** jayg has quit IRC19:57
dmsimardI'm not sure the issue is just kolla19:57
dmsimardthose os-refresh/collect config errors look like a problem19:57
dmsimardthere was something about python requests/urllib earlier ?19:57
dmsimardboth jobs have the same failures19:58
jpenayes, we had the puppet-murano issue with the newer urllib319:58
dmsimardso we downgraded it or something ?19:59
jpenawe kept the same version we had till now20:00
dmsimardwe have requests 2.1120:00
dmsimardand os-collect-config has >2.1420:00
dmsimardnot saying that's the issue but /me shrug20:00
dmsimardlet me look at what we're using in a passing job20:00
dmsimardyeah it's passing with 2.11 so that's not it20:01
jpenadmsimard: wait, we have no kolla in -testing (!?)20:01
dmsimardjpena: that's apparently one of the things it's complaining about yes20:01
jpenathere's no build for it ?20:01
dmsimardhow come20:01
dmsimardwe missed it for some reason ?20:01
*** mpeterson has joined #rdo20:02
dmsimardadding that is probably a good start20:02
dmsimardI'm trying to compare the failing job with a passing one20:02
*** jayg has joined #rdo20:02
jpenathere's no pike-rdo branch for kolla-distgit, so we missed it during branching20:03
jpenanumber80: ^^20:03
*** jkilpatr has joined #rdo20:03
apevecwe don't have stable containers, why do we care about kolla?20:04
dmsimardpkg diff between failing (cbs -testing) and passing (trunk pike promotion)
dmsimardapevec: tripleoclient uses kolla20:04
dmsimardthat prepare something image thing stuff imports and wraps kolla-build20:04
dmsimardso it's probably a dep in one of those tripleo-common packages20:04
*** vnogin has joined #rdo20:05
dmsimardthe versions in trunk for the series of os-*-config packages aren't the same20:06
dmsimardI don't know if we missed a tag/bump20:06
dmsimardcould just be pbr/dlrn autobump I dunno20:06
jpenadmsimard: that's autobump20:06
dmsimardah, yeah, i think that's just autobump20:06
dmsimardthe pkgdiff is not very conclusive20:08
*** imcleod has joined #rdo20:08
rdogerrithguemar created openstack/kolla-distgit rpm-master: Pike requirements sync
dmsimardit's possible that due to kolla installation failure puppet failed dependencies and lead to the os-*-config failure20:08
jpenamakes sense20:08
jpena"always go for the first error"20:09
dmsimardI hate puppet's "let's fail as late as possible"20:09
dmsimardthankfully ansible is "fail as early as possible" :)20:09
dmsimardif kolla is the only issue20:09
dmsimardthe weirdo jobs should come back green20:10
dmsimardweirdo jobs should tell us if there's anything else in the meantime20:10
dmsimardso if we're very optimistic, we're looking at a minimum 3hrs if everything is green all the way through (rdoinfo check, rdoinfo check, ci.centos, tagging, buildlogs sync for kolla, etc.)20:10
dmsimardare we going to push for a release today ? otherwise we're delaying until monday20:11
dmsimardeveryone is tired but that would break my heart :(20:11
rbowenMonday is Labo(u)r day20:11
dmsimardthat's correct, so tuesday even20:12
dmsimardugh :(20:12
apeveclet's fix that kolla issue first20:12
apevecdid they have pike release tag?20:12
apevecor we need snap?20:12
*** mpeterson has quit IRC20:13
apevecmilestone3 is kolla last tag20:13
jpenathey haven't branched yet, the newest tag they have is from Jul 2720:13
apevecso snpa?20:13
jpenaI'd say snap20:13
*** rpioso has quit IRC20:14
apevecI guess we should revisit feasibility of current DoD with deployment project20:15
apevecooo is also rc1 only20:15
apevecwe need to define staged GA20:15
apevecwith weirdo only DoD for GA +2h20:15
apevecand ooo, kolla etc. +2 weeks20:16
dmsimardeven weirdo depends on packstack and p-o-i20:16
apeveci.e. Pike cycle-trailing release deadline which is Sep11-1520:16
dmsimardwhich are not technically released20:16
apevecbut in better shape20:17
dmsimardno failures in yet20:17
apevecok, who is preparing kolla snap ?20:17
jpenaI'm going for dinner, brb20:17
*** jpena is now known as jpena|brb20:18
apevecI'll take it20:18
rdogerritMerged openstack/kolla-distgit rpm-master: Pike requirements sync
*** imcleod has quit IRC20:21
apevecnumber80, ^ are you sending pike-rdo patch in sf config?20:23
apevecok, I'm on standy for reviews20:23
rdogerrithguemar created config master: Create pike-rdo branch for kolla
number80apevec: ^20:25
*** egallen has joined #rdo20:25
*** egallen has quit IRC20:25
*** kgiusti has left #rdo20:27
rdogerritMerged config master: Create pike-rdo branch for kolla
rdogerritDavid Moreau Simard created gating_scripts master: Temporarily not enable the release repo
dmsimardapevec, jpena|brb, number80 ^ wdyt20:27
dmsimardhah, hang on20:28
rdogerritDavid Moreau Simard proposed gating_scripts master: Temporarily not enable the release repo
dmsimardtake two20:28
apevecnumber80, pike passed-ci is bbeda30a9df9021bf463b90d864e39cd6d7870df20:28
*** vnogin has quit IRC20:29
number80okalternative would be testing if release repo is existing but whatever20:29
dmsimardlast rdoinfo check failed on same 40420:29
dmsimardnumber80: that's a hack that will be reverted20:29
dmsimardI still have the node held if anyone wants to look20:30
apevecdmsimard, not sure about COMMON_TESTING change20:30
dmsimardapevec: it's copy pasta from the block right above20:31
dmsimardit's a if/elif20:31
number80dmsimard: yeah, I gave it a +220:31
dmsimardapevec: if we don't put that there, -testing won't be enabled20:31
apevecyeah but here we are actually testing common-release20:31
dmsimardas far as I can undertand20:31
apevecyeah, but COMMON_RELEASE will be set iiuc ?20:31
* apevec reads more bash20:31
dmsimardbut common release doesn't exist20:31
dmsimardright ?20:31
apevecthis is envvar20:32
apevecrepo is temp-common20:32
apevecit's coming from changed_repos.txt20:33
apeveci.e. what changes are detected in rdoinfo20:33
apevecso in our release review COMMON_RELEASE will be set20:33
rdogerritDavid Moreau Simard proposed gating_scripts master: Temporarily not enable the release repo
dmsimardlike this ^ ?20:33
apevecyep that should work20:34
*** tosky has quit IRC20:35
dmsimardwe need to ninja merge that, there's no gate jobs20:35
apevecor not... we might confuse further logic if it sees builds we are trying to push in the enabled repo?20:35
rdogerritMerged gating_scripts master: Temporarily not enable the release repo
apeveclet's try, how long that job takes?20:35
rdogerrithguemar created openstack/kolla-distgit pike-rdo: Pike update 5.0.0 pre-release (bbeda30a9df9021bf463b90d864e39cd6d7870df)
dmsimardapevec: ~1hr20:35
dmsimardbut we need the kolla fix20:36
dmsimardbefore doing a recheck20:36
dmsimardand I guess rebase20:36
apevecalmost there20:36
apevecyes, we need kolla in testing first20:36
dmsimardnumber80: added a comment in your review20:37
dmsimardnumber80: nevermind20:38
*** jayg is now known as jayg|g0n320:38
dmsimard3/6 green in
dmsimardremaining 3 in progress20:38
apevecnumber80, hguemar@benihime.seireitei - is that a trap for spammers? :)20:39
dmsimard4/6 green20:39
number80forgot that20:39
number80it just my laptop crashed and my emacs config is half-broken20:39
* number80 is on temporary laptop20:39
apevecoh fun20:40
rdogerrithguemar proposed openstack/kolla-distgit pike-rdo: Pike update 5.0.0 pre-release (bbeda30a9df9021bf463b90d864e39cd6d7870df)
*** peter-os has joined #rdo20:40
number80Yeah, I haven't find yet why my config is broken20:40
*** scorcoran is now known as scorcoran_biab20:41
apevecnumber80, need autosetup -S git20:43
dmsimardapevec, number80, EmilienM: we have full upstream p-o-i passing on standalone -testing
EmilienMdmsimard: do you want us to merge it?20:44
rdogerrithguemar proposed openstack/kolla-distgit pike-rdo: Pike update 5.0.0 pre-release (bbeda30a9df9021bf463b90d864e39cd6d7870df)
apevecEmilienM, hold on20:44
apevecactually you don't care about kolla20:44
EmilienMapevec: ok20:45
*** eharney has quit IRC20:45
apevecso up to you20:45
dmsimardEmilienM: for the time being I'm using it as a sanity check20:45
EmilienMno I don't20:45
EmilienMin that context I mean20:45
EmilienMdmsimard: ok20:45
apevecyeah, in poi20:45
EmilienManyway let me know20:45
dmsimardapevec: so that means weirdo is all green20:45
dmsimardonly ooo now...20:45
dmsimardhopefully it's that kolla issue20:45
*** scorcoran_biab has quit IRC20:46
apevecyes, review needs one more fix and we're good to merge it20:46
*** artom has quit IRC20:47
*** florianf has quit IRC20:50
number80apevec: you fixed the tagging review?20:51
*** spanglywires has quit IRC20:51
apevecnumber80, no, fix what?20:51
apevecwe need kolla build20:51
apevecthen push to testing20:51
apevecthen rebase release review on that20:52
number80this one + the release one20:52
rdogerrithguemar created rdoinfo master: Add kolla to pike-testing
*** imcleod has joined #rdo20:56
apevecnumber80, cbs build is not finished, ^ will fail20:57
number80I somehow did things in parallel and my fingers just submitted reviews on their own20:57
*** eze has quit IRC20:58
* number80 has fingers with their own will20:58
apevecit's good test for CI :)20:58
apevecit did fail20:58
number80I had to create a noop cbs alias to avoid manually tagging builds w/o thinking after we added automation ...21:00
*** scorcoran has joined #rdo21:03
*** julim_ has quit IRC21:04
*** trown is now known as trown|outtypewww21:04
*** wolverineav has quit IRC21:04
*** wolverineav has joined #rdo21:05
*** imcleod has quit IRC21:06
rdogerritMerged openstack/kolla-distgit pike-rdo: Pike update 5.0.0 pre-release (bbeda30a9df9021bf463b90d864e39cd6d7870df)
*** wolverineav has quit IRC21:09
*** jprovazn has quit IRC21:11
apevecdmsimard, doesn't want into gate ?21:12
apevecmessing with votes confused zuul?21:12
dmsimardParent needs to merge21:13
*** Goneri has quit IRC21:13
rdogerritMerged rdo-infra/rdo-release pike-rdo: Use HTTPS for the testing and pending repositories
rdogerritMerged rdo-infra/rdo-release pike-rdo: Enable the release repository for RDO Pike
rdogerritMerged rdo-infra/rdo-release ocata-rdo: Use HTTPS for the testing and pending repositories
dmsimardapevec, number80: do we care about check queue for kolla rdoinfo21:15
rdogerritMerged rdo-infra/rdo-release newton-rdo: Use HTTPS for the testing and pending repositories
dmsimardcause I'll enqueue gate right away21:15
*** jpena|brb is now known as jpena21:15
*** dustins has quit IRC21:15
dmsimardotherwise we're gonna wait >1hr for weirdo and ooo to come back21:16
dmsimardjust for kolla21:16
dmsimardwhich is gonna get tested in release anyway21:16
apeveclet's just push it21:16
dmsimarddon't push/submit21:16
dmsimardneeds to go through gate21:16
dmsimardsubmitting gate now.21:16
rdogerritAlan Pevec created rdo-infra/centos-release-openstack pike-rdo: Enable the release repository for RDO Pike
*** paramite has quit IRC21:18
dmsimardhmmm, not seeing the job in queue gate21:19
dmsimardbut I see in the logs that it should have worked21:19
dmsimard2017-08-31 21:18:16,578 INFO zuul.Scheduler: Adding rdoinfo, <Change 0x7fcc340f1e50 9192,1> to <Pipeline gate>21:20
dmsimardoh, it needs verified21:20
jpenayes, it's running weirdo tests21:20
*** imcleod has joined #rdo21:21
apevecso what time do we save ?21:21
dmsimardI guess it doesn't work :/21:21
dmsimardpabelanger: it's possible to bypass check queue right ?21:21
dmsimardpabelanger: enqueuing directly to gate ?21:21
dmsimard"zuul enqueue --trigger gerrit --pipeline gate --project rdoinfo --change 9192,1" doesn't generate any errors, I see "2017-08-31 21:20:54,650 INFO zuul.Scheduler: Adding rdoinfo, <Change 0x7fcc340f1e50 9192,1> to <Pipeline gate>" in logs21:22
dmsimardbut there's nothing in the gate queue21:23
dmsimardI manually +2/+v and +W the patch21:23
dmsimardapevec: we're saving the time it takes to run the weirdo and ooo jobs21:24
dmsimardapevec: which are needless for the kolla tag21:24
dmsimardbut I guess we're stuck with that21:24
dmsimardand then we'll have to run the /same/ weirdo and ooo jobs in rdoinfo GA21:24
dmsimardwe could disable the jobs temporarily21:25
dmsimardand rely on cico instead ?21:25
rdogerritrdo-trunk created openstack/tempest-distgit rpm-master: openstack-tempest: failed to build 5ee9e19
dmsimardI mean, weirdo is known to be fully green already21:25
apevecfor testing21:25
apevecunknown for proposed release tags21:26
pabelangerdmsimard: you can, but I usually like to make sure it passed checked first21:26
dmsimardI guess..21:26
dmsimardI need to lie down a bit, my allergy pills are knocking me out21:27
*** spanglywires has joined #rdo21:27
*** racedo has quit IRC21:33
*** karimb has joined #rdo21:35
*** abregman has quit IRC21:35
*** egonzalez has quit IRC21:38
rdobot[sensu] NEW: - check-delorean-master-current @ |#| Build failure on centos7-master/current: nova, tempest:
*** imcleod has quit IRC21:42
*** oidgar has quit IRC21:42
*** morazi has quit IRC21:53
*** rlandy is now known as rlandy|bbl21:58
*** jackNemrod has quit IRC21:58
*** [1]rpioso has joined #rdo22:00
*** peter-os has quit IRC22:01
rdogerritMerged rdo-infra/centos-release-openstack pike-rdo: Enable the release repository for RDO Pike
*** apevec has quit IRC22:02
*** spanglywires has quit IRC22:06
*** scorcoran has quit IRC22:06
*** nyechiel has joined #rdo22:10
*** pradk has quit IRC22:13
*** apevec has joined #rdo22:15
*** apevec has quit IRC22:15
*** apevec has joined #rdo22:15
*** jcoufal has quit IRC22:16
*** nyechiel has quit IRC22:16
*** dprince has quit IRC22:20
number80so what should we do now?22:24
*** rbowen has quit IRC22:25
number80"ERROR testing new configuration for cloud7-openstack-pike-release"22:25
number80as-is, automation is blocked22:25
jpena is finishing tests, then it will merge. After that, we can update 9162 with kolla, and while it's running test the promotion as well22:25
number80oh, I missed that one22:26
jpenadmsimard sent a hack for the ci scripts22:26
* number80 is confused22:26
jpenalet me recap22:26
number80I guess I will be out of commission tomorrow morning :)22:27
jpenakolla is being tagged in -testing22:27
jpenastill in ci22:27
*** tosky has joined #rdo22:27
jpenaafter that, we can update the big-fat -release change in rdoinfo and try again22:27
jpenaCI scripts will behave a bit different because of
jpenain parallel, we'll test the updated -testing repo with
jpenaif everything goes  fine, we'll be in one of two situations22:28
jpenaa) we don't have to ninja-merge anything and we're happy22:29
jpenab) repos work but we have to ninja-merge the rdoinfo change and tag manually22:29
*** vnogin has joined #rdo22:29
number80as for b) we can reuse the cbs_actions generated by CI to have consistent results22:30
number80so not too bad22:30
*** rcernin has quit IRC22:30
number80Yeah, that's why there was a two-step tagging, it's easier to review things or manually fix things22:31
* number80 had learnt this tricks from Releng who has to deal with this kind of stuff22:31
number80always allow manual intervention while keep things consistent with automation22:31
jpenait's going to be a while until rdoinfo-tripleo-pike-testing-centos-7-multinode-1ctlr-featureset005-nv finishes, I'll be away for a few mins22:32
*** vnogin has quit IRC22:34
*** thegreenhundred has quit IRC22:38
*** aludwar has joined #rdo22:38
*** [1]rpioso has quit IRC22:40
*** tosky has quit IRC22:41
*** richm has quit IRC22:42
*** richm has joined #rdo22:42
mnaserdmsimard fyi yum update was all that was needed once stuff was out in pike22:52
apevecjpena, number80 - we should have just tagged kolla snap manuall for -testing23:00
apevecwe missed few 2h cycles for buildlogs sync already :(23:00
apevecnext one is in ~1h23:00
apevecbut ok, now we have actual oooq job running in
apevecit's non-voting23:01
apevecbut result will be interesting to have23:02
apevecnm, it failed :)23:02
jpenait has failed in tempest run23:02
apevecsort: write failed: standard output: Broken pipe ?!23:03
apevecconfig_tempest failed23:04
rdogerritMerged rdoinfo master: Add kolla to pike-testing
jpenaok, I'll update 916223:07
rdogerritJavier Peña proposed rdoinfo master: Push all pike-testing to pike-release
*** jmelvin has quit IRC23:08
apevecnumber80, jpena - then we stop and continue in the morning, another all nighter will not help anyone23:08
*** eaguilar has joined #rdo23:08
number80at some point, my brain stopped working23:09
jpenaok, let's get some sleep23:09
jpenasee you tomorrow23:09
*** jpena is now known as jpena|off23:09
*** ihrachys has quit IRC23:09
*** apevec has quit IRC23:10
*** vnogin has joined #rdo23:13
*** eaguilar has quit IRC23:14
*** eaguilar has joined #rdo23:14
*** vnogin has quit IRC23:27
*** artom has joined #rdo23:50
rdogerritJon Schlueter proposed openstack/os-win-distgit rpm-master: Let RPM handle requirements

Generated by 2.15.3 by Marius Gedminas - find it at!