Thursday, 2018-05-17

openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044500:00
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Fix redis healthcheck  https://review.openstack.org/56898600:09
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097200:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154900:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163400:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169200:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)00:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)00:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)00:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]00:10
openstackgerritAlex Schultz proposed openstack/tripleo-common master: Fix redis healthcheck  https://review.openstack.org/56898600:31
*** gvrangan has quit IRC00:31
*** dxiri has joined #tripleo00:32
*** gvrangan has joined #tripleo00:34
*** dxiri has quit IRC00:42
*** dxiri has joined #tripleo00:42
*** cameron_chuang has quit IRC00:45
*** jcoufal has joined #tripleo00:47
*** dxiri has quit IRC00:47
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: overcloud plan deployment failures  https://review.openstack.org/56867301:03
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: overcloud plan deployment status  https://review.openstack.org/56434101:04
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: overcloud plan deployment failures  https://review.openstack.org/56867301:05
openstackgerritMerged openstack/tripleo-upgrade master: add container minimal check and gate  https://review.openstack.org/56873301:07
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097201:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154901:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163401:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169201:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)01:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)01:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)01:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]01:10
*** slaweq has joined #tripleo01:11
*** zshi has joined #tripleo01:14
*** slaweq has quit IRC01:15
*** fragatina has quit IRC01:16
*** psahoo has joined #tripleo01:17
*** fragatina has joined #tripleo01:18
*** fragatina has quit IRC01:19
*** liverpooler has joined #tripleo01:19
openstackgerritMerged openstack/tripleo-quickstart-extras master: Revert "Add test_autoscaling tests to skip list"  https://review.openstack.org/56886001:26
openstackgerritMerged openstack/tripleo-common stable/pike: Script to update RPM's in containers.  https://review.openstack.org/56815101:26
*** psahoo has quit IRC01:33
*** psahoo has joined #tripleo01:36
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Support networkx 2.0  https://review.openstack.org/50652401:39
*** olap has joined #tripleo01:39
*** olap has quit IRC01:43
*** fragatin_ has joined #tripleo01:52
*** owalsh_ has joined #tripleo01:58
*** owalsh has quit IRC02:02
*** itlinux has joined #tripleo02:02
*** rwsu has joined #tripleo02:03
*** thrash is now known as thrash|g0ne02:03
EmilienMstevebaker: can you look https://review.openstack.org/#/c/568933/ please?02:07
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097202:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154902:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163402:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169202:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)02:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)02:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)02:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]02:10
weshaymwhahaha, WAHOOO!02:17
weshayI think EmilienM knew that was going to happen but we also thought it was rebased to avoid.. but meh.. thanks!02:18
mwhahahaweshay: it took a while for the tripleo-common package to build02:18
EmilienMsorry02:19
mwhahahaYes all EmilienM's fault02:19
EmilienMmwhahaha: well, sorry to fix the role_name thing ;-)02:19
mwhahahaPfft like we need ansible anyway02:20
mwhahahaBash++ for life02:20
weshayI need something kgnu02:20
weshayopenstack needs to rewrite ansible..02:20
weshayoh wait02:20
weshaywhat's up west side folks!02:21
weshaythis week has SUUUUUUCKED02:21
weshayEmilienM, picked the wrong week to come back02:21
EmilienMI missed you02:21
EmilienMand all the bugs02:21
weshayI'm like a dirty dog w/ flea's to you gents02:22
weshayreally02:22
weshayall I do is show up and present bugs02:22
EmilienMthat's not false :-P02:22
weshayat least I can tell jokes02:23
EmilienMpretty good ones02:23
EmilienM1/10 are good02:23
weshayWHICH by the way.. ali wong on netflix02:23
weshayoh crap02:23
*** moshele has quit IRC02:24
cmystermorning02:27
*** itlinux has quit IRC02:30
*** moshele has joined #tripleo02:32
*** jcoufal has quit IRC02:35
*** moshele has quit IRC02:37
*** itlinux has joined #tripleo02:38
*** agopi has quit IRC02:40
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: doc on which tests to skip  https://review.openstack.org/56899602:41
*** agopi has joined #tripleo02:42
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: doc on which tests to skip  https://review.openstack.org/56899602:43
*** itlinux has quit IRC02:46
*** liverpooler has quit IRC02:46
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: DNM: build image in every OVB job  https://review.openstack.org/56825802:48
*** gbarros has joined #tripleo02:55
*** tiswanso has joined #tripleo02:56
*** tzumainn has quit IRC02:56
*** agopi has quit IRC02:57
*** bkopilov has quit IRC02:57
*** agopi has joined #tripleo02:59
*** tiswanso has quit IRC02:59
*** psachin has joined #tripleo03:00
*** tiswanso has joined #tripleo03:00
*** itlinux has joined #tripleo03:08
*** ramishra has joined #tripleo03:09
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097203:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154903:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163403:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169203:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)03:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)03:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]03:10
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: undercloud: change upgrade steps  https://review.openstack.org/56899703:10
*** bkopilov has joined #tripleo03:10
*** slaweq has joined #tripleo03:11
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: undercloud: change upgrade steps  https://review.openstack.org/56899703:11
*** jcoufal has joined #tripleo03:12
*** slaweq has quit IRC03:16
*** agopi has quit IRC03:22
*** cmyster has quit IRC03:24
*** bkopilov has quit IRC03:29
*** jcoufal has quit IRC03:30
*** itlinux has quit IRC03:34
*** jbadiapa has quit IRC03:35
*** jbadiapa has joined #tripleo03:35
*** ykarel|away has joined #tripleo03:36
*** gvrangan has quit IRC03:38
*** gvrangan has joined #tripleo03:39
*** gvrangan has quit IRC03:42
*** udesale has joined #tripleo03:49
*** gbarros has quit IRC03:51
openstackgerritShangXiao proposed openstack/tripleo-heat-templates master: [Trivialfix]Modify a grammatical error  https://review.openstack.org/56900603:51
*** pdeore has joined #tripleo03:52
*** ykarel|away has quit IRC03:58
*** gyankum has joined #tripleo03:59
*** udesale has quit IRC04:10
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097204:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154904:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163404:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169204:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)04:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)04:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)04:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]04:10
*** janki has joined #tripleo04:12
*** shreshtha has joined #tripleo04:13
*** udesale has joined #tripleo04:13
Tenguhello there :)04:15
*** olap has joined #tripleo04:21
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Support networkx 2.0  https://review.openstack.org/50652404:22
*** gvrangan has joined #tripleo04:22
*** anilvenkata has joined #tripleo04:23
*** links has joined #tripleo04:24
jaosoriorTengu: sup dude!04:24
Tengujaosorior: still readin docs :)04:26
Tengujaosorior: you?04:26
*** olap has quit IRC04:26
jaosoriorTengu: checking out a fix sileht did. He found the solution to the issue we were checking out yesterday04:26
Tenguoh? link?04:27
jaosoriorTengu: today I plan to experiment a bit on how to deploy keystone with kerberos auth04:27
jaosoriorwant to pair it?04:27
Tengujaosorior: I should get my build machine today :). so I will be able to help/check/foo04:28
*** fpan has quit IRC04:28
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Revert "Revert "Change default endpoint map entries to use TLS""  https://review.openstack.org/56873604:28
Tenguwill just be a bit afk in order to actually build and install it :)04:28
jaosoriorTengu: sure, no biggie04:28
jaosoriorI was thinking of opening a tmate session, but if you want to focus on building your machine (which is also very relevant) that's OK too04:28
Tenguhave to wait for the post to deliver ;).04:29
Tengubetween 7:30am - 1pm. Maybe even later -.-'04:29
Tengutmate? what's this? Didn't heard about that tool04:29
Tenguaahhh. I think I actually did hear about it04:30
Tenguyep.04:30
jaosoriorowalsh_: so... folks didn't like the fact that CloudName is now required https://review.openstack.org/#/c/568899/304:32
*** holser__ has joined #tripleo04:33
*** fpan has joined #tripleo04:34
*** udesale has quit IRC04:36
*** udesale has joined #tripleo04:37
*** ykarel|away has joined #tripleo04:40
*** gvrangan has quit IRC04:44
*** agopi has joined #tripleo04:44
*** cmyster has joined #tripleo04:45
*** cmyster has joined #tripleo04:45
openstackgerritMerged openstack/puppet-tripleo master: Neutron wrappers: lookup for THT parameter  https://review.openstack.org/56673704:46
*** holser__ has quit IRC04:48
*** ratailor has joined #tripleo05:01
*** yprokule has joined #tripleo05:05
*** marios has joined #tripleo05:05
*** radeks has joined #tripleo05:07
*** saneax-_-|AFK is now known as saneax05:09
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097205:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154905:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163405:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169205:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)05:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)05:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)05:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]05:10
*** slaweq has joined #tripleo05:11
*** bkopilov has joined #tripleo05:13
*** slaweq has quit IRC05:15
*** udesale has quit IRC05:20
*** udesale has joined #tripleo05:21
*** dparkes has quit IRC05:23
openstackgerritMerged openstack/tripleo-common stable/pike: Add in a STOPSIGNAL configuration  https://review.openstack.org/56555205:24
openstackgerritMerged openstack/tripleo-common stable/pike: Add yum update to base  https://review.openstack.org/56829205:25
*** udesale has quit IRC05:26
*** masco has quit IRC05:26
*** cshastri has joined #tripleo05:33
*** jtomasek has quit IRC05:36
*** masco has joined #tripleo05:38
*** paramite_ has joined #tripleo05:42
openstackgerritMerged openstack/tripleo-common stable/pike: Run yum clean to reduce size of docker image layer  https://review.openstack.org/56851005:42
*** udesale has joined #tripleo05:43
*** agurenko has joined #tripleo05:44
*** ssbarnea_ has joined #tripleo05:47
*** slaweq has joined #tripleo05:47
*** aufi has joined #tripleo05:47
*** quiquell|off is now known as quiquell05:49
*** slaweq has quit IRC05:51
*** agopi has quit IRC05:54
*** ykarel|away is now known as ykarel05:56
*** rlandy|bbl has quit IRC05:56
*** aufi has quit IRC06:00
*** jfrancoa has joined #tripleo06:04
*** olap has joined #tripleo06:10
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097206:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154906:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163406:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169206:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)06:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)06:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)06:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]06:10
*** slaweq has joined #tripleo06:11
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/56803806:12
openstackgerritNuman Siddique proposed openstack/tripleo-quickstart master: Enable OVN DB HA profile in fset30  https://review.openstack.org/52380406:13
*** olap has quit IRC06:14
*** jaganathan has joined #tripleo06:15
*** slaweq has quit IRC06:16
*** dparkes has joined #tripleo06:16
*** marios has quit IRC06:16
*** marios has joined #tripleo06:17
*** dxiri has joined #tripleo06:17
linkscan we used {{role}} in resource_registry yaml for 'OS::TripleO::<Role-Name>::NodeUserData:' ? https://review.openstack.org/#/c/408374/ . I want to reference some parameters created specific to a role.06:21
*** dxiri has quit IRC06:22
*** ssbarnea_ has quit IRC06:27
*** psahoo has quit IRC06:28
*** jaganathan has quit IRC06:28
EmilienMjaosorior: https://review.openstack.org/#/c/568997/ is also ready, I just finished to manually test it (the CI job is not actually testing it, it's a bug and we're on it)06:29
* EmilienM off to bed06:30
*** jaganathan has joined #tripleo06:31
*** nyechiel_ has joined #tripleo06:34
*** psahoo has joined #tripleo06:35
*** holser__ has joined #tripleo06:36
jaosoriorEmilienM: done06:39
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use rabbitmq service principal in metadata settings  https://review.openstack.org/56903306:39
*** dxiri has joined #tripleo06:48
*** mdnadeem has joined #tripleo06:48
Tenguoh. btw, jaosorior, why use rabbitmq instead of activeMQ, especially since all python lib in use are referencing activeMQ.. ?06:50
Tengu(of course, if you know the answer, it's probably an old choice from upstream :))06:50
jaosoriorold choice indeed06:51
TenguI kind of hate rabbitmq -.-. it was always the source of issues in my previous job....06:51
*** dxiri has quit IRC06:53
*** lvdombrkr has joined #tripleo06:57
*** ffiore has joined #tripleo07:00
*** tesseract has joined #tripleo07:01
*** shardy has joined #tripleo07:02
bandiniTengu: well there are a ton of AMQ brokers. initial work to decouple rabbitmq from tripleo has landed for rocky (courtesy of ansmith)07:03
*** ccamacho has joined #tripleo07:03
Tengu\o/07:03
*** kopecmartin has joined #tripleo07:06
*** slaweq has joined #tripleo07:08
*** bogdando has joined #tripleo07:09
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097207:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154907:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163407:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169207:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)07:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)07:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]07:10
chandankumarjaosorior: Hello07:11
chandankumarjaosorior: I need some help on tempest logging07:12
chandankumarjaosorior: https://review.openstack.org/566364 -> enables logging for tempest07:12
chandankumarjaosorior: https://review.openstack.org/566526 -> does the same in TQE07:12
chandankumarjaosorior: https://review.openstack.org/566332 -> here it is getting tested  but logging is not getting enabled07:13
*** ramishra has quit IRC07:13
*** florianf has joined #tripleo07:14
chandankumarjaosorior: please have a look07:14
*** zoli is now known as zoli|wfh07:14
*** zoli|wfh is now known as zoli07:15
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Mount /dev for chrooted environment  https://review.openstack.org/56883807:15
*** sshnaidm|off is now known as sshnaidm|rover07:15
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates master: Enable logging for tempest container  https://review.openstack.org/56636407:20
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Build image in every OVB job  https://review.openstack.org/56825807:20
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Fix certificate ownership of rabbitmq  https://review.openstack.org/56904107:20
Tengubandini: any doc for that new feature?07:21
bandinisec07:22
bandiniTengu: https://github.com/openstack/tripleo-specs/blob/master/specs/queens/tripleo-messaging.rst07:22
Tenguthank you!07:23
bandininp ;)07:23
jaosoriorbandini: could you check this out https://review.openstack.org/#/c/569033/ ?07:25
*** ramishra has joined #tripleo07:25
Tengujaosorior: wouldn't "rabbitmq_<type>" be best ?07:26
Tengulike rabbitmq_rpc and rabbitmq_notify07:27
jaosoriorTengu: so, this is a quick fix to get it working again (might need to backport it too)07:29
jaosoriorcurrently we're requesting certificates for rabbitmq (that's done in puppet)07:29
jaosoriorultimately what I wanna do is to get a more generic resource for that07:29
Tenguok :)07:29
jaosoriorinstead of having to write a manifest per-service07:29
*** leitan has joined #tripleo07:30
jaosoriorbandini: currently the rabbitmq alternative is not using TLS, is it?07:31
bandinijaosorior: we'd have to check with ansmith, I know qrouter (for rpc) does support tls. but not sure if we already configure it07:31
bandinijaosorior: let me shoot an email to him and put you in CC07:31
bandinijust so we don't lose track of this07:32
jaosoriorthat would be great07:32
jaosoriorthanks07:32
jaosoriorbandini: is there support for that in OSP13?07:32
bandinino, not yet. it landed for rocky upstream but not sure when (if) we will support it07:32
*** dxiri has joined #tripleo07:32
jaosoriorok07:32
*** ramishra has quit IRC07:34
*** ssbarnea_ has joined #tripleo07:34
chandankumarjaosorior: I am making it more configurable07:34
chandankumarhttps://review.openstack.org/#/c/566526/07:34
*** olap has joined #tripleo07:35
openstackgerritHarald Jensås proposed openstack/puppet-tripleo master: Fix duplicate entries in /etc/sysconfig/iptables  https://review.openstack.org/56835107:35
*** olap has quit IRC07:35
openstackgerritHarald Jensås proposed openstack/instack-undercloud master: Fix duplicate entries in /etc/sysconfig/iptables  https://review.openstack.org/56835207:35
*** olap has joined #tripleo07:35
*** ramishra has joined #tripleo07:36
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Enable tempest.log in /var/log/tempest for container  https://review.openstack.org/56652607:37
chandankumarjaosorior: sshnaidm|rover I am a big idiot I need prize for that07:37
chandankumarone typo container/containers07:37
openstackgerritNir Magnezi proposed openstack/tripleo-common master: Make lb-mgmt-subnet a class B subnet  https://review.openstack.org/56808907:37
*** dxiri has quit IRC07:37
chandankumarin validate-tempest role that's why logging is not happening07:37
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Enable tempest.log in /var/log/tempest for container  https://review.openstack.org/56652607:38
jaosoriorTengu: are you familiar with ldap setups?07:41
Tengujaosorior: errr... more or less?07:42
Tengujaosorior: details, I might be of some use on that subject, but I'm not fluent with ldap.07:42
jaosorioralright07:43
jaosoriorno biggie07:43
*** jpena|off is now known as jpena07:46
Tengujaosorior: still, you may ask :).07:50
* Tengu pops out of the Deep Dive videos07:51
Tengupretty intersting, it's heavy stuff :).07:51
*** amoralej|off is now known as amoralej07:54
*** tosky has joined #tripleo07:55
jaosoriorTengu: just getting a pretty weird error and trying to debug it07:56
jaosoriorbogdando: are you around?07:56
*** cylopez has joined #tripleo07:56
openstackgerritwangqi proposed openstack/tripleo-common-tempest-plugin master: Update links in README  https://review.openstack.org/54910707:57
*** lvdombrkr89 has joined #tripleo07:58
*** radeks_ has joined #tripleo07:58
bogdandojaosorior: hi07:59
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044508:00
*** lvdombrkr has quit IRC08:00
jaosoriorbogdando: how's your ldap knowledge? :D08:00
Tengujaosorior: well, share your ldap error ;)08:01
bogdandojaosorior: 0Km not ok, but rather https://en.wikipedia.org/wiki/Absolute_zero08:01
bogdandoso it's 0K :D08:02
jaosoriorTengu: http://paste.openstack.org/show/721145/08:02
Tengujaosorior: TLS in place? known CA by the ldap server? anything in ldap logs itself?08:03
jaosoriorTengu: trusted CA08:04
*** ykarel is now known as ykarel|lunch08:05
*** skramaja has joined #tripleo08:05
Tengujaosorior: there are some issues on the Net with that same error - I guess you already checked them?08:05
Tengufor instance: https://ask.openstack.org/en/question/30232/keystone-an-unexpected-error-prevented-the-server-from-fulfilling-your-request-http-500/08:05
jaosoriorTengu: so, I can actually do ldapsearch from that host08:05
bogdandoPTAL folks https://review.openstack.org/#/c/568837/08:05
Tengujaosorior: ok. and do you have any ldap logs?08:06
jaosoriorTengu: trying to find them08:06
Tengumight be in jourland08:06
shardyyeah I think you need the server side of the 500 error to debug this?08:06
Tengushardy: oh, hello :). I just watched your Deep Dive about heat in tripleo. Nice job, really.08:07
jaosoriorTengu: nothing from journald08:08
Tenguo_O nothing for "slapd" ?08:08
Tenguor some /var/log/slapd directory maybe?08:09
jaosoriorTengu: found it08:09
jaosorior/var/log/dirsrv08:09
Tenguah, right.08:09
TengufreeIPA then right?08:10
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097208:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154908:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163408:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169208:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)08:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)08:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]08:10
jaosoriorTengu: got it08:11
jaosoriorit was actually a trust issue08:11
Tengu\o/08:11
jaosorior[17/May/2018:11:11:07.295261379 +0300] conn=510 op=-1 fd=126 closed - Peer does not recognize and trust the CA that issued your certificate.08:11
Tenguwith ldap, it's almost always a trust issue.08:12
*** rcernin has quit IRC08:12
Tenguare you doing the ldapsearch over TLS in your test?08:12
jaosoriorTengu: which I don't understand, cause we do mount most of the CA certs in /etc/pki/ca-trust08:12
jaosoriorTengu: I did it from the host, not the container08:12
Tengu"most" - maybe the same issue as I had with keystone<>freeipa?08:12
jaosoriorTengu: in my case, LDAP is FreeIPA, and all my hosts are enrolled08:12
Tenguyou know, the missing anchor.08:12
jaosoriorTengu: didn't that anchor get added already?08:13
Tengushould be yep.08:13
jaosoriorTengu: this is from master08:13
jaosoriorso it should be there08:13
jaosoriorTengu: the funky thing is that I've always tested LDAP with FreeIPA, so I was sure we had the right anchor :/08:13
Tengujaosorior: hm, but you have the error on the server side in fact.08:14
Tenguso it's more client not sending the right thing at some point.08:14
Tenguas nodes are enroled, you might want to check the content of /etc/ipa or something like that, there might be some certificate in there you'll need in the container as well.08:14
jaosorioruhm... would LDAP be requiring mutual TLS now?08:14
Tengumight be for enroled nodes?08:14
Tenguyou might want to check the ldap.conf, probably in /etc/ldap directory08:15
Tenguon the client08:15
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations stable/queens: Fix docker img filter in rabbitmq validation  https://review.openstack.org/56905408:15
jaosoriorTengu: there's no such directory neither on the host nor the container :/08:15
*** jtomasek has joined #tripleo08:16
Tengujaosorior: hmm. there should be something. Maybe /etc/ldap.conf directly?08:16
Tengunot sure, I don't have anything at hand to check that :(08:16
Tengupost delivery sucks lately. Guess having to move about 20kg of randome material split in 2 packages isn't cool enough for the postman :D08:17
Tengualso, s/ramdome/radom/08:17
bogdandojaosorior: http://lists.openstack.org/pipermail/openstack-dev/2018-May/130601.html08:18
bogdandoWDYT?08:18
jaosoriorbogdando: I've actually brought up using Vault before08:19
shardymandre: Hey, good morning - why you get time please can you review https://review.openstack.org/#/c/568603/ ?08:19
jaosoriorwe're trying to get approval to take it into use (actually it was for another use-case)08:19
Tengubogdando: seems lovely :).08:19
bogdandoindeed08:19
shardymandre: it's decoupling the inner inventory for openshift so the role specific parameters can be used to set OpenShiftGlusterDisks per-role08:20
bogdandoTLS and everything encrypted and pink ponies all around!08:20
Tenguhey, that would also help me for the infamous certodia thingy.08:20
mandregood morning shardy, adding it to my review list08:20
shardymandre: thanks!08:20
bogdandooh, I forgot selinux enforcing08:20
jaosoriorbogdando: it would sure be nice :)08:20
*** dtantsur|afk is now known as dtantsur08:26
*** leitan has quit IRC08:27
*** cshastri has quit IRC08:27
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates stable/queens: FFU Add cinder-backup missing fast_forward_upgrade_tasks  https://review.openstack.org/56858408:28
openstackgerritMerged openstack/tripleo-heat-templates master: Add missing UndercloudUpgrade to environment  https://review.openstack.org/56893308:29
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Convert nic templates to new format before starting fast forward  https://review.openstack.org/56829308:29
*** etingof|afk is now known as etingof08:30
sshnaidm|roverjaosorior, hi, did you see such errors maybe? SSL exception connecting to https://192.168.24.2:13000/v3/auth/tokens: hostname '192.168.24.2' doesn't match '192.168.24.2' (tried 44, for a total of 170 seconds)08:30
sshnaidm|roverjaosorior, it's from failing undercloud in ocaa job08:31
sshnaidm|roverjaosorior, https://logs.rdoproject.org/58/568258/4/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata-branch/Z4ee27e8656254777b535a02afeab8dbb/undercloud/home/jenkins/undercloud_install.log.txt.gz08:31
jaosoriorsshnaidm|rover: I'll check it out08:32
*** links has quit IRC08:32
*** gfidente has joined #tripleo08:33
*** gfidente has quit IRC08:33
*** gfidente has joined #tripleo08:33
jaosoriorsshnaidm|rover: what is ocaa ?08:33
jaosorioraaaah ocata08:33
*** gyankum has quit IRC08:33
sshnaidm|roverjaosorior, yeah :)08:34
*** radeks has quit IRC08:34
jaosoriorsshnaidm|rover: I think the issue is how the cert was generated08:34
jaosoriorit's using an IP in the DNS entry of the subjectAltName, while it should be IP instead08:34
jaosoriorthat was fixed in a subsequent version of puppet-certmonger; I'm guessing this is using the older one08:34
sshnaidm|roverjaosorior, hmm.. afaik it's generated in oooq in the same way for all branches08:35
sshnaidm|roverjaosorior, oh, I see08:35
jaosorioralso, python-requests changed the way they verified certs to actually take IPs into account in the subjectaltname08:35
*** gkadam has joined #tripleo08:35
jaosoriorthat's why it didn't fail before08:35
sshnaidm|roverjaosorior, so we need to update version of puppet-certmonger?08:36
*** ykarel|lunch is now known as ykarel08:37
*** jtomasek has quit IRC08:37
jaosorioryes08:37
*** mdnadeem has quit IRC08:37
sshnaidm|roverjaosorior, https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-ssl/templates/overcloud-create-ssl-cert.sh.j2#L5608:38
sshnaidm|roverjaosorior, seems like we use IP in subjectaltname08:38
*** nyechiel has joined #tripleo08:38
*** nyechiel_ has quit IRC08:38
jaosoriorsshnaidm|rover: that's for the overcloud08:39
jaosoriorsshnaidm|rover: the undercloud uses certmonger to get the cert08:39
jaosoriorand in the initial implementation I fucked up that part :D08:39
sshnaidm|roverjaosorior, ah, right08:39
*** jtomasek has joined #tripleo08:39
*** mdnadeem has joined #tripleo08:40
*** wolverineav has joined #tripleo08:40
*** nyechiel_ has joined #tripleo08:45
*** nyechiel has quit IRC08:45
*** jcoufal has joined #tripleo08:45
*** cshastri has joined #tripleo08:47
*** owalsh_ is now known as owalsh08:48
*** links has joined #tripleo08:48
*** jcoufal has quit IRC08:50
*** d0ugal__ has quit IRC08:51
*** d0ugal has joined #tripleo08:51
*** d0ugal has quit IRC08:51
*** d0ugal has joined #tripleo08:51
*** derekh has joined #tripleo08:51
*** akrivoka has joined #tripleo08:57
*** salmankhan has joined #tripleo09:02
*** dxiri has joined #tripleo09:03
*** gyankum has joined #tripleo09:04
*** dxiri has quit IRC09:07
*** wolverineav has quit IRC09:08
*** wolverineav has joined #tripleo09:08
*** gyankum has quit IRC09:08
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097209:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154909:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163409:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169209:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)09:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)09:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)09:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]09:10
derekhsshnaidm|rover: so the yum fix didn't work?09:12
sshnaidm|roverderekh, well, it solved problem with updating images, but not for introspection09:12
*** wolverineav has quit IRC09:13
derekhssbarnea: ok, I'll keep seeing if I can narrow it down here09:13
sshnaidm|roverssbarnea, think about changing nickname :D09:27
*** rwsu has quit IRC09:28
*** zoli is now known as zoli|lunch09:28
derekhsshnaidm|rover: the image update process runs {{ modify_script }} on the image, where is {{ modify_script }} ?09:29
ssbarneasshnaidm|rover: haha, impossible, used it almost everywhere, sooner or later it was going to happen :p ... funny to have a nickname starting with "ssh" :)09:29
derekhssbarnea: opps, sorry09:30
derekhwe need a bigger alphabet09:30
owalshjaosorior: commented... does it make sense? I've forgotten all the gory details.... but I think that's what we were thinking at the time09:30
*** tvignaud has quit IRC09:31
sshnaidm|roverderekh, it's repo-setup.sh09:32
derekhsshnaidm|rover: ack thanks09:33
sshnaidm|roverderekh, https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/tasks/inject_repos_into_image.yml#L609:34
derekhok09:35
Tenguis there a proper way to deactivate a tripleo thing, like lbaasv2? Apparently, removed its definition/install from the env file doesn't drop it...09:35
*** cylopez has left #tripleo09:40
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Fix generation of docsDocs for tempest are not being generated properlyDepends-On: https://review.openstack.org/#/c/566527/Change-Id: I0cc8d0487b144026a6dadd4345bcc97800fbdcfc  https://review.openstack.org/56907609:44
*** dhill_ has quit IRC09:44
*** gyankum has joined #tripleo09:46
*** dxiri has joined #tripleo09:48
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Verify the Swift container exists with a small utility workflow  https://review.openstack.org/52821309:49
*** moshele has joined #tripleo09:50
*** gyankum has quit IRC09:50
*** sshnaidm|rover has quit IRC09:52
*** mdnadeem has quit IRC09:52
*** dxiri has quit IRC09:53
openstackgerritRoman Gorshunov proposed openstack/diskimage-builder master: Remove non-maintained ubuntu-core element  https://review.openstack.org/56907709:53
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Fix generation of docs Docs for tempest are not being generated properly  https://review.openstack.org/56878309:53
jaosoriorowalsh: thanks!09:54
ratailorjaosorior, jistr, slagle  could you pl review https://review.openstack.org/#/c/564179/09:54
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Fix generation of docs  https://review.openstack.org/56878309:54
*** moshele has quit IRC09:54
*** sshnaidm|rover has joined #tripleo09:55
jaosoriorgerrit isn't working for me :/09:55
*** gyankum has joined #tripleo09:56
jaosoriorbogdando: could you take a look at this https://bugs.launchpad.net/tripleo/+bug/1771543 ?09:56
openstackLaunchpad bug 1771543 in tripleo "Allow logrotate configuration to control log retention based on age " [Critical,Triaged]09:56
*** ssbarnea_ has quit IRC09:59
*** pkovar has joined #tripleo10:01
*** ssbarnea_ has joined #tripleo10:02
*** ssbarnea_ has quit IRC10:03
*** dbellant has joined #tripleo10:05
*** dbellant has quit IRC10:05
*** dbellant has joined #tripleo10:05
*** salmankhan has quit IRC10:05
*** panda|off is now known as panda10:07
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097210:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154910:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163410:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169210:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)10:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)10:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)10:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]10:10
*** gfidente has quit IRC10:10
*** ssbarnea_ has joined #tripleo10:14
*** salmankhan has joined #tripleo10:18
*** kopecmartin is now known as mkopec|lunch10:22
*** ktibi has joined #tripleo10:30
*** dxiri has joined #tripleo10:33
ratailorcould you pl review https://review.openstack.org/#/c/568832/10:36
ratailorjaosorior, jistr, RFE required in OSP1410:36
ratailor^^10:36
*** dxiri has quit IRC10:37
* jistr not core on puppet-* except puppet-tripleo10:38
jistr+2+A for the nova nfs backend at least :)10:38
jaosoriorratailor: gerrit isn't working for me still :(10:38
ratailorjaosorior, np :-)10:39
ratailorjistr, Thanks!10:39
*** gfidente has joined #tripleo10:40
*** gfidente has quit IRC10:40
*** gfidente has joined #tripleo10:40
jaosoriorjistr: can't pass you the link cause gerrit isn't working for me. But could you review some TLS patches I posted?10:41
jaosoriorjistr: one is a revert of a revert, the other two are fixes for tLS everywhere10:41
*** ktibi_ has joined #tripleo10:42
jaosoriorbogdando: I just did a deployment... and I don't see where logrotate would clean the container logs10:43
Tengualmost have my build machin. There's only one thing missing in the delivery. Nothing important: just the RAM. fuuuuuuu10:44
Tenguanyway. Will make room for it and start building it.10:44
Tengubuild-ception10:44
jaosoriorbogdando: nevermind10:44
jaosoriorfound it10:44
jaosoriorlol10:45
*** ktibi has quit IRC10:45
*** ktibi has joined #tripleo10:46
*** ktibi_ has quit IRC10:46
*** jcoufal has joined #tripleo10:46
*** suuuper has joined #tripleo10:48
*** jcoufal has quit IRC10:51
*** rwsu has joined #tripleo10:51
*** shreshtha has quit IRC10:51
*** bkopilov has quit IRC10:54
*** jpena is now known as jpena|lunch10:59
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Add method for getting dlrn_hash from release and hash_name  https://review.openstack.org/56732011:00
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Add releases script pytest tests to tox.ini  https://review.openstack.org/56764911:00
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: [WIP] Add CLI argument parser and YAML file parser  https://review.openstack.org/56793611:00
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: [WIP] Implement --output-file to write the bash script  https://review.openstack.org/56828511:00
*** jaosorior has quit IRC11:01
*** ansmith has quit IRC11:04
*** jaosorior has joined #tripleo11:05
jaosoriorwtf, for a very strange reason I can access gerrit from my phone, but not from my laptop11:05
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097211:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154911:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163411:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169211:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)11:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)11:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)11:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]11:10
openstackgerritPranali Deore proposed openstack/tripleo-heat-templates master: Add support of mounting stating location for glance-direct  https://review.openstack.org/56293011:14
*** gyan_ has joined #tripleo11:14
jistrjaosorior: ok will take a look, was also wondering if we should have the call about the upgrade CI? turns out the TLS->TLS upgrade fails on the initial deployment, too11:14
*** jcoufal has joined #tripleo11:15
jistrjaosorior: i think we're missing call to the overcloud-ssl role in the multinode playbook maybe. Have we had any CI for `ssl_overcloud: true` before we made it default?11:15
*** gyankum has quit IRC11:16
*** cshastri has quit IRC11:16
jistrbecause if not, i'm starting to think that the TLS->TLS upgrade job might not be the simplest way after all... if it would mean that it's our *first* multinode TLS job running on Queens, and it's an upgrade job at the same time :D11:16
*** links has quit IRC11:16
*** pkovar has quit IRC11:16
jistr2 complex things together...11:17
*** zoli|lunch is now known as zoli|wfh11:17
*** zoli|wfh is now known as zoli11:17
jistrjfrancoa: thanks for the comment @ https://review.openstack.org/#/c/568843/1/config/general_config/featureset051.yml11:19
*** pkovar has joined #tripleo11:19
jistrjfrancoa: the role which is supposed to generate those files is simply not running it seems. This one https://github.com/openstack/tripleo-quickstart-extras/tree/master/roles/overcloud-ssl11:19
derekhsshnaidm|rover: so I'm trying to update the image in the same way that quickstart does to narrow down the problem, any idea what I'm missing ? http://paste.openstack.org/show/721157/11:20
*** mdnadeem has joined #tripleo11:20
derekhsshnaidm|rover: images updated with ^^ work fine, but updated by quickstart don't11:20
*** jcoufal has quit IRC11:20
jistrjfrancoa: it's not in the multinode playbooks at all, which makes me think as i wrote above, if the upgrade job would be the first TLS job for Queens, then maybe we'd rather go the non-TLS way. Trying to make work both TLS and Upgrades for the first time in one job -- not a great idea IMO.11:21
*** mkopec|lunch is now known as mkopec11:21
*** mkopec has quit IRC11:21
*** kopecmartin has joined #tripleo11:22
*** aufi has joined #tripleo11:22
jistrjfrancoa: do you know if fs042 and fs035 map to actual jobs that are running & passing on stable/queens?11:23
jaosoriorjistr: I think it's missing the tag11:23
*** rh-jelabarre has joined #tripleo11:24
*** tvignaud has joined #tripleo11:24
jistrjaosorior: i don't see it anywhere in the playbook -- https://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/multinode.yml11:25
slaglejaosorior: for https://review.openstack.org/#/c/568899/, how is the fqdn supposed to get resolved?11:26
*** psachin has quit IRC11:28
sshnaidm|roverderekh,  and this one (without update)? https://images.rdoproject.org/master/rdo_trunk/current-tripleo/11:28
*** links has joined #tripleo11:29
*** psahoo has quit IRC11:30
*** ssbarnea_ has quit IRC11:31
*** cshastri has joined #tripleo11:32
sshnaidm|roverderekh, maybe cpio work different... I don't know what to suspect already after this chroot issue11:32
chandankumarsshnaidm|rover: weshay here is the tempest container logging https://review.openstack.org/#/q/topic:tempest_log+(status:open+OR+status:merged)11:32
chandankumarweshay: sshnaidm|rover http://logs.openstack.org/32/566332/10/check/tripleo-ci-centos-7-undercloud-containers/b477abd/logs/undercloud/var/log/containers/tempest/tempest.log.txt.gz11:33
derekhsshnaidm|rover: more info added, I got onto a node while introspection is failing https://bugs.launchpad.net/tripleo/+bug/177097211:33
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)11:33
sshnaidm|roverchandankumar, nice!11:33
chandankumarsshnaidm|rover: rdo jobs failing11:34
*** abishop has joined #tripleo11:36
jaosoriorslagle: at some point, folks do end up configuring DNS if they want they're users to take the cloud into use.11:36
*** wolverineav has joined #tripleo11:36
jaosoriorslagle: what I propose is to return the default to what it was, and create a warning via validations. Would that be a good middle-ground?11:37
*** psachin has joined #tripleo11:38
sshnaidm|roverderekh, is root password in /httpboot/inspector.ipxe? I don't see..11:38
*** bfournie has quit IRC11:38
slaglejaosorior: there's no guaranteed "at some point"11:39
derekhsshnaidm|rover: you add it as a kernel param, https://docs.openstack.org/ironic-python-agent/pike/admin/troubleshooting.html#id811:39
slaglejaosorior: we've never required dns before. we can't just flip the switch on that11:39
jaosoriorslagle: sure, understood11:39
slaglejaosorior: anyone trying to deploy tripleo follow our docs, can't.11:40
*** moshele has joined #tripleo11:40
slaglejaosorior: also, services that try and talk to these endpoints to do configuration (octavia) are broken because they can't resolve the hostname11:40
openstackgerritRaoul Scarazzini proposed openstack/tripleo-ha-utils master: Add pre checks before installing instance HA  https://review.openstack.org/56789211:40
jaosoriorslagle: don't we persist the hostname for every overcloud node in /etc/hosts?11:41
slaglejaosorior: yes, on the overcloud. not on the undercloud11:41
slagleoctavia runs config from the undercloud11:42
jaosoriorslagle: ok11:42
*** nyechiel_ has quit IRC11:42
*** ssbarnea_ has joined #tripleo11:42
jaosoriorslagle: so, what about the alternative I proposed? would that be an acceptable middleground?11:42
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Run undeploy_plan workflow to delete deployment  https://review.openstack.org/56636611:42
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Add Recover deployment status button and actions  https://review.openstack.org/56909211:42
slaglejaosorior: just adding the default back for CloudName? that only fixes part of the problem, you still can't resolve the hostname11:43
openstackgerritRaoul Scarazzini proposed openstack/tripleo-ha-utils master: Make instance HA uninstallation idempotent  https://review.openstack.org/56829611:43
slaglejaosorior: if CloudName was the only issue, i'd be fine with a fix for that. but i think the bigger issue is the hostname resolution11:43
jaosoriorslagle: I see11:43
jaosoriorslagle: well, then we can't have TLS by default11:44
*** ssbarnea_ has quit IRC11:44
jaosorioreither we require DNS, require predictable IPs, or we don't have TLS11:44
slaglejaosorior: in it's current form, that's correct. it needs more work to work "out of the box"11:44
*** ssbarnea_ has joined #tripleo11:44
jaosoriorwhat more work?11:44
slaglejaosorior: something to make sure hostname resolution works11:45
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Add Recover deployment status button and actions  https://review.openstack.org/56909211:45
jaosoriorslagle: the only solution I can think of is to trigger a mistral action that persists the host entries in the undercloud's /etc/hosts11:46
slaglejaosorior: that's part of the reason why I think we should revert to properly evaluate the options11:47
*** amoralej is now known as amoralej|lunch11:47
*** gfidente is now known as gfidente|pto11:47
*** leanderthal has joined #tripleo11:48
*** shreshtha has joined #tripleo11:48
openstackgerritJiri Stransky proposed openstack/tripleo-upgrade master: Prevent trying to enable TLS when upgrading  https://review.openstack.org/56909311:48
*** dxiri has joined #tripleo11:48
jaosoriorI see11:50
jaosoriorOK, I'll explore another way to get it to work11:50
jaosoriorslagle: I can't remove the -1 cause gerrit isn't working for me. I can't even pull code. I'll do that once I get access again.11:50
honzajtomasek: should we move the recover workflow from deployment.yaml to plan_management.yaml?11:53
*** udesale has quit IRC11:53
*** dxiri has quit IRC11:53
jtomasekhonza: I don't think so, IMHO anything related to deployment should reside in deployment.yaml11:54
jtomasekplan_management.yaml is already quite crowded11:54
honzajtomasek: k, cool11:54
openstackgerritHonza Pokorny proposed openstack/tripleo-common master: Add recover_deployment workflow  https://review.openstack.org/56669911:54
openstackgerritDerek Higgins proposed openstack/tripleo-quickstart-extras master: Nothing to see here  https://review.openstack.org/51736811:55
jfrancoajistr: well, I didn't check it in deep if they're being in use. let me do a quick search11:55
beaglesmatbu: will there be an upgrades squad meeting tomorrow?11:55
*** dprince has joined #tripleo11:56
jaosoriorjistr: an option with the TLS update is the EnablePublicTLS flag11:56
jaosoriorwe set that to true by default11:56
openstackgerritMarios Andreou proposed openstack/tripleo-docs master: WIP: Add docs for Q upgrade workflow  https://review.openstack.org/53585911:56
jaosoriorand it reads a true by default in tripleo-common11:56
jaosoriorchanging that to false (since it's not present in queens) should do the trick11:56
jfrancoajistr: so that was what I was missing, the lack of overcloud-ssl role in the job. Because for more that I searched I couldn't find the place where enable-tls.yaml was being set in the working_dir11:56
jaosoriorjistr: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/deployment.py#L27711:57
matbubeagles: yep11:58
jistrjaosorior: ack, i'll try a WIP patch for sc000, we might need to edit it elsewhere (some version-specific tweak in Quickstart i guess) but for testing a default param in scenario should be good11:58
beaglesmatbu: I'd like to join, what time is it scheduled for?11:58
matbubeagles: you can add stuffs in the agenda if you want11:58
beaglesmatbu: ack11:58
matbubeagles: 1:30pm utc11:59
beaglesmatbu cool thanks11:59
matbubeagles: in 1h30 as now11:59
beaglesmatbu: ack12:00
beaglesI'll modify etherpad to reflect same if that's okay :)12:00
beaglesI got burned by that last week12:00
matbusure :)12:00
jistrjaosorior: noob question -- TLS by default we mean only public tls by default, right? internal TLS is not the default, right?12:01
beaglesmatbu: cool thanks!12:01
jaosoriorjistr: correct12:01
jistrthanks12:01
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add CLI argument parser and YAML file parser  https://review.openstack.org/56793612:01
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Deploy scenario000 without public TLS  https://review.openstack.org/56909612:02
*** trown|outtypewww is now known as trown12:03
*** pkovar has quit IRC12:03
*** tcw has quit IRC12:03
*** pkovar has joined #tripleo12:04
*** ktibi has quit IRC12:05
*** wolverineav has quit IRC12:06
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci master: Add keystone only upgrade jobs  https://review.openstack.org/55842512:06
*** wolverineav has joined #tripleo12:06
jistrjaosorior, jfrancoa: ^ two (not ready to merge) patches in depends-on, we'll see if it gets us further12:07
*** jcoufal has joined #tripleo12:07
*** liverpooler has joined #tripleo12:08
*** tcw has joined #tripleo12:09
*** quiquell is now known as quique|lunch12:09
*** rbowen has joined #tripleo12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097212:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154912:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163412:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169212:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)12:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)12:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)12:10
*** moshele has quit IRC12:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]12:10
*** jaosorior has quit IRC12:12
*** nyechiel_ has joined #tripleo12:16
Tenguback. pfiouuu12:17
openstackgerritPranali Deore proposed openstack/tripleo-heat-templates master: Add support of mounting stating location for glance-direct  https://review.openstack.org/56293012:17
Tenguit's a monster :). A pity I didn't get all the components, it would really be cool for quickstart testing/breaking/others :)12:17
*** thrash|g0ne is now known as thrash12:18
*** atoth has joined #tripleo12:19
*** jaosorior has joined #tripleo12:19
*** bfournie has joined #tripleo12:21
*** moshele has joined #tripleo12:24
*** jpena|lunch is now known as jpena12:24
*** tzumainn has joined #tripleo12:26
*** bkopilov has joined #tripleo12:26
weshaythanks chandankumar12:29
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add CLI argument parser and YAML file parser  https://review.openstack.org/56793612:31
*** raildo has joined #tripleo12:31
*** tosky has quit IRC12:32
*** mcornea has joined #tripleo12:32
*** mdnadeem has quit IRC12:33
*** pchavva has joined #tripleo12:33
*** rlandy has joined #tripleo12:33
*** tosky has joined #tripleo12:36
derekhmrunge: you mentioned that centos are putting a fix into the image for dbus, is there a bug with details somewhere?12:36
mrungederekh: uhm, the new centos image (based on centos 7.5) will be released somewhere in the future12:37
mrungederekh: in that way, you won't have to restart services after update12:38
derekhmrunge: ya, I'm just wondering if there was a specific problem with upgrading dbus, as we're see dbus problems during introspection in CI12:39
derekhwhich may be irrelevant12:40
*** tiswanso has quit IRC12:40
jtomasekhonza: are you also going to update on-complete to on-success here? https://review.openstack.org/#/c/566699/4/workbooks/deployment.yaml12:42
honzajtomasek: yes12:42
*** moshele has quit IRC12:43
jtomasekhonza: ok, thanks!12:44
jtomasekhonza: I am working on sending the initial network config patch(es)12:44
openstackgerritHonza Pokorny proposed openstack/tripleo-common master: Add recover_deployment workflow  https://review.openstack.org/56669912:45
honzajtomasek: done, sorry for the delay12:46
*** Goneri has joined #tripleo12:46
*** moshele has joined #tripleo12:48
*** quique|lunch is now known as quiquell12:48
*** ansmith has joined #tripleo12:50
mrungederekh: I somehow understood, the service just needs a restart after upgrade.12:50
derekhmrunge: ok, thats not likely to be our problem so as introspection nodes have freshly booted, thanks anyways12:51
mrungederekh: ah, right. I was confusing the introspection issue with the other one preventing ovb to deploy12:52
mrungederekh: do you rememember me asking you about ovb back ad PTG in DUB? I believe, that's the same issue12:53
bogdandofolks, any ideas for https://bugs.launchpad.net/tripleo/+bug/1771543/comments/1 ? jaosorior?12:53
openstackLaunchpad bug 1771543 in tripleo "Allow logrotate configuration to control log retention based on age " [Critical,Triaged]12:53
mrungeI've pasted extensinve logs12:53
bogdandoit seems not that simple as it looks like12:53
*** cylopez has joined #tripleo12:54
derekhmrunge: yes I do, I assumed you solved that as I never heard from you, once I get this introspection thing out of the way I'll take a look12:54
mrungederekh: the interesting thing is: I was using a different setup, which means, *two* IPA images on different clouds would have to get changed at the same time12:54
*** gyan_ has quit IRC12:54
mrungederekh: assuming, both are the same issue12:54
*** cylopez has left #tripleo12:55
mrungeI *think*, this is more a timing thing12:55
bogdandoI'd say the only solution is use FS types for logs storage which supports creation time well and shows it in stat -s %w12:55
derekhmrunge: ok, your error has Exceptions in ironic logs, correct? I'm not seeing anything like that here12:56
mrungederekh: ok, right. that's true and I shut up immediately. sorry for the noise then12:57
derekhmrunge: no need to shut up ;-), just saying I think we got 2 different problems12:58
sshnaidm|roverweshay, trown mwhahaha can you vote please? https://review.openstack.org/#/c/568838/12:59
mrungederekh: yes, I got that. Since you're working on the other issue, and mine is unrelated, I'm trying to keep you focused then ;-) Noise is just distracting12:59
mrunge;-)12:59
jaosoriorbogdando: reading the man of logrotate.conf13:00
jaosoriormight be something there13:00
derekhmrunge: ack, I'll circle back after13:00
openstackgerritStanislav Makar proposed openstack/diskimage-builder master: Add Ubuntu 18.04 support  https://review.openstack.org/56910313:00
jaosoriorbogdando: what about maxage?13:01
jaosoriorbogdando: if we expose maxage to folks, they could set it to 2 or 3 weeks, thus complying with gdpr13:01
*** olap has quit IRC13:02
*** olap has joined #tripleo13:02
bogdandojaosorior: The age is only checked if the logfile is to be rotated. Technically this allows some short files persist users' data for ever13:02
*** olap has quit IRC13:02
*** olap has joined #tripleo13:02
*** olap has quit IRC13:03
*** olap has joined #tripleo13:03
bogdandojaosorior: also, collisions of size based vs age based are mistery, it never works as expected and never had been properly documented13:04
*** moshele has quit IRC13:04
jaosoriorbogdando: right, in that case we would also need to expose the time-based rotation ones13:04
sshnaidm|roverderekh, btw, when running yum update in initramfs image it updates dhclient and dhcp packages: https://logs.rdoproject.org/38/568838/5/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/Z12badebeb6aa44fea1610bb56946c500/undercloud/home/jenkins/repo_setup.sh.1526542651.log.txt.gz#_2018-05-17_07_37_4813:04
jaosoriorbogdando: oh crap :/13:04
sshnaidm|roverderekh, did it update the same  in your tests? could it be a problem..?13:04
*** dhill_ has joined #tripleo13:04
*** amoralej|lunch is now known as amoralej13:05
bogdandonot showing create time of files is really not GDPR compatible :)13:06
bogdandolet's patch linux first? :D13:06
jaosoriorhaha13:07
derekhsshnaidm|rover: I'm not sure, I'm rebuilding my undercloud will check when its done13:07
bogdandoanother idea, let's just purge all logs except those tar files created by logrotate, jaosorior13:07
*** olap has quit IRC13:07
bogdandoin postrotate13:08
jaosorioruhm13:08
bogdandojust rm -rf /var/log13:08
jaosoriorbogdando: why not expose all those parameters via puppet, and let the deployer decide? that would be another option13:08
bogdandofrom logrotate containers its only clears /var/log/containers13:08
openstackgerritStanislav Makar proposed openstack/diskimage-builder master: Add Ubuntu 18.04 support  https://review.openstack.org/56910313:08
bogdandoand we do not care of other things on host , prolly13:08
*** links has quit IRC13:09
jaosoriorbogdando: well, we have the same issue for the journal13:09
jaosoriorbogdando: haproxy logs IPs to the journal13:09
bogdandoI have idea, let's just allow to put a whatever command into postroate13:09
bogdandoand pass in via tht13:09
bogdandorm -rf /var/log/ - a viable option to go with :D13:10
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097213:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154913:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163413:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169213:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)13:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)13:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]13:10
bogdandojaosorior: right, there is ovs, pacemaker, journald on host13:10
jaosoriorbogdando: yeah, we also need to come up with a solution for that :D13:11
bogdandobut we could deal with one disaster at a time13:11
jaosoriorprobably gonna do it via ansible though13:11
*** tiswanso has joined #tripleo13:11
jaosoriorjust configure journal to purge stuff after 2 weeks13:11
bogdandoyes, it has a config opt for that13:11
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart master: DNM: don't update dhcp packages with repo-setup  https://review.openstack.org/56910613:11
sshnaidm|roverderekh, worth to check ^13:11
bogdandobut there is still raw files, not binary one for journald13:11
*** tiswanso has quit IRC13:12
bogdandook, for puppet and tht containers logrotate I'd propose to do rm -rf from logrotate container, which only purges the host's /var/log/containers13:12
bogdandoand that's it13:12
*** tiswanso has joined #tripleo13:12
bogdandowell , excluding rotated tarballs13:13
derekhsshnaidm|rover: yup13:13
jaosoriorbogdando: itw ould only purge the rotated? or the new stuff?13:13
jaosoriorbogdando: anyway, why not expose the time based bits? and leave that off by default13:15
jaosoriorthen the deployer can choose if they want to use it or not13:15
*** alee has quit IRC13:15
*** nkinder has joined #tripleo13:16
owalshdtantsur: hey, do we expect the wol staging driver to work with tripleo?13:17
bogdandoplease read the update13:17
bogdandojaosorior:13:17
bogdandologrotate will take care of files it creates13:18
dtantsurowalsh: it's expected, but nobody is testing it13:18
jaosoriornkinder: this is the bug https://bugs.launchpad.net/tripleo/+bug/177154313:18
openstackLaunchpad bug 1771543 in tripleo "Allow logrotate configuration to control log retention based on age " [Critical,Triaged]13:18
bogdandoother files, to be purged on postscript13:18
jaosoriorbogdando: I agree with you, lets split the problem13:18
jaosoriorlets focus right now on /var/log/containers13:18
weshaymyoung|ruck, sshnaidm|rover fyi.. I voted and commented on https://review.openstack.org/#/c/56825813:18
bogdandook, will propose a patch in puppet and tht13:19
jaosoriorbogdando: but, wouldn't the compressed artifacts be older data that we also need to deal with?13:19
bogdandologrotate can do that13:19
jaosoriorbogdando: but we don't configure that currently13:19
owalshdtantsur: ok, get_supported_power_states doesn't include POWER_OFF, it should allow but just log that it can't... need a bug or just push a fix since nobody seems to be using it?13:19
bogdandoor another postscript command13:19
jaosoriorthat's why I want to expose the time-based options from logrotate13:20
bogdandobut technically just daily and roate 14 works as well13:20
jaosoriornot enable them by default, just expose them, so folks can take them into use if needed13:20
bogdandoit shifts out older compressed files13:20
jaosoriorI see13:20
bogdandoor weekly and rotate 213:20
jaosoriorso we would only keep logs for a maximum of 2 weeks13:20
jaosoriorOK, I dig that13:20
bogdandook13:21
bogdandopathces WIP13:21
dtantsurowalsh: power off is not possible with the wol driver (IIRC), so it's not a bug13:21
owalshdtantsur: either is REBOOT, but it's in the allowed list and just logged13:21
bogdandojaosorior: sight, we shall disable delaycompress then13:22
bogdandosigh13:22
bogdandoor cannot purge13:22
dtantsurowalsh: reboot is required by ironic13:22
jaosoriorzzzeek: I'm almost done for the day, should we schedule the meeting with bandini to talk about multi-site keystone for next week?13:22
jaosoriorbogdando: so, purge only works with uncompressed?13:22
owalshdtantsur: needed power off for introspection13:22
dtantsuroverall, the wol driver is not a real driver.. it really can only power on13:22
zzzeekjaosorior: we should, I haven't gotten to it yet b.c. i've been swamped w/ upstream work13:22
bogdandojaosorior: that purge is just a find .. -exec rm -f in postrotate13:23
myoung|rucksshnaidm|rover, weshay: I had thought we had RCA for https://bugs.launchpad.net/tripleo/+bug/1770972 as https://review.openstack.org/#/c/568838 ?13:23
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)13:23
jaosoriorzzzeek: don't worry, whenever you have time.13:23
owalshyea, I just need it to not raise an exception and log so I can push the button13:23
bogdandoit works like we say to it13:23
owalshdtantsur: ^^^13:23
bogdandobut then we need always compress files13:23
zzzeekjaosorior: i have an outside appointment in half an hour so I could do it more late AM here like 11:30 EDT13:23
jaosoriorbogdando: I see13:23
owalshdtantsur: e.g https://github.com/openstack/ironic-staging-drivers/blob/master/ironic_staging_drivers/wol/power.py#L16413:23
sshnaidm|rovermyoung|ruck, RCA?13:23
bogdandoI'l make puppet throwing delaycompress is not GDPR compatible haha13:24
jaosoriorzzzeek: so... I'll end my day in an hour... so lets do it next week :D. Would monday work for you? what's your timezone?13:24
owalshdtantsur: so it needs to be on the list https://github.com/openstack/ironic-staging-drivers/blob/master/ironic_staging_drivers/wol/power.py#L20113:24
myoung|rucksshnaidm|rover: root cause analysis13:24
* myoung|ruck rereads the bug/cards13:24
sshnaidm|rovermyoung|ruck, OIUN13:24
sshnaidm|rovermyoung|ruck, ok I understand now13:24
jaosoriorbogdando: well... puppet itself only has info about the deployment, not IPs coming from users.13:24
myoung|rucksshnaidm|rover:  lolz... "OIUN" ?13:24
weshayoh I understand now?13:24
myoung|ruckasked and answered heh...IRC client lag13:25
sshnaidm|rovermyoung|ruck, NINRCBDB13:25
dtantsurowalsh: yeah, I guess you're right13:25
myoung|rucksshnaidm|rover: you win...i have no idea13:25
myoung|ruck:)13:25
sshnaidm|rovermyoung|ruck, not, it's not a root cause, but different bug :)13:25
sshnaidm|rovermyoung|ruck, very easy13:25
sshnaidm|rovermyoung|ruck, see 177175513:26
owalshdtantsur: ok, I'll push a quick fix, already tested on my local env13:26
myoung|ruckaye reading it now thx for updating LP.13:26
*** ratailor has quit IRC13:26
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: execute the build / install of zuul changes in undercloud-upgrade  https://review.openstack.org/56894613:27
myoung|rucksshnaidm|rover, weshay, the sun nearly never sets on our team!13:27
bandinijaosorior, zzzeek: next week I'll be at the summit, but dciabrin will be around13:28
zzzeekbandini: we have to do this this week, today or tomorrow, is that possible13:28
zzzeekbandini: oh, jaosorior can't13:28
zzzeekblah13:28
jaosorioryeah13:28
jaosoriorthat's the issue13:28
bandiniyou can't tomorrow?13:29
jaosoriorI was available the whole week, but today I end my day in an hour, and tomorrow even earlier :/13:29
jaosoriorgot a train to catch13:29
zzzeekjaosorior: okey doke it's fine, i think the thing we want on your side is to figure out who will write that blueprint spec13:29
bandinioh ok, can you give me 30mins?13:29
bandiniI absolutely need to finish one thing13:29
zzzeekbandini:  i have to go somewhere at 10 AM13:29
bandinizzzeek: and now is 9:30 am?13:29
zzzeekbandini: yes :)13:29
*** mdnadeem has joined #tripleo13:29
zzzeekbandini: catastrophic failure of meeting13:30
bandinialright, let's do it now then? :D13:30
zzzeekbandini: jaosorior ?13:30
zzzeekwe DO IT LIVE13:30
myoung|rucksshnaidm|rover: i see, you pulled out the chroot urandom issue to a discreet bug...ack13:30
bandinijaosorior, zzzeek: https://redhat.bluejeans.com/4884650097/13:30
sshnaidm|rovermyoung|ruck, yep, it doesn't solve introspection problem13:31
jaosoriorok13:32
*** pchavva has quit IRC13:32
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: Fix typo in ceph-ansible  https://review.openstack.org/56910913:32
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: Add tag for post fast forward upgrade tasks  https://review.openstack.org/56911013:33
dmsimard|offsshnaidm|rover, myoung|ruck, weshay, trown, EmilienM, mwhahaha: by the way this vscode extension for Ansible is REALLY good: https://github.com/VSChina/vscode-ansible13:34
myoung|ruckdmsimard|off: yes it is...been liking it very much13:34
myoung|ruckdmsimard|off: the snippits are great, and the integration to run playbooks locally is pretty smooth too13:34
*** cshastri has quit IRC13:35
myoung|ruckdmsimard|off: vscode has been hitting the sweet spot for me too.  (unrelated) but the debugging for python/javascript/typescript is also pretty killer13:36
*** abishop_ has joined #tripleo13:36
* trown smh13:37
*** abishop has quit IRC13:37
trown#VIM4LYFE13:37
dmsimard|offI've historically been using pycharm for things but this might tip the scales13:37
dmsimard|offtrown: I don't live in my shell :(13:37
trowndmsimard|off: I like living in a shell13:37
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: Add validation-errors-nonfatal in ceph upgrade.  https://review.openstack.org/56753913:37
trownIm kind of a turtle13:37
openstackgerritMerged openstack/tripleo-common master: Add validation for empty file system backup on Undercloud  https://review.openstack.org/56824513:38
dmsimard|offI can understand why some people use mutt, emacs, irssi, etc., I guess I use more GUI apps :)13:38
myoung|ruckdmsimard|off: ya pycharm is pretty nice...the one thing I think it did better was python venv niceness...but vscode does that pretty well (lately).  i still go back and forth but increasingly i like the speed/lightweight aspects of vscode13:39
* myoung|ruck forgets about all this and looks at world on fire13:39
*** zoli is now known as zoli|afk13:39
jaosoriorslagle: do you see benefit in making ansible have become: true in the undercloud?13:39
*** sdake_ is now known as sdake13:39
myoung|ruckdmsimard|off: thanks for update re: container push bug, looking at the promotion jobs now to see if the prune helped13:40
sshnaidm|roverdmsimard|off, niiice, I use a few for Atom, but this seems like better13:40
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: Fix search for compute roles  https://review.openstack.org/56911413:40
*** ratailor has joined #tripleo13:40
sshnaidm|roverdmsimard|off, and for pycharm completely sucks :(13:40
dmsimard|offI actually don't use much of the fancy IDE features in pycharm but the syntax highlighting and automation doesn't get in the way and is spot on13:41
*** ratailor has quit IRC13:41
*** lblanchard has joined #tripleo13:42
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: [WIP] Wire in get_dlrn_hash  https://review.openstack.org/56911513:42
*** beagles is now known as beagles|mtg13:43
*** shreshtha_ has joined #tripleo13:43
*** shreshtha_ has quit IRC13:44
*** shreshtha_ has joined #tripleo13:45
*** shreshtha has quit IRC13:46
openstackgerritMatt Young proposed openstack-infra/tripleo-ci master: Build image in every OVB job  https://review.openstack.org/56825813:49
myoung|rucksshnaidm|rover: updated commit message to include related bug13:49
sshnaidm|rovermyoung|ruck, ok13:49
EmilienMsshnaidm|rover, myoung|ruck : the commit message is very small, tbh13:52
*** alee has joined #tripleo13:52
EmilienMmissing context, problem description, how we solve the problem and why13:52
*** skramaja has quit IRC13:52
EmilienMalso, building the image for each ovb job will cost a lot13:52
EmilienMit'll really increase the job runtime13:52
*** ssbarnea_ has quit IRC13:52
*** gbarros has joined #tripleo13:52
mwhahahait should add about 15 mins now13:54
slaglejaosorior: we can't right now. mistral doesnt have sudo13:54
mwhahahait used to be more but since we dropped the openstack/ceph images it is down to 15-20 mins13:54
EmilienM15min is a lot13:54
slaglejaosorior: we'd have to keep poking holes in sudoers13:55
EmilienMand still I feel like we're not solving the real root cause13:55
sshnaidm|roverEmilienM, 10 min13:55
EmilienMwe take a shortcut and rebuild the image13:55
EmilienMit would take 2 minutes, I would -1 the patch13:55
openstackgerritMerged openstack/tripleo-heat-templates master: Allow configuration of NFS backend for Nova  https://review.openstack.org/56417913:55
sshnaidm|roverEmilienM, we don't leave this issue, but this could be a temporary workaround13:55
sshnaidm|roverEmilienM, otherwise it will continue to fail always13:55
sshnaidm|roverEmilienM, like we did a workaround in containers just a few days ago13:56
slaglejaosorior: i'm not totally opposed it, i had tried to use become:true to update /etc/hosts on the undercloud, but there's no sudo13:56
sshnaidm|roverEmilienM, we still work on that with derekh13:56
sshnaidm|roverEmilienM, I don't insist, but I think it could be an option13:57
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: Enable undercloud_enable_tempest to true  https://review.openstack.org/56633213:57
chandankumarweshay: sshnaidm|rover done^^13:57
*** shreshtha_ has quit IRC13:57
sshnaidm|roverEmilienM, especially when we have to fix 001 job too, it fails because different reasons13:57
*** kopecmartin_ has joined #tripleo13:57
*** ssbarnea_ has joined #tripleo13:58
sshnaidm|roverEmilienM, other option is not to update image with packages, it also should work13:58
sshnaidm|roverweshay, ^^13:58
sshnaidm|roverEmilienM, weshay but then we won't have built packages in overcloud image13:59
sshnaidm|roverand it'll be job 10 min less :)13:59
*** artom has quit IRC13:59
*** kopecmartin has quit IRC14:00
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add newton->ocata undercloud upgrades to unsupported list  https://review.openstack.org/56912014:00
*** rbrady__ is now known as rbrady14:00
cmysterEmilienM: for fun and giggles, what about a 2nd job that runs evey now and then and creates a ready image to be consumed elswhere?14:00
*** rbrady has quit IRC14:00
*** rbrady has joined #tripleo14:00
cmysterevery*14:01
*** ccamacho has quit IRC14:01
*** ccamacho has joined #tripleo14:02
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: disable update of IPA image in jobs  https://review.openstack.org/56883314:03
sshnaidm|roverweshay, EmilienM another option ^^14:03
weshayah..14:04
weshayhrm14:04
*** pdeore has quit IRC14:04
EmilienMsshnaidm|rover: will look after mtgs14:05
EmilienMbut this patch is more than just a workaround14:05
EmilienMwe should build images in our promotion pipeline, not in our gate.14:06
sshnaidm|roverEmilienM, patch with building images?14:06
sshnaidm|roverEmilienM, yeah, but it seems like only +10 min issue..14:06
*** zoli|afk is now known as zoli|wfh14:06
EmilienMwhat triggered this ironic issue? why did it work fine before?14:06
EmilienM"only 10min"14:06
*** zoli|wfh is now known as zli14:06
*** zli is now known as zoli14:06
sshnaidm|roverEmilienM, because CentOS 7.5 happened recently14:07
EmilienMwe're working hard to reduce runtime and make things more simple, not to add 10min just for workarounds14:07
mcorneaweshay: hi is the -2 still needed in https://review.openstack.org/#/c/567539/ ?14:07
EmilienMand why rebuilding the image once and consume it doesn't help?14:08
weshaymcornea, sorry fixed14:08
*** shreshtha has joined #tripleo14:08
sshnaidm|roverEmilienM, ?14:08
weshaymcornea, /me looks for others14:08
mcorneaweshay: thanks14:08
derekhEmilienM: something about the way quickstarrt/ci updates the image is breaking it, we don't know what/why yet14:09
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097214:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154914:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163414:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169214:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)14:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)14:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)14:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]14:10
EmilienMderekh: can't we just update the image in RDO and use this new image?14:10
EmilienMinstead of updating the current image for every job?14:10
sshnaidm|roverEmilienM, we build patches in jobs14:10
sshnaidm|roverEmilienM, and inject these patches to images14:11
derekhEmilienM: no because we may need packages in the image that are being tested14:11
EmilienMI know but I don't think we need to rebuild the whole image14:11
sshnaidm|roverEmilienM, what do you mean?14:11
derekhEmilienM: nobody things we should rebuild the whole image, we think we should only update it,14:12
derekhEmilienM: buty the update process is what broken14:12
sshnaidm|roverderekh, interesting, when yum update didn't work because of chroot, introspection failed.. all we do there it's just copying/moving files14:14
derekhsshnaidm|rover: yup, its weird14:15
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Initial Network configuration views  https://review.openstack.org/56912414:17
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: [WIP] Add Network Topology diagram  https://review.openstack.org/56912514:17
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: don't make any changes on image  https://review.openstack.org/56912614:18
sshnaidm|roverderekh, let's not do anything ^^14:18
sshnaidm|roverderekh, if it fails, maybe something wrong with way we pack it back14:18
sshnaidm|roverderekh, or extract..14:18
*** kopecmartin_ has quit IRC14:20
derekhsshnaidm|rover: yup14:20
derekhsshnaidm|rover: has anything changed there recently?14:20
sshnaidm|roverderekh, not in 3 last years14:21
derekhsshnaidm|rover: it was probably me so ...14:22
sshnaidm|roverderekh, https://github.com/openstack-infra/tripleo-ci/commit/bf770edd1287e8316d9229288ee040dd73cfc5ab14:23
* derekh wonders what he broke14:24
*** moshele has joined #tripleo14:27
*** hamzy_ has joined #tripleo14:28
*** hamzy has quit IRC14:29
*** dxiri has joined #tripleo14:29
*** kopecmartin_ has joined #tripleo14:33
aleesshnaidm|rover, addressing your comments now.14:33
Tengusee you tomorrow folks :)14:34
aleesshnaidm|rover, the job was set up to just create the relevant files so I could make sure it was working correctly - it is :)  now will modify to actually run the re-deploy script14:35
sshnaidm|roveralee, ack, it's better to set "WIP" in commit message if it's not final version14:35
*** yprokule has quit IRC14:35
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Don't use crudini to get rhn server  https://review.openstack.org/56913314:37
openstackgerritMerged openstack/tripleo-heat-templates master: Fix cinder-backup image wrangling on update  https://review.openstack.org/56780614:39
*** beagles|mtg is now known as beagles|brb14:41
*** ykarel is now known as ykarel|away14:43
*** hamzy_ is now known as hamzy14:43
*** dxiri has quit IRC14:45
*** abishop_ has quit IRC14:46
*** asbishop has joined #tripleo14:47
*** ykarel|away has quit IRC14:49
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates stable/queens: [Queens only] Allow mixed version deployment of scenario000  https://review.openstack.org/56913614:53
*** agurenko has quit IRC14:56
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci master: Add keystone only upgrade jobs  https://review.openstack.org/55842514:56
*** dxiri has joined #tripleo14:58
openstackgerritBogdan Dobrelya proposed openstack/puppet-tripleo master: Force GDPR compliance of containers logs  https://review.openstack.org/56913914:59
*** artom has joined #tripleo15:00
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Update HostnameMap generation  https://review.openstack.org/56795115:03
mwhahahaEmilienM: -^15:03
*** fragatin_ has quit IRC15:08
mwhahahamyoung|ruck, sshnaidm|rover, weshay : is anyone looking into why rdo-kolla-build-integration is failing all the time? I looked and it looks like we're missing https://review.openstack.org/#/c/568126/15:09
*** lvdombrkr89 has quit IRC15:09
jristflorianf: https://review.openstack.org/#/c/568603/ fyi15:09
*** ykarel|away has joined #tripleo15:09
mwhahahamyoung|ruck, sshnaidm|rover, weshay: example https://logs.rdoproject.org/86/568986/2/openstack-check/rdo-kolla-build-integration/Zfed565a6b56348719f6c735feb294e89/kolla/logs/base.log15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097215:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154915:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163415:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169215:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)15:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)15:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)15:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]15:10
aleesshnaidm|rover, your comments in the tripleo-upgrade patch have to do mostly with setting of tags -- and just setting them at the top level rather than individually.  I'm cool with that - its just different from all the other roles.15:10
*** fragatina has joined #tripleo15:10
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: [WIP] Implement --output-file to write the bash script  https://review.openstack.org/56828515:10
aleemcornea, any comment/ preference ?15:10
EmilienMrlandy: thanks for your work on the upgrade/delorean thing15:11
weshaymwhahaha, looking15:12
sshnaidm|rovermwhahaha, I don't think we watch this job..15:12
weshaymwhahaha, it looks that way.. :(15:12
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Implement --output-file to write the bash script  https://review.openstack.org/56828515:12
mwhahahaIt's part of the 3rd party jobs on tripleo-common15:13
mcorneaalee: in a meeting, I'll comment after I finish15:13
mwhahahaSo we should probably monitor it15:13
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Implement --output-file to write the bash script  https://review.openstack.org/56828515:14
*** pblaho1 has joined #tripleo15:14
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: Add ability to use a different release per playbook  https://review.openstack.org/56656515:15
rlandyEmilienM: sure - group effort from the CI sprint team15:16
*** eck` is now known as eck`gone15:16
openstackgerritBrad P. Crochet proposed openstack/tripleo-common master: Allow undercloud to use 'become' tasks  https://review.openstack.org/56914415:16
thrashslagle: ^^^ would appreciate a look at that.15:16
thrashslagle: in response to jaosorior's inquiry15:16
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Implement --output-file to write the bash script  https://review.openstack.org/56828515:16
*** pblaho has quit IRC15:16
thrashslagle: it worked in a very limited test that I did15:17
*** pchavva has joined #tripleo15:17
*** quiquell is now known as quiquell|off15:17
openstackgerritmathieu bultel proposed openstack/tripleo-heat-templates stable/queens: Fix cinder-backup image wrangling on update  https://review.openstack.org/56914615:19
*** thrash is now known as thrash|biab15:19
*** eck`gone is now known as eck`15:20
*** saneax is now known as saneax-_-|AFK15:21
aleesshnaidm|rover, hi - question about your comment in https://review.openstack.org/#/c/567888/2/config/general_config/featureset053.yml15:21
*** jpena is now known as jpena|off15:22
aleesshnaidm|rover, if I look at the patch you pointed to - it looks like certain parts of what you highlighted were removed from the jobs15:22
sshnaidm|roveralee, yeah, I think it's release dependent15:22
aleesshnaidm|rover, are you saying I dont need the whole thing ? or just the parts that are removed ..15:23
sshnaidm|roveralee, just part that were removed15:23
aleesshnaidm|rover, ok cool - thanks15:23
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add method for getting dlrn_hash from release and hash_name  https://review.openstack.org/56732015:23
sshnaidm|roveralee, just bring it to match current featuresets15:23
aleeack15:23
*** kforde has joined #tripleo15:23
*** saneax-_-|AFK is now known as saneax15:23
*** dprince has quit IRC15:24
openstackgerritAde Lee proposed openstack/tripleo-quickstart master: Add job to test password changes  https://review.openstack.org/56788815:26
openstackgerritAde Lee proposed openstack/tripleo-quickstart master: Add job to test password changes  https://review.openstack.org/56788815:27
*** saneax is now known as saneax-_-|AFK15:27
*** gbarros has quit IRC15:29
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates stable/queens: Fix cinder-backup image wrangling on update  https://review.openstack.org/56914615:29
*** nkinder has quit IRC15:29
*** nkinder has joined #tripleo15:29
openstackgerritJose Luis Franco proposed openstack/tripleo-validations master: Run validation only if overcluod exists.  https://review.openstack.org/56915115:30
openstackgerritAde Lee proposed openstack/tripleo-quickstart-extras master: Add playbook for overcloud-config-change  https://review.openstack.org/56886515:31
*** jpena|off is now known as jpena15:33
*** beagles|brb is now known as beagles15:34
*** dhill_ has quit IRC15:36
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: WIP: Check container health as part of the deploy  https://review.openstack.org/56915315:36
mwhahahaEmilienM, dpeacock -^ fyi a thought15:36
myoung|ruckmwhahaha: weshay: looking now (was in scrum).  Will capture an action item for tripleo-ci squad to add  rdo-kolla-build-integration to what we monitor (e.g. ruck/rover guide, sova, etc)15:37
*** dhill_ has joined #tripleo15:38
aleesshnaidm|rover, on your comment in https://review.openstack.org/#/c/567897/5/zuul.d/multinode-jobs.yaml , I was just using the job using featureset037 as a guide.  If its not needed, I'm happy to remove it.15:38
EmilienMmwhahaha: that looks really cool15:40
EmilienMmwhahaha: we'll see what healthcheck are not working15:41
sshnaidm|roveralee, let's try, I'm sure 90% it's not needed15:41
aleesshnaidm|rover, ack trying ..15:41
*** cshastri has joined #tripleo15:41
*** sshnaidm|rover is now known as sshnaidm|bbl15:41
openstackgerritAde Lee proposed openstack-infra/tripleo-ci master: Add password change job  https://review.openstack.org/56789715:42
*** lvdombrkr has joined #tripleo15:43
*** dprince has joined #tripleo15:43
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Allow custom time constraints to rotate logs  https://review.openstack.org/56915415:43
*** ssbarnea_ has quit IRC15:44
openstackgerritMichael Chapman proposed openstack/tripleo-quickstart-extras master: Add opendaylight to collect-logs  https://review.openstack.org/49404315:44
*** pblaho1 has quit IRC15:44
*** marios has quit IRC15:45
*** leanderthal has quit IRC15:45
openstackgerritMichael Chapman proposed openstack/tripleo-heat-templates master: Add OPNFV scenario environment  https://review.openstack.org/48690515:45
*** aufi has quit IRC15:46
openstackgerritMichael Chapman proposed openstack/tripleo-quickstart master: Updates OpenDaylight feature set 31  https://review.openstack.org/50087215:46
*** gbarros has joined #tripleo15:46
*** sshnaidm|bbl has quit IRC15:46
*** juriarte has quit IRC15:48
*** lucasagomes is now known as lucas-afk15:48
*** juriarte has joined #tripleo15:48
*** dparkes has quit IRC15:50
michchapany ideas why the overcloud kernel and ramdisk might not be uploaded? I'm getting "No image with the name 'bm-deploy-kernel' found - make sure you have uploaded boot images."15:50
EmilienMrlandy: I checked and indeed tripleoclient was built and deployed, nice15:50
rlandyEmilienM: yeah - we had some other failure down the line so hard to tell if the whole upgrade behaved as it should. I am going to work on this more later today15:52
EmilienMrlandy: the upgrade job is broken until https://review.openstack.org/568997 is landed15:54
EmilienM(containerized undercloud upgrade)15:54
openstackgerritMerged openstack/tripleo-ha-utils master: Add pre checks before installing instance HA  https://review.openstack.org/56789215:54
weshaymyoung|ruck, don't we monitor that as part of the promotion jobs?15:58
*** agopi has joined #tripleo15:58
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add releases script pytest tests to tox.ini  https://review.openstack.org/56764916:00
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add linter jobs for releases file script  https://review.openstack.org/56828716:00
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add CLI argument parser and YAML file parser  https://review.openstack.org/56793616:00
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Implement --output-file to write the bash script  https://review.openstack.org/56828516:00
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add newton->ocata undercloud upgrades to unsupported list  https://review.openstack.org/56912016:00
*** dbellant has quit IRC16:01
*** trown is now known as trown|lunch16:02
EmilienMweshay: not sure who from CI team goes at Summit but https://www.openstack.org/summit/vancouver-2018/summit-schedule/events/21764/machine-learning-for-ci-results-analysis looks interesting16:02
openstackgerritRaoul Scarazzini proposed openstack/tripleo-ha-utils master: Make instance HA uninstallation idempotent  https://review.openstack.org/56829616:03
weshaytristan's work EmilienM ?16:03
weshayit's f.. awesome if so16:03
weshayEmilienM, we're not sending anyone16:04
EmilienM:/16:04
weshayyay.. didn't get approval16:04
EmilienMI'm not sure it's tristanC's work16:04
jristmwhahaha: have you been here? http://www.gcdonuts.com/16:05
*** nkinder has quit IRC16:06
jristwe should bring some for PTG16:06
*** nkinder has joined #tripleo16:06
weshaydo they have croissants?16:07
jristweshay: oh man you should go there16:07
jristweshay: I don't know but they did have CRONUTS last time I was there16:07
weshayjrist, we need to bike!16:08
jristIKR16:08
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097216:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154916:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177163416:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177169216:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)16:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)16:10
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)16:10
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]16:10
weshaymwhahaha, I should probably remove alerts on 1771634 1771692, unless you want to keep them16:12
weshaymyoung|ruck, removing alert on https://bugs.launchpad.net/tripleo/+bug/1771634 not much anyone on this channel can fix16:14
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Triaged] - Assigned to Matt Young (halcyondude)16:14
weshaymyoung|ruck, same w/ https://bugs.launchpad.net/tripleo/+bug/177169216:15
openstackLaunchpad bug 1771692 in tripleo "hubbot check jobs are failing (timing out on OC deploy)" [Critical,Triaged]16:15
mwhahahaweshay: yea that's fine16:15
weshayalert = "tripleo folks need to read and help fix" :)16:15
weshayif they want to :P16:15
* weshay waits for things to be thrown16:15
weshayccccccdlvuefdgurheftrbkjdukjjiijuceldtulkfjj16:15
weshaystupid fingers16:16
openstackgerritMatthew Thode proposed openstack/diskimage-builder master:   IPA requires iptables  https://review.openstack.org/56916016:16
beaglesmwhahaha: with respect to https://review.openstack.org/#/c/567641/  what are your thoughts on merging this to backport to queens and then make it proper with a follow up for rocky16:18
beaglesmwhahaha: like use an external deploy task on the keystone container or similar?16:18
beaglesmwhahaha: it's not optimal ...16:19
*** moshele has quit IRC16:20
mwhahahabeagles: I guess16:21
mwhahahabeagles: really Octavia needs to be out of the services tenant16:22
mwhahahaBut that's and upgrade problem as well16:22
beaglesmwhahaha: yeah...16:22
mwhahahaSoooo it would be way better to fix this now but yea...16:23
*** jaganathan has quit IRC16:23
*** jaganathan has joined #tripleo16:23
*** nkinder has quit IRC16:23
*** masco has quit IRC16:24
*** jrist has quit IRC16:24
*** kopecmartin_ has quit IRC16:26
slagleEmilienM: mwhahaha : we may want to consider merging https://review.openstack.org/#/c/568899/ and the depends-on as currently default deployments are broken16:28
slagleEmilienM: mwhahaha : would like some more feedback on it16:29
EmilienMI see jaosorior's +1, so I approved it16:30
myoung|ruckweshay: re: alert, ack.16:35
* myoung|ruck will be biaf, this.enableMode(ACQUIRE_CALORIES)16:35
*** sshnaidm|bbl has joined #tripleo16:35
*** salmankhan has quit IRC16:36
*** suuuper has quit IRC16:38
*** tesseract has quit IRC16:38
*** cshastri has quit IRC16:39
*** mdnadeem has quit IRC16:41
*** holser__ has quit IRC16:41
*** cshastri has joined #tripleo16:42
*** mdnadeem has joined #tripleo16:43
*** dparkes has joined #tripleo16:45
openstackgerritwes hayutin proposed openstack/tripleo-upgrade stable/queens: WIP, update scen000 job on queens  https://review.openstack.org/56916816:47
openstackgerritwes hayutin proposed openstack/tripleo-upgrade stable/queens: DNM, test update on queens  https://review.openstack.org/56916916:49
*** bogdando has quit IRC16:50
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add dry run option to toci_quickstart  https://review.openstack.org/56706016:51
*** agopi has quit IRC16:53
*** dxiri has quit IRC16:55
*** amoralej is now known as amoralej|off16:58
*** mdnadeem has quit IRC17:00
*** tiswanso_ has joined #tripleo17:01
*** jpena is now known as jpena|off17:01
*** ccamacho has quit IRC17:02
*** trown|lunch is now known as trown|mtg17:02
*** florianf has quit IRC17:02
*** dtantsur is now known as dtantsur|afk17:04
*** tiswanso has quit IRC17:04
*** cshastri has quit IRC17:04
*** ykarel|away has quit IRC17:05
*** derekh has quit IRC17:05
*** jtomasek has quit IRC17:07
openstackgerritAde Lee proposed openstack/tripleo-upgrade master: Add config_change role  https://review.openstack.org/56730017:07
*** ykarel has joined #tripleo17:09
*** ramishra has quit IRC17:09
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add dry run option to toci_quickstart  https://review.openstack.org/56706017:09
*** pblaho has joined #tripleo17:10
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097217:11
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)17:11
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154917:11
*** ooolpbot has quit IRC17:11
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)17:11
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: [WIP] Add Network Topology diagram  https://review.openstack.org/56912517:14
*** panda is now known as panda|off17:14
*** pblaho has quit IRC17:16
*** cshastri has joined #tripleo17:17
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Implement fetching of networks  https://review.openstack.org/54868517:18
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/queens: Don't use crudini to get rhn server  https://review.openstack.org/56917717:18
*** pblaho has joined #tripleo17:18
*** tiswanso_ has quit IRC17:20
*** tiswanso has joined #tripleo17:21
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Don't use crudini to get rhn server  https://review.openstack.org/56917817:23
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/ocata: Don't use crudini to get rhn server  https://review.openstack.org/56918017:23
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/newton: Don't use crudini to get rhn server  https://review.openstack.org/56918117:23
*** lvdombrkr has quit IRC17:23
*** jfrancoa has quit IRC17:24
openstackgerritwes hayutin proposed openstack/tripleo-upgrade stable/queens: WIP, update scen000 job on queens  https://review.openstack.org/56916817:25
*** ykarel_ has joined #tripleo17:25
*** jfrancoa has joined #tripleo17:27
*** ykarel has quit IRC17:27
*** gvrangan has joined #tripleo17:28
*** pkovar has quit IRC17:29
*** janki has quit IRC17:29
*** ykarel_ has quit IRC17:30
*** udesale has joined #tripleo17:32
*** ykarel has joined #tripleo17:33
*** rmascena has joined #tripleo17:33
*** raildo has quit IRC17:35
*** radeks_ has quit IRC17:36
*** psachin has quit IRC17:37
*** shreshtha has quit IRC17:37
*** cshastri has quit IRC17:37
openstackgerritwes hayutin proposed openstack/tripleo-upgrade stable/queens: WIP, update scen000 job on queens  https://review.openstack.org/56916817:38
*** masco has joined #tripleo17:39
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Deprecate tripleo::profile::base::docker(_registry)  https://review.openstack.org/56650717:40
*** thrash|biab is now known as thrash17:48
*** ffiore has quit IRC17:48
*** dxiri has joined #tripleo17:49
*** gfidente|pto has quit IRC17:49
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade stable/queens: (queens) add container minimal check  https://review.openstack.org/56919117:51
*** jfrancoa has quit IRC17:52
*** moshele has joined #tripleo17:53
*** dprince has quit IRC17:56
openstackgerritBrad P. Crochet proposed openstack/tripleo-common master: Allow undercloud to use 'become' tasks  https://review.openstack.org/56914417:56
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Let scenario000-multinode-oooq-container-updates run on queens  https://review.openstack.org/56919517:57
*** links has joined #tripleo17:58
*** ykarel has quit IRC17:59
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade stable/queens: (queens) add container minimal check  https://review.openstack.org/56919117:59
*** olap has joined #tripleo17:59
EmilienMweshay: here we go, it works: see zuul18:00
EmilienMweshay: look for 56919118:00
EmilienMweshay: we'll need to do https://review.openstack.org/#/c/569191/ for all projects in http://codesearch.openstack.org/?q=tripleo-multinode-container-minimal&i=nope&files=&repos= in their stable/queens branch18:00
weshaylooking18:00
weshayEmilienM, ok.. thanks! that makes sense.. I kind of started down that path and removed queens from !18:01
weshaybut backed away.. thinking it may go right into being a gate job18:01
weshaythanks18:01
*** ssbarnea_ has joined #tripleo18:02
EmilienMweshay: I haven't found a better way although it might exists something simpler18:03
EmilienMweshay: note that overriding templates is still under discussion AFIK and not in production18:03
weshayEmilienM, hrm.. https://review.openstack.org/#/c/569195/1/zuul.d/multinode-jobs.yaml18:04
*** zoli is now known as zoli|gone18:04
*** zoli|gone is now known as zoli18:04
weshayseems like we could just put branches: ^(stable/queens)18:04
weshayzuul is odd w/ the project branches.. but this makes sense to me18:04
weshayso if it passes I guess we can merge18:04
EmilienMweshay: so when you'll patch stable/queens, make sure you override a project18:05
openstackgerritMarius Cornea proposed openstack/tripleo-quickstart-extras master: DNM: Run fast forward upgrade for multinode-overcloud-upgrade  https://review.openstack.org/56919818:05
EmilienMweshay: also read my comment on https://review.openstack.org/56919518:05
EmilienMwe need to be careful in landing this one18:06
EmilienMin fact, I added depends-on on tripleo-ci patch just for testing18:06
EmilienMbut we should remove it18:06
EmilienMbecause https://review.openstack.org/569195 will ahve to land at the very end18:06
weshayya.. had the same thought18:06
EmilienMotherwise we'll have all queens running this voting job18:06
EmilienMnot sure we want that :D18:06
EmilienMweshay: please take over my stuff18:06
weshaythanks18:06
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: DNM: Test fast forward gate job  https://review.openstack.org/56920018:07
EmilienMmwhahaha: we might want to have a look at https://review.openstack.org/#/c/568976/ in the future18:08
EmilienMsee if there is any potential collaboration18:08
EmilienMflaper87: ^ also18:08
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097218:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154918:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)18:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)18:10
openstackgerritJaganathan Palanisamy proposed openstack/tripleo-common master: Prevent updating derived_parameters in plan environment  https://review.openstack.org/56714418:11
*** salmankhan has joined #tripleo18:12
openstackgerritwes hayutin proposed openstack/tripleo-upgrade stable/queens: (queens) add container minimal check  https://review.openstack.org/56919118:15
*** salmankhan has quit IRC18:16
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: execute the build / install of zuul changes in upgrade  https://review.openstack.org/56894618:18
*** trown|mtg is now known as trown18:22
*** anilvenkata has quit IRC18:22
EmilienMbfournie: hey, could you please justify https://review.rdoproject.org/api/13806 ?18:25
mcorneaEmilienM: weshay re https://review.openstack.org/#/c/569191/ do we know that the update job works on queens?18:25
EmilienMno we don't18:25
EmilienMbut the job will be non voting18:25
EmilienMbut I guess weshay is testing it18:25
EmilienMbefore we waste CI jobs18:26
weshayaye18:26
mcorneaah, missed the non-voting part, kk18:26
bfournieEmilienM: yes its the Ceph entitlement fix, to remove Ceph nodes from needing OSP entitlements, one thing that had to removed was ovs from os-net-config since ceph nodes still need os-net-config, but they don't need to create ovs bridges/bonds and hence don't need ovs.  Much more detail here - https://docs.google.com/document/d/1roq3lGg4fYZhjPvRxGwn5p6lQeWcgWJ1MugEDw0BKlo/edit?ts=5af9a905#heading=h.22tuhdsz35xa, specifically scroll down18:30
bfournie to "PLAN recap"18:30
EmilienMbfournie: if you update with a commit message I'll go ahead and approve this patch immediately :)18:30
bfournieEmilienM: will do18:30
EmilienMthx18:32
*** links has quit IRC18:35
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Initial Network configuration views  https://review.openstack.org/56912418:42
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: [WIP] Add Network Topology diagram  https://review.openstack.org/56912518:43
*** gvrangan has quit IRC18:43
*** dxiri has quit IRC18:47
*** lblanchard has quit IRC18:47
*** lblanchard has joined #tripleo18:47
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Deploy Docker via Ansible and not Puppet  https://review.openstack.org/56137718:48
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Deploy Docker via Ansible and not Puppet  https://review.openstack.org/56137718:48
*** lblanchard1 has joined #tripleo18:48
openstackgerritEmilien Macchi proposed openstack/ansible-role-container-registry master: Handle Docker rpm updates  https://review.openstack.org/56871418:49
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: docker: cleanup update tasks  https://review.openstack.org/56871518:49
*** lblanchard has quit IRC18:51
openstackgerritwes hayutin proposed openstack/tripleo-specs master: WIP, first pass at a policy for new git repos to TripleO  https://review.openstack.org/56922418:56
EmilienMbfournie: merged. Do you need backports?18:59
*** rlandy is now known as rlandy|brb19:01
bfournieEmilienM: thanks, yes I will backport to queens19:01
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Temporary disable update of IPA image in jobs  https://review.openstack.org/56883319:01
sshnaidm|bblEmilienM, weshay ^^19:01
* weshay looks19:02
sshnaidm|bblEmilienM, weshay I hope this workaround won't affect us at all19:02
sshnaidm|bbl035 passes with it, 001 passes introspection19:02
EmilienMlooking19:02
weshayya.. we need to get these through so we can get eyes on why 001 is failing19:03
EmilienMI prefer this workaround19:03
weshayvs.. building the images in the job?19:03
EmilienMyes19:03
weshayjust to get a sense to why you think that.. why? time?19:03
EmilienMthe time added to the job, and the risk of build failure19:04
* weshay tries to get inside the mind of a frenchman19:04
EmilienMthe risk of regression in the IPA image is I guess low?19:04
sshnaidm|bblEmilienM, very low19:04
weshayEmilienM, ok.. fair.. the risk of a build failure I think afaict is fairly low..19:04
sshnaidm|bblEmilienM, I'm not even sure we gate these changes..19:04
weshayagree w/ excluding the IPA image is also low19:05
EmilienMsshnaidm|bbl: +219:05
weshaysshnaidm|bbl, just via promotion19:05
weshaysshnaidm|bbl, thank you sir.. dam.. please pick up the ovb work on sunday19:05
weshaysshnaidm|bbl, have a great weekend19:05
*** ramishra has joined #tripleo19:09
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097219:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154919:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)19:10
*** olap has quit IRC19:11
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: [WIP] Add Network Topology diagram  https://review.openstack.org/56912519:11
*** olap has joined #tripleo19:12
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Implement fetching of networks  https://review.openstack.org/54868519:12
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Initial Network configuration views  https://review.openstack.org/56912419:13
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Initial Network configuration views  https://review.openstack.org/56912419:14
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: [WIP] Add Network Topology diagram  https://review.openstack.org/56912519:14
sshnaidm|bblweshay, sunday is holiday19:14
sshnaidm|bblshavuot19:15
*** ramishra has quit IRC19:16
*** olap has quit IRC19:16
*** moshele has quit IRC19:19
*** matbu has quit IRC19:19
*** moshele has joined #tripleo19:21
*** matbu has joined #tripleo19:25
*** zaneb has quit IRC19:26
*** rlandy|brb is now known as rlandy19:27
*** dtantsur|afk has quit IRC19:29
*** moguimar has quit IRC19:30
*** pblaho has quit IRC19:31
*** moguimar has joined #tripleo19:32
*** slaweq has quit IRC19:34
*** radeks has joined #tripleo19:34
*** slaweq has joined #tripleo19:35
*** rbrady is now known as rbrady-afk19:35
*** dtantsur has joined #tripleo19:36
*** jrist has joined #tripleo19:36
*** pblaho has joined #tripleo19:36
*** moguimar has quit IRC19:37
openstackgerritRonelle Landy proposed openstack/python-tripleoclient master: DNM Testing releases script upgrades with zuul change  https://review.openstack.org/56922919:38
*** moguimar has joined #tripleo19:39
*** paramite_ has quit IRC19:39
*** dtantsur has quit IRC19:40
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: Enable undercloud_enable_tempest to true  https://review.openstack.org/56633219:41
openstackgerritAde Lee proposed openstack/tripleo-upgrade master: Add config_change role  https://review.openstack.org/56730019:42
*** pblaho has quit IRC19:42
*** dxiri has joined #tripleo19:46
*** olap has joined #tripleo19:46
*** jrist has quit IRC19:48
*** zaneb has joined #tripleo19:48
*** moshele has quit IRC19:50
*** dtantsur has joined #tripleo19:51
*** olap has quit IRC19:51
*** jrist has joined #tripleo19:51
*** zshi has quit IRC19:54
myoung|ruckmwhahaha, weshay, sshnaidm|bbl: fyi pike 340f99e18169457f4ca51b543c62805bf0f08172_ddd29a3f promotion in flight19:54
*** zshi has joined #tripleo19:54
*** dhill_ has quit IRC19:55
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Wire in get_dlrn_hash  https://review.openstack.org/56911519:56
*** pblaho has joined #tripleo19:57
*** tiswanso_ has joined #tripleo20:01
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add CLI argument parser and YAML file parser  https://review.openstack.org/56793620:01
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add linter jobs for releases file script  https://review.openstack.org/56828720:01
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Implement --output-file to write the bash script  https://review.openstack.org/56828520:01
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add newton->ocata undercloud upgrades to unsupported list  https://review.openstack.org/56912020:01
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Wire in get_dlrn_hash  https://review.openstack.org/56911520:01
*** tiswanso has quit IRC20:04
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: WIP: Check container health as part of the deploy  https://review.openstack.org/56915320:05
*** tiswanso_ has quit IRC20:06
*** nyechiel_ has quit IRC20:06
*** gkadam has quit IRC20:08
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097220:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154920:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)20:10
openstackgerritAlex Schultz proposed openstack/puppet-tripleo master: Force GDPR compliance of containers logs  https://review.openstack.org/56913920:16
*** wolverineav has quit IRC20:16
*** tiswanso has joined #tripleo20:24
*** akrivoka has quit IRC20:25
openstackgerritRonelle Landy proposed openstack/python-tripleoclient master: DNM Testing releases script upgrades with zuul change  https://review.openstack.org/56922920:26
*** tiswanso has quit IRC20:29
*** dhill_ has joined #tripleo20:29
*** fragatina has quit IRC20:32
openstackgerritAndy Smith proposed openstack/tripleo-heat-templates master: [DNM] testing separate messaging backends  https://review.openstack.org/56648320:32
*** moshele has joined #tripleo20:33
*** dprince has joined #tripleo20:39
*** ccamacho has joined #tripleo20:40
*** pchavva has quit IRC20:43
*** ssbarnea_ has quit IRC20:44
*** rmascena has quit IRC20:44
*** ansmith has quit IRC20:46
*** jcoufal has quit IRC20:50
*** artom has quit IRC20:50
openstackgerritMarius Cornea proposed openstack/tripleo-quickstart-extras master: Run fast forward upgrade for multinode-overcloud-upgrade  https://review.openstack.org/56919820:51
mcorneaweshay: EmilienM can you please check ^ right now the ffu gate  job skips all the ffu upgrade tasks20:52
EmilienMmcornea: ok20:52
mcorneaI tested it here https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-multinode-1ctlr-featureset036-oc-ffu-queens-nv/132/console20:53
weshaymcornea, we checked about 1.5 months ago and found several of the jobs skip upgrade but report green20:53
weshaymcornea, we had an etherpad on it20:53
weshaymcornea, https://etherpad.openstack.org/p/upgrade-ci20:54
mcorneaweshay: I'll try to bring the ffu one up to date, I'll send a follow up patch for tripleo-upgrade to make it pass20:54
weshaymcornea, awesome... thank you!!20:54
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Mount /var/run/redis for the redis socket  https://review.openstack.org/56925620:55
mcorneaweshay: thank you, I'm getting more familiar with how the upstream ci works20:55
weshayvery good :)20:56
weshaymcornea, fwiw.. I don't think we had adequate doc on how to get started when we push tripleo-upgrade up20:56
weshaytrying to fix that now for upcoming breakouts .. like tempest and collect-logs20:57
*** lblanchard1 has quit IRC20:58
*** fragatina has joined #tripleo20:59
*** fragatina has quit IRC21:01
*** vpickard is now known as vpickard_21:01
*** fragatina has joined #tripleo21:02
*** dtantsur has quit IRC21:02
*** dtantsur has joined #tripleo21:03
*** fragatina has quit IRC21:03
*** moguimar has quit IRC21:04
*** pblaho has quit IRC21:04
*** moguimar has joined #tripleo21:04
*** fragatina has joined #tripleo21:04
*** wolverineav has joined #tripleo21:07
*** Goneri has quit IRC21:08
*** trown is now known as trown|outtypewww21:09
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097221:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154921:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)21:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)21:10
*** alee has quit IRC21:14
*** rh-jelabarre has quit IRC21:15
*** rh-jelabarre has joined #tripleo21:15
weshayis something reseting the gate?21:17
*** pblaho has joined #tripleo21:18
*** toure is now known as toure|gone21:24
*** dprince has quit IRC21:28
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: WIP: Do not run nic template conversion on default tht location  https://review.openstack.org/56920021:29
*** asbishop has quit IRC21:29
*** bfournie has quit IRC21:30
*** bfournie has joined #tripleo21:31
*** rbowen_ has joined #tripleo21:31
*** rbowen has quit IRC21:34
*** ansmith has joined #tripleo21:35
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: WIP: Do not run nic template conversion on default tht location  https://review.openstack.org/56920021:37
mwhahahasdoran: so is every value in ansible tasks run through a jinja parser?21:37
sdoranPretty much.21:39
sdoranVery few things aren’t templates.21:39
mwhahahasdoran: ok so i ran into a problem with bash comments in a shell: |21:39
sdoran*templated21:39
*** fragatina has quit IRC21:39
mwhahahawhich was totally not obvious21:39
EmilienMweshay: not afik what would you make you think about that?21:40
mwhahahasdoran: http://logs.openstack.org/53/569153/2/check/tripleo-ci-centos-7-undercloud-containers/5b2a4c6/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-05-17_20_36_2521:40
mwhahahasdoran: figured out if i ## or # comment # it doesn't error21:40
mwhahahasdoran: http://logs.openstack.org/53/569153/2/check/tripleo-ci-centos-7-undercloud-containers/5b2a4c6/logs/undercloud/home/zuul/undercloud-ansible-E8TdcI/common_deploy_steps_tasks.yaml.txt.gz for reference21:40
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: WIP: Do not run nic template conversion on default tht location  https://review.openstack.org/56927121:41
weshayEmilienM, certainly when statements and the variables used there are21:41
* weshay reads21:41
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: WIP: Do not run nic template conversion on default tht location  https://review.openstack.org/56927221:42
EmilienMweshay: my question was about reset gate21:42
weshayOH21:42
weshayEmilienM, oh.. so.. I see the same silly tox jobs respinning... I think21:42
weshaynot sure21:42
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: WIP: Do not run nic template conversion on default tht location  https://review.openstack.org/56927121:43
weshayEmilienM, one of the things I'm going to insist on that ruck/rover tool is to capture a chronilogical list of gate failures21:43
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade stable/queens: WIP: Do not run nic template conversion on default tht location  https://review.openstack.org/56927221:43
weshayhopefuly realtime21:43
openstackgerritMerged openstack/tripleo-quickstart-extras master: Mount /dev for chrooted environment  https://review.openstack.org/56883821:44
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Add tag for post fast forward upgrade tasks  https://review.openstack.org/56911021:44
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Fix typo in ceph-ansible  https://review.openstack.org/56910921:44
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Fix search for compute roles  https://review.openstack.org/56911421:44
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Add validation-errors-nonfatal in ceph upgrade.  https://review.openstack.org/56753921:44
openstackgerritMerged openstack/tripleo-heat-templates master: undercloud: change upgrade steps  https://review.openstack.org/56899721:44
weshayEmilienM, sova tells me the gate is at 95% today so.. maybe I'm full ofit21:44
weshayof it21:44
weshayqueens is still down though EmilienM21:45
EmilienMyea21:45
weshaygood thing we don't have to merge anything related to queens21:45
weshaywait.. DOH21:45
EmilienMweshay: B-21:46
*** fragatina has joined #tripleo21:46
weshayheh21:46
*** wolverineav has quit IRC21:46
weshayat least you are honest21:47
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: WIP: Check container health as part of the deploy  https://review.openstack.org/56915321:49
openstackgerritwes hayutin proposed openstack/tripleo-upgrade master: DNM, test update on queens  https://review.openstack.org/56927821:50
*** mcornea has quit IRC21:53
*** matbu has quit IRC21:53
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add dry run option to toci_quickstart  https://review.openstack.org/56706021:54
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: undercloud/upgrade: suggest backup before upgrade  https://review.openstack.org/56927921:54
*** mburned has quit IRC21:55
openstackgerritMerged openstack/tripleo-heat-templates master: FFU Add cinder-backup missing fast_forward_upgrade_tasks  https://review.openstack.org/56852021:56
openstackgerritMerged openstack/python-tripleoclient master: Convert enable-ssh-admin.sh to python  https://review.openstack.org/56612921:56
openstackgerritMerged openstack/tripleo-common master: Revert "TLS by default for the overcloud"  https://review.openstack.org/56896421:56
openstackgerritMerged openstack/tripleo-heat-templates master: Revert "Switch public endpoints to use FQDNs by default"  https://review.openstack.org/56889921:56
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add dry run option to toci_quickstart  https://review.openstack.org/56706021:57
mwhahahaEmilienM: hey it worked http://paste.openstack.org/show/721193/21:59
openstackgerritMerged openstack/tripleo-quickstart-extras master: doc on which tests to skip  https://review.openstack.org/56899621:59
openstackgerritMerged openstack/tripleo-heat-templates master: Don't use crudini to get rhn server  https://review.openstack.org/56913321:59
*** wolverineav has joined #tripleo22:00
EmilienMmwhahaha: a likely story22:03
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add ability to use a different release per playbook  https://review.openstack.org/56656522:03
*** mcornea has joined #tripleo22:04
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add dry run option to toci_quickstart  https://review.openstack.org/56706022:04
*** rcernin has joined #tripleo22:04
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Check container health as part of the deploy  https://review.openstack.org/56915322:08
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Check container health as part of the deploy  https://review.openstack.org/56915322:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097222:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154922:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)22:10
*** wolverineav has quit IRC22:29
*** myoung|ruck is now known as myoung|ruck|off22:31
*** threestrands has joined #tripleo22:37
*** rlandy is now known as rlandy|biab22:38
*** rpioso|afk is now known as rpioso22:41
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: WIP - Prompt before running upgrades  https://review.openstack.org/56928822:42
*** slaweq has quit IRC22:44
*** slaweq has joined #tripleo22:44
*** leitan has joined #tripleo22:54
*** moshele has quit IRC22:55
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Prompt before running undercloud/all-in-on- upgrades  https://review.openstack.org/56928822:55
*** tcw has quit IRC22:56
*** tcw has joined #tripleo22:57
*** moshele has joined #tripleo22:59
*** alee has joined #tripleo23:01
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Prompt before running undercloud/all-in-one upgrades  https://review.openstack.org/56928823:03
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: undercloud/upgrade: suggest backup before upgrade  https://review.openstack.org/56927923:05
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177097223:10
openstackLaunchpad bug 1770972 in tripleo "CI: Images introspection fails in OVB jobs" [Critical,Triaged] - Assigned to Derek Higgins (derekh)23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177154923:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1771549 in tripleo "Containers multinode jobs fails on stable queens with overcloud deploy timeout" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)23:10
*** radeks has quit IRC23:11
openstackgerritAde Lee proposed openstack/tripleo-upgrade master: Add config_change role  https://review.openstack.org/56730023:13
*** leitan has quit IRC23:16
*** leitan has joined #tripleo23:17
*** leitan has quit IRC23:21
*** moshele has quit IRC23:23
*** artom has joined #tripleo23:24
*** bfournie has joined #tripleo23:35
*** tosky has quit IRC23:36
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Update HostnameMap generation  https://review.openstack.org/56795123:50
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Update HostnameMap generation  https://review.openstack.org/56795123:51
*** rlandy|biab is now known as rlandy23:53
*** rbowen_ has quit IRC23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!