Friday, 2016-08-05

*** manous has quit IRC00:00
*** sdake_ has joined #rdo00:06
*** stanchan has quit IRC00:07
*** sdake has quit IRC00:09
*** nixpenguin has quit IRC00:14
*** nixpenguin has joined #rdo00:14
*** aortega has joined #rdo00:17
*** sarath has quit IRC00:19
*** limao has joined #rdo00:25
*** chlong has quit IRC00:32
*** stanchan has joined #rdo00:33
*** mburned is now known as mburned_out00:34
*** dmacpher has joined #rdo00:43
*** rhallisey has joined #rdo00:43
*** aortega has quit IRC00:46
*** maeca1 has quit IRC00:47
*** stanchan has quit IRC00:54
*** gildub has joined #rdo01:05
*** ayoung has joined #rdo01:07
*** jcoufal has joined #rdo01:13
ggilliespabelanger: ping01:50
zodbotggillies: Ping with data, please:
rdobot[sensu] NEW: - check-delorean-newton-head-current @ |#| Build failure on centos7-master-head/current: nova:
ggilliespabelanger: is there any plans for to expand beyond offering oauth201:51
ggilliessuch as openidc connect or saml or something01:51
rdobot[sensu] RESOLVED: - check-delorean-newton-head-current @ |#| No build failures detected:
*** fzdarsky_ has joined #rdo01:56
*** fzdarsky has quit IRC01:59
dmsimardggillies: I've seen ipsilon (same thing as fedora account system) mentioned somewhere01:59
dmsimardi.e, to help replace launchpad and integrate openstackid02:01
*** sdake has joined #rdo02:03
*** sdake_ has quit IRC02:04
*** rhallisey has quit IRC02:05
ggilliesdmsimard: interesting02:05
ggilliesjust looking at federation options for RDO Cloud because I really don't want to be stuck doing password management :V02:05
ggillieshmmm not sure if I am a fan of ipsilons "lets just be this intermediary wrapped in a black box installer for you"02:06
*** myoung is now known as myoung|afk02:06
*** mburned_out is now known as mburned02:08
*** myoung|afk is now known as myoung02:10
*** fragatina has joined #rdo02:12
*** stanchan has joined #rdo02:13
*** sdake_ has joined #rdo02:14
*** danielbruno has quit IRC02:16
*** sdake has quit IRC02:17
*** jhershbe has joined #rdo02:17
*** ashw has joined #rdo02:21
*** maeca1 has joined #rdo02:22
*** stanchan has quit IRC02:24
*** fragatina has quit IRC02:27
*** julim has joined #rdo02:28
dmsimardggillies: chat with fungi about it02:32
dmsimardggillies: in #openstack-infra02:32
dmsimardggillies: anything would be better than trystack's facebook auth :D02:33
dmsimardand looks like our friends ayoung and richm have already discussed keystone/ipsilon
ayoungdmsimard, RHSSO is going to be based on Keycloak.  I am actively testing it right now02:35
pabelangerggillies: IIRC, we are planning on implementing ipsilon upstream. As is supports more backends02:36
*** crossbuilder has joined #rdo02:36
*** crossbuilder_ has quit IRC02:36
*** jhershbe has quit IRC02:41
ggilliesdmsimard: where are the links to DLRN repos that have passed CI? I'm not seeing them on
ayoungIpsilon is the Fedora AS replacement, and it is pretty nice and light weight.  DOes not do OpenIDC yet last I checked.  Keycloak is a hell of a lot more full featured, which is a mixed blessing.  It does both SAML2 and OpenIDC, with OIDC being the first and better supported.02:44
ayoungAnd it is the basis for RHSSO, so we kindof have to make sure it works02:44
ggilliesayoung: from a very quick glance02:45
ggilliesit looks like it's based off you having to run these "magic scripts" (ipsilon-client-install) on hosts to set it up02:45
ayoungggillies, no magic.  Just python02:45
ggilliesit is magic02:45
ggilliesthat pattern in software is cancer02:45
ggilliesI don't need magic scripts or buttons02:45
ayoungggillies, most of it is configuring HTTPD02:45
*** nyechiel has joined #rdo02:45
ggilliesyes that02:46
ggilliesthat is wrong02:46
ayoungit is the complexity of SAML that calls for it02:46
ggillies*I* want to configure httpd based off how I need02:46
ggilliesthrough my configuration management engine02:46
ggilliespeople *think* it's too complicated, but it's not, the doco just sucks02:46
ayoungggillies, I think you are reading it backwards02:46
ayoungyou can configure SAML or OIDC however you want02:46
ayoungthese scripts are designed to do it the least surpsing way02:46
ggilliesso is ipsilon just providing a saml endpoint I can integrate to02:47
ggilliesor openidc, or oauth, or whatever the flavor of the month is02:47
ayoungggillies, I'd watch the word "just" there02:47
ayoungSAML is like short term X50902:48
ayoungwith all the key signing and 2 way auth02:48
ayoungdone in the HTTP layer.  I'd be much happier with X509 client Auth for the most part, or Kerberos, but those suck for other reasons.02:48
ggilliesbut all of those are possible for me to configure and integrate with using my configuration management system of choice02:49
ayoungBut what the scripts do is not magic.  They  make a call to the IdP and registers the service provider, fetch  the files from the IdP, and generate the local files to get them signed02:49
ggilliesthat is configuration management02:50
ayoungggillies, I'm actually doing somethig like that right now02:50
ggilliesI have an ecosystem to solve that problem02:50
ggilliesbut developers keep thinking they have to solve this problem over and over and keep writing their own systems to do it for their software02:50
ggilliesoften poorly02:50
*** rasca has quit IRC02:50
ayoungggillies, so the way I see this working is that  for Tripleo, we would run this on the undercloud, cache the fiels in a tarball, and delivber them to the keystone nodes02:50
*** rasca has joined #rdo02:50
ggilliesno thankyou02:51
ayoungggillies, with HA, you need to make sure the three (or more) nodes all respond as the same SP02:51
ggilliesI can deliver it to the nodes through heat/puppet and the appropriate hooks02:51
ayoungggillies, yes, that is what I mean02:51
ayoungHeat will do the delivery02:52
ayoungbut the files will be pre-created02:52
ggilliesthat's fine, but I want it all done through the configuration management and remote execution frameworks I have02:52
ayoungHeat has the ability to deliver the tarball...however, I think it might deliver it too early.  We need to make sure, at a minimum, that mod_auth_mellon or mod_auth_oidc is installed in the webserver before we try to run02:53
ayoungggillies, that is a give02:53
ayoungggillies, part of my problem right now is that Heat does not provide an easy way to run pre-execution scripts to generate things like this prior to an openestack server install.  Is there a way to do that, or do I need to wait for Mistral?02:54
ayoungI'd rather not have to do it manually02:54
ggilliesthere are hooks when the server is booted before any further configuration is done02:54
ayoungNope too late02:55
ayoung I need it on the Heat server prior to running on the booted node02:55
ggillieswhy should it be done on the undercloud02:55
ggilliesalso anything like x509 certs and related02:55
ggilliesI want to generate and store in my config management system so I can track them02:55
ggilliesI don't want them generated on the fly at deploy time02:56
ggilliesthat's a horrible idea02:56
ayoungTHe registration of Identity needs to be done by a trusted server02:56
*** steveg_afk has quit IRC02:57
ayoungggillies, of course they get stored in the config management system02:57
*** Tenhi has quit IRC02:57
ayoungbut I don't want to generate them on one node, them pull them into the CMS to be stuck on another02:57
ggilliesright I'm saying I would pregenerate and then inject into my deploy02:59
ggillieswhich I think is what you were saying?02:59
ayoungggillies, yep02:59
*** Tenhi has joined #rdo03:00
ayoungggillies, I've just started getting my head around how Heat and the os-*-config programs work to drive puppet.03:00
ayoungI was doing Ansible all last year, and the mapping is kindof hard from one to the other.03:00
ayoungBut I think I get it now.  I was able to work through the SSL deploy steps with FQDN earlier today03:01
ggilliesI'm actually working on an example now I hope to post publically (groundwork for RDO Cloud) that shows using tripleo for full static ip assignment for all interfaces on all networks, static assignment of vips, and 1:1 node placement03:02
ggilliesthe step after that is federation integration03:02
ggillieshence my question to begin with03:02
ayoungggillies, I should have something for you with Federation integration shortly03:03
ayoungHelp getting it production ready would be greatly appreciated03:03
ayoungWe had a proof of concept of Ipsilon with Packstack back in Tokyo.  The trick is mapping those steps to Tripleo, and cleaning up some of the demo-isms03:03
ggilliesI've done most the work I've needed already in puppet-keystone03:04
ggilliesit's just a bunch of stuff passed as an environment template03:04
ggilliesthen some extra steps to upload the mapping etc03:04
ayoungggillies, have you submitted it to puppet-keystone, at least as a WIP?  I'd like to look at it.03:04
ggilliesayoung: yeah it's been merged03:05
ggilliesthe first bit03:05
ggilliesit isn't the complete set, you still need to run some keystone commands03:06
ayoung#  The allowed values are: 'external', 'password', 'token', 'oauth1', 'saml2'03:06
ggilliesbut I'm pondering the best way to automate that with puppet-keystone at the moment03:06
ayoungI think we allow oidc now too03:06
ggilliesyou can put in anything there03:06
ggilliesyou can put in asdf123403:06
ggilliesas long as you have a section for it to map to03:07
ayoungThe way we configure auth mechanisms in Keystone is too centralized. It should be on the protocol URL, not one for the whole server03:07
ayoungyeah, they all use the mapped Keystone auth plugin03:07
ggilliesyeah fair enough03:07
ggilliesit seems like a lot of this federation stuff is in flux03:08
ggillieslike slowly getting reworked03:08
ggilliesbut I dunno03:08
ayoungggillies, well, I should be working on it, but Learning Tripleo has eaten my time for this release03:08
*** steveg_afk has joined #rdo03:09
ayoungI don't think any radical changes are queued up for Federation, though.  I just think a lot of people have not tried to really make it work yet.  We are hitting some of the snarls03:09
ayoungThe puppet code was obviously written with only K2K federation in mind.  There was some work done on untangling that03:10
ggilliesIt seems keystone to keystone is a lot stronger03:10
*** Son_Goku has quit IRC03:11
ayoungAnyway,  with Keycloak/RHSSO, we should have a single IdP that can handle either Saml2 or OIDC03:11
ayoungI've not put a lot of time into OIDC, as the demand from our side has been on SAML.03:12
ayoungGood to know that you;'ve been beating on it.03:12
*** Son_Goku has joined #rdo03:12
ayoungThere was far more interest in OIDC at the last summit than SAML03:12
ayoungggillies,  is it necessary to have secrets stored in the Hiera data? That really is poor security hygene.03:18
ayoungWe are starting to get beat up on compliance issues...rightfully so, too.03:19
*** ashw has quit IRC03:25
*** ayoung is now known as ayoung_ZZZzzz03:26
*** Tenhi has quit IRC03:27
ggilliesunfortunately we haven't got much of a choice03:29
ggilliesand what are the compliance issues your talking about?03:29
ayoung_ZZZzzzKey management for the most part.  PCI for passwords,  etc.03:30
ayoung_ZZZzzzggillies, Gotta crash.  Thanks for leetting me know what you are working on, I'll catch up and sync with you once I have something functional.03:31
ggilliesso is the problem unencrypted at rest?03:31
*** julim has quit IRC03:31
*** Tenhi has joined #rdo03:35
*** vimal has joined #rdo03:36
*** stanchan has joined #rdo03:41
*** sdake_ has quit IRC03:42
*** ohochman has left #rdo03:46
*** CustosL1men has quit IRC03:47
*** dmacpher has quit IRC03:48
*** CustosL1men has joined #rdo03:48
*** chlong has joined #rdo03:49
*** JonSnowcl has joined #rdo03:49
*** dmacpher has joined #rdo03:50
*** JonSnowcl has quit IRC03:50
*** richm has quit IRC03:51
*** stanchan has quit IRC03:52
*** sdake has joined #rdo03:57
*** jdob has quit IRC04:01
*** jcoufal has quit IRC04:02
*** rlandy has quit IRC04:08
*** nyechiel has quit IRC04:10
*** vimal has quit IRC04:12
*** sdake has quit IRC04:14
*** numans has joined #rdo04:16
*** chandankumar has joined #rdo04:22
*** sdake has joined #rdo04:24
*** fragatina has joined #rdo04:27
*** fragatina has quit IRC04:31
*** limao has quit IRC04:31
*** limao has joined #rdo04:32
*** anilvenkata has joined #rdo04:32
*** vimal has joined #rdo04:32
*** dhill_ has quit IRC04:38
*** jaosorior has joined #rdo04:45
*** jaosorior has quit IRC04:51
*** nehar has joined #rdo04:52
*** jaosorior has joined #rdo04:52
*** Poornima has joined #rdo04:53
*** jaosorior has quit IRC04:54
*** jaosorior has joined #rdo04:55
*** hynekm has joined #rdo04:55
*** eliska has joined #rdo05:09
*** jaosorior has quit IRC05:14
*** jaosorior has joined #rdo05:15
*** dmacpher has quit IRC05:15
*** stanchan has joined #rdo05:16
*** maeca1 has quit IRC05:17
*** jaosorior has quit IRC05:17
*** dmacpher has joined #rdo05:18
*** jaosorior has joined #rdo05:18
*** rcernin has joined #rdo05:31
*** livelace has joined #rdo05:34
*** links has joined #rdo05:36
*** limao_ has joined #rdo05:42
*** nmagnezi has joined #rdo05:44
*** limao has quit IRC05:45
*** mcornea has joined #rdo05:47
*** pgadiya has joined #rdo05:57
*** ganesh has joined #rdo06:01
*** ganesh is now known as Guest6538406:02
*** Guest65384 is now known as gkadam06:02
*** chlong has quit IRC06:08
*** aufi has joined #rdo06:09
*** coolsvap has joined #rdo06:14
*** jhershbe has joined #rdo06:23
*** pnavarro has joined #rdo06:35
*** stanchan has quit IRC06:36
*** chlong has joined #rdo06:40
*** danpawlik has joined #rdo06:44
*** jtomasek has joined #rdo06:44
*** chlong has quit IRC06:52
*** kaminohana has quit IRC06:55
*** Poornima has quit IRC06:56
*** nyechiel has joined #rdo06:56
*** jhershbe has quit IRC06:59
*** pnavarro has quit IRC06:59
*** nyechiel has quit IRC07:03
*** belmoreira has joined #rdo07:03
*** satya4ever has joined #rdo07:04
*** dgurtner has joined #rdo07:05
*** mosulica has joined #rdo07:06
*** gildub has quit IRC07:07
*** Poornima has joined #rdo07:12
*** saneax_AFK is now known as saneax07:12
*** stanchan has joined #rdo07:12
*** nyechiel has joined #rdo07:13
*** paramite has joined #rdo07:14
*** nyechiel has quit IRC07:15
*** nyechiel has joined #rdo07:17
*** nyechiel has quit IRC07:23
*** panda|zz is now known as panda07:24
*** rdas has joined #rdo07:24
*** jpena|off is now known as jpena07:26
jpenagood morning!07:27
jaosoriorjpena: Hey; yesterday a pin was done to mistral due to an error in it? Is it still in place? The fix just landed07:28
jpenajaosorior: we ended up not doing it, apevec wanted to give the fix some time to land (
flepiedgood morning!07:30
*** stanchan has quit IRC07:31
flepiedjpena: do you know where is the jjb repo for the RDO promotion CI?07:31
*** gfidente has joined #rdo07:32
jpenaflepied: I remember some talks yesterday about it using gerrithub, let me check07:32
flepiedthx jpena07:33
*** flaper87 has joined #rdo07:36
*** flaper87 is now known as Guest7354807:36
*** jpich has joined #rdo07:39
*** dmacpher has quit IRC07:43
*** eliska has quit IRC07:44
*** eliska has joined #rdo07:51
*** eliska has quit IRC07:55
*** shardy has joined #rdo07:55
*** eliska has joined #rdo07:55
*** dtantsur|afk is now known as dtantsur07:56
*** ohamada has joined #rdo07:58
*** dgurtner has quit IRC07:58
chandankumarjpena, flepied number80 \o/08:00
*** migi_ is now known as migi08:02
*** openstackgerrit has quit IRC08:03
*** mosulica has quit IRC08:03
*** openstackgerrit has joined #rdo08:04
*** ChanServ sets mode: +v openstackgerrit08:04
*** stanchan has joined #rdo08:04
*** jhershbe has joined #rdo08:05
*** Amita has joined #rdo08:08
*** pkovar has joined #rdo08:09
*** leanderthal|afk is now known as leanderthal08:09
*** jaosorior has quit IRC08:13
*** shardy has quit IRC08:14
*** jaosorior has joined #rdo08:14
*** toanju has joined #rdo08:14
*** shardy has joined #rdo08:14
amoralejgood morning08:14
*** nmagnezi has quit IRC08:15
*** tesseract- has joined #rdo08:16
*** Goneri has joined #rdo08:16
jpenawelcome back amoralej!08:18
amoralejthanks jpena!!!08:18
number80amoralej: hey, welcome back :)08:19
chandankumaramoralej, \o/08:23
amoralejhi chandankumar!!08:23
*** jhershbe has quit IRC08:25
*** pnavarro has joined #rdo08:27
*** lucas|dinner is now known as lucasagomes08:28
*** stanchan has quit IRC08:31
*** sasha2 has quit IRC08:31
*** d0ugal has quit IRC08:32
*** d0ugal has joined #rdo08:33
*** sasha2 has joined #rdo08:33
*** d0ugal has quit IRC08:33
*** d0ugal has joined #rdo08:33
*** sdake has quit IRC08:37
*** Guest73548 is now known as flaper8708:39
*** flaper87 is now known as Guest7864108:39
*** jlibosva has joined #rdo08:39
*** jbadiapa has joined #rdo08:39
*** Guest78641 has quit IRC08:44
rdogerritAlan Pevec proposed rdoinfo: Add openstack-tripleo-ui
rdogerritAlan Pevec proposed rdoinfo: Add openstack-tripleo-validations
*** mdavidson has joined #rdo08:52
*** pkovar has quit IRC08:59
*** iranzo has joined #rdo09:06
*** iranzo has quit IRC09:06
*** iranzo has joined #rdo09:06
*** dgurtner has joined #rdo09:08
*** dgurtner has quit IRC09:08
*** dgurtner has joined #rdo09:08
rdogerritMerged openstack/oslo-vmware-distgit: Update dependencies
*** Goneri has quit IRC09:15
*** mdavidson has quit IRC09:16
*** Ryjedo_ has joined #rdo09:17
*** Ryjedo has quit IRC09:17
*** Ryjedo_ is now known as Ryjedo09:17
*** Goneri has joined #rdo09:18
socialhmm mistral db-sync is broken?09:27
* social ;.;
*** imcleod has joined #rdo09:29
*** abehl has joined #rdo09:30
*** jbadiapa has quit IRC09:31
*** degorenko has joined #rdo09:31
*** puzzled has joined #rdo09:31
d0ugalsocial: Yeah, I think it is09:31
*** iberezovskiy has joined #rdo09:32
*** dgurtner has quit IRC09:33
d0ugalsocial: From #openstack-mistral earlier today.
d0ugalI think that is related anyway...09:34
*** iranzo has quit IRC09:35
*** paragan has quit IRC09:36
*** iranzo has joined #rdo09:37
sociald0ugal: yes it seems to be the issue09:38
flepiedadarazs: I started a pad to keep track of the best practices around debugging:
*** saneax is now known as saneax_AFK09:39
adarazsflepied: ack, nice.09:39
*** flaper87 has joined #rdo09:40
*** flaper87 has quit IRC09:40
*** flaper87 has joined #rdo09:40
*** iranzo has quit IRC09:40
*** iranzo has joined #rdo09:41
*** iranzo has quit IRC09:41
*** iranzo has joined #rdo09:41
*** iranzo has quit IRC09:41
*** limao_ has quit IRC09:41
rdogerritFabien Boucher created config: Activate experimental CBS non-scratch build in the gate pipeline
lucasagomes(not sure if I should ask here but...) I have a BZ to include a new package for ironic-staging-drivers hanging for a while. Any chance someone can take a look at when you find the right time? Thanks09:45 bug 1312328 in Package Review "New Package: openstack-ironic-staging-drivers" [Unspecified,New] - Assigned to hguemar09:45
*** jbadiapa has joined #rdo09:45
*** amoralej is now known as amoralej|brb09:46
chandankumarlucasagomes, I am doing a initial review of the package09:46
lucasagomeschandankumar, cool, thank you :-)09:46
*** saneax_AFK is now known as saneax09:48
*** iranzo has joined #rdo09:49
*** paragan has joined #rdo09:51
*** paragan has joined #rdo09:51
*** nyechiel has joined #rdo09:51
*** Amita has quit IRC09:53
*** chandankumar_ has joined #rdo09:54
*** chandankumar has quit IRC09:54
*** jbadiapa has quit IRC09:54
*** dgurtner has joined #rdo09:59
*** chandankumar_ is now known as chandankumar09:59
*** jcoufal has joined #rdo10:00
jpenaapevec: would it be a good time now to purge the old centos-master instance in the dlrn server? It'll probably take a long time, not sure if leaving it running all over the weekend would be best10:02
*** amoralej|brb is now known as amoralej10:06
*** nyechiel has quit IRC10:08
rdogerritJavier Peña created rdo-infra/puppet-dlrn: Make cron schedule for workers configurable
rdogerritJavier Peña created rdo-infra/puppet-dlrn: Use virtualenv --system-site-packages
rdogerritJavier Peña proposed rdo-infra/puppet-dlrn: Use virtualenv --system-site-packages
*** alexismonville has joined #rdo10:21
*** akrivoka has joined #rdo10:23
*** Goneri has quit IRC10:29
*** Goneri has joined #rdo10:31
*** imcleod has quit IRC10:40
*** tosky has joined #rdo10:43
*** chandankumar has quit IRC10:48
*** chandankumar has joined #rdo10:51
*** dgurtner has quit IRC10:56
*** alexismonville has quit IRC10:56
gfidentehey guys, looks like we're unable to build python-tripleoclient DLRN10:58
gfidenteas apparently some of the deps added in are missing10:58
gfidenteanybody willing to help? I am unsure why some of those should be missing and also unsure about in which repo10:59
jpenagfidente: link?10:59
gfidenteah sorry,
gfidentejpena, thanks10:59
dmsimardamoralej: woohoo, welcome back11:01
dmsimardjpena: not sure you saw this before you left yesterday11:01
*** jcoufal has quit IRC11:01
dmsimardjpena: but why is -head so much faster than s/-head// ?11:01
jpenagfidente: the error happened yesterday, while we were switching repos. Let me check if the missing packages are there now, just in case, before rechecking11:02
amoralejthanks dmsimard11:04
jpenadmsimard: that's what I was wondering yesterday11:05
*** rhallisey has joined #rdo11:07
dmsimardbahahaha.. troubleshooting an issue with a browser dev and the guy assumed I was on Windows11:09
jpenagfidente: I think the issue is in the repos the CI job is using for dependencies while building python-tripleoclient. I see mock is using the following repo configuration:
jpenait's finding some packages (such as python-cliff) because they are available from the openstack-newton repo at buildlogs, but the failing ones are not, because they're at the DLRN repos11:10
dmsimardflepied: did you get the answers you needed for the rdo ci promotion ?11:11
jpenagfidente: where is the job expecting the dlrn packages to be present?11:11
*** lucasagomes is now known as lucas-hungry11:11
gfidentejpena, sec I think I can get the .repo files from the tripleo-ci11:12
flepieddmsimard: I'm trying to collect the best practices here: . if you can add some content for the weirdo jobs, it could be cool.11:12
dmsimardflepied: I have something basic for weirdo jobs, I'll put it there, sec11:13
*** pnavarro is now known as pnavarro|lunch11:13
*** spr1 has quit IRC11:19
*** fultonj has joined #rdo11:19
*** alexismonville has joined #rdo11:24
*** nyechiel has joined #rdo11:26
*** florianf has joined #rdo11:27
*** limao has joined #rdo11:29
*** nehar has quit IRC11:32
*** nyechiel has quit IRC11:32
*** limao has quit IRC11:32
JehaneI'm still fighting with keystone.11:32
JehaneWhen trying to install a qualif platform (I have a free server who wasn't used) we are hitting bug 1330289 in openstack-packstack "Failure to install Controller/Network&&Compute Cluster on RDO Mitaka with keystone API V3" [High,Post] - Assigned to jpena11:33
*** imcleod has joined #rdo11:34
Jehaneis the up-to-date package not (yet) pushed ?11:34
*** gszasz has joined #rdo11:36
jpenaJehane: I see the latest openstack-puppet-modules build on dates back to early May, so the official packages are not updated11:36
jpenayou can use the updated packages from RDO Trunk stable/mitaka in the meantime11:36
jpenathose would be, and
Jehanethanks, I was searching the link11:37
jpena(openstack-packstack, openstack-packstack-puppet, and openstack-puppet-modules)11:37
JehaneI force it manually but the version number of the trunk package is lower than the one on the centos repos11:43
Jehanebe careful with that if you're the one pushing the update11:43
jpenammm... that means we have something wrong with tagging, /me thinks11:45
*** paramite has quit IRC11:46
*** dgurtner has joined #rdo11:47
*** dgurtner has joined #rdo11:47
Jehanethere is 0 who shouldn't be here or something close11:47
*** morazi has joined #rdo11:49
rdogerritDavid Moreau Simard created rdo-infra/weirdo: When building docs for readthedocs, pre-fetch the roles
rdogerritDavid Moreau Simard created rdo-infra/weirdo: Remove kolla from ansible-role-requirements.yml
*** weshay has joined #rdo11:59
Jehaneok, new error (yet another), packstack wan't to install the non-available "python-aodhclient" rpm12:00
JehaneI will open a bug12:00
*** Poornima has quit IRC12:01
rdogerritDavid Moreau Simard proposed rdo-infra/weirdo: Always make sure the WeIRDO roles are there before building the docs
gfidentejpena, so I think it's using the following12:02
gfidente (as delorean-current.repo locally)12:02
rdogerritDavid Moreau Simard proposed rdo-infra/weirdo: Remove kolla from ansible-role-requirements.yml
*** toanju has quit IRC12:02
*** Goneri has quit IRC12:03
gfidentejpena, is it missing something or ?12:03
jpenagfidente: no, but... it's like it isn't using the current-tripleo repo when building. Let me check why12:04
gfidentejpena, delorean-current you mean?12:06
gfidentenot sure what is current-tripleo?12:06
jpenagfidente: I mean
gfidenteah right it's not using that12:07
jpenathat repo contains the missing deps12:07
jpenawhich repo contains
*** rodrigods has quit IRC12:07
*** rodrigods has joined #rdo12:08
*** Son_Goku has quit IRC12:08
*** saneax is now known as saneax_AFK12:08
*** Son_Goku has joined #rdo12:09
gfidentejpena, those three12:09
jpenaok, I think I know what it is12:09
jpenaI thought this was fixed :?12:09
*** amoralej is now known as amoralej|lunch12:09
jpenathe command line uses --build-env DELOREAN_DEV=1 , which relies on dlrn internals (which changed some time ago)12:10
jpenait should be --dev instead12:10
gfidentejpena, ack can you push a change for this?12:10
jpenagotta run (lunch time), but I can propose a patch to tripleo-ci, then recheck your patch. Sounds like a plan?12:10
gfidentesure, thanks12:10
*** julim has joined #rdo12:12
*** dprince has joined #rdo12:13
jpenagfidente: I'm not sure if it could break some other stuff (maybe that's why it used the env var to begin with), will check after lunch12:13
*** jpena is now known as jpena|lunch12:13
*** Goneri has joined #rdo12:14
*** saneax_AFK is now known as saneax12:16
*** lmiccini_ has joined #rdo12:17
*** lmiccini has quit IRC12:17
*** pradk has joined #rdo12:18
*** jayg|g0n3 is now known as jayg12:20
*** nstrug has quit IRC12:24
*** lucas-hungry is now known as lucasagomes12:24
*** nstrug has joined #rdo12:25
*** nstrug_ has joined #rdo12:26
*** nstrug_ has quit IRC12:27
*** ashw has joined #rdo12:30
*** nyechiel has joined #rdo12:31
*** mlammon has joined #rdo12:31
sociald0ugal: I'm quite unhappy from that, isn't there rule that master never breaks?12:33
d0ugalsocial: Not that I know of, you could try asking in #openstack-mistral12:35
d0ugalsocial: I mean, I don't make the rules but I don't know of them being written down anywhere for the project12:35
*** chandankumar has quit IRC12:36
*** jdob has joined #rdo12:36
*** jhershbe has joined #rdo12:37
*** chandankumar has joined #rdo12:37
*** rlandy has joined #rdo12:43
*** aortega has joined #rdo12:45
*** jhershbe has quit IRC12:48
*** Goneri has quit IRC12:48
*** jmelvin has joined #rdo12:49
*** nyechiel has quit IRC12:51
*** jcoufal has joined #rdo12:52
EmilienMdmsimard: barbican is now part of scenario002 in our gate12:53
*** ccamacho is now known as ccamacho|lunch12:53
*** jaosorior has quit IRC12:54
*** nyechiel has joined #rdo12:55
EmilienMtrying to re-enable mistral again on scenario00312:57
EmilienMbut tempest tests are super flappy12:57
*** paramite has joined #rdo12:59
*** rbowen has joined #rdo13:00
*** ChanServ sets mode: +o rbowen13:00
*** stanchan has joined #rdo13:00
EmilienMdmsimard: what happens with designate?13:01
*** imcleod has quit IRC13:01
*** pnavarro|lunch has quit IRC13:01
*** stanchan has quit IRC13:02
*** Goneri has joined #rdo13:03
*** saneax is now known as saneax_AFK13:03
*** nyechiel has quit IRC13:04
weshayadarazs, we tracking
adarazsweshay: I was just about to add it to the issue list :/13:08
weshayadarazs, rockin13:08
weshayadarazs, hey panda and I are chatting about the instack-virt-setup blueprint13:08
weshaydo you want to join?13:08
dmsimardEmilienM: nothing much I'm focused on other things right now13:10
dmsimardEmilienM: but I had insomnia last night so I rebased all my p-o-i things :p13:10
EmilienMoh ok13:10
EmilienMbut why is it failing?13:10
dmsimardEmilienM: a bunch of random stuff, remember how it makes tempest not being able to find the aodh endpoint and mistral failures and things like that13:10
*** sdake has joined #rdo13:11
adarazsweshay: oh sorry I didn't see the notification. okay.13:12
adarazsweshay: you're bluejeans?13:12
EmilienMthat is super wweird13:12
*** saneax_AFK is now known as saneax13:12
dmsimardEmilienM: it is13:13
dmsimardEmilienM: it's something from designate interfering for sure, just gotta find what13:13
dmsimardmaybe something from the designate tempest plugin or I don't know13:13
dmsimardEmilienM: it wouldn't be the first time a tempest plugin broke tempest (see: neutron monkey patching)13:15
*** nyechiel has joined #rdo13:15
*** lmiccini has joined #rdo13:15
hewbroccaYeah I think that's pretty much the expected behavior for tempest plugins, isn't it?13:17
EmilienMdmsimard: i'll look into it13:17
*** lmiccini_ has quit IRC13:18
*** dustins has joined #rdo13:18
dmsimardhewbrocca: I see what you did there :D13:20
*** amoralej|lunch is now known as amoralej13:20
dmsimardEmilienM: Appreciate the help. I'm fairly certain the designate tempest failures are (to some extent) legit and we'll need to troubleshoot them eventually but if we could at least not have random aodh/mistral/etc errors ..13:20
dmsimardEmilienM: an interesting experiment would be to see if Ubuntu Cloud Archive packages the plugin and install it (to see if it would be something weird with RDO packaging)13:22
EmilienMyou're asking me to test ubuntu13:22
EmilienMdude it's Friday, it's supposed to be a good day.13:22
dmsimardEmilienM: Does UCA package the tempest plugins at all ?13:23
dmsimardzigo: what about Debian ? ^13:23
EmilienMdmsimard: they include it in the services13:23
*** saneax is now known as saneax_AFK13:24
dmsimardEmilienM: oh, so you apt-get install python-tempest (or something) and the plugin is (maybe?) installed in-tree ?13:24
dmsimardor er, python-nova13:24
dmsimardor python-designate, whatever13:24
* dmsimard tired13:24
EmilienMdmsimard: yes13:24
zigodmsimard: I'm not testing designate in my Tempest CI yet, because it's very complicated to install.13:24
*** stanchan has joined #rdo13:25
EmilienMzigo: use puppet13:25
*** jpena|lunch is now known as jpena13:25
dmsimardzigo: it was mostly a general question around how (if at all) tempest plugins in general are packaged13:25
EmilienMI think I told you that 92384 times13:25
zigodmsimard: I do package them all.13:25
*** gkadam has quit IRC13:25
zigodmsimard: Though Designate is Canonical maintained, they don't sync it from Debian, so I can't tell what they do.13:26
zigo(you'd have to check by yourself)13:26
zigoEmilienM: I always wanted to do *both* tempest *and* packaged install.13:26
*** rdas has quit IRC13:26
zigoEmilienM: The packaging-deb project is finally getting into shape, with the few last infra patches going through, so we'll get there ! :)13:27
*** egafford has joined #rdo13:27
*** ohochman has joined #rdo13:32
*** ccamacho|lunch is now known as ccamacho13:33
pandapabelanger: Hi, can you send a brief email with an outline of all the changes you want to make in quickstart, and what is the long-term goal ? I'm a bit lost ..13:34
jpenagfidente: I've updated with what I think is the right fix.13:34
*** jeckersb_gone is now known as jeckersb13:34
*** ohochman has quit IRC13:34
*** ohochman has joined #rdo13:35
*** eliska has quit IRC13:36
*** richm has joined #rdo13:38
Jehanejpena: there is a small issue with the packstack rpm you show me a few hours ago13:38
*** stanchan has quit IRC13:38
* jpena is afraid of "small issues"13:38
*** Son_Goku has quit IRC13:38
Jehanefor aodh, it's trying to install the rpm python-aodhclient but the package is called python2-aodhclient13:38
Jehanejpena: easy fix but I don't know where to look in the code :)13:39
*** dcain has joined #rdo13:39
jpenaoh damn virtul packages13:39
*** dgurtner has quit IRC13:39
jpenavirtual, I mean13:39
Jehaneyep, that was my supposition13:39
*** stanchan has joined #rdo13:39
pabelangerpanda: Sure, for the most part I am experimenting with some things in quickstart.  End game would be to run tripleo-quickstart upstream in OpenStack, which we just landed an experimental job to help test that.13:40
*** maeca1 has joined #rdo13:41
jpenaJehane: the fastest fix is to install it manually, then re-run packstack with the same answer file.13:41
pabelangerpanda: but, yes, I plan to send out an email explaining some of the changes once I have examples of it working13:41
jpenaJehane: I proposed for a more permanent fix some time ago13:41
Jehanejpena: already done ;) but I wanted to report it can be fixed for everyone13:41
pandapabelanger: ok, I'll wait for rewiewing more then13:42
*** danpawlik has quit IRC13:42
*** chandankumar has quit IRC13:45
*** kgiusti has joined #rdo13:47
pabelangerpanda: sure, if you want to see progress on the experimental job, you can use 'check experimental' in a quickstart review which will start the experimental job.13:48
pabelangerpanda: it won't work right now, but its the first step to running it in the gate13:48
*** jhershbe has joined #rdo13:49
*** dcain1 has joined #rdo13:58
*** dcain has quit IRC14:00
*** jhershbe has quit IRC14:02
pabelangeris == ?14:02
*** openstackgerrit has quit IRC14:03
*** satya4ever has quit IRC14:03
jpenapabelanger: no. The first one is the official RDO repo, built from CBS. The second one is current stable/mitaka status, packaged by DLRN14:04
*** openstackgerrit has joined #rdo14:04
*** ChanServ sets mode: +v openstackgerrit14:04
pabelangerjpena: okay, does dlrn eventually flow into
*** stanchan has quit IRC14:07
jpenapabelanger: it should, but it's a manual process. dlrn follows the git repos, and CBS uses tagged versions, but eventually any fix added to dlrn should make its way into CBS14:08
pabelangerjpena: great, that helps. Thanks14:08
*** stanchan has joined #rdo14:08
*** lmiccini has quit IRC14:09
*** dgurtner has joined #rdo14:11
*** coolsvap has quit IRC14:11
*** dgurtner has quit IRC14:13
*** lmiccini has joined #rdo14:19
*** vimal has quit IRC14:21
*** mlammon1 has joined #rdo14:23
*** pgadiya has quit IRC14:25
*** jeckersb is now known as jeckersb_gone14:26
*** mlammon has quit IRC14:26
*** stanchan has quit IRC14:28
*** jeckersb_gone is now known as jeckersb14:29
*** shaunm has joined #rdo14:29
*** stanchan has joined #rdo14:30
*** stanchan has quit IRC14:31
*** aortega has quit IRC14:32
*** stanchan has joined #rdo14:34
rdogerritMerged rdo-infra/weirdo: Always make sure the WeIRDO roles are there before building the docs
rdogerritMerged rdo-infra/weirdo: Remove kolla from ansible-role-requirements.yml
*** aortega has joined #rdo14:36
*** Goneri has quit IRC14:36
myoungweshay, dmsimard: it looks like some of the image gates were failing last night because the overcloud validate is pushing job times > 180 mins.  Via web UI I increased the max job time --> 240 to get things unstuck & passing,but that's a temporary stop-gap measure. IMHO 3 hours is already too long.  also, who pushes JJB for those jobs (trown|outtypewww)?14:38
*** pgadiya has joined #rdo14:39
dmsimardmyoung: 3 hours is pretty damn long, yeah. What is that job doing anyway ? Just installing packages inside an image ?14:39
dmsimardor is that the whole undercloud image thing14:39
myoungdmsimard: no, for the image build role for tripleo-quickstart, we build an image, then verify that a basic tripleo-quickstart deploy works (basic tempest smoke, and some ping tests to make sure overcloud is actually up and operational)14:40
myoungthis is when "images-ci-check" appears in a comment (gerrit hook)14:40
dmsimardmyoung: so basically it's almost a full pipeline job14:40
dmsimardi.e, build image, then test that image14:40
myoungas policy for ansible-role-tripleo-image-build we like to see green gate jobs for things that affect images since breaking that is bad news :)14:41
dmsimardthat does take pretty long in the promote pielines ..14:41
dmsimardfair, there needs to be a way to make ooo-q in pipeline faster in general14:41
myoungthe jobs used to hover around 2:40 total, but it looks like a little while ago we added some additional checks in the overcloud validate (this is a good thing btw) and it just pushes things over by a bit14:41
dmsimardtimeouts have been an issue for many jobs, increasing them just hides the issue under the carpet14:42
*** chandankumar has joined #rdo14:42
dmsimardwe can blame the (slower) CPUs all we want but >1:30hr for a job is pretty darn long14:42
*** ayoung_ZZZzzz is now known as ayoung14:42
myoungdmsimard: aye...I've done some initial analysis.  we already do some good things (like building IPA and overcloud DIB passes in parallel), but there is lots more that can be done.  I have a card from months ago enumerating ways to speed things up I'll revive and port over to RDO board.14:42
dmsimardmyoung: we could work with a cached base image or something14:43
dmsimardmyoung: like iirc we download the centos cloud image and then install stuff in it14:43
dmsimardmyoung: we could install everything that is not openstack-related in that image, and then download that "pre-installed" image instead14:43
dmsimardmyoung: for example for weirdo jobs, there are >300 packages that aren't openstack related that are installed throughout the jobs14:45
myoungdmsimard: {nod} there's a variety of things we can do.  Perhaps we should just schedule a bit of time to chat, I've been looking into this and have some ideas.  I'll move/create the card, and here's how it works today:
* myoung looks14:46
*** fragatina has joined #rdo14:46
myoungooooo.  i like.  I'll spin a patch for artib to leverage this.  How much time in practice do you see saved?14:47
*** iranzo has quit IRC14:47
dmsimardmyoung: whatever time it took to install 365 packages14:47
*** fragatin_ has joined #rdo14:48
myoungbtw the actual image building is only one component of the 3+ hour job times, but anything we can do to speed up image building (pile of network + i/o on slow machines...) makes sense to do14:48
*** eharney has joined #rdo14:49
*** nyechiel has quit IRC14:49
*** anilvenkata has quit IRC14:49
dmsimardmyoung: so what you need to do to leverage something like that is to 1) run that script on a oooq journalctl log 2) recover list of pacakges 3) download base image 4) install those packages in base image 5) upload base image to artifacts ... 6) download that base image in relevant jobs (instead of original cloud image) 7) profit14:50
*** lmiccini_ has joined #rdo14:51
*** fragatina has quit IRC14:51
dmsimardthere's an assumption in that script that expects a gzipped journalctl (that's how it ends up in weirdo..) not sure about ooq14:53
*** jeckersb is now known as jeckersb_gone14:54
*** lmiccini has quit IRC14:54
myoungdmsimard: hrm...i kind of like the idea of a periodic job that does all of that (with minimal tire kicking).    presently the image building role starts with a user image (starting point) that can be just passed as a parameter:  It currently defaults to
myoungdmsimard ^ s/user//14:57
dmsimardyeah that default to could be our new pre-built image14:58
dmsimardThat would be generated by a periodic job.. could be once per 24hr or something14:58
*** aortega has quit IRC14:59
*** aortega has joined #rdo15:01
*** jeckersb_gone is now known as jeckersb15:02
*** dgurtner has joined #rdo15:02
*** d0ugal has quit IRC15:05
*** aufi has quit IRC15:07
*** shardy is now known as shardy_afk15:07
*** dgurtner has quit IRC15:07
*** hynekm has quit IRC15:07
hewbroccaflepied: řéřččá15:08
flepiedhewbrocca: you speak yubikey natively? :-)15:08
*** dhill_ has joined #rdo15:09
*** dhill_ has quit IRC15:10
*** dhill_ has joined #rdo15:10
*** fragatin_ has quit IRC15:10
*** gszasz has quit IRC15:13
*** jistr is now known as jistr|mtg15:14
*** stanchan has quit IRC15:14
*** gszasz has joined #rdo15:16
*** stanchan has joined #rdo15:18
*** d0ugal has joined #rdo15:18
*** tesseract- has quit IRC15:21
*** hewbrocca is now known as hewbrocca-afk15:24
*** rcernin has quit IRC15:26
*** stanchan has quit IRC15:28
*** stanchan has joined #rdo15:29
*** links has quit IRC15:33
*** belmoreira has quit IRC15:40
*** stanchan has quit IRC15:42
*** beagles is now known as beagles_food15:42
*** stanchan has joined #rdo15:42
*** mdnadeem has quit IRC15:43
*** stanchan has quit IRC15:43
*** dgurtner has joined #rdo15:44
*** dgurtner has joined #rdo15:44
*** pgadiya has quit IRC15:48
*** florianf has quit IRC15:48
*** florianf has joined #rdo15:49
*** stanchan has joined #rdo15:50
*** florianf has quit IRC15:53
*** abehl has quit IRC15:54
*** iberezovskiy is now known as iberezovskiy|off15:55
*** sdake has quit IRC15:56
*** sdake has joined #rdo15:56
*** stanchan has quit IRC16:01
*** paragan has quit IRC16:03
*** dgurtner has quit IRC16:04
*** stanchan has joined #rdo16:05
*** pcaruana has quit IRC16:05
*** paragan has joined #rdo16:06
*** beagles_food is now known as beagles16:07
*** dtantsur is now known as dtantsur|afk16:08
*** fragatina has joined #rdo16:08
*** fragatina has quit IRC16:09
*** jeckersb is now known as jeckersb_gone16:12
*** shardy_afk is now known as shardy16:12
*** jeckersb_gone is now known as jeckersb16:12
*** amoralej is now known as amoralej|off16:13
*** READ10 has joined #rdo16:15
*** stanchan has quit IRC16:16
*** stanchan has joined #rdo16:17
*** mcornea has quit IRC16:17
rdogerritJavier Peña created DLRN: [WIP] Parallel mock builds in DLRN
*** tumble has quit IRC16:20
*** degorenko has quit IRC16:21
*** hrw has quit IRC16:21
*** hrw has joined #rdo16:23
*** fragatina has joined #rdo16:24
*** rasca has quit IRC16:27
*** jlibosva has quit IRC16:27
*** Tenhi has quit IRC16:28
*** rwsu has quit IRC16:28
*** coolsvap has joined #rdo16:29
*** myoung has quit IRC16:29
*** Tenhi has joined #rdo16:30
*** myoung has joined #rdo16:30
*** ohamada has quit IRC16:30
rdogerritJavier Peña proposed DLRN: [WIP] Parallel mock builds in DLRN
*** snecklifter has joined #rdo16:31
*** rwsu has joined #rdo16:31
*** paramite has quit IRC16:33
*** lucasagomes is now known as lucas|dinner16:36
*** jpich has quit IRC16:36
*** gszasz has quit IRC16:39
*** stanchan has quit IRC16:41
*** d0ugal has quit IRC16:44
*** paragan has quit IRC16:47
imcsk8jpena: just merged:
*** fragatin_ has joined #rdo16:53
*** fragatina has quit IRC16:55
jpenaimcsk8: thanks! if you have time, pls have a look at, we need it to unblock packstack CI16:55
imcsk8jpena: i will, i'm checking reviews16:56
jpenacool, thx!16:56
*** paragan has joined #rdo16:56
*** paragan has joined #rdo16:56
*** mlammon1 has quit IRC16:56
jpenatime to go, have a nice weekend everyone!16:57
*** jpena is now known as jpena|off16:58
*** numans has quit IRC17:00
*** stanchan has joined #rdo17:01
*** bandini has quit IRC17:03
rdogerritrdo-trunk created openstack/horizon-distgit: python-django-horizon: failed to build 216e673
*** d0ugal has joined #rdo17:12
rdobot[sensu] NEW: - check-delorean-newton-current @ |#| Build failure on centos7-master/current: horizon:
*** itlinux has quit IRC17:15
*** Tenhi has quit IRC17:17
*** dustins has quit IRC17:17
*** Tenhi has joined #rdo17:25
*** puzzled has quit IRC17:26
*** tosky has quit IRC17:27
*** pradk has quit IRC17:30
*** akrivoka has quit IRC17:32
*** pradk has joined #rdo17:33
*** stanchan has quit IRC17:34
*** paragan has quit IRC17:36
*** chandankumar has quit IRC17:40
rdogerritrdo-trunk created openstack/cliff-distgit: python-cliff: failed to build 9f42088
*** dgurtner has joined #rdo17:43
openstackgerritMerged openstack/packstack: Fix pep8 and py27 tests in Ubuntu Xenial
openstackgerritMerged openstack/packstack: Create Gnocchi legacy resources
*** abehl has joined #rdo17:45
*** Tenhi has quit IRC17:47
*** leanderthal is now known as leanderthal|afk17:47
*** ddmitriev has quit IRC17:48
*** paragan has joined #rdo17:48
*** eharney has quit IRC17:49
*** Tenhi has joined #rdo17:50
*** dustins has joined #rdo17:51
*** fragatin_ has quit IRC17:52
*** fragatina has joined #rdo17:53
rdobot[sensu] NEW: - check-delorean-newton-head-current @ |#| Build failure on centos7-master-head/current: cliff:
*** weshay has quit IRC17:58
*** sarath has joined #rdo17:59
*** weshay has joined #rdo18:01
*** shardy is now known as shardy_afk18:03
*** eharney has joined #rdo18:03
*** stanchan has joined #rdo18:10
*** JuanDRay has joined #rdo18:10
*** eharney_ has joined #rdo18:13
*** JuanDRay has quit IRC18:14
*** eharney has quit IRC18:15
*** eharney_ has quit IRC18:19
*** stanchan has quit IRC18:26
*** eharney_ has joined #rdo18:33
*** gfidente has quit IRC18:36
*** nstrug has quit IRC18:40
*** iranzo has joined #rdo18:44
*** READ10 has quit IRC18:44
*** dcain1 has quit IRC18:46
*** iranzo has quit IRC18:52
*** livelace has quit IRC18:52
*** stanchan has joined #rdo18:53
*** dhill_ has quit IRC18:54
*** dyasny has quit IRC18:58
*** fcoelho has quit IRC18:58
*** dhill_ has joined #rdo18:58
*** dgurtner has quit IRC19:02
*** stanchan has quit IRC19:05
*** dyasny has joined #rdo19:07
*** shaunm has quit IRC19:08
rdobot[sensu] NEW: - check-delorean-newton-current @ |#| Build failure on centos7-master/current: horizon:
dmsimardmyoung: protip
myoungdmsimard: oooooo.  neat.19:10
* myoung thanks dmsimard19:10
*** eharney_ is now known as eharney19:16
dmsimardI added a tl;dr description to the card19:16
*** Son_Goku has joined #rdo19:17
*** dneary has quit IRC19:21
*** RoozbehShafiee has joined #rdo19:24
*** sdake has quit IRC19:26
*** sdake has joined #rdo19:28
pabelangermyoung: dmsimard: so, would be interesting to talk to both of you about some jobs upstream I am wanting to try. Specifically building the overcloud-full.qcow2 image.  While it might not do everything that ansible-role-tripleo-image-build will, I think there is an option to make better pipelines then have a single job build for 3+ hours19:31
pabelangermyoung: dmsimard: review in question is and adds an experimental job into python-tripleoclient19:32
dmsimardpabelanger: I'm not very knowledgeable about triple-o (and ooo-q) but I can try to help19:33
myoungpabelanger: ack, rope me in.  note that the bulk of that time isn't spent building an image but testing it...19:33
myoungactual image build times are more like 35ish mins19:33
myoungand i'm working on ways to make that even faster19:34
myoungalso these gates are only run when we need to make a change to the image building role *itself*19:34
pabelangermyoung: dmsimard: how I see it working, is we'll launch parallel jobs to build all the images, and only start validation once all image builds are green. Then maybe at a later time or in another pipeline, do that validation of the images19:34
dmsimardmyoung: it takes around 1hr on average to build the image, not negligible
pabelangermyoung: right, this is POC to see how things look. controling when we run the jobs can be updated to be when ever we want19:35
pabelangerdmsimard: is that all images?19:35
dmsimardpabelanger: hum, that's just the job that builds the undercloud image and uploads it, I think.19:35
dmsimardfor one release (i.e, in this case delorean newton)19:36
pabelangermyoung: another thing we have upstream, is a pretty good caching system in place.  So we can help reduce the time downloading bits, but adding things to the cache19:36
pabelangerdmsimard: I can see that, it takes us about 1 hour to build a new DIB for nodepool per label19:36
pabelangerhave our images are 5G, not sure what they are for delorean19:37
dmsimardpabelanger: not built with dib however19:37
pabelangerdmsimard: oh?19:37
dmsimardpabelanger: it downloads base centos image and builds on top of that19:37
myoungpabelanger, dmsimard: we're talking about a few different things here.  build role gates themselves actually build am image, then use it in place for the validation, so there are no copies19:38
pabelangerdmsimard: Ya, I want to change that. and use centos-minimal, since your basically using a binary as your base.  If that image was to be deleted, you have zero way of reproducing it19:38
myoungthere is also a gating role (oooq) that can be used to modify existing images quickly in response to a patch...quick virt-customize --> rpm changes --> test19:38
dmsimardpabelanger: well to be fair, dib images aren't exactly guaranteed to be reproducible either19:39
pabelangermyoung: right, I'm looking at just building images right now and only building them in the gate. Validation will only happen after I get that process finished19:39
dmsimardpabelanger: repositories can change and all that19:39
*** alexismonville has quit IRC19:40
myoungdmsimard: re: 38 vs 1 hr, correct, I had wires crossed.  RDO machines seem to be a little slower :(19:40
pabelangerdmsimard: Right, but using -minimal images, gets you closer to that.   We had to switch to it after canonical started deleting ubuntu cloud qcow2 images from upstream19:40
*** alexismonville has joined #rdo19:42
*** alexismonville has quit IRC19:43
*** rbowen has quit IRC19:48
*** gfidente has joined #rdo19:50
rdobot[sensu] NEW: - check-delorean-newton-head-current @ |#| Build failure on centos7-master-head/current: cliff:
*** RoozbehShafiee has quit IRC19:51
*** aortega has quit IRC19:53
bnemecIt looks like we're using master gnocchiclient in Mitaka, and they just merged a patch that broke us:
bnemecosc-lib doesn't exist in Mitaka so that can't work.19:54
rdogerritDavid Moreau Simard proposed openstack/cliff-distgit: Remove announce.rst from packaging
dmsimardbnemec: we're probably not using master gnocchiclient, no19:56
bnemecExample failure caused by this:
dmsimardbnemec: swhich repo, delorean or stable ?19:56
bnemecdmsimard: It's a mitaka CI job.19:56
dmsimardbnemec: mitaka delorean or mitaka stable ? :)19:56
*** ppowell has joined #rdo19:57
bnemecI believe we use all dlrn repos.19:58
dmsimardbnemec: hrm19:59
dmsimardpradk: o/19:59
dmsimardpradk: should gnocchiclient have source branches set ?
pradkdmsimard, we could, but generally client doesnt go by releases. gnocchiclient is expected to be compatible with multiple releases.. unless otherwise specified.. so i guess we just set it to latest source version?20:02
dmsimardpradk: bnemec mentioned that a recent commit might've broken mitaka20:03
* pradk reads back20:03
bnemecYeah, the osc-lib commit breaks older releases.20:03
bnemecBecause osc-lib is a shiny new thing. :-)20:03
dmsimardbnemec: telemetry likes shiny new things :P20:04
pradkhmm i see.. we could set mitaka to 2.1 if that alleviate this issue20:04
dmsimardpradk: gnocchiclient is somehow at 2.4.1 in dlrn, not sure how that can happen20:05
pradkdmsimard, thats building off master?20:06
dmsimardoh, nevermind20:06
dmsimardyeah I guess it takes the latest (2.4.0) and builds off of that20:06
dmsimardhence 2.4.120:06
pradkyea we just grab --version20:06
dmsimardprobably 2.3.1 would be fine ? /me shrugs20:06
pradkdmsimard, i would think it so .. we need atleast 2.1 for sure, we can just set it to 2.220:07
dmsimardyou're the pro20:07
pradkdmsimard, so if we set the source to 2.2 it will only build from that tag?20:08
pradksource here is tag or branch?20:08
pradkdmsimard, reason i ask is.. in gnocchiclient we have
pradkthats the latest stable branch20:09
pradkif we can point mitaka to stable/2.2 we should be safe20:10
*** ohochman has left #rdo20:12
dmsimardah, yes, 2.2... I was looking at the releases20:12
dmsimardwhich goes up to 2.420:12
*** pnavarro|lunch has joined #rdo20:13
*** sarath has quit IRC20:13
rdogerritMerged openstack/cliff-distgit: Remove announce.rst from packaging
pradkdmsimard, ok i can set that in rdoinfo.yml and submit a patch?20:16
dmsimardsounds sane to me, I think we would need to do something similar for liberty -- or was gnocchiclient not there yet for liberty ?20:16
pradkso want to clarify whats source-branch here referring to?20:18
pradkthe tag or the branch?20:18
dmsimardit's a ref, it can be either20:19
dmsimardexcept if you put a tag, dlrn is effectively not going to be building new commits20:20
dmsimarda branch, however, will have new commits in it20:20
pabelangerdmsimard: 36af0d94f98ec7f4ade790ba7382a76b3ac1f70f would have been a perfect commit to test the 3rdparty CI jobs were talking about. I would expect the %check macro from python-gnocchiclient to fail20:20
dmsimardpradk: we just, for example, created the newton-uc concept where libs and clients are pinned to the tags defined in upstream uppper-constraints20:20
pabelangerdmsimard: also, I'm free if you are to work on zuul.conf.ergb20:21
dmsimardpabelanger: more or less, gnocchiclient would have built successfully20:21
dmsimardpabelanger: it's a runtime dependency, not a build dependency20:21
pradkdmsimard, k so i'll set source-branch: stable/2.0 then20:21
dmsimardpabelanger: or else we would've got a failure to build from source long before it would've landed in bnemec's broken CI :)20:21
pabelangerdmsimard: right, but hopefully unit tests would have caught it20:21
dmsimardpabelanger: I'm not sure if we typically run unit test systematically in the builds20:22
pabelangerdmsimard: since we need to install Requires into spec file20:22
*** jayg is now known as jayg|g0n320:22
*** Son_Goku has quit IRC20:22
dmsimardpabelanger: right, but that's a runtime dependency20:22
pabelangerdmsimard: right, I think the number is 24% atm of spec files run testr20:22
*** beagles is now known as beagles_bl8r20:22
pabelangergoing to look at spec file now20:22
rdogerritPradeep Kilambi created rdoinfo: Set gnocchiclient mitaka to stable/2.2
pradkdmsimard, ^^20:23
dmsimardpabelanger: i.e, gnocchiclient could have for example sqlalchemy as a runtime dependency.. but since it's not used/imported at build time, dlrn will build it successfully and it will crash rather horrible once the package lands in the repos20:23
dmsimardpradk: can you add a note that this is to pin before the introduction of osc_lib or something which does not exist in <newton ?20:24
dmsimardpradk: I can't quite +2 rdoinfo reviews but I'll +1 and add number80 and apevec :)20:24
pradkdmsimard, note in the yml or commi message?20:24
dmsimardpradk: commit20:24
pradkwill do20:24
dmsimardpradk: put the review too:
pabelangerdmsimard: I would expect it to look like which number80 recently did20:25
dmsimardpradk: and btw just noticed aodhclient is doing the same thing in
pabelangerdmsimard: also, I cannot find the python-gnocchiclient spec file. Are you able to help with that?20:26
dmsimardahhh, jschlueter :)20:26
pradkhe's doing it for everything :)20:26
pabelangerdmsimard: ah, thanks. Looking in the wrong repo20:26
dmsimardpradk: so he's going to break everything20:26
dmsimardand then jschlueter comes to us, all sad, when we break things !20:27
dmsimard /friday20:27
pabelangerdmsimard: yes, we don't run testr for python-gnocchiclient. Let me see if I can fix that20:27
dmsimardpabelanger: you're probably going to need quite a lot more build dependencies for that20:28
rdogerritPradeep Kilambi proposed rdoinfo: Set gnocchiclient mitaka to stable/2.2
pabelangerdmsimard: yup20:28
rdogerritDavid Moreau Simard proposed rdoinfo: Set gnocchiclient mitaka to stable/2.2
jschlueterdmsimard, pradk: sorry if I mixed things up but that deprecation warning that is in master of openstackclient really does mess up installs, if it's too soon we can put a -2 on the reviews or -1 workflow20:30
* jschlueter has to run but will check back later20:30
*** null_ref has quit IRC20:32
jschlueterthere are 4 or 5 instances across the packages that may be hit ... and a revert of that patch would then be in order ... why was that not caught at gate?20:32
dmsimardjschlueter: which gate ?20:33
*** aortega has joined #rdo20:33
dmsimardthat's a tripleo failure, there's no tripleo CI on gnocchiclient :)20:34
*** ohochman has joined #rdo20:34
*** ohochman has left #rdo20:35
pabelangerjschlueter: were working on a POC to run rpmbuild from against patches to, 36af0d94f98ec7f4ade790ba7382a76b3ac1f70f would have been a perfect commit to potential be exposed if the POC was working20:36
pabelangerjschlueter: and alert people from 18days ago to 13hour when it was merged20:37
*** shardy_afk is now known as shardy20:38
*** zaneb has quit IRC20:39
jschlueterdmsimard: see for related patches20:40
jschlueterpabelanger: yep it would have been20:40
pabelangerjschlueter: it won't take long to get the POC working, working with dmsimard on that. But should give downstream plenty of heads on of breaking changes hit review.o.o20:42
*** ppowell has quit IRC20:42
dmsimardpabelanger: finishing up on a failure to build and I'm all yours20:42
pabelangerdmsimard: \o/20:43
rdogerritHonza Pokorny proposed rdoinfo: Add openstack-tripleo-ui
dmsimardjruzicka: you're probably gone by now, yes ? :)20:44
dmsimardpabelanger: crap, this might take a while :/20:45
dmsimardpabelanger: and your CI would've also caught this one :)20:45
rdobot[sensu] RESOLVED: - check-delorean-newton-head-current @ |#| No build failures detected:
*** jeckersb is now known as jeckersb_gone20:46
rdogerritHonza Pokorny proposed rdoinfo: Add openstack-tripleo-ui
pabelangerdmsimard: ya, for catching more things20:47
dmsimardpabelanger: horizon just introduced 3 new deps that are not packaged anywhere20:47
dmsimardthis is the worst kind of failures :(20:47
rdogerritpabelanger created openstack/gnocchiclient-distgit: Run unit tests during package build
pabelangerdmsimard: which commit?20:48
rdogerritGabriele Cerami created openstack/horizon-distgit: Increase systemd start timeout
dmsimardpabelanger: also TIL gerrit can render looks of disapproval in reviews
dmsimardpabelanger: I shall make use of that.20:49
pabelangerdmsimard: making an etherpad to track some of this20:50
rdogerritHonza Pokorny proposed rdoinfo: Add openstack-tripleo-ui
*** Son_Goku has joined #rdo20:51
*** zaneb has joined #rdo20:52
*** beagles_bl8r is now known as beagles20:54
dmsimardpabelanger: I'm going to... try to pick up the three new deps20:55
dmsimardpabelanger: sorry, I have to get the ball rolling so we're not too far behind20:55
dmsimardpabelanger: if you want I can paste you the zuul config so we have a starting point or sometihng20:56
dmsimardnumber80: three new xstatic dependencies to package.. I'll give it a shot20:56
pabelangerdmsimard: sure, let me get the syntax to you that needs to be added20:57
*** jdob has quit IRC20:58
*** Son_Goku has quit IRC20:58
*** shardy has quit IRC20:58
*** kgiusti has left #rdo21:00
*** marrusl has quit IRC21:02
*** pradk has quit IRC21:03
*** rbrady is now known as rbrady-afk21:07
pabelangerdmsimard: lets use this as a metric to track failures we think 3rdparty CI would catch:
rdobot[sensu] NEW: - check-delorean-newton-current @ |#| Build failure on centos7-master/current: horizon:
*** sdake has quit IRC21:12
*** dprince has quit IRC21:12
*** pnavarro|lunch has quit IRC21:12
dmsimardpabelanger: probably a deal of those too:
pabelangerdmsimard: yes, maybe I'll add the info their21:16
dmsimardpabelanger: some of those aren't packaging-specific issues (some are)21:16
* pabelanger nods21:16
dmsimarda lot of them stem from using master against master21:16
rdogerritrdo-trunk created openstack/horizon-distgit: python-django-horizon: failed to build 87818bc
pabelangerdmsimard: this is basically the syntax we need to add to zuul.conf.erb:
pabelangerhardcoded values obviously21:23
pabelangerbobsmith is our user in review.openstack.org21:23
pabelangerdmsimard: once that is in place, and zuul has been restarted, we should see events from gerrit but no jobs will build, since we haven't updated layout.yaml21:23
pabelangerlets do that first, then setup the pipelines21:24
*** Son_Goku has joined #rdo21:24
*** stanchan has joined #rdo21:24
*** fzdarsky_ has quit IRC21:25
rdobot[sensu] NEW: - check-delorean-newton-head-current @ |#| Build failure on centos7-master-head/current: horizon:
*** ashw has quit IRC21:32
*** bnemec is now known as beekneemech21:36
rdogerritMerged rdoinfo: Set gnocchiclient mitaka to stable/2.2
dmsimardpabelanger: you minimum familiar with koji ?21:44
dmsimardpabelanger: I'm a super duper koji noob and all the pros are already out for the weekend :)21:45
dmsimardpabelanger: tried a scratch build and it failed, says to read build.log but where is that ?
*** stanchan has quit IRC21:46
dmsimardpabelanger: ok, found it21:51
*** fultonj has quit IRC21:53
*** sdake has joined #rdo21:54
*** dustins has quit IRC21:54
*** morazi has quit IRC21:58
*** tdasilva has quit IRC21:59
*** rlandy has quit IRC22:00
*** jcoufal_ has joined #rdo22:00
*** weshay has quit IRC22:02
*** jcoufal has quit IRC22:04
*** paragan has quit IRC22:04
*** paragan has joined #rdo22:05
*** paragan has joined #rdo22:05
*** panda is now known as panda|zz22:06
*** stanchan has joined #rdo22:11
*** milan has quit IRC22:12
*** jcoufal_ has quit IRC22:15
*** thrash is now known as thrash|wknd22:16
*** jcoufal has joined #rdo22:20
*** maeca1 has quit IRC22:21
*** gfidente has quit IRC22:22
*** stanchan has quit IRC22:25
*** tdasilva has joined #rdo22:27
*** stanchan has joined #rdo22:32
*** jcoufal has quit IRC22:35
*** jcoufal has joined #rdo22:40
*** imcsk8 has quit IRC22:50
*** imcsk8 has joined #rdo22:50
*** scorcoran has joined #rdo23:01
*** stanchan has quit IRC23:08
*** stanchan has joined #rdo23:09
rdobot[sensu] NEW: - check-delorean-newton-current @ |#| Build failure on centos7-master/current: horizon:
*** rhallisey has quit IRC23:13
*** stanchan has quit IRC23:23
rdobot[sensu] NEW: - check-delorean-newton-head-current @ |#| Build failure on centos7-master-head/current: horizon:
*** jcoufal has quit IRC23:30
*** sdake has quit IRC23:31
*** aortega has quit IRC23:31
EmilienMdmsimard: I guess you're aware about
openstackLaunchpad bug 1610411 in tripleo "Mitaka jobs broken by gnocchiclient move to osc-lib" [Critical,Triaged]23:32
EmilienMI haven't followed all the bits23:32
EmilienMbut is there anything I can do?23:32
dmsimardEmilienM: pradk submitted an update to rdoinfo and it was merged
dmsimardtl;dr gnocchiclient should build 2.2 instead of 2.4.1 for mitaka23:33
dmsimardbut I don't know if/how/when it would be rebuilt 2.223:33
EmilienMso fixed now?23:33
*** manous has joined #rdo23:40
*** jcoufal has joined #rdo23:41
*** stanchan has joined #rdo23:41
*** dtrainor has quit IRC23:41
*** dtrainor has joined #rdo23:41
*** myoung is now known as myoung|afk23:46

Generated by 2.14.0 by Marius Gedminas - find it at!