Thursday, 2016-05-26

*** openstack has joined #tripleo00:03
*** chem`` has quit IRC00:07
*** rbrady has quit IRC00:07
*** r-mibu has quit IRC00:26
*** r-mibu has joined #tripleo00:26
*** tzumainn has quit IRC00:35
*** penick has quit IRC00:42
*** chlong has quit IRC00:46
*** openstackstatus has quit IRC00:46
*** panda has quit IRC00:48
*** thrash is now known as thrash|g0ne00:48
*** panda has joined #tripleo00:49
*** openstack has joined #tripleo00:54
*** dkelson has joined #tripleo01:10
dkelsonI'm trying to do some customization of my overcloud controller using NodeExtraConfigPost with puppet, but when I run "openstack service list" (after setting the appropriate env variables using hieradata for the password, and URL) I get "ERROR: openstack The service catalog is empty.". However logging into the controller, setting the same env variables and running the same command results in the expected output.01:15
*** fzdarsky_ has joined #tripleo01:18
openstackgerritBen Swartzlander proposed openstack/tripleo-heat-templates: Add integration with NetApp Manila driver
*** fzdarsky|afk has quit IRC01:22
*** TSCHAK has joined #tripleo01:27
openstackgerritLiu Qing proposed openstack/diskimage-builder: Fix variable unbound error while REPOREF="*"
*** TSCHAK has quit IRC01:44
*** openstackgerrit has quit IRC01:49
*** TSCHAK has joined #tripleo01:52
*** rhallisey has quit IRC01:53
*** TSCHAK has quit IRC01:55
*** dmacpher-afk has joined #tripleo01:57
*** openstackgerrit has joined #tripleo01:57
*** chlong has joined #tripleo01:59
*** TSCHAK has joined #tripleo02:08
*** saneax_AFK is now known as saneax02:24
*** myoung has joined #tripleo02:24
*** openstack has joined #tripleo02:38
*** openstackstatus has joined #tripleo02:40
*** ChanServ sets mode: +v openstackstatus02:40
*** myoung has quit IRC02:40
*** dmacpher-afk is now known as dmacpher03:11
*** saneax is now known as saneax_AFK03:14
*** julim has joined #tripleo03:24
*** julim has quit IRC03:30
*** saneax_AFK is now known as saneax03:30
*** links has joined #tripleo03:34
*** karthiks has quit IRC03:45
*** ramishra has quit IRC03:59
*** rlandy has quit IRC04:10
*** saneax is now known as saneax_AFK04:14
*** chlong has quit IRC04:15
*** yamahata has joined #tripleo04:17
*** masco has joined #tripleo04:21
*** chlong has joined #tripleo04:34
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Tempest support for quickstart
*** panda has quit IRC04:48
*** panda has joined #tripleo04:49
*** gumbyblake has joined #tripleo04:59
*** jaosorior has joined #tripleo05:05
*** stendulker has joined #tripleo05:12
*** dsariel has joined #tripleo05:15
*** yolanda_ has quit IRC05:23
coolsvapmorning folks05:29
*** bnemec has quit IRC05:30
*** bnemec has joined #tripleo05:31
jaosoriorcoolsvap: Hey man05:33
*** mikelk has joined #tripleo05:33
coolsvapjaosorior, hello05:34
*** apetrich has joined #tripleo05:37
*** apetrich has quit IRC05:39
*** apetrich has joined #tripleo05:40
openstackgerritMerged openstack/tripleo-image-elements: Remove tuskar element
*** bnemec has quit IRC05:44
*** ramishra has joined #tripleo05:47
*** chlong has quit IRC05:47
*** dsariel has quit IRC05:47
*** bnemec has joined #tripleo05:52
*** yolanda_ has joined #tripleo05:52
*** rcernin has joined #tripleo05:54
*** florianf has joined #tripleo05:56
*** karthiks has joined #tripleo05:57
*** chlong has joined #tripleo05:59
*** bvandenh has quit IRC06:00
*** itamarl has joined #tripleo06:03
*** bnemec has quit IRC06:12
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add MysqlInternal endpoint to enable-tls
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap
*** bnemec has joined #tripleo06:14
*** bvandenh has joined #tripleo06:15
*** fragatina has quit IRC06:18
*** bnemec has quit IRC06:23
*** yolanda_ has quit IRC06:29
*** numans has joined #tripleo06:31
*** yolanda has joined #tripleo06:31
*** pkovar has joined #tripleo06:36
*** bnemec has joined #tripleo06:36
*** jprovazn has joined #tripleo06:39
*** dsariel has joined #tripleo06:51
*** openstack has joined #tripleo06:54
*** ccamacho|hungry is now known as ccamacho06:59
*** mcornea has joined #tripleo07:01
*** oshvartz has joined #tripleo07:01
*** paramite has joined #tripleo07:06
*** greghaynes has quit IRC07:08
*** mikelk has quit IRC07:14
*** ebarrera has joined #tripleo07:20
*** greghaynes has joined #tripleo07:21
*** athomas has joined #tripleo07:22
*** olap has joined #tripleo07:26
*** liverpooler has joined #tripleo07:27
*** gfidente has joined #tripleo07:30
jaosoriormarios: +1 on landing heat07:30
*** dmacpher has quit IRC07:30
gfidentejaosorior, so I had to skip tests in 4.007:31
gfidenteI think it's best we can do for now07:31
jaosoriorgfidente: Yeah man, I couldn't think of anything else myself07:32
jaosoriorthose backward incompatible changes in puppet are pretty harsh to deal with :/07:32
*** olap has quit IRC07:32
gfidentewell it works with puppet > 4.107:32
mariosjaosorior: yeah, my hesitation is cos i missed that cfn endpoitn stuff on the tht side - but I think they are good to go now07:32
gfidentebecause they fixed the visibility issue07:32
gfidenteso you *can* use call_functions(hiera from a 3x puppet and it will see the 3x version of hiera()07:33
gfidenteso the tests pass on -latest- and on 3x07:33
mariosjaosorior: +Ad the puppet-tripleo07:33
gfidentethey just fail in 4.007:33
mariosgfidente: that review you mentioned only has +1 still07:33
gfidentemarios, yeah I know :(07:34
mariosgfidente: oh isee07:34
mariosgfidente: yeah ok, just saw what it was07:34
*** gumbyblake has quit IRC07:34
*** olap has joined #tripleo07:34
gfidentemarios, thanks! :)07:35
*** dtantsur|afk is now known as dtantsur07:36
*** aufi has joined #tripleo07:36
openstackgerritMerged openstack/puppet-tripleo: Add Heat profiles
*** chem`` has joined #tripleo07:41
*** ifarkas has joined #tripleo07:42
jaosoriorby the way07:43
jaosoriorI found the issue with the database uris in the MysqlInternal endpoint patch07:43
*** hjensas has quit IRC07:43
jaosoriorgfidente: It needed the addition of that endpoint to tripleo-ci's enable-tls
openstackgerritMartin André proposed openstack/tripleo-validations: Initialize README
jaosoriorthat environment is a sneaky bastard X_x which makes me want to push more for this we would avoid all those issues if that merged07:44
*** bvandenh has quit IRC07:45
gfidentejaosorior, oh right07:45
gfidentein tripleo-ci07:45
gfidenteI faced this a week ago07:45
bandiniis the ci completely fubared atm?07:46
gfidentebut it didn't come up to my ming07:46
gfidentejaosorior, great so let's merge that07:46
*** zoli_gone-proxy is now known as zoliXXL07:46
gfidentebut yeah agreed ideally we need to get rid of that custom map in CI or find a different way to customize things07:46
bandini done a third recheck but it never seems to find any vms07:46
gfidentejaosorior++ :)07:46
jaosoriorbandini: yeah, CI is pretty flaky :/07:47
bandinijaosorior: any bug open about or do I need to take out my voodoo dolls?07:47
jaosoriorthere should be a bug, I think, shardy and derekh were working on it the other day07:48
jaosoriorbandini: But the voodoo dalls and a rabbit foot would sure help07:48
bandiniack, thanks will look for it and cc myself07:48
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add MysqlInternal endpoint to enable-tls
ccamachohey! quick question, for stable/liberty all gates are also broken? Or is a temporary issue?07:51
jaosorioronly upgrades was broken07:51
jaosoriorthe rest of them being broken is an actual problem07:51
bandiniccamacho: I see the *same* issue07:52
ccamachoi have pushed a patch to tht in stable/liberty without changes, and upgrades, ha and nonha are getting issues..07:52
*** dbecker has joined #tripleo07:52
mariosccamacho: nice patch :)07:52
jaosoriorccamacho: the error seems to be in the undercloud deployment07:52
ccamachoyeahp, I thought it was a temporary issue07:54
jaosoriornot sure if this is related07:54
jaosoriorMay 25 15:46:46 instack kdumpctl: No memory reserved for crash kernel.07:54
bandiniI saw the issue yesterday as well07:54
jaosoriorMay 25 15:46:46 instack kdumpctl: Starting kdump: [FAILED]07:54
jaosoriorMay 25 15:46:46 instack systemd: kdump.service: main process exited, code=exited, status=1/FAILURE07:54
jaosoriorMay 25 15:46:46 instack systemd: Failed to start Crash recovery kernel arming.07:54
bandiniit is okay if kdump fails, we can ignore that07:54
jaosoriorguys, any idea where we can get the puppet logs from the undercloud?07:55
*** athomas has quit IRC07:55
jaosoriorfound them07:55
*** bvandenh has joined #tripleo07:55
jaosoriorceilometer is broken07:55
jaosoriorccamacho: tried posting a message in #rdo about it, seems it's an issue with the ceilometer-alarm package07:57
ccamachojaosorior I will follow it, thanks!07:58
jaosoriorwell, maybe it's the package, or maybe it's the repo setup for stable/liberty07:58
*** zoliXXL is now known as zoli|wfh07:59
*** paramite is now known as paramite|afk08:00
*** athomas has joined #tripleo08:00
zoli|wfhgood morning08:01
*** chlong has quit IRC08:04
*** cmyster has quit IRC08:07
*** dtantsur is now known as dtantsur|bbl08:07
*** paramite|afk is now known as paramite08:08
*** cmyster has joined #tripleo08:12
*** cmyster has quit IRC08:12
*** cmyster has joined #tripleo08:12
*** afazekas is now known as afazekas|sick08:12
bandinijaosorior: I will file a BZ against RDO, ok? (at least we track this)08:14
*** paramite is now known as paramite|afk08:14
jaosoriorbandini: You mean for the ceilometer issue?08:14
jaosorioryeah, only thing is that I'm not sure if it's an RDO issue, or is it that we misconfigured the repos. shardy and derekh would be more capable of debugging that08:15
*** paramite|afk is now known as paramite08:16
*** electrofelix has joined #tripleo08:18
bandinijaosorior: ack, I'll start with an LP bug and then we can create an rdo bz if needed08:21
jaosoriorbandini: Sounds good to me08:21
bandinialthough I havent seen any changes related to liberty in tripleo-ci recently08:22
jaosoriorbandini: I think it's either an error in packaging, or an error in our repo setup08:22
*** lucas-dinner is now known as lucasagmes08:22
*** lucasagmes is now known as lucasagomes08:22
*** dmk0202 has joined #tripleo08:23
bandinijaosorior: I could not find the repolist in the logs though08:31
openstackLaunchpad bug 1585937 in tripleo "liberty ci is broken (undercloud install fails)" [Undecided,New]08:31
jaosoriorbandini: thanks08:32
bandiniI need to install a liberty system anyway, so I guess I'll get to the same error08:33
*** sshnaidm has quit IRC08:34
jaosorior*bandini, let me know when you have a liberty deployment08:36
jaosoriorhave an unrelated question about that08:36
bandinijaosorior: claro ;)08:37
* bandini flexes his spanish brain muscles08:38
jaosorior jajaja va pues. Gracias08:38
*** derekh has joined #tripleo08:39
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: [EARLY WIP] Composable Upgrades
jistrmarios, social: if you have a moment at some point, it would be great if you could check out ^^^08:40
*** tosky has joined #tripleo08:40
jistri was able to "no-op upgrade" keystone and glance with that08:41
mariosjistr: ack thx08:41
jistrmarios: i also thought about the increased flexibility of the whole solution we talked about, and i think it could be doable at least to some extent08:41
*** paramite is now known as paramite|afk08:42
jistralso i got back to the per-service rather than per-component structuring on the lowest level, to fully map 1-1 to the composable services08:42
jaosorioranybody has a stable/mitaka environment around?08:43
jistrit's not the best at some spots (e.g. as we discussed, one cannot update RPMs for glance-api and glance-registry separately, as they're one RPM), but it shouldn't break, as those two services *must* be upgraded at the same time always anyway08:43
jistrto keep AMQP messaging formats in sync08:44
*** jaosorior is now known as jaosorior_lunch08:45
openstackgerritCarlos Camacho proposed openstack/tripleo-docs: Composable roles within services Tutorial
*** panda has quit IRC08:48
mariosjistr: i see ok thanks for the info, will have a look and we can catchup later/tomorrow again08:49
*** panda has joined #tripleo08:49
jistrmarios: sure thing, thanks08:49
*** dsariel has quit IRC08:52
ansiwenEmilienM: ping nova composable roles08:55
*** saneax_AFK is now known as saneax08:55
jistrmarios, social: one possible complication wrt Mistral->Ansible connection is that Ansible is very filesystem focused, we won't be able to feed it URLs for playbooks/inventories etc., we'd have to generate these things to some tmp directory on the undercloud (possibly not under /tmp itself b/c security, but something similar)08:59
*** devvesa has joined #tripleo09:00
*** jtomasek_ has joined #tripleo09:00
*** jtomasek_ has quit IRC09:02
*** tesseract has joined #tripleo09:02
*** jtomasek_ has joined #tripleo09:02
*** zoli|wfh is now known as zoli|mtg09:03
*** paramite|afk is now known as paramite09:03
jistrmarios, social: as a next step i'll start looking into Mistral a bit to see how we could generate these things, and it will give me a chance to think about "could we use *directly* Mistral without Ansible for upgrades". On high level Ansible seems better suited for system management operations, but if the Mistral->Ansible connection turns out to be too costly, we might investigate Mistral-only approach too.09:04
mariosjistr: i like the first pass... you *have* decomposed into per service-component upgrade definitions, but can orchestrate with a higher template09:04
jistrmarios: yea we'll have to approach the component's one way or another09:05
jistrlemme fpaste sth09:05
*** nijaba has quit IRC09:06
mariosjistr: did you get a chance to see  - going to go work on that some more in a bit and tidy up09:06
jistrmarios: ah not yet, will look at that today09:06
*** dsariel has joined #tripleo09:07
mariosjistr: k thanks (just reminder ;) )09:07
*** nijaba has joined #tripleo09:07
*** jtomasek_ has quit IRC09:08
socialjistr: here, sorry for being late09:10
jistrmarios: if we have a bit of per-service info and a list of upgrade phases, we could autogenerate what a component means essentially:
jistr^^ this alone can be used to autogenerate the glance.yaml and keystone.yaml ansible playbooks09:10
socialjistr: definetly not /tmp but /var/tmp ,)09:10
jistrso e.g. if there's a vendor plugin that needs to update at the same time as neutron09:11
jistrit just says `upgrade_phase: neutron`09:11
jistrand its upgrade runs together with neutron09:11
mariosjistr: phase for grouping09:11
*** paramite is now known as paramite|afk09:12
jistrthe list of phases is still static in what i pasted above09:12
jistrbut it might be a good start09:12
mariosjistr: service manifest it is :)09:12
mariosjistr: right? I mean to say, the 'upgrade_phase' could be added/carried by the service into the service manifest09:13
ansiwenshardy somewhere explained the difference of composable services  vs. composable role. somebody remembers where that was?09:13
jistrmarios: well... in theory yea but i'm not sure if we should put this info into the service's Heat templates09:13
jistrmarios: the main issue is that to get any sort of sane info from the files we have in t-h-t, they have to be crunched through Heat, which is bleargh09:14
*** anshul has joined #tripleo09:14
*** paramite|afk is now known as paramite09:14
*** anshul is now known as Guest4548109:14
jistrmarios: i.e. if we put service manifests where i put them until now, to read that manifest after updating something, you have to do a stack-update... yuck09:14
mariosjistr: but the service manifest would be generated by the deployment itself09:15
mariosjistr: ok so i guess you mean, at time of upgrade, we'd need a 'fresh' service manfiest or sthing09:15
mariosjistr: and to get that we'd have to 'redeploy' (stack update)09:15
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Initial zaqar websockets
jistrmarios: yea an ability to have fresh info for what was in the fpaste would be better i think. We still have to read something from the Heat stack (at least the list of roles+services deployed, and a list of machines and their roles) but the smaller the interface, the better the chance you don't need to stack-update to get the info you need.09:17
jistrmarios: yes exactly as you said, "redeploy"09:17
*** devvesa has quit IRC09:17
*** devvesa has joined #tripleo09:18
*** sambetts|afk is now known as sambetts09:18
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart: Add options to download and handle overcloud images
*** larstobi has quit IRC09:19
*** larstobi has joined #tripleo09:24
bandinijaosorior_lunch: so with these steps liberty install actually worked
matbuhi, is someone know if there is some prebuilt OC images somewhere ?09:26
matbupanda: cool thx09:28
*** dtantsur|bbl is now known as dtantsur09:28
socialjistr: what about related packages for example you usually have some deps, those will be updated when?09:29
*** sshnaidm has joined #tripleo09:30
jistrsocial: whenever something pulls them in i'm afraid. The basic guideline is "whenever you start a per-service upgrade, don't restart any unupgraded services" as upgraded deps can break them on restart. I mentioned it in the spec too. Unfortunately i don't think there's a way around this with "traditional packaging".09:31
jistrthis would be a non-issue after we switch to containers09:31
socialjistr: and that's the issue depending on config you have soft deps in openstack packages09:32
jistroof soft deps09:32
socialjistr: this is solved in puppet by installing if some class is set up :/09:32
socialyet the service might not start if we don't update that09:32
socialeven db-sync failed on neutron if you didn't have fwaas09:33
jistrwe can always list more packages to update, provided that we *know* what those are09:34
jistrsocial: thanks for bringing this up09:34
socialjistr: which will mean we'll reproduce the logic from puppet ;.;09:34
*** jaosorior_lunch is now known as jaosorior09:35
jaosoriorbandini funky...09:35
jaosoriorshardy, derekh are you around?09:35
socialjistr: i would still go with your approach for now but we should have this possibility in mind09:35
jistrheh i guess. "a list of packages" is not a huge amount of logic though09:36
jistrsocial: you mean upgrading via Puppet?09:36
socialjistr: no, even though I'm for using puppet if we are using it to deploy and just overriding package from installed to latest, but just stick with ansible and if issue with soft deps shows up add it to ansible09:37
*** devvesa has quit IRC09:38
derekhjaosorior: whats up?09:39
jaosoriorderekh: Hey dude, we've been seeing failures in the stable/liberty branch09:40
openstackLaunchpad bug 1585937 in tripleo "liberty ci is broken (undercloud install fails)" [Undecided,New]09:40
jistrsocial: well just overriding from installed to latest won't cut it, there's also making sure that things are only running at the same time if they actually can talk the same AMQP to each other, and they don't run when DB sync is being performed underneath, and the PM requirement of doing a per-service (per-component) upgrade. Puppet won't do much good there, unless we do some serious hacks.09:40
jaosoriorand what's strange enough is taht bandini actually was able to deploy liberty with pretty similar steps as we do in the CI09:40
jaosoriorany idea what's up? O_o09:40
jaosoriorderekh: This is what bandini did
derekhjaosorior: looking09:41
jistrsocial: though to be fair, per-service upgrade is a concern regardless of the approach we pick. I'm still kinda untrusting towards OpenStack itself that it will flawlessly run with some components upgraded and some not.09:41
bandinijaosorior, derekh: looking at the ci, I see I did not do " --delorean-setup", not sure how relevant that is09:43
derekhbandini: which log file are you looking at?09:44
bandinijaosorior: I see though that my undercloud has instack-undercloud-2.2.8-0.20160523200751.121f79f.el7.centos.noarch as a verision. how do I check if I really did get a liberty undercloud?09:45
bandiniderekh: logs from the following review
bandinijaosorior: these are the repos I end up with . Seems sane for liberty I guess09:50
*** devvesa has joined #tripleo09:54
derekhbandini: jaosorior I suspect maybe the squid has cached an old repository, going to restart it now09:54
derekhbandini: jaosorior we used to have a rule in the squid server to prevent repomd.xml from being cached in yum repositories but its gone, os-apply-config probably stomped over it09:56
derekhjaosorior: bandini  putting it back now09:56
bandiniderekh: ack, let me know when you want me to recheck a liberty job of mine09:56
derekhbandini: go for it, let me know if it helps09:58
bandiniderekh: ack, thanks09:58
derekhbandini: restarted it again, noticed another config difference, your just should be ok unless the timing was unlucky10:02
derekhjaosorior: bandini I've updated the os-apply-config template on the squid server so that a restart shouldn't cause this to happen again10:03
derekhhopefuly it solves the problem10:03
*** mikelk has joined #tripleo10:04
*** rhallisey has joined #tripleo10:16
hoobamancan i combine external and floating into one subnet/vlan?10:17
hoobamanthe manual states10:18
hoobamanNetwork 6 - External, Floating IP (mapped after Overcloud creation)10:18
socialmarios: about dbsync triggered from puppet, you'd have to notify it and if we update the package only thing that can notify it is config change10:18
hoobamanhowever i only see a working possibility when i am using 2 vlans/subnets?10:18
hoobamanthe manual staes 6 vlans are sufficiant, however how do i see , i need 710:22
jaosoriorHey dude10:25
jaosoriorthere seems to be some issue now that we added the MySQLInternal10:25
jaosoriorupgrades is failing since it complaigns about a hiera value10:25
jaosoriorand the value it complaigns about it this one10:25
jaosoriormysql_virtual_ip: [fd00:fd00:fd00:2000::11]10:25
hoobamani just dont understand how i can have external/floatin ip on the same vlan (virtual vlan tagged interface)10:25
*** jprovazn has quit IRC10:37
bandiniderekh: same error
bandinijaosorior: ^10:37
sshnaidmdoes anybody know why we have two delorean repos in CI - delroean-current.repo with special packages and delorean.repo with different hashes?10:40
*** zoli|mtg is now known as zoli|wfh10:47
*** rhallisey has quit IRC10:49
derekhsshnaidm: the one with special packages is because we use current master packages for all of the tripleo packages10:50
derekhsshnaidm: the other one is the non tripleo packages10:50
ccamachosshnaidm, Might be delorean.repo from current-tripleo and delorean-current.repo from RDO (for core openstack packages)?10:50
derekhsshnaidm: for these we have a pinned repository that is updated periodically if the periodic job passes with them10:50
derekhbandini: ok, looking10:51
sshnaidmthanks, now it's clear..10:53
*** yolanda has quit IRC10:57
*** jprovazn has joined #tripleo10:57
openstackgerritMarios Andreou proposed openstack/puppet-tripleo: Adds the puppet-tripleo manifests for manila
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Composable manila tripleo-heat-templates side
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration
jaosoriorgfidente: so the issue is that most of the usages of that IP are with the uri map, which puts IPv6 in brackets. While only one value (mysql_virtual_ip) takes the bracketless11:00
hewbroccathis stuff makes me insane11:01
jaosoriorhewbrocca: ??11:01
hewbroccathe fact that we have AFAICT 17 different ways to write an IPv6 address, and they're all incompatible11:02
jaosoriorah that11:02
jaosorioryep, sucks11:02
hewbroccacould we not have picked *one way* for OpenStack and stuck with it?11:02
jaosoriorwell, nah11:03
jaosoriorthat would be too easy :P11:03
jaosoriorit's openstack, remember11:03
*** yolanda has joined #tripleo11:04
*** pkovar has quit IRC11:07
*** devvesa has quit IRC11:08
*** links has quit IRC11:08
hoobamanwhat ports should be open on the pubapi/external side?11:10
hoobamanonly keystone or more?11:10
jaosoriorhoobaman: well, depends if you expect users to be able to use nova and such from the outside network or if you expect them to only use horizon11:12
jaosoriorit also depends if you're using the default SSL settings, because then the ports change11:13
derekhbandini: jaosorior I *think* the problem is that  openstack-ceilometer-common is being installed from the delorean-deps repository
derekhversion 5.0.3-111:13
derekhthe undercloud install tries to install11:13
derekhopenstack-ceilometer-alarm and is trying to pull it from the delorean-liberty repository11:13
derekhthe version in there is less then what is in delorean-deps 5.0.3-0.2016050616275611:13
derekhjaosorior: bandini ^11:13
*** stendulker_ has joined #tripleo11:13
derekhI didn't know the delorean deps repo contained openstack packages11:14
jaosoriorneither did I11:14
bandinisurprising yes, I thought it was for deps only11:15
*** stendulker has quit IRC11:17
*** ccamacho is now known as ccamacho|lunch11:18
*** stendulker_ has quit IRC11:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap
*** qasims has joined #tripleo11:25
*** wendar has quit IRC11:31
*** wendar has joined #tripleo11:31
*** rhallisey has joined #tripleo11:32
*** julim has joined #tripleo11:32
bandiniderekh: is it an rdo issue in your view?11:32
*** remix_tj has joined #tripleo11:33
*** rbrady has joined #tripleo11:34
derekhbandini: possibly, although I can't reproduce it outside of our CI11:35
remix_tjHello, i'm installing an overcloud composed by 3 controller nodes. Do i need to specify something additional to --control-scale 311:35
remix_tji've succesfully deployed with only one controller, but with 3 fails due to mysql errors (tables already existing, columns already existing and similar)11:36
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Nothing to see here
remix_tji'm using latest rdo-release (not delorean stable)11:38
bandiniremix_tj: likely you need to add -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml to your deploy command11:38
bandinithat will give you an ha control plane on the three controllers11:38
derekhbandini: jaosorior ok, found out exactly whats going on now11:42
derekhwhen we build the undercloud image we pre install a load of packages on it, here
derekhsee the package list in that line11:42
derekhthe key here is that one of the packages that is getting installed is yum-plugin-priorities11:43
mandrehello, can I get a couple of cores to reviews patches in so that we can initiate work on tripleo-validations?11:43
derekhso the priorties we have set on the repositories don't have any effect during this package install11:43
*** dsariel has quit IRC11:44
derekhand openstack-ceilometer-common ends up coming from the delorean-deps repo11:44
derekhbecausde its release is greater11:44
*** pkovar has joined #tripleo11:44
derekhthen after this any package installs try to respect the priorities set on the db (as now yum-plugin-priorities is installed)11:45
dkelsonI'm trying to do some customization of my overcloud controller using NodeExtraConfigPost with puppet, but when I run "openstack service list" (after setting the appropriate env variables using hieradata for the password, and URL) I get "ERROR: openstack The service catalog is empty.". However logging into the controller, setting the same env variables and running the same command results in the expected output.11:46
derekhso while the undercloud is being installed it trys to install openstack-ceilometer-alarm from the delorean/liberty repo as it has a higher priority11:46
derekhand fails becasue the release is lower11:46
derekhbandini: jaosorior ^11:46
remix_tjbandini: ok. This may solve the problems i'm experiencing with the deploy? (these are the errors that has been found on a failing controller node)11:46
jaosoriorderekh: So, can we pre-install packages in two steps there?11:47
derekhso I suggest you talk to rdo to see if they expect that the delorean/liberty repo should always have a NVR hight then whats in the delorean-deps repo and if so they might need to fix somthing11:48
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Initial rework of pacemaker_common_functions for M..N upgrades
hewbroccaderekh: that'đ entirely excellent11:48
derekhif not we can change CI to may not preinstall packages in stable jobs, or see if we can install pluging-priorities before the other packages as jaosorior suggests11:49
jaosoriorNVR hight?11:49
hewbroccaor we could get real package repositories11:49
bandiniremix_tj: it's likely yes11:50
derekhjaosorior: NVR - name/version/release   , and hight should have been higher11:50
remix_tjgrazie bandini11:52
*** Goneri has quit IRC11:53
derekhhewbrocca: real package repositories?11:54
hewbroccawhere we don't have to use hacks like yum-plugin-priorities11:55
hewbroccaI realize that is entirely impossible11:55
*** devvesa has joined #tripleo11:55
hewbroccabut I don't have to like it...11:55
*** thrash|g0ne is now known as thrash11:56
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Add master configuration
derekhhewbrocca: ahh, crazy talk ;-), actually I think this was an aim for RDO, its why we put the timestamps into the release numbers in the delorean packages so the the NVR would always be higher but that seems to have gone wrong here11:57
derekhtrown|outtypewww: isn't that correct? ^11:57
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Fix ceph keyring setting for gnocchi
gfidenteanyone wants to merge ?12:00
*** jaosorior has quit IRC12:00
*** jaosorior has joined #tripleo12:00
derekhjaosorior: bandini to reproduce(if you want),
slaglelucasagomes: dtantsur : hey, wondering if i can get some help on
openstackLaunchpad bug 1585641 in tripleo "CI: jobs failing with Message: No valid host was found. There are not enough hosts available., Code: 500"" [High,Triaged] - Assigned to James Slagle (james-slagle)12:03
*** julim has quit IRC12:03
slaglelucasagomes: dtantsur : a fair amount of tripleo-ci jobs have been failing with the the dreaded "No Valid Host" error, and it appears to be some inconsistency between the nova scheduler and ironic12:03
bandiniderekh: oh I see12:04
slaglelucasagomes: dtantsur : from what i can tell, i think it's related to this nova commit:, b/c i'm seeing that some nodes are suddenly reporting memory_mb=0 in the compute resource tracker, even though they aren't in use12:04
lucasagomesslagle, hi there. Lemme take a look12:05
*** pradk has joined #tripleo12:05
dtantsurI remember someone complaining about that..12:05
lucasagomesslagle, there was a bug recently at how the resource manager does compute the available resources12:05
slaglei think there is something racey going on, b/c it doesnt happen all the time in our CI12:06
bandiniderekh: so who do you think is best suited of the rdo overlords to help out here?12:06
openstackLaunchpad bug 1572555 in OpenStack Compute (nova) "Nova reports memory_mb=0 for available Ironic node" [High,In progress] - Assigned to Anton Arefiev (aarefiev)12:06
lucasagomesslagle, the bug was introduced by
* bandini not all too familiar with how/who does what in packaging rdo12:06
derekhbandini: probably worth asking number80 on #rdo12:06
lucasagomesslagle, here's the fix for nova, it's not merged yet tho:
bandininumber80: since you're here too ;) we have the following issue in liberty ci
openstackLaunchpad bug 1585937 in tripleo "liberty ci is broken (undercloud install fails)" [Undecided,New]12:07
derekhbandini: if there isn't a solution in #rdo we can just remove that list of pre installed packages for stable jobs12:07
* number80 looking12:07
*** zoli|wfh is now known as zoli|lunch12:08
slaglelucasagomes: hmm, yea, that might be the issue12:08
number80bandini: there's maybe a problem w/ the delorean build of ceilo12:09
bandiniderekh: so basically in our jobs we always assume that "delorean/liberty packages have later NVR than the ones in delorean-deps", correct? or the other way around?12:09
derekhnumber80: essentially the problem is that the NVR of openstack-ceilometer in is lower then the package in
* derekh goes for lunch12:10
slaglelucasagomes: i think we see the issue after the first nodes get successfully scheduled to, so this sounds like it might be the same problem12:10
jaosoriorwent for coffee12:10
*** trown|outtypewww is now known as trown12:10
trownderekh: bandini, unfortunately we can not guarantee NVR in deps is lower than DLRN, hence the need for yum plugin priorities12:11
number80bandini: ok, it tries to install openstack-ceilometer-alarm from delorean but the one in rdo-liberty-testing is newer12:12
number80but what I don't get is why puppet doesn't take the openstack-ceilometer-alarm from rdo-liberty-testing?12:12
*** yamahata has quit IRC12:12
*** jayg|g0n3 is now known as jayg12:12
trownthe telemetry project in particular has had other issues with release tagging that have caused this before12:13
bandiniI think derekh mentioned that it depends on the when yum-plugin-priorities is instaled12:13
trownthe solution is just to always install yum-plugin-priorities in its own transaction before anything else12:13
jaosoriornumber80, bandini: So the thing is that we do pre-installment of packages for the images here
bandiniderekh: can we do what trown ^ suggests for the stable jobs at least?12:14
*** fultonj has joined #tripleo12:14
trownbandini: right if you install ypp in the same transaction as a bunch of other packages, it wont have affect12:14
trownfor that transaction12:14
number80bandini: yes, but then it should take all subpackages from the same repo12:14
number80not mixing them12:14
trownhmm effect rather (I think)12:14
trownnumber80: well, not if later you update, since priorities is then in effect12:15
bandininumber80: yes I think the reason is the line that jaosorior mentioned in the link above, by preinstalling some stuff before we effectively take stuff from two separate repos12:15
lucasagomesslagle, yes exactly. Once nova subtract the resources when a node was pick, it them subtract for other nodes too12:15
lucasagomesslagle, what I observed was, now memory_mb, local_gb and vpcus lives in the inventory database table in nova12:16
openstackgerritMerged openstack/tripleo-quickstart: Add master configuration
lucasagomesslagle, and if you have one or more nodes with the exactly same amount of a given resource they will have the same id in that inventory table12:18
lucasagomesslagle, so once nova sets it to 0 because it's taken, it mistakenly update other nodes12:18
*** dprince has joined #tripleo12:19
jtomasekrbrady: do you agree that the proposed solution is a good one?12:19
trownbandini: derekh, I am 100% sure the only solution to that issue is installing yum plugin priorities before anything else12:21
rbradyjtomasek: what proposed solution?12:22
trownsince RDO could hack something for ceilometer, but that just kicks the can down the road to the next time a project does something wonky with a release12:22
jtomasekrbrady: to make the changes so the instackenv.json can match the ironic nodes structure12:23
trownnote that is how the RDO image is made:
rbradyjtomasek: I think it would be good to investigate to see if anything else depends on that file first.12:24
EmilienMslagle: about release -- how to we handle release notes usually?12:24
slagleEmilienM: we haven't12:26
openstackgerritMerged openstack/tripleo-heat-templates: composable heat services
slagleEmilienM: we don't track anything, so it would be a manual process of scouring the changelog to see what has changed12:27
jistrmarios: reviewed the upgrade script change, just found a few issues, but the overal approach is very nice12:28
EmilienMslagle: ok -- in the future we might want to use reno12:28
*** MaxPC has joined #tripleo12:29
slagleEmilienM: yea, agreed we probably need to12:30
jtomasekrbrady: problem I am facing now is how/where do I get the information about what drivers we support and what are the properties I need to set for each of the driver? If it was using Ironic, that would be clear, but for some reason instackenv.json nodes format is different12:30
mariosjistr: thx for looking - still early days but i am trying to make it work with 'what we know'. for example once we know exactly which services _are_ pcmk we could be more explicit/exact12:31
mariosjistr: thanks for reminding me about the ${@[*]} wanted to sanity check it12:31
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap
*** rodrigods has quit IRC12:34
*** rodrigods has joined #tripleo12:34
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap
*** myoung has joined #tripleo12:46
*** olap has quit IRC12:47
pandaderekh: what packages are taken from master to create triplo-ci images, even when testing stable branches ?12:47
*** rlandy has joined #tripleo12:48
*** julim has joined #tripleo12:49
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: loadbalancer: make sure controller_* are array-typed
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Explode loadbalancer role in 2 sub-roles
pandaderekh: because using the images "as is" it's showin some weird behaviours. For example I'm hitting while testing mitaka12:50
openstackLaunchpad bug 1572467 in tripleo "Cannot load scheduler driver from configuration" [Critical,Fix committed] - Assigned to John Trowbridge (trown)12:50
d0ugaljtomasek: Does the ironic format cover the virt case?12:51
jtomasekd0ugal: not sure I follow, virt case is some driver type?12:53
*** myoung has quit IRC12:56
*** paramite has quit IRC12:56
*** trown is now known as trown|mtg12:59
derekhpanda: the cached image is for master only, you shouldn't be using it for mitaka13:01
*** liverpooler has quit IRC13:01
*** lucasagomes is now known as lucas-hungry13:03
derekhbandini: trown|mtg if yum-plugin-priorities can't easily be installed first we can just remove the list of packages for the stable jobs.13:03
*** myoung has joined #tripleo13:04
*** pradk has quit IRC13:05
trown|mtgderekh: it seems like we would just want to add `yum install -y yum-plugin-priorities` to the repo setup step in tripleo.sh13:05
trown|mtgit is a hard requirement for dlrn repos13:05
derekhtrown|mtg: the problem is that its happening inside DIB here , we'de have to add a yum-plugin-priorities element or something13:07
trown|mtgderekh: or we inject repos into the base image before running dib13:07
trown|mtgthat is how it is done in RDO13:08
pandaderekh: thanks, I was missing that part of information. I though that all periodic jobs were creating an image, and indeed I did not understand what image would land on the repository since periodic jobs for stable are launched too13:08
*** rbrady_ has joined #tripleo13:08
*** rbrady has quit IRC13:08
trown|mtgwhich is I guess is what an element would do too... I just am biased against adding more DIB elements13:08
*** Goneri has joined #tripleo13:09
derekhtrown|mtg: the repos are availble and use when DIB is run, anyways will continue after meeting13:11
*** tzumainn has joined #tripleo13:15
*** rbrady_ has quit IRC13:15
*** rbrady_ has joined #tripleo13:15
*** MaxPC has quit IRC13:16
*** MaxPC has joined #tripleo13:16
*** akshai has joined #tripleo13:18
*** pradk has joined #tripleo13:18
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Configure ComputeServices via resource chains
sshnaidmmyoung, hi13:22
myoungsshnaidm: hi :)13:23
sshnaidmmyoung, did you try to build images on promoted undercloud.qcow2 ? like from here :
bnemecderekh: trown|mtg: We already are installing yum-plugin-priorities in  It's a gross hack that needs to go away.13:24
sshnaidmmyoung, just running "overcloud build images --all"13:24
bnemecIMHO it should just be added to the image build command, but I got pushback when I proposed that.13:24
derekhbnemec: yes we are but the problem we're trying to solve is in the indercloud install13:25
bnemecderekh: We're also doing it there:
bnemecThat one is slightly less gross since it's actually documented that way.13:26
derekhbnemec: the reason its in and not anywhere else is because devs and CI are the only cases we're layer repositories the we we do13:26
bnemecMore than slightly, in fact. should just reflect what's documented, so it's not gross at all.13:26
derekhbnemec: users who consume tripleo shoudl only be using one repository13:26
*** masco has quit IRC13:27
derekhbnemec: but having said all that I got no objection to a cleaner solution13:27
*** absubram has quit IRC13:27
myoungsshnaidm: no...not sure I have context...13:27
*** akshai_ has joined #tripleo13:33
sshnaidmI was able to build overcloud images with only, including repo-setup though.. But building images on undercloud fails, it seems like a bug...13:33
sshnaidmtrown|mtg, derekh ^^13:33
*** lucas-hungry is now known as lucasagomes13:36
*** gfidente has quit IRC13:36
*** akshai has quit IRC13:37
*** gfidente has joined #tripleo13:37
ramishrahi guys, what's this error "ERROR: The Stack (overcloud-VipConfig-df2v3qligao4) could not be found." for gate-tripleo-ci-f22-nonha. I don't see any error in the heat engine log of undercloud.13:38
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Configure ComputeServices via resource chains
ramishraderekh: hey any idea?13:39
*** lblanchard has joined #tripleo13:41
*** zoli|lunch is now known as zoli|mtg13:45
*** rbrady_ has quit IRC14:01
*** rbrady has joined #tripleo14:01
derekhramishra: I don't see anything obvious, is the problem consistently happening on that patch?14:04
derekhsshnaidm: whats the error?14:04
ramishraderekh: yeah twice14:04
jaosoriorEmilienM: Hey dude, in this CR I'm trying to pass the mysql_virtual_ip as a string, since in IPv6 we're using brackets for the addresses14:07
derekhramishra: hmm, maybe its possible to get "openstack overcloud deploy" to output more debug info, but if there is I don't know it <--- anybody know ?14:07
jaosoriorand that seems to fail down in
jaosoriorthe address ends up something like this in the hieradata that's persisted in the controller:14:08
*** trown|mtg is now known as trown14:08
jaosoriormysql_virtual_ip: '[fd00:fd00:fd00:2000::11]'14:08
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - NTP
jaosoriorand the error I'm seeing is:14:08
jaosoriorError: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type openstacklib::db::mysql::host_access at /etc/puppet/modules/openstacklib/manifests/db/mysql.pp:68 on node overcloud-controller-0.localdomain14:08
jaosoriorany idea what could be causing that?14:08
EmilienMjaosorior: it only happens during upgrades?14:09
jaosoriorEmilienM: Yes, because we only use IPv6 in the upgrades jobn14:09
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB
openstackgerritMerged openstack/puppet-tripleo: Add lookup_hiera_hash function
d0ugaljtomasek: hmm, I think I am confusing myself.14:11
d0ugaljtomasek: So nevermind, I should look at how Ironic does it. Do you have a link handy?14:11
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Composable roles within services - NTP
EmilienMjaosorior: I don't have any clue after a quick look, I'll have to investigate a bit further, let me finish my current task first14:12
jaosoriorEmilienM: No problem, just wanted to know if you had an idea or had seen something like that before14:14
jtomasekd0ugal: yes, one sec14:15
EmilienMjaosorior: I'll first look why it worked before14:15
*** egafford has joined #tripleo14:16
*** MaxPC has quit IRC14:17
jtomasekd0ugal: this is kind of helpful
jtomasekd0ugal: but doing ironic node-show is better imo14:18
d0ugaljtomasek: oh, I see, so just the API format.14:18
jtomasekd0ugal: yes, important parts are driver and driver_info14:18
jtomasekd0ugal: because that is what changes14:19
jtomasekd0ugal: dependent on selected driver, the driver_info properties are changing14:19
*** MaxPC has joined #tripleo14:20
jtomasekd0ugal: so the instackenv.json would need to use attribute names that Ironic uses and specified driver_info json object with properties based on selected driver14:21
*** tserong has quit IRC14:22
jtomasekd0ugal: in the same manner for manual nodes registration, GUI could provide means to display list of available drivers (provided by Ironic API) and get the list of driver_info properties (again using Ironic API) so no driver related logic would be hardcoded either on the backend or frontend side14:23
jaosoriorEmilienM: It worked before cause it didn't have the brackets14:23
jaosoriorso it looked exactly like this14:23
jaosoriormysql_virtual_ip: fd00:fd00:fd00:2000::1114:23
jaosoriorbut the change towards getting the mysql virtual IP in the endpoint map kinda makes us use them14:23
d0ugaljtomasek: Right, changing this makes sense, but I guess it isn't totally trivial14:24
*** noslzzp has joined #tripleo14:24
jtomasekd0ugal: I agree, probably not. I'll need to adjust GUI to use attributes defined in instackenv.json, but I am not sure what is the list of drivers and relevant driver properties supported by tripleocommon/utiles/ which I can let user choose in the GUI14:25
*** eggmaster has joined #tripleo14:26
*** cwolferh has quit IRC14:26
*** noslzzp has quit IRC14:27
*** noslzzp has joined #tripleo14:27
openstackgerritThomas Herve proposed openstack-infra/tripleo-ci: DNM: check heat translation patch
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Gnocchi composable roles
d0ugaljtomasek: That may help14:33
hoobamanwhy is there also a br-ex bridge being attached on every compute node?14:34
hoobamani did not make any dvr configurations14:35
jtomasekd0ugal: thanks14:37
*** dmacpher has joined #tripleo14:39
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: loadbalancer: make sure controller_* are array-typed
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: Composable Sahara services
dtantsurfolks, we've put together a spec on ironic in overcloud, can someone please take a look:
dtantsurdprince, ^^^14:40
dtantsurhewbrocca, ^^14:40
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Explode loadbalancer role in 2 sub-roles
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: loadbalancer: explode hieradata to separate HAproxy & keepalived
dprincedtantsur: nice14:43
*** myoung has quit IRC14:43
*** oshvartz has quit IRC14:45
egaffordgfidente: Saw your feedback on the Trove patches; sounds reasonable. I'm noting though that the CI jobs are consistently failing at the moment, in ways that don't seem related to the patch. Is this par for the course at the moment?14:48
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades
*** dc has quit IRC14:53
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart: Add options to download and handle overcloud images
*** myoung has joined #tripleo14:55
*** myoung has quit IRC15:00
*** zoli|mtg is now known as zoli|gone15:01
*** zoli|gone is now known as zoli_gone-proxy15:02
*** dtrainor has quit IRC15:03
*** dmk0202 has quit IRC15:04
*** dc has joined #tripleo15:07
*** ebarrera has quit IRC15:08
*** sshnaidm has quit IRC15:11
*** itamarl has quit IRC15:15
*** rcernin has quit IRC15:19
*** penick has joined #tripleo15:22
d0ugalmandre: I think it would be better to squash
d0ugalinto the parent, just to keep things green15:24
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades
d0ugalmandre: actually, it might be best to just squash all three of those, as they are all really just the initial cookiecutter15:29
mandred0ugal: sure, I'll do that15:30
*** tesseract has quit IRC15:34
*** trozet has quit IRC15:36
openstackgerritMartin André proposed openstack/tripleo-validations: Initial Cookiecutter Commit.
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Refacor input error and description
bnemechoobaman: The bridge may not be necessary in all configurations, but I believe it is in some and since it doesn't hurt anything to have it, that's the default configuration.15:37
bnemecIf you're using network isolation you can certainly remove it from the compute template if you want.15:37
*** trozet has joined #tripleo15:38
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Update FormErrorList component
gfidentedprince, cephstorage chain pls15:39
hoobamanbnemec: ok thx15:40
*** dc has quit IRC15:41
*** mgarciam has joined #tripleo15:41
*** ccamacho|lunch is now known as ccamacho15:43
*** mikelk has quit IRC15:47
*** noslzzp has quit IRC15:48
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Configure CephStorage services via resource chains
*** dc has joined #tripleo15:53
*** Guest45481 has quit IRC15:55
*** noslzzp has joined #tripleo15:55
*** dprince has quit IRC15:57
*** dc has quit IRC15:58
*** dprince has joined #tripleo15:58
*** dc has joined #tripleo15:58
*** sshnaidm has joined #tripleo16:03
*** yamahata has joined #tripleo16:03
*** pradk has quit IRC16:07
*** trown is now known as trown|lunch16:07
*** mcornea has quit IRC16:09
*** jaosorior has quit IRC16:11
*** aufi has quit IRC16:14
*** sambetts is now known as sambetts|afk16:16
*** ifarkas has quit IRC16:16
*** noslzzp has quit IRC16:18
*** numans has quit IRC16:28
*** dtrainor has joined #tripleo16:29
*** anshul has joined #tripleo16:31
*** anshul is now known as Guest6926316:31
*** dprince has quit IRC16:35
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Set rabbitmq loopback_users explicitely
*** dprince has joined #tripleo16:35
*** devvesa has quit IRC16:35
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.
*** lucasagomes is now known as lucas-afk16:41
*** pradk has joined #tripleo16:43
*** panda has quit IRC16:48
*** panda has joined #tripleo16:48
greghaynesianw: re: - Something that would be neat to protect us even further is we could run all of our 'outside the chroot' phases inside the (or a differente) chroot but bind mount the rootfs in to it. That would require all operations to have the target_dir prefix or risk nuking the chrooot rather than the build host16:52
*** absubram has joined #tripleo16:55
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker.
*** derekh has quit IRC16:56
*** absubram has quit IRC16:56
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: integration of the new puppet pacemaker.
*** florianf has quit IRC16:57
*** qasims has quit IRC16:58
*** absubram has joined #tripleo16:59
*** dtrainor has quit IRC17:01
*** diazjf has joined #tripleo17:01
openstackgerritOpenStack Proposal Bot proposed openstack/os-apply-config: Updated from global requirements
openstackgerritOpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements
pkovardoes one still need to enable epel when setting up environment?
openstackgerritOpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements
openstackgerritOpenStack Proposal Bot proposed openstack/os-refresh-config: Updated from global requirements
*** julim_ has joined #tripleo17:03
*** cwolferh has joined #tripleo17:04
openstackgerritOpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements
*** julim has quit IRC17:07
*** numans has joined #tripleo17:09
*** Guest69263 has quit IRC17:14
*** fragatina has joined #tripleo17:15
*** cwolferh has quit IRC17:19
*** fragatina has quit IRC17:20
*** cwolferh has joined #tripleo17:20
remix_tja very simple question: does pacemaker need to be already installed on the image overcloud-full? because i'm experiencing issue on deploying the cluster and i see that no packages are installed17:20
*** trown|lunch is now known as trown17:20
*** dtrainor has joined #tripleo17:21
trownpanda: wrt do you plan to submit a patch to tripleo-ci to create md5's on the mirror server?17:23
dtrainorshould I expect to be able to use heat templates from Kilo in Liberty?  how about introducing just what i changed from kilo in to liberty instead of *all* the templates?  How does one approach this?17:24
trownI can do it if not, I can17:24
* trown kicks brain17:24
*** isq has quit IRC17:25
*** isq has joined #tripleo17:25
trowndtrainor: the heat templates are very coupled to the puppet modules, I doubt that Kilo templates could deploy Liberty17:25
pandatrown: already did17:27
*** dprince has quit IRC17:28
trowndtrainor: how much did you change? Ideally, changes are included via extra environment files rather than direct modification of the templates, and those are supposed to work from release to release17:28
dtrainoryay i got more info about what i'm asking.  we're still using extra environment config files which is great.17:29
dtrainorvery cool, thanks17:29
*** mgould is now known as mgould|afk17:29
trownthat said, I know the interface in Kilo days was less than stable, so I would not be surprised if some environments for Kilo have non existent variables in later versions17:29
dtrainorjust trying to get the temperature on such a subject17:29
dtrainorthank you17:30
trownya I am pretty sure extra environment files from n-1 is *supposed* to work on n17:30
openstackgerritMerged openstack/diskimage-builder: Add a best-effort sudo safety check
dtrainori'll start crossing fingers17:31
*** diazjf1 has joined #tripleo17:32
*** ebalduf has joined #tripleo17:33
trowneasy tripleo-ci review:
*** diazjf has quit IRC17:33
*** noslzzp has joined #tripleo17:33
chem``hi, I've got this error Could not find data item mysql_clustercheck_password in any Hiera data file ... I cannot find where this is supposed to be set, could someone point me in the right direction ?17:35
*** noslzzp has quit IRC17:36
*** electrofelix has quit IRC17:41
*** david-lyle has quit IRC17:47
*** david-lyle has joined #tripleo17:47
*** pkovar has quit IRC17:50
*** athomas has quit IRC17:53
*** lblanchard has quit IRC18:05
*** lblanchard has joined #tripleo18:06
*** mgarciam has quit IRC18:07
openstackgerritHonza Pokorny proposed openstack/tripleo-common: Add node assignment workflow
dkelsonchem``, look in /etc/puppet/hieradata/ either on your undercloud or overcloud box18:10
EmilienMslagle, trown: do we have a logstash query for the no valid host found issue?18:16
EmilienMin elasticrecheck thing18:16
EmilienMI'm working on it18:17
EmilienMbnemec: looking at system/config modules/openstack_project/files/logstash/jenkins-log-client.yaml18:22
EmilienMit's really outdating, I'm proposing a change18:22
bnemecEmilienM: sshnaidm had a change up to improve that.  Let me see if I can find it.18:22
*** noslzzp has joined #tripleo18:23
EmilienMbnemec: ok I was about proposing the same kind of patch, but with a bit more things, it's fine do go with this one though18:23
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: Change DLRN promote method
*** dtrainor has quit IRC18:27
*** chem`` has quit IRC18:28
dkelsonI need some help some custom configuration failing, see:
*** chem`` has joined #tripleo18:29
*** gfidente has quit IRC18:34
slagleEmilienM: yea i filed a bug and linked to the logstash query18:35
slagleEmilienM: lucas also helped me identify the patch in nova that will hopefully fix it18:36
slagleit's not landed yet, but it sounds like our issue18:36
EmilienMslagle: do you have url handy?18:36
slagleEmilienM: here is the tripleo bug:18:37
slagle> OpenStack use to be more of a product then it is now. When there were common problems to be solved, there was pressure applied to solve them in a way everyone (OpenStack Project, OpenStack Users, and Openstack Operators) would benefit from.18:37
slaglegrr sorry18:37
slaglebad copy/paste18:37
slagleobviously i was reading openstack-dev :)18:37
EmilienMhopefully it was SFW18:37
slaglethe exciting golang thread!18:37
EmilienMah, that18:37
slaglehere is the bug
openstackLaunchpad bug 1585641 in tripleo "CI: jobs failing with Message: No valid host was found. There are not enough hosts available., Code: 500"" [High,Triaged] - Assigned to James Slagle (james-slagle)18:38
slaglethe nova bug and patch is linked in a comment18:38
slagleyep, we need that to land, then we need a dlrn promote18:39
EmilienMslagle: I dropped a comment in the review18:39
EmilienMto wake up people a bit18:39
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Remove the delay resource and its constraints
trownlarsks: do you by chance know a better way to hide a URL parameter using curl?
*** dsariel has joined #tripleo18:43
* larsks looks18:43
trownlarsks: I think what I have will be fine, because that token is very narrowly scoped, but if there is a better way... that would be better :)18:44
pradki'm trying to test an upgrade heat template and getting this error ERROR: Failed to validate: : resources.UpdateWorkflow: : Value must be a string   ... the error message is clear but i have no idea what or where its referring to18:45
pradkany suggestions?18:45
larskstrown: that seems sufficient.  You could also just put the curl command in a separate script, and call out to that; the trace settings wouldn't propogate.18:45
trownawesome thanks18:47
*** lucas-afk has quit IRC18:48
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: add NAT rules for external network when network_isolation=true
larsksayoung: ought to be fix the problem with outbound network access from the overcloud nodes when using network_isolation=true.  If you have the chance to test to see if that resolves your problem that would be awesome.18:48
*** lucasagomes has joined #tripleo18:54
openstackgerritayoung proposed openstack/tripleo-quickstart: add NAT rules for external network when network_isolation=true
*** cwolferh has quit IRC18:57
ayounglarsks, will do18:57
*** paramite has joined #tripleo19:04
*** hoobaman has quit IRC19:05
*** saneax is now known as saneax_AFK19:06
openstackgerritEmilien Macchi proposed openstack/puppet-pacemaker: Add other-requirements.txt for bindep
*** dbecker has quit IRC19:13
*** fragatina has joined #tripleo19:16
*** fragatina has quit IRC19:17
*** fragatina has joined #tripleo19:17
*** fultonj has quit IRC19:26
*** dbecker has joined #tripleo19:26
*** dbecker has quit IRC19:26
*** fzdarsky_ has quit IRC19:27
*** dbecker has joined #tripleo19:30
colonwqEmilienM, I have made  headway for my tripleo install of radosgw. Once I get it fully working, I'll need to make it work the new right way19:35
EmilienMcolonwq: with composable roles?19:36
colonwqEmilienM, Yes. Currently it is the old way but I will rework it as needed.19:37
dsneddonslagle, I have a question. If I wanted to modify the 20-os-net-config script in tripleo-image-elements, what would I have to do to use the modified element in the next deploy?19:37
dsneddonslagle, Or would modifying the file in /usr/share be sufficient?19:37
*** pradk has quit IRC19:38
dsneddontrown, bnemec: ^^^ any idea?19:42
bnemecdsneddon: You would have to modify the one in /usr/share, then rebuild your images.19:43
dsneddonbnemec, That's what I thought, so for OSP I'd need to use virt-customize. Thanks.19:43
bnemecAlthough I thought there was work going on to stop using that script from the elements so it could be updated.19:44
bnemecdsneddon: Ah, for OSP I think that would be correct.19:44
dsneddonbnemec, Yeah, there's a review to replace it, but I don't know if it's merged yet.19:44
bnemecYeah, and it definitely wouldn't be in OSP yet.19:44
rbradyhey trown, do you have any time this afternoon to talk about release tagging?19:45
trownrbrady: sure, I am not sure I will have the answers19:49
EmilienMrbrady: tripleo release?19:49
rbradytrown: I was asked to look into why mistral isn't tagged.  when I asked the mistral folks, they asked why it would be important.  I just don't have the answer.  Did you see their comments in the email you were forwarded?19:50
rbradyEmillien: mistral19:50
rbradytrown: my mistake, the email you got wasn't detailed19:51
rbradytrown: just sent you the ongoing conversation19:52
*** absubram has quit IRC19:55
openstackgerritAndreas Florath proposed openstack/diskimage-builder: partitioning-parted: extraced and adapted element
trownrbrady: that is pretty odd to me... every openstack project tags releases either manually or through the release:managed tag19:56
*** pradk has joined #tripleo19:56
rbradytrown: what I'm looking for is the justification to them of why they should use the release:managed tag19:57
*** lblanchard has quit IRC19:58
trownseems pretty odd to need justification for something every other project does... but the main reason is to have a clear indicator in the git repository corresponding to a release19:58
trowndo they need justification on why to release software?19:58
trownI am pretty confused actually... mistral does not have pypi releases either?
openstackgerritAndreas Florath proposed openstack/diskimage-builder: [WIP] Add: lvm element
*** cwolferh has joined #tripleo20:01
*** jprovazn has quit IRC20:01
trownrbrady: ok so they at least have tarballs on ... as a consumer of those though, I want to be able to look up in the git repo the commit that tarball was made from20:03
trownthat is what a release tag in git is for20:03
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates: Enable firewall by default on the overcloud
*** ayoung has quit IRC20:07
*** qasims has joined #tripleo20:07
openstackgerritMerged openstack/instack-undercloud: Only run mistral commands if mistral is enabled
rbradytrown: I can see there are tags in the git repo that correspond to tarballs.  why is it important for them to used release:managed tag?  I'm not finding anything that describes the value or purpose at
trownrbrady: to me the fact that there is confusion around why releases are a good thing is a good argument that mistral folks should let experienced release engineers take care of it for them with the release:managed tag20:10
*** absubram has joined #tripleo20:11
trownalthough release:managed is not a requirement to tag releases20:12
openstackgerritDan Prince proposed openstack/instack-undercloud: Only validate SSL VIPs if SSL is enabled
pandaI'm developing a CREATE_FAILED panic syndrome20:16
vkmchey guys, I'm getting python-hardware-detect package not found when trying to build a Centos overcloud image20:17
vkmcprecisely, when running "openstack overcloud image build --all"20:18
vkmcis that a known issue? is there a workaround for it?20:18
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart: Do not archive root directory
vkmcEmilienM, ^?20:22
*** jayg is now known as jayg|g0n320:22
EmilienMslagle, bnemec: could you review - and please?20:22
EmilienMvkmc: I'm not aware about it, let me check20:22
EmilienMvkmc: in your undercloud.conf, what is the value of "inspection_extras" ?20:24
EmilienMTrue requires python-hardware-detect20:24
trownrbrady: re-reading that email I am still confused is the question "what is the benefit of release:manged" or is it "why should we tag a release in git"20:24
trownvkmc is there a reason to not use pre-validated images?20:25
EmilienMvkmc: you might need to build with --agent-dib-extra-args20:25
vkmcit's commented out20:25
vkmcso I assume the default is true20:25
vkmctrown, not really, giving my first steps with tripleo20:26
EmilienMtrown ++ why not using pre-built images?20:26
rbradytrown: well, maybe I should prepare with answers to both.  the idea is to bring this subject up at the next mistral irc meeting20:26
vkmcI'm following this guide
EmilienMtrown: if you have time to review it also ^20:26
trownvkmc: ok cool there are images from tripleo-ci on the mirror server for master:
vkmcthanks trown, EmilienM20:27
vkmcwill try those20:27
trownvkmc: and there are images built in RDO for both master and older releases:
trownvkmc: I think the docs around building images have fallen out of date from what is used in CI20:28
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Update profiles for loabalancer split
vkmcI see :)20:29
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: loadbalancer: update hiera parameters for HAproxy/keepalived split
openstackgerritQasim Sarfraz proposed openstack/puppet-tripleo: add plumgrid neutron profile
*** julim_ has quit IRC20:33
trownEmilienM: don't we need tests for ?20:37
rbradytrown: I have to go for the day, but maybe we could pick this discussion up tomorrow?20:37
EmilienMtrown: well, there are no tests for loadbalancer class20:38
trownEmilienM: ya just saw that...20:38
EmilienMbut yes, we would need to write them one day (for haproxy & keepalived)20:38
EmilienMin this patch I did copy paste20:38
trownya that is fair20:38
trownrbrady: sure, though I think it is probably a better thing to engage openstack-infra folks on20:39
bnemecrbrady: trown: I would suggest a discussion with #openstack-release20:39
rbradytrown: the request came from tripleo folks20:39
trownrbrady: from a packaging perspective it makes it much easier when a project has sane release practices, but I don't really have justification beyond that20:39
EmilienMbnemec: your comment is nice20:39
*** jdob has quit IRC20:40
EmilienMbnemec: why didn't I think about it?20:40
rbradytrown, bnemec: thanks20:40
*** numans has quit IRC20:41
openstackgerritMerged openstack/puppet-tripleo: loadbalancer: make sure controller_* are array-typed
*** panda has quit IRC20:48
*** panda has joined #tripleo20:48
larskstrown: well, though I would check to see how our playbooks worked with ansible 2.1.20:49
trownI am not very optimistic :p20:50
larskstrown: well, it went about as well as expected :)  There was a failure early on that I thought was another awful regression, but it's actually just a chance in the default type applied to module params that don't declare one explicitly.  So I am going to add a line to our generate_macs module and try again...20:52
openstackgerritMerged openstack/puppet-pacemaker: Add other-requirements.txt for bindep
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart: Add options to download and handle overcloud images
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Explode loadbalancer role in 2 sub-roles
EmilienMbnemec: better ? :-)20:58
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Update profiles for loabalancer split
*** Goneri has quit IRC21:03
*** paramite has quit IRC21:03
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: loadbalancer: explode hieradata to separate HAproxy & keepalived
*** apetrich has quit IRC21:08
*** MaxPC has quit IRC21:08
*** fragatina has quit IRC21:09
*** yamahata has quit IRC21:11
*** trown is now known as trown|outtypewww21:11
ianwyolanda: where's the fedora-atomic element coming from?21:12
ianwi'm guessing magnum ... let me see if i can repro21:12
*** rhallisey has quit IRC21:18
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: make compatible with ansible 2.1
larskstrown|outtypewww: with the quickstart seems to work okay.21:26
*** myoung has joined #tripleo21:27
*** myoung has quit IRC21:28
*** myoung has joined #tripleo21:28
*** fragatina has joined #tripleo21:29
*** jrist has quit IRC21:32
*** jrist has joined #tripleo21:45
*** akshai_ has quit IRC21:48
*** fultonj has joined #tripleo21:51
*** noslzzp has quit IRC21:53
*** dc has quit IRC22:00
*** fultonj has quit IRC22:03
*** ayoung has joined #tripleo22:12
*** Goneri has joined #tripleo22:23
Erming_larsks: Are your already off now? Just wondering if there is a dashboard in the virtual overcloud? how to access it?22:23
Erming_larsks: I tried nearly the possible IPs but none work.22:25
*** noslzzp has joined #tripleo22:26
*** qasims has quit IRC22:30
ayoungErming_, yes there is22:33
ayoungyou need to set up some way to access it.22:33
ayoungBy default it is not reachable from outside (assuming you are doing quickstart)22:34
Erming_ayoung: I am trying with the default IPs, but don't find one that works for me. I knew it's not accessible from outside. so I use a vnc to the host but still doesn't work22:34
ayoungErming_, I had used sshuttle22:34
Erming_ayoung: is it https or http?22:35
ayoungshuttle   -e "ssh -F $HOME/.quickstart/ssh.config.ansible"   -r undercloud -v
ayoungErming_, I'm not sec22:35
ayoungErming_, heh...not running on mine. It is suppsed to though22:36
ayoungErming_, nothing is running on mine22:36
Erming_ayoung: is shuttle a X terminal?22:36
ayoungEmilienM, nah. Just something you run from one that sets up routes22:37
Erming_ayoung: is is what?22:37
ayoungErming_, that is the subnet I want to route via sshuttle22:38
Erming_ayoung: I think you are talking to Erming instead of EmilienM :-)22:38
ayoungHeh Good evening EmilienM ....22:38
EmilienMI'm going to run brb22:39
Erming_>> iptables -t nat -A sshuttle-12300 -j RETURN --dest -p tcp22:42
Erming_ayoung: it gets stuck here:22:42
*** diazjf1 has quit IRC22:42
ayoungErming_, HTTPD even running on your overcloud, cuz mine isn't22:42
*** Goneri has quit IRC22:43
Erming_ayoung: it's running22:44
ayoungErming_, so you should be able to start by hitting it with curl locally22:45
ayoungprobably curl $HOSTNAME or summat22:45
Erming_ayoung:  Connection refused, neither lftp works. But telnet hostname 80 works22:46
Erming_ayoung: from one controller or from undercloud?22:47
Erming_ayoung: better. works on undercloud.22:48
Erming_but doesn't make sense?22:48
ayoungErming_, what is the overcloud HTTPD server listening on?  Its at the top of /etc/httpd/conf/httpd.conf22:49
Erming_ayoung: overcloud-controller-1.localdomain
Erming_ayoung: but I tried this IP too.22:50
Erming_ayoung: even on the localhost if I access the httpd22:51
ayoungwhat does route or traceroute give you?22:52
Erming_ayoung: sorry. it works locally.22:52
ayoungand is the port open?22:52
Erming_ayoung: maybe I should find a way to access this IP22:52
Erming_ayoung: from host to undercloud to a control (, I should build a path for this port.22:53
ayoungErming_, from undercloud can you curl
Erming_ayoung: no22:54
Erming_ayoung: that's what confuses me22:54
ayoungok,  looks like the undercloud httpd instance is listening on an interface that does not have routes to the outside world22:54
Erming_ayoung: it goes back to my original test: instead of using a specific IP, we should use the VIP. However, even I could telnet/curl from undercloud, I failed to use brower to access to it.22:57
ayoungErming_, I think it is a side effect of network isolation.  He's attemptingto put it on its own network, just that network is misconfigured?22:58
Erming_ayoung: I will try you sshuttle later. gotta go to daycare now. Thank you so much!22:59
*** cwolferh has quit IRC23:05
*** myoung has quit IRC23:07
*** ayoung has quit IRC23:07
*** absubram has quit IRC23:07
*** ayoung has joined #tripleo23:07
*** noslzzp has quit IRC23:13
*** tosky has quit IRC23:22
*** cwolferh has joined #tripleo23:23
*** rhallisey has joined #tripleo23:28
openstackgerritIan Wienand proposed openstack/diskimage-builder: Export die() function
*** yamahata has joined #tripleo23:32
*** julim has joined #tripleo23:33
*** ayoung has quit IRC23:34
*** thrash is now known as thrash|g0ne23:38
openstackgerritBrad P. Crochet proposed openstack/puppet-tripleo: Add Sahara profiles

Generated by 2.14.0 by Marius Gedminas - find it at!