Tuesday, 2016-08-09

*** Ryjedo_ has joined #tripleo00:06
*** Ryjedo has quit IRC00:08
*** Ryjedo_ is now known as Ryjedo00:08
*** myoung has quit IRC00:10
*** limao has joined #tripleo00:51
*** maeca1 has joined #tripleo01:03
*** dmacpher has joined #tripleo01:11
*** limao_ has joined #tripleo01:16
openstackgerritRatnakar Kolli proposed openstack/tripleo-heat-templates: Adding optional aci-opflex hiera data  https://review.openstack.org/35266101:18
*** limao has quit IRC01:19
*** ratnakar has joined #tripleo01:21
*** ratnakar has quit IRC01:22
*** chlong has joined #tripleo01:27
*** bana_k has quit IRC01:32
*** bana_k has joined #tripleo01:34
*** jrist has quit IRC01:36
*** ebalduf has quit IRC01:39
*** jrist has joined #tripleo01:50
*** yamahata has quit IRC02:02
*** yamahata has joined #tripleo02:07
jristthrash|g0ne: I am02:08
*** jdob has quit IRC02:08
*** jrist has quit IRC02:14
*** coolsvap has joined #tripleo02:19
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot  https://review.openstack.org/35048902:24
*** maeca1 has quit IRC02:25
*** jrist has joined #tripleo02:26
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot  https://review.openstack.org/35048902:34
*** Ryjedo_ has joined #tripleo02:40
*** fragatina has quit IRC02:44
*** Ryjedo has quit IRC02:44
*** Ryjedo_ is now known as Ryjedo02:44
openstackgerritMerged openstack/tripleo-heat-templates: Move *Flavor parameters into per-role templates  https://review.openstack.org/34726502:45
openstackgerritMerged openstack/instack-undercloud: Revert "glance: disable swift backend"  https://review.openstack.org/33665202:49
*** ebalduf has joined #tripleo02:53
*** coolsvap is now known as coolsvap_02:59
*** milan has quit IRC03:00
*** julim has quit IRC03:00
*** tzumainn has quit IRC03:03
*** limao_ has quit IRC03:03
*** limao has joined #tripleo03:04
*** limao has quit IRC03:09
*** milan has joined #tripleo03:10
openstackgerritXiang Chen proposed openstack/diskimage-builder: check APT_SOURCES earlier  https://review.openstack.org/34660503:19
*** saneax_AFK has quit IRC03:19
*** yamahata has quit IRC03:21
*** saneax_AFK has joined #tripleo03:22
*** saneax_AFK is now known as saneax03:22
*** saneax is now known as Guest7997703:22
openstackgerritXiang Chen proposed openstack/diskimage-builder: check APT_SOURCES earlier  https://review.openstack.org/34660503:25
*** Guest79977 is now known as saneax_AFK03:40
*** myoung has joined #tripleo03:42
*** limao_ has joined #tripleo03:59
*** fultonj has quit IRC04:00
*** limao__ has joined #tripleo04:00
*** limao_ has quit IRC04:03
*** links has joined #tripleo04:11
*** numans has joined #tripleo04:13
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot  https://review.openstack.org/35048904:25
*** yamahata has joined #tripleo04:28
*** bana_k has joined #tripleo04:33
*** bana_k has quit IRC04:45
*** trozet has quit IRC04:45
*** trozet has joined #tripleo04:45
*** coolsvap_ is now known as coolsvap04:51
*** r-mibu has joined #tripleo05:02
*** aqkhan__ has joined #tripleo05:02
*** skramaja has joined #tripleo05:05
*** aqkhan_ has quit IRC05:07
jtomasekawayne: hi05:07
*** david-lyle_ has joined #tripleo05:11
jtomasekawayne: it is the intermediate tripleo-api solution which is unfortunately not opensource at the moment :/ It is going to be removed in couple of weeks - replaced by Mistral driven api, which is in progress05:11
*** david-lyle has quit IRC05:13
ccamachohey coolsvap good morning! Mind to review https://review.openstack.org/#/c/351625/ when having some time? Its a long but trivial change, ordering a little bit things in puppet-tripleo05:15
coolsvapccamacho: ack will do sometime later05:16
openstackgerritSaravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bond  https://review.openstack.org/34918305:18
ccamachocoolsvap thanks dude!05:20
*** rcernin has joined #tripleo05:21
*** pgadiya has joined #tripleo05:21
*** jaosorior has joined #tripleo05:21
*** rlandy has quit IRC05:23
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Remove KeystoneRegion from controller.yaml  https://review.openstack.org/35233805:24
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Move neutron-related options for nova to base profile  https://review.openstack.org/35240205:24
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Move nova's kestone::auth parameters to API profile  https://review.openstack.org/35233305:24
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Move gnocchi's kestone::auth parameters to API profile  https://review.openstack.org/35231905:24
*** jtomasek_ has joined #tripleo05:26
*** jtomasek has quit IRC05:26
openstackgerritJuan Antonio Osorio Robles proposed openstack/diskimage-builder: DO NOT MERGE - TESTING KEYSTONE ENDPOINTS  https://review.openstack.org/35270505:27
*** jtomasek_ is now known as jtomasek05:37
*** trozet has quit IRC05:44
openstackgerritsandhya proposed openstack/diskimage-builder: Add support for building images capable of UEFI  https://review.openstack.org/28778405:46
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Include heat base profile outside of step  https://review.openstack.org/35271305:48
ccamachojaosorior! morning! man passing CI https://review.openstack.org/#/c/351625/ and https://review.openstack.org/#/c/352373/  :)06:02
jaosoriorccamacho: Hey dude, I'll check em out06:06
jaosoriorhow's it gonig?06:07
*** rbrady has quit IRC06:07
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Remove deprecated neutron auth options  https://review.openstack.org/35272406:08
ccamachothanks, but when having time :)  Fine man :) investing all day finding things to fix, improve and also add, also Ill like to make a deep dive session so im digging into a few few things that might be interesting, and you? How its the cold up there?06:08
ccamachojaosorior ^06:08
jaosoriorccamacho haha, not really cold06:09
*** jprovazn has joined #tripleo06:10
ccamachoIm at 30 degrees all day all night... ;(06:10
jaosoriorsomewhere between 17 degrees or something. So not cold but not hot either. Pretty nice if you ask me :P06:10
*** ebalduf has quit IRC06:12
ccamacholucky you06:12
*** ccamacho_ has joined #tripleo06:17
*** ccamacho_ has quit IRC06:18
*** ccamacho_ has joined #tripleo06:19
*** akuznetsov has joined #tripleo06:20
openstackgerritsandhya proposed openstack/diskimage-builder: Add support for building images capable of UEFI  https://review.openstack.org/28778406:22
*** ccamacho_ has quit IRC06:23
jaosorioreasy review moving some hieradata https://review.openstack.org/#/c/352313/406:24
*** radeks has joined #tripleo06:24
*** ccamacho_ has joined #tripleo06:24
*** ccamacho_ has quit IRC06:25
*** ccamacho_ has joined #tripleo06:28
*** bana_k has joined #tripleo06:28
*** ccamacho_ has quit IRC06:31
*** chlong is now known as chlong|mtg06:32
*** ccamacho_ has joined #tripleo06:33
jaosoriorccamacho_: I'm getting some mem usage errors lately too06:33
jaosoriorI think it's due to the fact that sahara was added to the list of services not too long ago06:33
ccamacho_jaosorior yeah, we should disable non mandatory services by default06:35
jaosoriorccamacho_: problem is, sahara is meant to be a default06:35
ccamacho_Mmm :(06:36
openstackgerritsandhya proposed openstack/diskimage-builder: Add support for building images capable of UEFI  https://review.openstack.org/28778406:39
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Set Sahara workers to 1  https://review.openstack.org/35272906:39
jaosoriorccamacho_: Lets see if this helps ^^06:39
*** pcaruana has joined #tripleo06:39
*** oshvartz has joined #tripleo06:43
*** tesseract- has joined #tripleo06:45
jaosoriorccamacho_: unfortunately I don't think it'll make much difference06:45
jaosorioras the default in the sahara configuration seems to be one already06:45
*** liverpooler has joined #tripleo06:45
ccamacho_jaosorio yeah man, anyways I.e. in my case I'll like to be able to deploy a sort of naked overcloud..06:45
jaosoriorseems that there is another option06:46
jaosoriorthe same one06:46
jaosoriorbut giving 0 there06:46
jaosoriorseems to run all sahara in one thread06:46
jaosoriormaybe that helps06:46
jaosoriorit'll be hella slow06:46
ccamacho_And enable other services or all on demand06:46
jaosoriorbut lets see06:46
*** ccamacho_ has quit IRC06:50
*** coolsvap is now known as coolsvap_06:51
*** ccamacho_ has joined #tripleo06:51
*** ccamacho_ has quit IRC06:52
*** ccamacho_ has joined #tripleo06:55
*** ccamacho_ has quit IRC06:57
*** ccamacho_ has joined #tripleo06:59
*** ccamacho_ has quit IRC07:00
*** ccamacho_ has joined #tripleo07:01
*** mcornea has joined #tripleo07:02
*** ccamacho_ has quit IRC07:04
*** ccamacho_ has joined #tripleo07:04
jaosoriormarios: Hey dude, if you have time can you check this out? https://review.openstack.org/#/c/352312/5 one of the commits needed to finally get keystone endpoints in puppet :D07:05
jaosoriord0ugal: Hey dude, how's it going?07:05
d0ugaljaosorior: Good good, just waking up :-D You?07:05
jaosoriord0ugal: having coffee deploying the overcloud :P07:06
jaosoriorbasic morning07:06
jaosoriord0ugal: Have you had a chance to check out latest mistral?07:06
jaosorior* the latest07:06
jaosoriorto see if they fixed it :D07:07
*** ebarrera has joined #tripleo07:09
*** ccamacho_ has quit IRC07:10
d0ugaljaosorior: No, not yet. I am avoiding busting my dev env07:10
jaosoriord0ugal: lol, crap, same here07:11
d0ugaljaosorior: Are the TripleO periodic jobs passing now?07:11
d0ugalThat is probably a good sign07:11
jaosoriord0ugal: No clue07:12
*** jpena|off is now known as jpena07:12
jaosoriord0ugal: Well... before celebrating we would need to ask in #rdo if they pinned mistral or not07:13
d0ugaljaosorior: good point07:13
jaosoriorasking now07:15
jaosoriord0ugal: They didn't pin it :D07:15
d0ugaljaosorior: I seen, thanks for checking!07:16
mariosjaosorior: sure thing07:16
*** stendulker has joined #tripleo07:19
*** florianf has joined #tripleo07:21
*** bootsha has joined #tripleo07:26
*** jbadiapa has joined #tripleo07:28
*** abehl has joined #tripleo07:28
*** leanderthal|afk is now known as leanderthal07:29
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Only db-sync sahara is included in the bootstrapnode  https://review.openstack.org/35274607:29
*** jpich has joined #tripleo07:29
*** tremble has joined #tripleo07:31
*** aufi has joined #tripleo07:32
*** sshnaidm|afk has quit IRC07:32
*** rbrady has joined #tripleo07:34
*** david-lyle has joined #tripleo07:44
*** hewbrocca-afk is now known as hewbrocca07:44
*** david-lyle_ has quit IRC07:46
*** bvandenh has quit IRC07:46
hewbroccaMorning folks07:46
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Only db-sync sahara is included in the bootstrapnode  https://review.openstack.org/35274607:48
jaosoriorccamacho: I think this ^^ will fix the db-sync issues in sahara that randomly happen07:48
*** pino|work has quit IRC07:48
jaosoriorfor instance, in that mistral patch, it failed because of that07:48
*** shardy has joined #tripleo07:54
*** ohamada has joined #tripleo07:55
*** devvesa has joined #tripleo07:55
*** ohamada has quit IRC07:55
*** bana_k has quit IRC07:56
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart: Stop using deprecated network range  https://review.openstack.org/34344307:56
*** ohamada has joined #tripleo07:56
*** pino|work has joined #tripleo07:58
*** coolsvap_ is now known as coolsvap07:58
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart: Persist ironic configuration changes  https://review.openstack.org/35168307:58
*** ifarkas_afk is now known as ifarkas07:59
d0ugalshardy: Hey07:59
*** lucas-dinner is now known as lucasagomes07:59
*** yamahata has quit IRC08:00
*** radeks has quit IRC08:04
*** athomas has joined #tripleo08:06
*** dmacpher has quit IRC08:06
*** mcornea is now known as mcornea|afk08:07
*** ramishra_ has quit IRC08:07
d0ugalshardy: un-ing, gotta go to a drs appt. I'll catch you after.08:07
*** ramishra has joined #tripleo08:08
*** sshnaidm|afk has joined #tripleo08:11
*** sshnaidm|afk is now known as sshnaidm08:11
*** akuznetsov has quit IRC08:13
jaosoriorHey guys, can we merge this? https://review.openstack.org/#/c/352312/ :D08:17
mariosjaosorior: am looking now man, but why is ovb failing didn'g get to it08:18
jaosoriormarios: Where?08:18
jaosoriormarios: Ah, it had failed because ever since sahara was enabled by default, we started seeing failures08:18
mariosjaosorior: oh i didn't refresh, was red on previous run08:18
jaosoriorthere's some OOM08:18
jaosoriorand there's a db-sync issue08:18
jaosoriormarios: Actually, in this commit I'm trying to fix the db-sync issue https://review.openstack.org/#/c/352746/08:19
jaosoriormarios: there appears to be race-conditions with sahara's db-sync. Since it's not limited to only run on the bootstrap node (like is the case for nova)08:20
*** Goneri has joined #tripleo08:20
jaosoriormarios: For the OOM I submitted this to tripleo-ci https://review.openstack.org/#/c/352729/ but I'm not sure if setting the sahara workers to 1 will help08:21
jaosoriormarios: By the way, regarding the neutron password, there is actually a commit that gets rid of it. It's part of a long series haha08:22
mariosjaosorior: hehe thanks :)08:22
*** chem has quit IRC08:22
jaosoriormarios: Anyway, as part of that series, can you check this one out too? https://review.openstack.org/#/c/352313/4 it's the one next in the chain08:23
*** amoralej|off is now known as amoralej08:23
mariosjaosorior: ack, i've a few requests to go through, added to list may be bumped though depending08:23
jaosoriormarios: thanks dude08:24
mariosjaosorior: doing my job like everyone else around here man08:24
mariosjaosorior: luckily red hat sends me some money every month to do this stuff :)08:25
openstackgerritMichele Baldessari proposed openstack/instack-undercloud: WIP galera and rabbit do not need to be accessible over the network  https://review.openstack.org/35248408:25
jaosoriormarios: we thanks taxi and bus drivers for taking us to places, and waiters from bringing food. Why not thank a fellow dev?08:26
mariosjaosorior: haha yeah i know man i am the same too I am just messing.08:27
openstackgerritMichele Baldessari proposed openstack/instack-undercloud: WIP galera and rabbit do not need to be accessible over the network  https://review.openstack.org/35248408:28
openstackgerritMerged openstack/tripleo-heat-templates: Move neutron's kestone::auth parameters to server profile  https://review.openstack.org/35231208:28
jaosoriorbandini: So mysql and rabbit are listening on ?08:28
jaosoriorin the undercloud08:28
jaosoriorcan we get rid of that? :O08:29
bandini_jaosorior: no they listen to, but I don't think they need to be accessible over that ip08:29
*** bandini_ is now known as bandini08:30
jaosoriorbandini_: if we can restrict them to listen in localhost, that would be alright08:30
bandinijaosorior: yeah my same thinking08:30
bandinijaosorior: dciabrin and I will scribble up some notes on general undercloud hardening ideas08:31
jaosoriorbandini: actually, that's pretty much what I want for all the services08:31
jaosorioronly haproxy should be listening on publicly accessible IPs08:31
bandinijaosorior: ah so only haproxy ? do we have a list of services that do need to listen to anything except localhost?08:31
jaosoriorbandini: not that I know of08:32
jaosoriorright now, the only ones that are not haproxy terminated that need to be accessible are ironic-inspector and zaqar08:32
jaosoriorI'm trying to get them into haproxy though08:32
jaosoriorthere are two commits out there08:32
jaosoriorthe only thing I'm missing is zaqar's websocket endpoint08:32
jaosorioronce that's done08:32
bandinioh I see, cool08:32
jaosoriorthen I can make all the services listen on localhost and get haproxy as a default for the undercloud08:33
jaosoriorcause right now, haproxy is only deployed if TLS is enabled08:33
*** dsariel has joined #tripleo08:33
jaosoriorof course, if you wanna step in and help out, that would be awesome :D08:33
bandinijaosorior: I am definitely interested. Do you have a link to those two reviews for ironic/zaqar?08:36
*** milan has quit IRC08:37
jaosoriorbandini: here https://review.openstack.org/#/c/332641/1208:37
jaosoriorbandini: That's for ironic, but you can see the series on the "Related changes" section to the right08:37
jaosoriorbandini: it was being blocked due to a mistral issue. But that got merged already. Now we just need to wait for promotion to happen in RDO08:38
*** dsariel has quit IRC08:38
ccamachobandini when having some free cycles mind to check https://review.openstack.org/#/c/351625/  ? Long but trivial submission  BTW jaosorior I think the parameters got scrambled as the puppet lint check is not checking the params order :P08:39
Jokke_morn ccamacho08:39
ccamachoJokke_ morning dude!08:40
*** myoung is now known as myoung|afk08:40
bandiniccamacho: will do so!08:42
bandinidoes anyone have recommendations for a console review-based tool? I kind of dislike doing it over the web interface08:43
jtomasekshardy: hi, could you please take a look at my comments here if it makes sense or not? https://review.openstack.org/#/c/298682/1308:43
bandiniis gerrty what all the cool kids use these days?08:43
jaosoriorbandini: Seems so08:44
jaosoriorI tried using it and couldn't get it running nicely08:44
openstackgerritJulie Pichon proposed openstack/python-tripleoclient: Fix 'no nodes found' message in introspect manageable nodes workflow  https://review.openstack.org/35278308:46
*** milan has joined #tripleo08:49
jaosoriorIf someone feels like merging some code https://review.openstack.org/#/c/352313/4 here's a small commit :D08:49
*** jaosorior is now known as jaosorior_brb08:49
*** limao__ has quit IRC08:49
openstackgerritLucas Alvares Gomes proposed openstack/instack-undercloud: Revert "Revert "Deploy and Upgrade Ironic to run in mod_wsgi""  https://review.openstack.org/35180308:49
*** limao has joined #tripleo08:50
*** vijaykc4 has joined #tripleo08:50
bandinijaosorior_brb: https://etherpad.openstack.org/p/undercloud-hardening-ideas08:53
*** gfidente has joined #tripleo08:57
openstackgerritMerged openstack/tripleo-docs: Upgrade documentation  https://review.openstack.org/30898508:57
shardyjtomasek: Hey, I think we're mixing two different definitions here, the parameters schema (returned by GetParametersAction) and the parameter values (modified by UpdateParametersAction)09:00
shardyjtomasek: I need to think about how we can improve that, but I'm not sure the parameter schema data can ever be from a mistral environment09:01
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud  https://review.openstack.org/34959109:02
jtomasekshardy:  no, I mean, the parameters values set using UpdateParameters are stored in mistral environment. I proposed 2 things in that comment:  UpdateParameters makes sure it stores there only the parameters which actually differ from what template processing outputs09:04
jtomasekand 2. that GetParameters makes use of this so it processes the templates + mistral environment and then it checks the mistral environment and identifies which parameters values come from the mistral environment09:05
shardyjtomasek: yup, but GetParameters doesn't return any parameter values, other than the template defaults09:06
shardyit only returns the schema, and nothing related to the mistral environment09:06
jtomasekshardy: yeah, that is the only way to get parameter values though09:06
shardywe could change that, but I'm not sure that we should09:06
jtomasekshardy: mistral environment gets passed as an additional environment to heat-validate09:07
shardyjtomasek: Ah, yeah I guess we can do that09:07
jtomasekshardy: so the parameter schema returns correct value (but as a parameter_default)09:07
*** milan has quit IRC09:07
shardyit's not very clear from that patch that processed_data contains the mistral environment09:08
shardyI'll have to pull this and test, but what you're requesting sounds OK09:08
*** electrofelix has joined #tripleo09:08
jtomasekshardy: and when we receive such parameter schema, we can compare it with what is stored in mistral env. and in action output make it clear with some flag, that a specific environment has been set by user via mistral environment09:08
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot  https://review.openstack.org/35048909:08
jtomasekups "specific parameter" not "specfic environment"09:08
shardyjtomasek: I guess what I'm not super-keen on is we're combining two different heat interfaces into a single category of action09:09
shardye.g heat parameters, and heat environments09:09
shardyin heat those are distinct interfaces09:09
*** coolsvap is now known as coolsvap_09:09
shardyhere we combine them in *Parameters actions09:09
shardyI'm wondering how that will work when we need to consider other environment interfaces09:09
shardye.g not just parameters, but resource_registry overrides, event sinks, hooks, etc09:10
jtomasekshadower: yeah, but it is the only way to determine the parameter values - as environments actually hold the parameter values09:10
*** chlong|mtg has quit IRC09:10
jtomasekshardy: resource registry updates would also get stored in mistral environment. I am considering it as an additional environment which is applied as last one and overrides any previous values defined in other environments09:12
ccamachohey bandini are you using pycharm? Im using the gerrit pluging for Pycharm and works really well and you have IDE+gerrit in one single window09:13
bandiniccamacho: ah no I don't use pycharm but I can give it a try :)09:13
*** akrivoka has joined #tripleo09:16
*** mgould|afk is now known as mgould09:19
*** abehl has quit IRC09:19
*** abehl has joined #tripleo09:20
*** bvandenh has joined #tripleo09:20
openstackgerritJiri Stransky proposed openstack/tripleo-docs: Fix environment-specific limiting for upgrade docs  https://review.openstack.org/35280009:24
jistrmarios: here you go ^ :)09:24
openstackgerritMerged openstack/tripleo-heat-templates: Move cinder's kestone::auth parameters to API profile  https://review.openstack.org/35231309:24
openstackgerritMerged openstack/tripleo-heat-templates: Configure project name for keystone::auth resource in heat  https://review.openstack.org/35243409:25
openstackgerritMerged openstack/tripleo-heat-templates: Move setting of Glance's keystone region to API profile  https://review.openstack.org/35231409:25
mariosjistr: ha.. so make this the parent review /me looks at the review09:26
mariosjistr: k whatever it is afollow on but thanks jistr09:29
*** mcornea|afk is now known as mcornea09:29
jistrmarios: yea i didn't want to block the main one on nits b/c it was quite large and it got finally like 4 + votes :)09:30
mariosjistr: sure np just firing up sphinxy to sanity check09:31
*** tosky has joined #tripleo09:33
openstackgerritMerged openstack/tripleo-heat-templates: Move nova's kestone::auth parameters to API profile  https://review.openstack.org/35233309:35
mariosjistr: lgtm thanks09:35
d0ugalshardy: ping09:39
shardyd0ugal: Hi!09:39
d0ugalshardy: I am getting myself stuck with the CLI work and wanted to talk it through with you09:40
d0ugalshardy: From what I can tell, the files need to be processed locally - to find relative files in environments etc.09:40
d0ugalshardy: but the result of processed files is then quite different from the original, so uploading that to swift seems wrong09:40
d0ugalshardy: An example that wont work: uploading environments with absolute paths (which we document btw) will break when in swift.09:41
*** jaosorior_brb is now known as jaosorior09:42
jaosoriorshardy: Thanks for the reviews dude09:42
shardyd0ugal: Yeah, I think we'll need to strip the path prefix so that everything is relative to the root of the swift container09:43
d0ugalshardy: so re-writing the environments is better than using heatclient to merge?09:43
d0ugalI felt a bit stumped as I didn't like either option09:44
shardyd0ugal: Yeah, ideally we want to avoid any client side merging, as we need to use server side merging to do e.g   https://review.openstack.org/#/c/330414/09:44
d0ugalshardy: Okay, cool - I guess I also then need to walk the envs to find all the referenced files and upload them.09:45
openstackgerritMerged openstack/python-tripleoclient: Make validation errors fatal by default  https://review.openstack.org/34960909:45
*** numans has quit IRC09:46
d0ugalshardy: I couldn't see any code in heatclient that would do that for me easily without doing the full merge. Do you know if that can be done or should I just write it?09:46
*** limao has quit IRC09:46
shardyd0ugal: heatclient should already do it, sec09:46
shardyd0ugal: basically you need the keys from the "files" map we pass to heat:09:47
*** chlong|mtg has joined #tripleo09:47
shardyd0ugal: and the values are what you need to upload to swift (unless they're already in the plan)09:47
shardyWe also need to store "environment_files" which is a list of keys09:48
shardythat exist in the files map09:48
shardyI assume that will end up in the mistral environment09:48
d0ugalshardy: okay, cool. I think I can work with that.09:48
d0ugaland then I need to make sure the UI can get those environments too09:49
jtomasekshardy, d0ugal: Do I understand it right, that certain environments in tht point to the templates that are outside the actual tht path?09:49
d0ugalI thought this bit would be easy :)09:49
d0ugaljtomasek: They can do, yes.09:49
shardyd0ugal: FWIW I'd probably work backwards from that hc.stacks.create, dump the **fields and see where each kwarg comes from in the mistral flow09:49
shardyend result should be the exact same request to heat09:50
d0ugaljtomasek: Well, not templates in t-h-t, but envs that the user adds can09:50
*** paramite has joined #tripleo09:50
shardyjtomasek: Yes, someone can do -e my_custom_env.yaml and it can reference any file09:50
shardyor even a URL09:50
jtomasekd0ugal: that is going to break the functionality then. lets say there is a tht tarball, that user uses to create plan (uploads to swift). if that tarbal includes such environments, then the plan is broken and the templates processing will fail09:50
jtomasekd0ugal: ah, ok09:50
jaosoriorshardy, marios: By the way, I proposed a commit to tripleo-quickstart changing the default network range to https://review.openstack.org/#/c/34344309:50
d0ugaljtomasek: http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/network_isolation.html?highlight=environment#create-network-environment-file09:50
d0ugaljtomasek: ^ That is an example of where we document doing that09:51
jtomasekd0ugal: ok, makes sense, thanks09:51
d0ugaljtomasek: so I just need to re-write the paths - simples ;)09:51
shardyjtomasek: This is the way heat environments have always worked - basically we just need to translate the files map into the swift container, and I think all should be OK09:51
jtomasekd0ugal: such case will probably not be possible to support with GUI09:51
jaosoriorshardy, marios: It's not enforced in any way in tripleo, but I thought quickstart would be a good starting place to have a discussion about a new default.09:51
mariosjaosorior: thanks adding to reviews for tomorrow09:52
d0ugaljtomasek: After the CLI re-writes the paths and uploads it to swift it should be fine for you09:52
jtomasekd0ugal: yeah09:52
d0ugaljtomasek: but yeah, if they are uploading a tar to the UI they need to make sure all the files are included - that I think should be expected by the user09:52
shardyd0ugal: Yup, I think all that's needed is an absolute to relative path conversion on the files map key, and upload all-the-things to swift09:52
d0ugalAlright, I'll see how I get on.09:53
d0ugalshardy, jtomasek - thanks!09:53
jtomasekd0ugal: I meant the case when user wants to update plan from GUI (add additional environment) -> such environment must point to templates that are in plan (or get uploaded together with that environment)09:53
jtomasekyep, thanks09:53
shardyjtomasek: Yeah, I think any time a user-provided environment is specified, we will have to handle passing additional files into the plan09:53
shardymaybe we'll need a user_files and user_environments subdirectory in the plan container or something09:54
shardyjaosorior: sure, sounds good, although we have to keep in mind that there are still a lot of non-quickstart users09:55
jtomasekshardy: I see it as a general updatePlan action, those user-provided ones can live normally next to the other templates/environments09:55
d0ugaljtomasek: Right, adding environments in the UI could be complicated. I guess you will need to check all the paths resolve to something.09:55
jtomasekshardy: template/environment metadata can be used to identify those if user wants09:55
shardyjtomasek: Yeah, we just need to be careful about path collisions, hence my suggestion re a subdir09:56
shardye.g if I do09:56
*** devvesa has quit IRC09:56
shardymkdir puppet && vim puppet/controller.yaml09:57
d0ugalshardy: Even with a subdir we could have collisions, so we might need to do unique names somehow.09:57
jtomasekd0ugal: that could get solved by a sane error message on templates processing action09:57
d0ugaljtomasek: Yup, that would be good.09:57
shardythen add puppet/controller.yaml to a user environment, we don't want it to overwrite the original controller.yaml in the plan09:57
jtomasekd0ugal: as that is the place where such potential problem gets uncovered09:57
jaosoriorshardy: yeah, for sure, but that's mostly meant to have a discussion about the defaults. And at least have somewhere that uses that default. Afterwards we can start pushing the rest of tripleo (mostly the undercloud and our CI) to use that09:57
d0ugaljtomasek: True, without downloading the full plan you can't tell. makes sense.09:57
shardyd0ugal: true, but just using a subdir which we know doesn't exist in t-h-t would be a good starting point09:58
d0ugalshardy: +109:58
d0ugalI'll get that working then worry about the next bit09:58
shardy++, sounds good, thanks!09:58
jtomasekshardy: considering a generic UpdatePlan action, I'd expect that action to fail if file with such path (key) is already presen in the plan, and notify user about it09:59
*** dtantsur|sick is now known as dtantsur10:00
d0ugaljtomasek: We might need to update the file in some cases10:00
d0ugali.e. running this twice: openstack overcloud deploy --templates -e myenv.yaml10:00
jtomasekd0ugal: hmm, right10:01
*** Nakato has quit IRC10:05
*** Nakato has joined #tripleo10:06
*** radeks has joined #tripleo10:07
*** athomas has quit IRC10:11
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375010:14
*** pblaho has quit IRC10:15
*** jcoufal has joined #tripleo10:15
*** athomas has joined #tripleo10:18
*** jeckersb is now known as jeckersb_gone10:32
Jokke_EmilienM: around?10:32
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: DO NOT MERGE: testing ironic  https://review.openstack.org/32987210:39
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: Add composable service for configuring Nova compute with Ironic  https://review.openstack.org/34143310:39
*** sshnaidm is now known as sshnaidm|lnch10:39
*** jcoufal has quit IRC10:44
*** Ryjedo_ has joined #tripleo10:48
*** Ryjedo has quit IRC10:49
*** Ryjedo_ is now known as Ryjedo10:49
jaosoriorJokke_: too early. give him a couple more hours10:50
jaosoriorHe's in quebec I think10:51
*** lucasagomes has quit IRC10:51
jaosoriorJokke_: How's it going? Visited Finland yet?10:51
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add puppet-lint for checking parameters order in puppet-tripleo manifests  https://review.openstack.org/35284710:52
*** vijaykc4 has quit IRC10:53
*** assassin is now known as ubijtsa10:54
*** florianf has quit IRC10:54
*** ubijtsa is now known as assassin10:54
*** vijaykc4 has joined #tripleo10:55
*** apetrich has joined #tripleo10:57
*** akrivoka has quit IRC10:57
*** akrivoka has joined #tripleo11:01
jaosoriorI'm seeing several post_failures in CI11:05
jaosorioranybody has an idea of what's going on?11:05
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates: Availability monitoring agents support  https://review.openstack.org/25478811:07
openstackgerritJulie Pichon proposed openstack/tripleo-specs: Add 'bug tagging' policy  https://review.openstack.org/35285211:08
*** r-mibu has quit IRC11:11
openstackgerritMerged openstack/puppet-tripleo: Fix parameters and headers inconsistency in the puppet manifests.  https://review.openstack.org/35162511:12
gfidentejaosorior, on https://review.openstack.org/#/c/352746/2/manifests/profile/base/sahara.pp11:12
gfidenteI think if we don't add the or condition11:12
gfidentethen sync_db might be running without proper sahar.conf ?11:13
gfidenteunless it includes ::sahara?11:13
jaosoriorgfidente: can you explain?11:13
*** stendulker has quit IRC11:16
*** akuznetsov has joined #tripleo11:16
*** devvesa has joined #tripleo11:19
*** r-mibu has joined #tripleo11:19
*** lucasagomes has joined #tripleo11:22
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Fix use of bootstrap_node in cinder base profile  https://review.openstack.org/35236511:23
*** sshnaidm|lnch is now known as sshnaidm11:25
*** jaosorior has quit IRC11:27
*** jaosorior has joined #tripleo11:28
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Fix use of bootstrap_node in cinder base profile  https://review.openstack.org/35236511:30
*** apetrich_ has joined #tripleo11:32
*** vijaykc4 has quit IRC11:32
*** vijaykc4 has joined #tripleo11:32
*** lucasagomes has quit IRC11:34
*** apetrich_ has quit IRC11:34
*** apetrich has quit IRC11:35
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add support for cinder::volume::rbd hiera parameters  https://review.openstack.org/33513711:35
*** lucasagomes has joined #tripleo11:35
*** dsariel has joined #tripleo11:36
paramiteHi guys, is there a way how to actually find out what is wrong with role_data, when I get: [overcloud]: CREATE_FAILED Resource CREATE failed: The Referenced Attribute (ControllerServiceChain role_data) is incorrect.11:37
*** social has quit IRC11:37
*** devvesa has quit IRC11:37
*** devvesa has joined #tripleo11:38
*** jeckersb_gone is now known as jeckersb11:39
*** lucasagomes has quit IRC11:39
*** coolsvap_ is now known as coolsvap11:39
*** lucasagomes has joined #tripleo11:41
gfidenteparamite, what is the submission?11:41
gfidenteprobably you're operating on a parameter which isn't defined for the service template11:41
paramitegfidente, https://review.openstack.org/#/c/254788/2811:41
paramitegfidente, so this is causing it: https://review.openstack.org/#/c/254788/28/puppet/services/services.yaml ?11:42
*** dprince has joined #tripleo11:43
shadowercan I have a review for https://review.openstack.org/#/c/352278/ ? It's short and it'll make the subsequent validation patches simpler + keep the docs up to date11:43
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: remove overcloud roles from oooq  https://review.openstack.org/35125911:43
sshnaidmpabelanger, hi11:44
*** lucasagomes is now known as lucas-hungry11:44
sshnaidmpabelanger, for running experimental jobs on tripleo-heat-templates project, should I add here "- name: experimental-tripleo-jobs" in https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L14139 ?11:46
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Configure keystone endpoints in service profiles  https://review.openstack.org/35236411:47
*** julim has joined #tripleo11:47
*** milan has joined #tripleo11:48
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: add debug to undercloud post  https://review.openstack.org/34555911:49
*** larsks has quit IRC11:51
*** dprince has quit IRC11:56
*** larsks has joined #tripleo11:56
*** dprince has joined #tripleo11:56
*** sshnaidm is now known as sshnaidm|afk11:56
*** weshay has joined #tripleo11:57
*** fultonj has joined #tripleo11:57
openstackgerritMerged openstack/tripleo-ui: Add npm version to webpack output  https://review.openstack.org/34897611:57
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Configure keystone endpoints in service profiles  https://review.openstack.org/35236411:57
jaosoriorshardy: Is there a way to debug what comes out of the specific service chain templates?11:58
jaosoriorshardy: In this commit the role_data output comes out as null https://review.openstack.org/#/c/352319/811:59
jaosoriorand I don't understand why :/11:59
*** jayg|g0n3 is now known as jayg12:00
*** pkovar has joined #tripleo12:01
*** akuznetsov has quit IRC12:01
*** ohamada_ has joined #tripleo12:02
*** ohamada has quit IRC12:02
*** numans has joined #tripleo12:02
*** jpena is now known as jpena|lunch12:02
*** devvesa has quit IRC12:03
*** rhallisey has joined #tripleo12:04
*** vijaykc4 has quit IRC12:04
*** jeckersb is now known as jeckersb_gone12:05
*** jcoufal has joined #tripleo12:06
shardyjaosorior: Yes, look at the heat stack-show output for the nested stack with the null output12:07
shardyjaosorior: I raised a heat bug about it, we should surface the error at validation time12:08
shardyjaosorior: another option is to create a stack only containing the service template12:08
*** ioggstream has joined #tripleo12:09
jaosoriorshardy: is it possible to do a heat resource-show for the specific element in the chain?12:11
shardyjaosorior: Yes, you just need the ID of the stack, sec12:11
*** yolanda_ has joined #tripleo12:11
*** amoralej is now known as amoralej|lunch12:12
shardyjaosorior: http://paste.openstack.org/show/552433/12:13
*** jeckersb_gone is now known as jeckersb12:14
shardyjaosorior: if you know the failing service you can just do heat resource-list -n5 overcloud | grep "OS::TripleO::Services::GlanceRegistry"12:14
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: WIP - Add puppet-lint for checking parameters order in puppet-tripleo manifests  https://review.openstack.org/35284712:15
*** yolanda_ has quit IRC12:15
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: fix teardown (tagging and ansible_user)  https://review.openstack.org/35211212:15
jaosoriorshardy: Excellent12:15
jaosoriorshardy: Right, so for a very strange reason the gnocchi API is giving a null role_data12:16
jaosoriorit's specifically that one12:16
shardyjaosorior: on t-h-t master, or an under review patch?12:16
jaosoriorunder review12:16
jaosoriorshardy: So when it was initially proposed, it had this same issue. But it was worked around...somehow12:17
shardyjaosorior: whats the link pls?12:17
jaosoriorshardy: https://review.openstack.org/#/c/352319/812:17
*** ccamacho is now known as ccamacho|lunch12:20
*** pradk has joined #tripleo12:20
*** jcoufal has quit IRC12:22
*** jcoufal has joined #tripleo12:22
*** akshai has joined #tripleo12:22
openstackgerritMerged openstack/tripleo-quickstart: Add dlrn-gate-testing job type for roles-deploy.sh  https://review.openstack.org/35175912:23
*** lucas-hungry is now known as lucasagomes12:24
*** jdob has joined #tripleo12:26
*** thrash|g0ne is now known as thrash12:26
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Include heat base profile outside of step  https://review.openstack.org/35271312:28
*** rlandy has joined #tripleo12:28
openstackgerritMerged openstack/tripleo-quickstart: Improve readability of the overcloud failure output  https://review.openstack.org/35115512:29
*** yolanda has quit IRC12:29
openstackgerritMerged openstack/tripleo-heat-templates: Move neutron-related options for nova to base profile  https://review.openstack.org/35240212:30
openstackgerritJiri Stransky proposed openstack/tripleo-docs: Fix environment-specific limiting for upgrade docs  https://review.openstack.org/35280012:31
*** ohamada_ has quit IRC12:31
*** ohamada_ has joined #tripleo12:31
*** ayoung has quit IRC12:33
pabelangersshnaidm|afk: yes12:33
*** sshnaidm|afk is now known as sshnaidm12:33
EmilienMcan someone approve https://review.openstack.org/#/c/352724/ please?12:34
EmilienMbefore puppet-neutron breaks tripleo12:34
jaosoriorEmilienM: Good thing that breaking CR depends on that change12:34
sshnaidmpabelanger, then I'll go wild: https://review.openstack.org/35288012:35
jaosoriorbut yeah, it should be approved soon12:35
EmilienMjaosorior: the last one is https://review.openstack.org/#/c/352407/12:35
*** maeca1 has joined #tripleo12:35
*** sshnaidm has quit IRC12:35
*** florianf has joined #tripleo12:35
shardyjaosorior: I think I spotted the issue, see comment on https://review.openstack.org/#/c/352319/812:35
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add cinder-backup service as role  https://review.openstack.org/30456312:36
jaosoriorshardy: Awesome!12:36
jaosoriorshardy: Will try to deploy with the changes you pointed out, lets see how it goes12:36
shardyjaosorior: I think just not removing the GnocchiPassword parameter should fix it12:36
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart: Run dstat on the virthost and the undercloud  https://review.openstack.org/34959112:36
*** egafford has joined #tripleo12:37
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] remove ci-scripts from the oooq repo  https://review.openstack.org/35132412:37
jistrmarios: FYI after 2nd round of downstream issues in compute vs. novacompute i think we may want to consider again merging https://review.openstack.org/#/c/30570512:37
jistrmarios: +2'd and rechecked12:38
jistrafter we started supporting upgrades upstream i think this one might be better than the other one12:39
openstackgerritMerged openstack/tripleo-heat-templates: Remove static hieradata  https://review.openstack.org/35174112:39
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: fix teardown (tagging and ansible_user)  https://review.openstack.org/35211212:40
shardyjistr: FWIW I've been thinking that long term this, and other similar workarounds for backwards compatibility, would be better handled via a mistral workflow that can adjust the stack parameters by looking at the already deployed stack12:40
shardyat least then we'd be able to maintain the list of hacks in one place12:41
mariosjistr: ok ... and then we point to it where applicable in docs i guess (if starting at ... include ...  in the upgrade init setting it once should do it)12:41
shardyvs an ever increasing variety of special environment files12:41
jistrshardy: yea that would be cool12:41
openstackgerritMerged openstack/instack-undercloud: Remove deprecated neutron auth options  https://review.openstack.org/35272412:41
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: [WIP] undercloud reboot  https://review.openstack.org/35048912:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Remove KeystoneRegion from controller.yaml  https://review.openstack.org/35233812:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Move gnocchi's kestone::auth parameters to API profile  https://review.openstack.org/35231912:42
openstackgerritMerged openstack/tripleo-heat-templates: Remove outdated Neutron auth options  https://review.openstack.org/35240712:43
jistrmarios: yea. Was thinking it would be nice if Newton in particular would have synced names on fresh deploys already, so that we don't have to carry the desync forward for much longer.12:44
jaosoriorshardy: Seems that the issue is still there :/12:44
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Use modern auth options for Neutron  https://review.openstack.org/35251812:44
shardyjaosorior: You've got the same issue with KeystoneRegion12:47
shardylocal testing with either stack-show or output-show would tell you that12:48
openstackLaunchpad bug 1599114 in heat "Outputs aren't correctly validated" [Medium,In progress] - Assigned to Oleksii Chuprykov (ochuprykov)12:49
shardythat's the heat bug btw, which will make these errors easier to debug12:49
shardywell, the fix for it will12:50
*** tzumainn has joined #tripleo12:50
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart: fix link in the readme file  https://review.openstack.org/35178712:51
*** jrist has quit IRC12:52
*** numan_ has joined #tripleo12:54
*** fultonj has quit IRC12:55
*** numans has quit IRC12:55
*** morazi has joined #tripleo12:56
*** fultonj has joined #tripleo12:56
*** yolanda has joined #tripleo12:57
EmilienMdo we have plans to disable sahara by default?12:57
EmilienMit's failing quite often in our CI12:58
EmilienMError: /Stage[main]/Sahara::Db::Sync/Exec[sahara-dbmanage]: Failed to call refresh: sahara-db-manage --config-file /etc/sahara/sahara.conf upgrade head returned 255 instead of one of [0]\u001b[0m\n\u001b[1;31mError: /Stage[main]/Sahara::Db::Sync/Exec[sahara-dbmanage]: sahara-db-manage --config-file /etc/sahara/sahara.conf upgrade head returned 255 instead of one of [0]12:58
slagleEmilienM: let's just do it12:58
EmilienMslagle: ok, I'm doing it12:58
EmilienMslagle: have you seen it also?12:58
*** sshnaidm has joined #tripleo12:58
slaglenot specifically12:58
*** coolsvap is now known as _coolsvap_12:58
*** jpena|lunch is now known as jpena12:58
slaglebut our run times are horrible12:58
slaglewe keep making everything slower12:59
slagleso +1 on disabling things we don't need12:59
openstackgerritJiri Stransky proposed openstack/tripleo-docs: Only show stable repositories in upgrade guide  https://review.openstack.org/35288313:00
EmilienMslagle: can you approve https://review.openstack.org/#/c/349408/ then?13:00
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Create per-role node_names in hieradata for all nodes  https://review.openstack.org/34897313:00
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: WIP Create composable mapping between enabled services and role ips  https://review.openstack.org/34897413:00
*** rook_ is now known as rook13:01
slagleEmilienM: done13:01
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Move AllNodesValidationDeployments into jinja template loop  https://review.openstack.org/33758713:01
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Convert overcloud.yaml to support jinja2 templating  https://review.openstack.org/31567913:02
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Remove Compute role parameters from overcloud.yaml  https://review.openstack.org/34792613:02
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Move AllNodesDeployments into jinja template loop  https://review.openstack.org/33726713:02
*** jeckersb is now known as jeckersb_gone13:03
*** myoung|afk is now known as myoung13:04
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Disable sahara services by default  https://review.openstack.org/35288613:05
EmilienMslagle: thanks. And last one ^13:05
*** jrist has joined #tripleo13:05
*** jrist has joined #tripleo13:05
openstackgerritMerged openstack/tripleo-heat-templates: Disable mistral services by default.  https://review.openstack.org/34940813:05
*** links has quit IRC13:06
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Convert EndpointMap to not require per-service VIP parameters  https://review.openstack.org/34290013:06
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Allow map_replace substitution of network names  https://review.openstack.org/34551013:06
*** jeckersb_gone is now known as jeckersb13:07
rcerninshardy: slagle: If we restart the heat-engine service it kills the parent stack, but doesn't clear any IN_PROGRESS which are nested. Could that cause problems with deployment or is it fine to just re-run the deployment, even if the tasks are still stuck in the IN_PROGRESS state?13:07
*** trozet has joined #tripleo13:07
shardyrcernin: No, all stacks including nested stacks should be either COMPLETE or FAILED, or the update won't work13:08
shardysounds like you have an old heat as I believe we fixed several issues around this13:09
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Validation to check the undercloud processes  https://review.openstack.org/35124813:09
openstackgerritJiri Stransky proposed openstack/tripleo-docs: Only show stable repositories in upgrade guide  https://review.openstack.org/35288313:10
shardyrcernin: Note it's important to distinguish between nested stacks that are IN_PROGRESS from deployments (e.g from heat deployment-list)13:10
shardyhaving IN_PROGRESS deployments is OK and a known issue - they will be replaced provided the owning resource is FAILED13:10
rcerninshardy: heat-templates and engine is 5.0.1-513:11
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Port the network environment validation  https://review.openstack.org/34158613:11
*** oshvartz has quit IRC13:12
*** bfournie has quit IRC13:13
*** bootsha has quit IRC13:14
rcerninshardy: The issue is that parent is FAILED but no new tasks are able to be created against that node with tasks that are stuck in IN_PROGRESS.13:14
*** julim has quit IRC13:14
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Validation to detect if undercloud disks use 512e  https://review.openstack.org/32217913:14
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates: DO NOT MERGE: testing ironic  https://review.openstack.org/32987213:15
shardyrcernin: When you say tasks, do you mean deployments or resources?13:15
shardye.g is heat giving you an error, or is the node stuck because it's still processing the old deployment?13:16
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Add network gateway validation  https://review.openstack.org/32938513:18
rcerninshardy: the deployment-list shows these IN_PROGRESS deployments.13:19
*** jpeeler has joined #tripleo13:19
shardyrcernin: right, and that is OK unless the node is wedged on one of them which never completes13:20
*** lblanchard has joined #tripleo13:21
shardySimple stable backport ready for review/approve if anyone has a moment: https://review.openstack.org/#/c/345650/13:25
*** amoralej|lunch is now known as amoralej13:25
jaosoriorshardy: So I'm doing local testing, which is why I mentioned that it didn't work. But debugging this stuff is really not trivial13:26
jaosoriorbecause the output it not something that's relevant to the issue13:26
*** ayoung has joined #tripleo13:26
jaosoriorEmilienM: wasn't sahara supposed to be included by default?13:26
EmilienMjaosorior: it's currently failing very often in CI13:27
jaosoriorshardy: I see13:27
shardyjaosorior: I agree we can improve it, which is why I raised that heat bug, but the output-show error is very clear13:27
jaosorioronly thing I got out was13:27
jaosorior2016-08-09 12:48:01 [overcloud]: CREATE_FAILED Resource CREATE failed: The Referenced Attribute (ControllerServiceChain role_data) is incorrect.13:27
EmilienMmaybe we could disable it now and investigate why it fails afterwards13:27
shardyjaosorior: Yeah, until that heat bug is fixed you have to dig in to the nested stacks to figure out which one broke role_data13:27
jaosoriorEmilienM: Actually this is my attempt to fix it https://review.openstack.org/#/c/352746/13:27
*** Goneri has quit IRC13:28
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Remove KeystoneRegion from controller.yaml  https://review.openstack.org/35233813:29
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Move gnocchi's kestone::auth parameters to API profile  https://review.openstack.org/35231913:29
EmilienMjaosorior: nice13:30
EmilienMjaosorior: I +213:30
EmilienMgfidente: can you look it again maybe?13:31
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Convert EndpointMap to not require per-service VIP parameters  https://review.openstack.org/34290013:33
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Allow map_replace substitution of network names  https://review.openstack.org/34551013:33
*** Goneri has joined #tripleo13:33
gfidentebnemec, was working on a doc update to add the ceph repos13:33
gfidentenot sure how familiar are you with those docs or if you stick to tripleo.sh as I do13:34
gfidentebut I don't see where we give the list of .repo files to be included in the image for the image building13:34
gfidentebnemec, any clue where OVERCLOUD_IMAGES_DIB_YUM_REPO_CONF is set in tripleo-docs? or if not, why?13:35
*** rodrigods has quit IRC13:35
*** rodrigods has joined #tripleo13:35
jaosoriorayoung: Hey man, saw your tweet about having fixed SAML federation behind HAProxy. What did you do?13:36
*** ccamacho|lunch is now known as ccamacho13:37
*** cdearborn has joined #tripleo13:39
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Remove NeutronPassword parameter from puppet/controller.yaml  https://review.openstack.org/35290713:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Only db-sync sahara is included in the bootstrapnode  https://review.openstack.org/35274613:42
openstackgerritGiulio Fidente proposed openstack/tripleo-docs: Document Ceph repositories to be enabled  https://review.openstack.org/35291013:44
*** ramishra has quit IRC13:45
*** bfournie has joined #tripleo13:45
*** bootsha has joined #tripleo13:46
ayoungjaosorior, Majik13:46
ayoungjaosorior, Blackest of majik.13:46
jaosoriorI see13:46
ayoungjaosorior, na, it was actually a few things we needed to sort13:47
openstackgerritMerged openstack-infra/tripleo-ci: upgrades job is renamed to updates  https://review.openstack.org/35193713:47
ayoungwe needed to get Keystone's public interface on TLS13:47
jaosoriorshardy: the KeystoneRegion parameter did the trick! it's now going forward with the deployment13:47
jaosoriorayoung: Whhat do you mean?13:47
jaosoriorYou mean we need to get keystone to stop being terminated by haproxy?13:47
ayoungjaosorior, and needed to force the interface to report itself as openstack.<hostname -d>13:47
ayoungjaosorior, horizon was using the non TLS13:48
ayoungneed to add to HA proxy the ability to rewrite the URLs for redirects13:48
jaosoriorso the public_endpoint (or something like that) parameter in the keystone configuration?13:48
jaosoriorthought that was being set13:48
ayoungjaosorior, /etc/openstack-dashboard/local_settings13:48
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton  https://review.openstack.org/34699513:48
ayoungrodrigods, discussing here13:49
jaosoriorayoung: tell me the changes you made and I can start setting up commits for tripleo13:49
ayoungjaosorior, working up a blog post.  There is a lot of little details13:49
jaosoriorayoung: Sure man13:49
*** pgadiya has quit IRC13:49
jaosoriorayoung: Can you also open a launchpad bug so we can backport this stuff?13:49
ayoungjaosorior, yep...all that13:49
rodrigodsayoung, nice13:50
ayoungjaosorior, I also need to take a better look at the current puppet support. I suspect I am not using it all properly13:50
EmilienMtripleo meeting in 10 min, https://etherpad.openstack.org/p/tripleo-meeting-items13:50
ayoungfor example, I need to set a bunch of vaules in local-settings, and I don't know if they are all supportted. I did it manually post deploy, but it really should be done via the heat template13:50
rodrigodsayoung, what steps are you doing automatically?13:51
ayoungrodrigods, I'll paste my current deploy-env13:51
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Implement non-ovb overcloud update job - Newton -> Newton  https://review.openstack.org/35133013:52
*** bfournie has left #tripleo13:52
*** bfournie has joined #tripleo13:52
jaosoriorayoung: Sure thing, when you have that blog post ping my and I'll start looking into that too13:52
ayoungrodrigods, jaosorior http://paste.openstack.org/show/552441/13:53
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375013:53
ayoungjaosorior, so I lumped all of my changes into a single file that I can revision control.  We should talk Certificate management, though.13:53
jaosoriorayoung: We should. What I'm doing at the moment is getting rid of the obstacles for the internal endpoints though (like finally getting keystone endpoints managed by puppet) ... and it's really near. at least I have that working in my local environment, just needs to pass CI and merge13:54
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: additional info about host in CI  https://review.openstack.org/35291613:55
ayoungjaosorior, so, we should provide a midground prior to the auto-registration of nodes. Let the user do it manually, and set an OTP/controller map13:55
rodrigodsayoung, so the mapping, idp, protocol, etc and also, running the keycloak client install is done manually?13:55
ayoungrodrigods, I have a Ansible to do that, but, yes, I ended up doing it manually13:56
jaosoriorayoung: well, without the service that rcrit was working on. I was planning on passing the OTP's via heat metadata13:56
ayoungrodrigods, that stuff goes in the database, and probably should not be done via puppet on each node. I also generated the metadata on the undercloud node, made it into a tarball, and expanded it manually, which is something that needs to happen after the yum install of mod_auth_mellon13:57
ayoungjaosorior, yes, exactly13:57
ayoungjaosorior, do you have a POC of doing that?13:57
*** julim has joined #tripleo13:57
jaosoriorayoung: Not at the moment but it shouldn't be a big deal.13:57
shardymeeting starts in ~3mins https://wiki.openstack.org/wiki/Meetings/TripleO13:58
*** rbrady has quit IRC13:58
ayoungjaosorior, excellent. I was doing the old steps, which include private keys in a heat template...I find that yucky13:58
shardyplease add any one-off items there13:58
*** rbrady has joined #tripleo13:58
jaosoriorayoung: well, that will probably stay supported for the public endpoints. There's not much to do about that.13:58
ayoungjaosorior, it is disgusting13:59
jaosoriorayoung: hopefully soon we can give the user an option to have certs provided by FreeIPA as al alternative13:59
ayoungits an antipattern, we should not be propagating it13:59
jaosoriorayoung: If you can figure out a better way to support user-provided certificates. I'm in.13:59
ayoungjaosorior, if we really need a Lowest comon denominator like that, we should think it through13:59
jaosoriordeployers will need to inject certificates somehow. That's a requirement14:00
*** jistr is now known as jistr|debug14:00
ayoungjaosorior, Anchor or FreeIPA is the simplest set of alternatives. Let Cermonger do it14:00
*** _coolsvap_ is now known as coolsvap14:00
ayoungjaosorior, yep...we can brainstorm.  I could see Heat playing the role of the CA proxy14:00
jaosoriorayoung: And what about the sharing of private keys? if we don't share them (which is what you guys had proposed) won't that break SSL pinning?14:01
bnemecgfidente: Right now we don't use DIB_YUM_REPO_CONF.  There's a patch up to start using it ( https://review.openstack.org/#/c/337669 ), but there's some contention over the move to a non-OSC image build command so it hasn't merged yet.14:01
jaosoriorayoung: So browsers will have a bad time when trying to access horizon14:02
ayoungjaosorior, shouldn't.  Heat should  be capable of getting the CSRs signed by a real CA in the case where certmonger is not capable of reaching it directly14:02
ccamachoEmilienM jaosorior please can you briefly check this? This got merged https://review.openstack.org/#/c/349408/ but in the tree https://review.openstack.org/#/c/323436/ isn't yet. So Im not sure if things will break14:02
ayoungthe current self-signed is no better anyway.14:03
jaosoriorayoung: That doesn't address the issue14:03
ayoungjaosorior, so long as we can "swap up" to a real cert, we could start with a self signed.14:03
*** limao has joined #tripleo14:03
ayoungjaosorior, anyway, as I said, we need to discuss...14:04
*** skramaja_ has joined #tripleo14:04
*** ebalduf has joined #tripleo14:04
jaosoriorayoung: If each node has a different private key. When a browser that has SSL pinning enabled accesses horizon, it will notice that the signature of the server certificate doesn't match the one that it stored and it will spit an error.14:05
ccamachojaosorior EmilienM should not break anything*** but just to be sure14:05
EmilienMccamacho: should not14:06
*** skramaja has quit IRC14:06
*** skramaja_ is now known as skramaja14:06
jaosoriorccamacho: I actually think that the db-sync in mistral is broken14:07
jaosoriorwas gonna submit a patch about it but got caught up with other stuff14:07
d0ugaljaosorior: still broken?14:07
ccamachojaosorior, Ill look it also14:07
gfidentebnemec, so how does DIB know about the repos? is there an element adding the delorean .repo ?14:07
jaosoriord0ugal: now I'm talking about the puppet-tripleo parts haha. Not mistral itself14:08
jaosoriorayoung: But yeah, if we can come up with a solution that replaces the current approach, that would be great14:08
*** limao_ has joined #tripleo14:08
d0ugaljaosorior: ah okay, cool :)14:08
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: Add undercloud reboot  https://review.openstack.org/35048914:08
jaosoriorI'm actually quite fond of rcrit's initial suggestion of using FreeIPA's Vault to fetch the keys. It could be somethign else too.14:09
jaosoriorayoung ^^14:09
bnemecgfidente: No, whatever paths you set in DIB_YUM_REPO_CONF get copied into the image and used as the yum repos verbatim.  The elements don't have to know anything about the repos you use.14:09
ayoungjaosorior, so all of the HA proxy impls need identical certs, not just certs signed by the same CA....nasty14:10
*** jcoufal has quit IRC14:10
gfidentebnemec, yeah so I mean if we're not using it, how is currently DIB provisioning the repos to build the images?14:10
jaosoriorayoung: Yeah dude. Actually my initial proposal had different certs with the same DNS and different keys. But then I stumbled upon SSL pinning.14:11
jaosoriorayoung: After going back and forth with was to to public SSL, ended up with what's there now14:11
*** limao has quit IRC14:11
ayoungjaosorior, Holy Carp.  Pinning seems like a spectacularly bad idea.  its like someone said "how can I best mess up security and force peopoe to use compromised keys?"14:11
jaosoriorayoung: I agree... Aaaand it's used (apparently)14:12
bnemecgfidente: Oh, yeah I think there's a delorean-repo element or something that is pulling them down.14:12
ayoungjaosorior, can we say that we won't support it?14:12
ayoungIts like,  wow...such bad.14:12
jaosoriorayoung: not something I can answer14:12
jaosoriorayoung: We could push the certmonger-based solution as an alternative, and document that SSL pinning won't work if you deploy with that.14:13
ayoungjaosorior, ++14:13
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: Composable Mistral services  https://review.openstack.org/32343614:13
openstackgerritMiles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package  https://review.openstack.org/34421414:14
gfidentebnemec, ack14:14
gfidenteso I'd need one for the ceph repos too14:14
bnemecgfidente: I'd really prefer that we just move to the method in thrash's review.  Actually, I'd prefer that we had a user-facing tool to handle repo management, but at least it's a step in that direction.14:16
*** bootsha has quit IRC14:16
*** r2bit__ is now known as r2bit14:16
thrashbnemec: yeah... it would be relatively easy to add some repo entries to the yaml.14:16
thrashif we wanted. :)14:17
openstackgerritMatt Young proposed openstack/tripleo-common: Add support to image build yaml input to handle env vars  https://review.openstack.org/31808714:17
*** r2bit has quit IRC14:17
*** r2bit has joined #tripleo14:17
gfidentethrash, bnemec ok adding me to https://review.openstack.org/#/c/337669 meantime14:17
*** limao_ has quit IRC14:19
*** liverpooler has quit IRC14:19
jaosoriorayoung: If you're at the openstack-meeting-alt channel we're talking about the default network range and trying to come up with an alternative14:20
openstackgerritBrad P. Crochet proposed openstack-infra/tripleo-ci: Use tripleo-build-images for CI  https://review.openstack.org/33631214:21
*** yamahata has joined #tripleo14:22
pabelangerweshay: If you didn't see: http://docs.openstack.org/developer/tripleo-quickstart/14:23
weshaypabelanger, nice!14:24
openstackgerritBrad P. Crochet proposed openstack-infra/tripleo-ci: Use tripleo-build-images for CI  https://review.openstack.org/33631214:24
pabelangerweshay: I'll finish up converting the other docs today14:24
bnemecgfidente: You could split out the repo setup part of thrash's review.  I don't think that will be contentious since it's what we're doing in CI anyway.14:24
*** tosky_ has joined #tripleo14:25
*** tosky has quit IRC14:27
*** akrivoka has quit IRC14:27
*** akrivoka has joined #tripleo14:28
*** oshvartz has joined #tripleo14:28
*** yamahata has quit IRC14:28
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Fix mistral sync_db profile steps  https://review.openstack.org/35293714:32
rcerninshardy: when doing the deployment the os-refresh-config has to run alongside the os-collect-config? when starting a deployment in ps output i see only os-collect-config no os-refresh-config nor any other puppet deployments, there is no deployed folder in /var/lib/heat-config when checking journalctl -u os-collect-config there is polling every 30s but only WARNINGS about no auth url for zaqar.14:34
gfidenteneeds votes pls :) https://review.openstack.org/#/c/352365/14:36
*** ramishra has joined #tripleo14:38
*** Goneri has quit IRC14:38
jaosoriorayoung: went through the configuration values you added http://paste.openstack.org/show/552456/14:40
*** abehl has quit IRC14:41
ayoungjaosorior, so...I am not a huge fan of having customer puppet modules for the values from Keystone. I would rather keystoneconfig be the source of truth there, and have a consistant way to specify all config values14:42
ayoungbut, I'm not going to make a fuss.  That is the least ugly part of my deploy.  The Horizon stuff needs far more work.14:42
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services  https://review.openstack.org/32343614:43
rcerninshardy: when doing deployment on the node being stuck only os-collect-config runs, it was run from the past, i restarted it manually but nothing changed.14:43
jaosoriorEmilienM: for federation, ayoung used this http://paste.openstack.org/show/552441/ which contains some values for keystone_config. Do you think it's alright to leave them as they are or should we enable the setting of those through puppet-keystone?14:44
EmilienMalways use puppet-keysne14:44
EmilienMkeystone::config::* should be avoided, I told it 1000 times maybe14:44
ayoungEmilienM, So, yeah, I am not sold14:45
ayoungEmilienM, I really want to not get in the business of having to worry about something in a puppet module shocins a different default than the true default14:45
EmilienMif you use keystone::config, one day something will break because someone will add the options in puppet-keystone14:45
ayoungand I don't want lag between changes in keystone config and puppet keeping us from using new options14:45
EmilienMthat's the reason why I don't like it14:45
EmilienMI don't understand the problem of patching puppet-keystone14:46
jaosoriorayoung: are we always gonna enable the federation options?14:46
*** chem has joined #tripleo14:46
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: DO NOT TEST CHECKING MISTRAL SUBMISSION  https://review.openstack.org/35294714:46
ayoungjaosorior, EmilienM anyway, let me get the whole process written down first, and we can do the incrementals to getting this productized14:46
ayoungright now, I am not properly setting vlaues in at least 3 places14:47
jaosoriorayoung: Sure; So first we patch puppet-* to enable the options we need. Then we set them in t-h-t.14:47
ayounghaproxy.cfg, local_settings, and the Apache config files14:47
ayoungI want to be able to do everytihg in Mitaka, so withoug backports etc14:47
ayoungwe can clean it up for Newton and later14:48
jaosoriorayoung: for haproxy.cfg there is already a place to set them https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp14:48
ayoungjaosorior, well, we will see. I suspect that what I need is not in that fi;le, but in the underlying puppet modules14:49
jaosoriorayoung: For keystone this is where you want to change it https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L52014:49
ayoungOk...fo rthat  I neededL14:49
ayoung  bind transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem14:49
ayoung  redirect scheme https code 301 if { hdr(host) -i } !{ ssl_fc }14:49
ayoung  rsprep ^Location:\ http://(.*) Location:\ https://\114:49
ayoungthose 3 values have to be added to the public endpoint.14:50
jaosoriorthe redirect and rsprep can be done without an issue14:50
ayoungWell...I added those values, I do not know if they *need* to be14:50
jaosoriorI'm not sure what you want from the bind though14:50
sshnaidmpabelanger, the rh2 cloud is still in use, right?14:50
ayoungLots a cargo culting going on!14:50
ayoungjaosorior,   I needed Keystone and HA proxy to agree on the port14:50
*** thrash is now known as thrash|biab14:51
ayoung13000 is strange14:51
jaosoriorayoung: Right, that's gonna be a bigger change and I'm working on that fix14:51
ayoungwe already have a 5000 internal without HTTPS14:51
ayoungbut this is external, and I added HTTPS on there,14:51
pabelangersshnaidm: no, it is currently disabled in nodepool14:51
jaosoriorbasically you'll get that once I get the keystone endpoints to be changed by puppet14:51
* ayoung might have broken something, though14:51
pabelangersshnaidm: there is some discussions on bring it back online, but we need to address the networking issues first. EG: ntpd getting blocked14:52
jaosoriorayoung: But anyway, you can already change those ports already, but it takes... several additions to hieradata14:52
jaosoriorwhat I'm working on is having that seamless (basically you would only change the endpoint map value and that's it)14:52
ayoungjaosorior, the one change that worries me is that in /etc/httpd/conf.d/10-keystone_wsgi_main.conf  I made it ServerName https://openstack.ayoung-dell-t1700.test14:53
ccamachojaosorior, I have pushed the same fix than for AODH for the sync_db steps14:54
ccamachomind to check it?14:54
ayoungHowever, the internal port is http, not https.  It needs to be the https value, however, to confirm the vlaue sent by the SAML IdP14:54
jaosoriorccamacho: pass the CR14:54
jaosoriorayoung: so you need internal TLS in the internal endpoints (which is what I'm working on too); or something else?14:55
ccamachojaosorior dude you are super fast :) https://review.openstack.org/#/c/352937/  thanks also gfidente14:55
jaosoriorccamacho haha14:56
*** bana_k has joined #tripleo14:57
pandabswartz: how do you run it ? and what failures did you see ?14:57
*** dtrainor has joined #tripleo14:57
bswartzpanda: http://paste.openstack.org/show/547837/14:58
bswartzpanda: I cloned the git repo, and ran the quickstart.sh script from that14:58
bswartzI tried both as myself and as root14:58
bswartzI pointed it at localhost14:58
ayoungjaosorior, I bet that if you get internal TLS working, it will be enough15:00
ayoungjust make sure you use the cloudname for the endpoints, not internals15:00
jaosoriorayoung: that will be needed for IPA anyway15:00
jaosoriorFreeIPA won't give us a cert if we request it for an IP15:01
jaosoriorayoung: I'm adding what you need for haproxy15:01
jaosoriorI did a very similar change (if not the same) for horizon. So I'm searching for that commit first, to see that I don't miss anything15:01
pandabswartz: can I see you command line that starts it ?15:02
pandathe command line you use to start it15:02
bswartz./quickstart.sh localhost15:02
bswartzthat's run as me (bswartz)15:02
bswartzin the /home/bswartz/tripleo-quickstart directory15:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Handle redirects for keystone  https://review.openstack.org/35295615:03
jaosoriorayoung: ^^15:03
pandabswartz: can you paste the entire log somewhere ? the cache directory should be create early in the workflow. Anyway using localhost as target for deployment is not heavily tested.15:05
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart: Fix InvocationError during ansible-lint run.  https://review.openstack.org/35295715:06
jaosoriorshardy: regarding the composable services work. I think stuff gets pretty complicated with HAProxy, since it needs to know which services are running on each node in order to proxy them. I was thinking... what if we have HAProxy as a different type of service. All the other services could output their endpoint template, port and such, and the haproxy service would get that as an input.15:07
*** bana_k has quit IRC15:07
*** bana_k has joined #tripleo15:08
*** bana_k has quit IRC15:09
bswartzpanda: sure I'll post log -- what do you recommend other than localhost? the readme file explicitly mentions using localhost15:09
*** skramaja has quit IRC15:09
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: [WIP] Used subunit to track metrics  https://review.openstack.org/35296115:10
pabelangerdprince: is your graphite server?15:12
dprincepabelanger: should be, yeah15:12
ayoungjaosorior, ++15:12
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Set Sahara workers to 0  https://review.openstack.org/35272915:13
pandabswartz: readme mentions it as a possible option, but the usual use is targeting a beefy machine for testing15:14
*** aufi has quit IRC15:14
jaosoriorayoung: Do you need anything else in haproxy.cfg?15:14
pabelangerdprince: okay, I wanted to talk to you about moving the metric data to start using generate-subunit over an external graphite server. We have a new tool called stackviz that will render the data from subunit at the end of a job, and visualize it. eg:15:15
*** [1]cdearborn has joined #tripleo15:15
pabelangerdprince: part of the effort to remove private infrastructure dependencies from tripleo-ci15:16
dprincepabelanger: sounds great to me15:16
dprincepabelanger: so long as we have sometime to profile individual chunks (of our choosing)15:16
*** links has joined #tripleo15:16
dprincepabelanger: like I'm really interested in walltimes of various taks15:16
dprincepabelanger: over time.15:16
dprincepabelanger: so like if we add a change that made DIB much slower for example, just for TripleO. It'd be nice to know that via a graph15:17
ayoungjaosorior, I think that was it.  I need to diff what was there before and what I put in.  It was defintely only for the Keystone public, so that should be good.15:17
jaosoriorayoung: I added those options in keystone admin too though. Should I remove them?15:17
ayoungjaosorior, we need to look at the Federation support in puppet modules altogether15:17
ayoungjaosorior, I don't know.  I would like to get Keystone down to a single port anyway15:18
dprincepabelanger: if the tool doesn't support similar ad-hoc walltime stuff then it would be nice to get something that did15:18
ayoungwith V3, there is no difference on wht is on main vs admin15:18
jaosoriorayoung: Lets do this. I'll remove them from keystone admin and if they're needed then I push another commit for that15:18
ayoungjaosorior, ++15:18
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Handle redirects for keystone  https://review.openstack.org/35295615:19
jaosoriorayoung ^^15:19
pabelangerdprince: right, there is actually a spec open for that in qa-specs: http://specs.openstack.org/openstack/qa-specs/specs/devstack/counter-inspection.html15:20
openstackgerritAna Krivokapic proposed openstack/instack-undercloud: Add default plan creation  https://review.openstack.org/34953215:21
pabelangerdprince: I think we could contribute code to that, since it is a common request for openstack projects15:21
ayoungjaosorior, I just realized that my deployment is more broken than I realized.  I only have identity service and endpoint definied15:21
ayoungjaosorior, yeah, the deploye succeeded, no error reported15:22
jaosoriorthat's funky15:22
dprincepabelanger: sure, sounds promising. But if it is just starting it might be a while ?15:22
*** jistr|debug is now known as jistr15:23
pabelangerdprince: I think QA has already started the effort, I'd have to follow up with them and see where they stand.15:23
ayoungjaosorior, let me document what I did, and then we can break down the specific tasks.  I certainly want a second set of eyes on some of the decisions I made15:23
jaosoriorayoung: Sure, I'll be online for a couple of hours more so we can go through it if you want15:25
*** radeks has quit IRC15:25
pabelangerbnemec: EmilienM: any reason not to bring https://review.openstack.org/#/c/346985/ online? all jobs have passed at one point in time15:26
EmilienMslagle: I recently find this error quite often in multinode job http://logs.openstack.org/46/352746/3/check/gate-tripleo-ci-centos-7-nonha-multinode-nv/03242b0/logs/postci.txt.gz#_2016-08-09_14_32_59_00015:26
EmilienMpabelanger: merged15:27
openstackgerritMerged openstack-infra/tripleo-ci: Add conditional check for tripleo-test-cloud-rh1  https://review.openstack.org/34698515:27
pabelangerEmilienM: thanks15:27
slagleEmilienM: i think that just happens in postci if no stack was actually created15:28
slagleEmilienM: so you'd need to check why the job failed before creating a stack15:28
jaosoriorpabelanger: I see this error in one commit that seems to have passed the gate... but was reported as a failure anyway15:30
*** ebarrera has quit IRC15:30
jaosorior2016-08-09 15:24:26.551223 | [Zuul] standard output/error still open after child exited2016-08-09 15:24:26.551562 | [Zuul] Task exit code: 115:30
*** cdearborn has quit IRC15:30
awaynejtomasek: does that mean that openstack-ui cannot be successfully deployed at the moment?15:31
slagleEmilienM: i think i see the issue, 1 sec15:32
*** tremble has quit IRC15:32
EmilienMslagle: http://logs.openstack.org/46/352746/3/check/gate-tripleo-ci-centos-7-nonha-multinode-nv/03242b0/console.html#_2016-08-09_14_32_47_11524315:32
openstackgerritMerged openstack/puppet-tripleo: Use modern auth options for Neutron  https://review.openstack.org/35251815:32
EmilienM/home/jenkins/overcloudrc: No such file or directory15:33
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Use --validation-errros-nonfatal in osinfra jobs  https://review.openstack.org/35297515:34
slagleEmilienM: ^15:34
jtomasekawayne: that's unfortunately right. It It will work but the pieces which use tripleo-api won't work. Removing of that stuff should be done in matter of days though. This is due to the fact that mistral workflows/action that are replacing that functionality are still in progress15:34
slagleEmilienM: ^ that should fix it15:34
slagleEmilienM: this is what broke it https://review.openstack.org/#/c/34960915:34
slagleEmilienM: it never passed multinode ci15:34
EmilienMslagle: ok but does it mean we'll ignore all validation failures?15:35
slagleEmilienM: we already were15:35
awaynejtomasek: ok, in that case I'll wait for the new API before I continue trying to get the UI stood up. thanks for the help.15:35
slagleEmilienM: but, yes.15:35
EmilienMmhh ok15:35
EmilienMslagle: what are those validations? I missed it15:36
jtomasekawayne: thanks! sorry about the trouble15:36
pabelangerjaosorior: mind linking the log file?15:36
slagleEmilienM: they're in tripleoclient if you want to see what they are15:36
slagleEmilienM: they are not new. look at the patch I linked15:36
slagleEmilienM: all that is doing is making them fatal by default15:36
jaosoriorpabelanger: http://logs.openstack.org/13/352713/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/1b11920/console.html#_2016-08-09_15_24_26_55122315:37
EmilienMslagle: yeah, I missed that thing15:37
slagleEmilienM: so we need to have them be non fatal in the multinode job15:37
bnemecEmilienM: If/when we make the multinode setup a supported configuration we need to fix the validations to handle that case somehow.15:37
shardyslagle, EmilienM: In CI validation warnings/errors should already be fatal15:37
*** ohamada_ has quit IRC15:37
*** leanderthal is now known as leanderthal|afk15:37
slagleshardy: not for the multinode job15:37
slagleshardy: we override it15:37
shardyslagle: ah, right15:37
slaglei actually meant to discuss making the multinode jobs voting in the meeting15:39
pabelangerjaosorior: looks like the job hit the timeout, cause the job to fail.  The zuul-launcher tried to write something to the console, which failed.15:39
slaglebut i forgot and then blew off the meeting15:40
slaglei'll mail the list15:40
EmilienMslagle: me too15:40
shardyslagle: Sorry I missed that failure before approving, +1 on fixing then making it voting15:40
EmilienMslagle: I think we should enable voting15:40
slagleshardy: no worries, i meant to do this patch when I reviewed it myself :)15:40
slaglebut forgot15:40
pabelangerjaosorior: Suspect if you address the timeout problem, zuul-launcher will be able to write properly to the console log15:40
*** numan_ has quit IRC15:41
jaosoriorpabelanger: thanks, makes sense15:41
*** numan_ has joined #tripleo15:42
jaosoriorshardy: You had a really useful link with the status of promotion in RDO, which was it?15:44
shardyjaosorior: ^^15:45
jaosoriorit's been a whle15:45
jaosoriorso I guess we still don't have the newest mistral :(15:46
*** amoralej is now known as amoralej|brb15:47
*** numan_ has quit IRC15:47
*** tosky_ has quit IRC15:48
*** pcaruana has quit IRC15:48
*** rcernin has quit IRC15:50
*** tesseract- has quit IRC15:50
*** fragatina has joined #tripleo15:51
bswartzpanda: in my case localhost IS a beefy machine (24 core 64GB RAM)15:51
bswartzpanda: I changed to using the actual hostname and that fixed my problems however15:52
bswartzpanda: thanks for you help -- maybe someone should file a bug about the localhost thing, if it can be reproduced elsewhere15:52
*** cdnchris has joined #tripleo15:54
paramiteshardy, Hey Steve, can I bother you for few minutes? I'm probably missing something obvious here. I'm trying to add another attribute to role_data of ResourceChain on the same level as service_names, but I'm getting following error: https://paste.fedoraproject.org/ . Does this ring a bell for you or something I did in the patch: https://review.openstack.org/#/c/254788/28 ?15:57
*** hewbrocca is now known as hewbrocca-afk16:02
*** dprince has quit IRC16:04
*** dprince has joined #tripleo16:05
*** jbadiapa has quit IRC16:07
shardyparamite: Hey, no paste referenced there, but assuming it's the same issue CI is failing on I think I spotted it, commented on the review16:08
paramiteshardy, thanks!16:08
paramiteshardy, and the paste was https://paste.fedoraproject.org/404923/07579341/ sorry16:09
shardyparamite: Ah, that's slightly different from the error in CI so could be a different problem, but the resource_registry mapping definitely needs to be fixed anyway16:10
paramiteshardy, ah-hah, in my testing I'm using https://review.openstack.org/#/c/254788/28/puppet/services/monitoring/sensu-client.yaml as a resource16:10
paramiteshardy, will fix it, of course, but it probably is not the cause of the problem16:11
EmilienMparamite: can you look https://review.openstack.org/352484 ?16:11
EmilienMparamite: I'm wondering if it will cause issue for sensu to connect to rabbitmq undercloud16:11
*** numan_ has joined #tripleo16:11
paramiteEmilienM, checking16:11
EmilienMor if we changed the design here16:12
shardyparamite: the error you're seeing is a generic error you see when there's something malformed in the role_data section of the *Services templates16:12
shardyparamite: I raised a heat bug to improve it, but for now you'll need to check the output role_data for each service until you spot which one is broken16:12
slagleEmilienM: i believe we said in the monitoring spec that it would not use the underclouds rabbit16:12
slagleEmilienM: we would deploy a new rabbit on the monitoring node16:13
paramiteEmilienM, nope, it's ok we won't use undercloud rabbit or redis16:13
EmilienMslagle: ah ok16:13
EmilienMgood, nevermind then16:13
paramiteshardy, damn ... ok, I browsed them several times already. I think I'm blind :) ... do you have a link to a patch which can I apply to make my debugging faster?16:14
*** jpich has quit IRC16:15
shardyparamite: There is a heat patch, but I've not yet tested/reviewed it16:17
shardyparamite: you could just look up all role_data by brute force:16:17
shardythen tee the result and search for Error16:17
paramiteshardy, I guess it's worth the time for me to test it, thanks!16:17
*** jcoufal has joined #tripleo16:17
shardyparamite: one or more of those output-shows will contain the error pointing to a broken *Services template16:18
*** mcornea has quit IRC16:18
paramiteshardy, ok, I'm checking it. Thanks for help16:18
shardyparamite: https://bugs.launchpad.net/heat/+bug/1599114 is the heat bug FYI16:19
openstackLaunchpad bug 1599114 in heat "Outputs aren't correctly validated" [Medium,In progress] - Assigned to Oleksii Chuprykov (ochuprykov)16:19
paramiteshardy, ack, thx16:19
*** amoralej|brb is now known as amoralej16:20
gfidentejaosorior, was the sahara db sync fix merged?16:21
jaosoriorgfidente: Not yet16:21
openstackgerritMerged openstack/instack-undercloud: Default to Ironic API v1.15 on undercloud  https://review.openstack.org/34832316:22
pandabswartz: mh interesting, could  *you* file a bug in launchpad then ? :) https://bugs.launchpad.net/tripleo-quickstart/+filebug16:25
EmilienMcan someone approve https://review.openstack.org/336538 please?16:33
EmilienMshardy: I have a dumb question, what are becoming the tripleo-puppet-elements defined for controller/compute etc, with composable services?16:34
*** ifarkas is now known as ifarkas_afk16:34
openstackgerritMerged openstack/tripleo-specs: tripleo-ovs-dpdk  https://review.openstack.org/31387116:35
openstackgerritMerged openstack/tripleo-image-elements: Deprecate old RDO release  https://review.openstack.org/33598416:35
*** bvandenh has quit IRC16:36
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: [WIP] Used subunit to track metrics  https://review.openstack.org/35296116:36
shardyEmilienM: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_image.py#L19216:36
EmilienMoh right16:36
EmilienMshardy: are we converging into overcloud-full ?16:36
shardyEmilienM: we include all the elements in overcloud-full16:36
shardyEmilienM: Yeah, I think a common image is kinda mandatory if folks want flexibility with service placement16:37
*** athomas has quit IRC16:37
EmilienMcool, I just wanted to make sure :)16:37
openstackgerritMerged openstack/tripleo-specs: Metal to Tenant: Ironic in Overcloud  https://review.openstack.org/32099516:37
*** milan has quit IRC16:38
*** fragatin_ has joined #tripleo16:38
*** dsariel has quit IRC16:39
openstackgerritMerged openstack/tripleo-incubator: Don't require EPEL  https://review.openstack.org/35061916:40
*** fragatina has quit IRC16:41
openstackgerritMerged openstack/tripleo-specs: Spec: Add Adapter Teaming Support to os-net-config  https://review.openstack.org/33788616:41
openstackgerritMerged openstack/tripleo-puppet-elements: Add puppet-ec2api module  https://review.openstack.org/33653816:41
openstackgerritMerged openstack/tripleo-puppet-elements: Add packages required for dpdk in compute node  https://review.openstack.org/34131316:42
*** dprince has quit IRC16:42
*** dprince has joined #tripleo16:43
openstackgerritMerged openstack/tripleo-docs: Add all puppet modules to includepkgs  https://review.openstack.org/35077616:45
*** paramite is now known as paramite|afk16:46
*** akrivoka has quit IRC16:48
*** bana_k has joined #tripleo16:48
*** yamahata has joined #tripleo16:54
*** dprince has quit IRC16:54
bswartztbarron: ping16:55
*** ebarrera has joined #tripleo16:55
*** dtantsur is now known as dtantsur|afk16:56
*** lucasagomes is now known as lucas-dinner16:58
EmilienMslagle: merging your fix, it passed multinode16:59
slaglei don't object17:00
openstackgerritMerged openstack-infra/tripleo-ci: Use --validation-errros-nonfatal in osinfra jobs  https://review.openstack.org/35297517:00
EmilienMok one last thing we need is https://review.openstack.org/#/c/352746/17:01
EmilienMboth ovb jobs passed17:01
EmilienMslagle: what about enabling voting on multinode job?17:02
EmilienMthe nonha at least for now17:02
*** dsariel has joined #tripleo17:03
*** numan_ has quit IRC17:03
*** jpena is now known as jpena|off17:07
*** jcoufal has quit IRC17:10
*** dsariel has quit IRC17:10
slagleEmilienM: chill. i'm working on it17:10
slaglei'm in the yaml, updating the yaml17:11
EmilienMslagle: it's all yaml17:11
slaglei need some yaml to define this yaml17:11
*** amoralej is now known as amoralej|off17:11
*** mgould is now known as mgould|afk17:11
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add manifests to inject and trust CA certificates  https://review.openstack.org/35302117:12
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: [WIP] Used subunit to track metrics  https://review.openstack.org/35296117:13
*** bana_k has quit IRC17:14
slagleEmilienM: Yo, check my yaml, https://review.openstack.org/#/c/353019/17:14
*** numan_ has joined #tripleo17:15
EmilienMslagle: yo I had to -117:16
*** thrash|biab is now known as thrash17:17
slagleit's b/c i can't yaml17:17
EmilienMthe alternative option is to use a template17:17
EmilienMwhere you define both jobs in check & gate and use the template in check-tripleo-jobs17:17
slagledo you think we need them in the gate?17:18
EmilienMif they are voting, yes17:18
slaglei had planned to leave them as check for now17:18
EmilienMI never saw a CI job voting that is not in gate17:18
EmilienMmaybe I'm wrong, pabelanger can you confirm?17:18
*** fragatin_ has quit IRC17:19
shardyEmilienM: won't that mean we have to run the whole job again every time a patch is approved?17:22
slagleyea that's what i'd like to avoid17:22
pabelangerEmilienM: correct, voting jobs should be in gate too17:22
slaglepabelanger: should be or have to be?17:23
pabelangerhave to be if they are voting17:23
slaglei'll update it17:24
slagleshardy: so i'll update the patch to add them to the gate too. i guess this will be a good litmus test to see if we can live with this as a project17:25
shardyslagle: Ok - I suspect it's going to be unworkable, but lets see I guess17:25
* shardy looks at the worn out r-e-c-h-e-c-k keys on his keyboard17:26
slagleshardy: certainly possible. though we've been saying for 3+ years we want gating jobs. so if it doesnt work, at least we can stop saying that17:27
slaglei'll clarify this on my mail as well17:28
bswartzshardy: if the c h or k keys are broken, you can use the synonym "reverify" ;-)17:29
*** lblanchard has quit IRC17:29
*** weshay has quit IRC17:30
*** lblanchard has joined #tripleo17:30
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Convert ServiceNetMap to a nested template  https://review.openstack.org/35303217:31
*** dprince has joined #tripleo17:32
tbarronbswartz: pong17:32
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add alternative environment to deploy CAs via hieradata  https://review.openstack.org/35303717:33
bnemecWe may be able to get away with it in multinode.17:34
bnemecThere's no baremetal deployment and no ha in that job.17:34
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add deployment of CAs via hieradata  https://review.openstack.org/35303717:34
bnemecThat's the vast majority of our failures in the other jobs.17:34
jaosoriorshardy, EmilienM: Hey guys, I did an alternative approach to deploy the CA certs via hieradata instead of the bash script that does it currently. I haven't tested it yet, but if you guys can take a quick look about the approach, would appreciate it https://review.openstack.org/#/c/353037/17:38
EmilienMjaosorior: ok17:39
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add deployment of CAs via hieradata  https://review.openstack.org/35303717:40
EmilienMslagle: +1 yo17:40
*** ioggstream has quit IRC17:41
*** Vijayendra has quit IRC17:41
*** Vijayendra has joined #tripleo17:42
*** yamahata has quit IRC17:42
EmilienMjaosorior: https://review.openstack.org/#/c/353037/3/puppet/ceph-storage.yaml17:43
EmilienMnot sure we want to keep this file on long term17:43
EmilienMwell, all the hiera files17:44
EmilienMsame for https://review.openstack.org/#/c/353037/3/puppet/manifests/overcloud_cephstorage.pp17:44
EmilienMwe don't want to keep old manifests17:44
EmilienMyou might want to use profiles17:44
EmilienMthink container !17:44
*** pcaruana has joined #tripleo17:45
jaosoriorEmilienM: right. So I thought of that, but my only fear is that the deployer will mess up and not include the CA parts in one of the roles17:46
*** weshay has joined #tripleo17:46
jaosoriorand then we'll get into trouble cause things will break17:46
jaosoriorand it's a "supported path"17:46
jaosorioreven if having the CA installed should be mandatory17:46
EmilienMjaosorior: in tripleo::init?17:47
jaosoriorEmilienM: Does that get run every time?17:47
jaosoriorif that's the case I'll just do it there17:47
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: WIP: Upgrade scripts to migrate aodh alarm data  https://review.openstack.org/35304217:48
jaosoriorEmilienM: Should I move tripleo::packages and tripleo::firewall there then?17:48
EmilienMno it doesn't but we should do it17:49
EmilienMthat's the way to write puppet manifests17:49
EmilienM::tripleo should be a common class to all nodes17:49
EmilienMbut we need to be very careful17:49
jaosoriortomorrow I'll give it a try then17:49
jaosoriorI gotta go17:50
*** yamahata has joined #tripleo17:50
*** pkovar has quit IRC17:50
jaosoriorhave a good one guys!17:50
*** jaosorior has quit IRC17:50
*** liverpooler has joined #tripleo17:53
EmilienMshardy: I agree with your statement, re- voting jobs. See now, check queue is 430 jobs... it means we'll have to wait in case of emergencies patches.17:53
EmilienMbut it's like all OpenStack projects17:54
EmilienMso I guess this is fair to give a try17:54
*** yamahata has quit IRC17:54
bnemecMost of our projects have to wait for the gate queue anyway because they have unit tests of some sort.18:00
*** sarath_ has joined #tripleo18:02
bnemec+A anyone? https://review.openstack.org/#/c/351763/18:03
EmilienMbnemec: right18:05
*** electrofelix has quit IRC18:05
EmilienMbnemec: +A18:05
bnemecEmilienM: Thanks18:05
gfidentebnemec, thanks18:05
slagledarn. you stole my Thanks18:05
bnemecThankses all around :-)18:06
*** gfidente is now known as gfidente|afk18:07
*** yamahata has joined #tripleo18:07
*** liverpooler has quit IRC18:08
*** bana_k has joined #tripleo18:08
openstackgerritMerged openstack/puppet-tripleo: Only db-sync sahara is included in the bootstrapnode  https://review.openstack.org/35274618:12
openstackgerritMerged openstack/tripleo-quickstart: allow RHEL virthosts to use an updated ipxe-roms-qemu  https://review.openstack.org/34801718:13
*** bana_k has quit IRC18:14
*** fragatina has joined #tripleo18:16
*** cdnchris has quit IRC18:17
*** chem has quit IRC18:18
openstackgerritMerged openstack-infra/tripleo-ci: Add 'ceph status' and tripleo-overcloud-passwords to host_info  https://review.openstack.org/34886218:19
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Implement non-ovb overcloud update job - Newton -> Newton  https://review.openstack.org/35133018:20
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton  https://review.openstack.org/34699518:21
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375018:22
openstackgerritMerged openstack/tripleo-heat-templates: Revert "We don't need to set a default for the CephX keys and cluster FSID"  https://review.openstack.org/35176318:24
*** akshai has quit IRC18:26
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: [WIP] Used subunit to track metrics  https://review.openstack.org/35296118:26
*** akshai has joined #tripleo18:27
*** fragatina has quit IRC18:29
*** fragatina has joined #tripleo18:30
*** bana_k has joined #tripleo18:32
EmilienMbnemec: do we still actually need https://review.openstack.org/#/c/348884/ ?18:37
pabelangerEmilienM: shardy: I think the wait will be worth it for voting in gate.  Brings tripleo jobs into the fold with other openstack projects, a good thing18:38
EmilienMpabelanger: I agree18:39
EmilienMlet's see how it works during the next days/weeks18:39
shardypabelanger: I agree, it's just that our runtimes and rate of false negatives are pretty high compared to most projects18:39
shardyhopefully it will work out OK tho18:39
*** myoung|alt has joined #tripleo18:40
*** links has quit IRC18:40
*** david-lyle has quit IRC18:41
pabelangershardy: right, have I some ideas how to fix that, by refactoring some of tripleo-ci. I hope to have a spec in place by barcelona to discuss options18:41
*** rkolli has joined #tripleo18:41
shardypabelanger: ack, sounds good!18:42
*** david-lyle has joined #tripleo18:42
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Print services statuses of nodes in CI  https://review.openstack.org/35291618:42
*** bana_k has quit IRC18:43
rkollishardy: in continuation to our discussion regarding aci_opflex in this morning's meeting. Can you please take a look at http://paste.openstack.org/show/552653/ where I have provided snippets of the out of tree heat templates18:45
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart: [WIP] Prototype tandem documentation generation  https://review.openstack.org/34759218:48
*** yamahata has quit IRC18:48
shardyrkolli: Hi, thanks - that's interesting18:48
shardyrkolli: so, an alternative to adding things to the hierarchy and using *ExtraConfigPre is this:18:49
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: [WIP] Used subunit to track metrics  https://review.openstack.org/35296118:49
shardythere are parameters which will inject any hieradata you need, and you can still use NodeExtraConfigPost to apply your manifests18:49
shardyrkolli: I'm not blocking the patches you have posted, but I wonder if this pattern may be more flexible for you, as it won't require any upstream patches at all?18:50
*** rcernin has joined #tripleo18:51
rkollishardy: I am a novice to tripleO, let me go thru the link you sent and try it out18:51
*** florianf has quit IRC18:53
shardyrkolli: Ok, thanks - I'll comment on the review and we can discuss further if needed18:53
* shardy dropping off for dinner now18:53
rkollishardy: I understand that this will write the hiera data, but the patch i put to controller.yaml file was so that puppet run loads the hiera file18:54
shardyrkolli: the patch to controller.yaml only writes the hieradata18:54
shardyyou can still run puppet via your NodeExtraConfigPost changes18:54
rkollishardy: thanks, i will try it out18:55
shardybasically this just replaces the ControllerExtraConfigPre part18:55
shardyyou can use NovaComputeExtraConfig to deploy the same data on the compute nodes18:55
*** myoung has quit IRC18:55
*** myoung|alt is now known as myoung18:55
beagleshas anyone tried building overcloud images for mitaka from a a mitaka based undercloud lately?18:57
shardyrkolli: there is also a generic ExtraConfig parameter which deploys the hieradata to all roles18:57
rkollishardy: thanks for your input, this is my first time with tripleO, I need to read up further. Appreciate your help18:57
shardyrkolli: No problem, if you try this approach and it doesn't work for you, we can still land the hierarchy addition18:58
openstackgerritDan Sneddon proposed openstack/os-net-config: Fix Linux bond/slave ifup ordering  https://review.openstack.org/35307218:58
rkollishardy: thanks, dont want hold up your dinner, we can chat later18:58
shardyack, thanks!18:58
*** shardy is now known as shardy_afk18:58
*** social has joined #tripleo19:07
*** akshai has quit IRC19:08
dsneddon^^^ Fix for serious os-net-config bug: https://bugs.launchpad.net/os-net-config/+bug/161147119:08
openstackLaunchpad bug 1611471 in os-net-config "os-net-config 0.2.4 fails to enable Linux bonds" [High,In progress] - Assigned to Dan Sneddon (dsneddon)19:08
*** akshai has joined #tripleo19:09
*** myoung has quit IRC19:09
*** myoung has joined #tripleo19:10
*** akshai has quit IRC19:14
*** paramite|afk is now known as paramite19:22
*** abehl has joined #tripleo19:27
*** paramite has quit IRC19:28
*** rkolli has quit IRC19:29
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: [WIP] Used subunit to track metrics  https://review.openstack.org/35296119:35
*** radeks has joined #tripleo19:37
*** abehl has quit IRC19:38
EmilienMayoung: what are plans to integrate fernets into OOO?19:38
EmilienMayoung: is uuid removed in Ocata?19:39
*** david-lyle_ has joined #tripleo19:41
*** numan_ has quit IRC19:41
sshnaidmpabelanger, do you know if there is a way to set more memory to nodes, now it has 4.7 GB per node and no swap, I meet more and more jobs killed by OOM19:42
pabelangersshnaidm: which node?19:42
*** david-lyle has quit IRC19:43
sshnaidmpabelanger, controller nodes19:44
larsksCan someone here answer some composable service design questions?  Particular w/r/t where I should put "required" overrides to puppet module parameters...19:44
sshnaidmpabelanger, and undercloud too19:46
*** penick has joined #tripleo19:46
pabelangersshnaidm: in general, the clouds openstack-infra jobs on have 8GB RAM configured.19:46
larsksFor setting up a 'fluentd' service, I need to override several of the parameter defaults.  I can put these into the config_settings of the tempalte...or I could put them in the puppet-tripleo module.19:46
pabelangersshnaidm: which cloud is your job running on?19:46
sshnaidmpabelanger, and how do I know now?19:46
*** jeckersb is now known as jeckersb_gone19:46
pabelangersshnaidm: also, if you are missing a swap, you have the ability to enable it19:47
pabelangersshnaidm: look for Building remotely on ...19:47
pabelangerin the console.log19:47
sshnaidmpabelanger, no, there is no swap at all on node19:47
sshnaidmpabelanger, tripleo-centos-7-tripleo-test-cloud-rh1-331670219:48
sshnaidmpabelanger, as I see in "free -h" it's 6 GB for undercloud and 5GB for controller19:48
sshnaidmpabelanger, controller-0 and undercloud are close to out of memory19:49
sshnaidme.g. undercloud: http://logs.openstack.org/98/352598/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/c3bf5b4/logs/undercloud/var/log/host_info.txt.gz19:49
pabelangersshnaidm: | 18e29a9b-8b32-4b61-bc76-6346dc7468f2 | undercloud | 6144 |   40 |         0 |     2 | True      |19:50
pabelangerthat is the flavor nodepool is using to launch VMs19:50
sshnaidmpabelanger, what is it?19:50
pabelangerI am not sure why it is 6144 RAM19:50
sshnaidmpabelanger, ok, 6GB, as I said19:50
pabelangerIt should be 8GB to be inline with other clouds19:51
pabelangeras for swap, I think devstack-gate has a function to enable it19:51
sshnaidmyeah, 8 would be very very nicer..19:51
*** coolsvap has quit IRC19:51
pabelangeryou'll need to ask the cloud admins, I am not sure why they chose that amount19:52
sshnaidmpabelanger, ok, and who are them? :)19:52
pabelangerslagle: ^19:52
sshnaidmslagle, hi19:52
pabelangersshnaidm: http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n324 will setup the swap too19:53
slaglewe could add more memory if there is capacity19:53
slaglethese instances aren't spawned by nodepool19:53
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Update hieradata parameters to match recent changes  https://review.openstack.org/35309019:54
slagleso that's why it doesnt match those instances that are19:54
slaglethat's the whole point of why we have ovb19:54
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: [WIP] Used subunit to track metrics  https://review.openstack.org/35296119:55
pabelangerslagle: sshnaidm: right, I was just looking at the undercloud node19:56
slaglepabelanger: sshnaidm : it's as simple as updating the baremetal flavor in the overcloud19:56
slaglewe just need to do some analysis of available capacity before we do that19:57
sshnaidmslagle, how can we know if we have a capacity?19:57
sshnaidmslagle, can you give me please an access to this cloud? at least ro19:57
pabelangersshnaidm: http://grafana.openstack.org/dashboard/db/nodepool-tripleo-test-cloud19:58
slaglesshnaidm: would have to check available memory when we are running all of our jobs19:58
pabelangeris the current capacity from nodepool19:58
slaglesshnaidm: i thought you had access already?19:58
sshnaidmslagle, nope19:59
sshnaidmpabelanger, do I understand right from this board that we are always in full nodes (50) mode?20:00
pabelangersshnaidm: not always, the last 6 hours20:00
pabelangersshnaidm: you can change the time range on the top right20:01
sshnaidmpabelanger, oh, right, sorry)20:01
pabelangersshnaidm: but 50 is currently the max nodes we launch in tripleo-test-cloud-rh120:01
slaglesshnaidm: we probably need to update this process, but can you propose a patch adding yourself here: https://github.com/openstack/tripleo-incubator/tree/master/tripleo-cloud20:01
pabelangerI don't have access to the cloud capacity20:01
slaglesshnaidm: to tripleo-cd-admins and include your ssh key under ssh-keys20:02
sshnaidmslagle, great, will do20:02
slaglethat's how we've been managing access for now20:02
slaglefrom what I can tell, we have a full queue right now with around 40 jobs in progress20:07
slagleand are consuming 1.5TB of 4TB of available memory20:07
slagleso I'd say we have some room20:07
openstackgerritAde Lee proposed openstack/puppet-tripleo: Add barbican profile - WIP  https://review.openstack.org/35244720:10
openstackgerritAde Lee proposed openstack/tripleo-heat-templates: Add Barbican to the overcloud - WIP  https://review.openstack.org/35245820:10
*** julim has quit IRC20:11
*** david-lyle_ is now known as david-lyle20:12
bnemecYeah, we have a ton of memory capacity.20:14
bnemecWe were getting uncomfortably high on CPU usage on the rh1 controller, but I made a change in OVB so it won't hit the host cloud apis quite as hard now.20:14
bnemecI was actually going to propose that we bump the number of testenvs since we're set up for at least 80 right now.20:15
slagleyea, there is plenty of ram :)20:15
bnemecWe have compute nodes that look completely unused, in fact.20:16
bnemecWhich seems like a waste. :-)20:16
slaglemy only concern is we keep bumping the resources and having longer job run times and I'd like to know why20:17
slaglei'd like to remove all of the optional services from the controller20:17
slaglein the main HA job20:17
slaglebnemec: we got a whole cloud sitting unused :)20:19
slaglewe should turn rh2 back on20:20
bnemecWell, that too.20:20
bnemecPart of the problem is that we're building images in every job right now.20:20
bnemecAnd for some reason it's taking an absurdly long time.20:20
bnemecLike 40+ minutes.20:20
bnemecI can build both images locally on basic hardware in ~16 minutes.20:21
*** radeks has quit IRC20:27
*** rhallisey has quit IRC20:27
*** lblanchard has quit IRC20:28
*** jprovazn has quit IRC20:30
*** ebarrera has quit IRC20:31
ayoungEmilienM, good question.  Need to come up with a story for Key rotation20:36
ayoungI think uuid will still be there Ocata and beyond.  There is some demand for UUID still in the community20:36
*** dprince has quit IRC20:37
*** ioggstream has joined #tripleo20:43
*** ioggstream has quit IRC20:49
openstackgerritSagi Shnaidman proposed openstack/tripleo-incubator: Add sshnaidm to tripleo-cloud  https://review.openstack.org/35310320:49
sshnaidmbnemec, I think we used promoted images before, at least in part of jobs. did we reutrn to building intentionally?20:49
bnemecsshnaidm: Yes, it was intentional.  There was a change in how we install puppet modules that required us to stop using cached images for a while.20:50
bnemecWe may be able to turn it back on now.  I'm not sure though.20:50
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE TESTING  https://review.openstack.org/31643620:53
pabelangerbnemec: does that mean we are not using the private mirror for puppet modules?20:54
shardy_afkwe could use DeployArtifactURLs to update the modules in a cached image, either from source or by passing in a list of RPMs from delorean-current plus delorean-ci20:55
bnemecpabelanger: I think that's correct, yes.  We're installing them from package now, so not using the git repos on the mirror server.20:55
*** shardy_afk is now known as shardy20:55
bnemecI thought we had a way to inject packages into the CI image?20:55
pabelangerbnemec: if so, I'd prefer not to switch back to it. And remove those bits from tripleo-ci and use openstack-infra if we need something20:55
bnemecWe had to do that for stuff like os-net-config too.20:56
pabelangerbnemec: looking for the commit now20:56
bnemecpabelanger: That's fine.  I don't think anyone is proposing we start pulling puppet from git again.20:56
*** radeks has joined #tripleo20:57
pabelangerbnemec: great20:57
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart: [WIP] Prototype tandem documentation generation  https://review.openstack.org/34759221:01
*** dergrunepunkt has joined #tripleo21:01
dergrunepunkthi guys, to deploy with tripleO is it absolutelly neccesary to have IPMI?21:02
sshnaidmif we install puppet modules from rpms, it could be just "yum update puppet-* --disablerepo=* enablerepo=delorean-current" in undercloud?21:05
openstackgerritPaul Belanger proposed openstack-infra/tripleo-ci: Remove git repo mirror scripts  https://review.openstack.org/35311221:07
pabelangerbnemec: ^ once that is landed, we can manually delete the apache folder21:07
bnemecsshnaidm: It's not the undercloud that is a problem.  It's the overcloud images.  If they're pre-built, we either need to crack them open and update the puppet packages (which is how we handle other packages), or we need to use the artifact deploy to update the puppet modules.21:10
sshnaidmbnemec, I see.. using virt-customize, etc?21:11
bnemecsshnaidm: Yeah, although we're actually just mounting the image: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/common_functions.sh#n7321:13
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart: Expand templated bash to allow for doc generation  https://review.openstack.org/34759221:15
*** thrash is now known as thrash|g0ne21:15
sshnaidmbnemec, oh, so it's almost ready, just one more line there21:15
*** bana_k has joined #tripleo21:15
sshnaidmmaybe it would be cool to have a task, that is triggered by every new build of puppet module and rebuild our overcloud images, and then uploads them to server where each job can take it21:16
bnemecsshnaidm: Actually, I think it might be fine already.  Since the puppet modules are just packages now, they'll be updated just like all the other packages were.21:17
bnemecThe main thing I don't know for sure is when we last updated our cached images.21:17
bnemecIf it's since we switched to packaged puppet we can probably turn back on caching and it will Just Work(tm)21:18
sshnaidmbnemec, worth a try..21:18
*** ayoung has quit IRC21:19
sshnaidmas I heard we have a pretty old undercloud, but overcloud is new.. maybe pabelanger knows better21:19
bnemecsshnaidm: We aren't uploading undercloud anymore because we don't build a separate undercloud image now.21:19
bnemecIt just runs on the Jenkins slave.21:19
*** radeks has quit IRC21:20
sshnaidmbnemec, uh, so that's why it's old :)21:20
sshnaidmso we just run yum update every job, right?21:21
bnemecBasically.  The base image gets updated periodically by infra, so it should always be pretty up to date in the first place.21:22
bnemecThe main place it hurts us timewise is installing the OpenStack packages.21:22
bnemecLooks like we're hovering a little under 25 minutes for the undercloud install right now.21:22
bnemecWhich is in line with what I see locally.21:22
*** dergrunepunkt has quit IRC21:23
pabelangernice images get build everyday and uploaded21:23
pabelangerwhich is done via nodepool21:23
slaglebnemec: i think that's because through the power of puppet we install 1 package at a time21:25
bnemecslagle: Yeah, it's pretty terrible.  I wonder if it would be worth pre-installing stuff we know we need in instack-undercloud.21:26
* bnemec is off to try21:26
*** bana_k has quit IRC21:28
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Convert ServiceNetMap to a nested template  https://review.openstack.org/35303221:28
*** bana_k has joined #tripleo21:31
*** zigo has quit IRC21:32
*** zigo has joined #tripleo21:35
*** yamahata has joined #tripleo21:36
*** weshay has quit IRC21:38
*** shardy has quit IRC21:38
*** julim has joined #tripleo21:40
*** bank_ has joined #tripleo21:43
*** bank_ has quit IRC21:43
openstackgerritRyan Brady proposed openstack/tripleo-common: Add Plan Deletion  https://review.openstack.org/29993721:44
*** bana_k has quit IRC21:46
*** morazi has quit IRC21:53
openstackgerritBen Nemec proposed openstack/instack-undercloud: Pre-install packages before running puppet  https://review.openstack.org/35312121:55
*** Goneri has joined #tripleo21:56
*** rbrady has quit IRC21:57
*** rbrady has joined #tripleo21:58
*** rbrady has quit IRC22:01
*** rcernin has quit IRC22:02
openstackgerritStephanie Miller proposed openstack/diskimage-builder: Explain the difference between 'DIB_OFFLINE' and 'DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE'  https://review.openstack.org/34593522:12
*** yamahata has quit IRC22:23
*** bfournie has quit IRC22:25
*** davidlenwell has quit IRC22:25
*** fragatin_ has joined #tripleo22:33
*** rkolli has joined #tripleo22:34
*** rkolli has quit IRC22:35
*** fragatina has quit IRC22:36
*** bana_k has joined #tripleo22:36
*** davidlenwell has joined #tripleo22:38
*** ChanServ sets mode: +v davidlenwell22:38
*** Goneri has quit IRC22:53
*** TSCHAK has quit IRC22:57
*** TSCHAK has joined #tripleo22:58
*** Ryjedo has quit IRC22:59
*** bana_k has quit IRC23:05
*** rlandy has quit IRC23:16
*** david-lyle has quit IRC23:20
*** sarath_ has quit IRC23:32
*** ayoung has joined #tripleo23:35
*** david-lyle has joined #tripleo23:39

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!