Tuesday, 2016-10-11

*** jdennis has quit IRC00:00
*** jdennis has joined #tripleo00:00
*** tiswanso has joined #tripleo00:04
*** tiswanso has quit IRC00:09
openstackgerritSteve Baker proposed openstack/os-collect-config: Treat ec2 collector data as immutable  https://review.openstack.org/38474300:10
openstackgerritSteve Baker proposed openstack/os-collect-config: Treat ec2 collector data as immutable  https://review.openstack.org/38471200:11
*** bana_k has quit IRC00:16
*** dsneddon_ has joined #tripleo00:26
*** dsneddon_ has quit IRC00:31
*** pmannidi is now known as pmannidi|brb00:33
*** apetrich has quit IRC00:48
*** apetrich has joined #tripleo00:49
*** bana_k has joined #tripleo00:49
*** limao has joined #tripleo00:53
openstackgerritDan Prince proposed openstack/tripleo-common: WIP: test the new hiera-heat-hook  https://review.openstack.org/38475100:56
openstackgerritDan Prince proposed openstack/tripleo-common: WIP: test the new hiera-heat-hook  https://review.openstack.org/38475100:59
openstackgerritgecong proposed openstack/instack-undercloud: Replace 'MagicMock' with 'Mock'  https://review.openstack.org/38475201:04
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Split out hosts config deployment  https://review.openstack.org/38475601:17
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Hiera optimization: use a new hiera hook  https://review.openstack.org/38475701:17
*** dmacpher-afk has quit IRC01:22
*** bana_k has quit IRC01:26
*** dmacpher has joined #tripleo02:05
*** mbozhenko has joined #tripleo02:35
*** mbozhenko has quit IRC02:39
*** rlandy|afk is now known as rlandy02:49
*** rlandy has quit IRC02:49
-openstackstatus- NOTICE: Jobs running on osic nodes are failing due to network issues with the mirror. We are temporarily disabling the cloud.02:49
*** xuao has joined #tripleo02:53
*** yamahata has quit IRC02:55
*** pradk has quit IRC02:59
*** apetrich has quit IRC03:06
*** apetrich has joined #tripleo03:10
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore  https://review.openstack.org/36544903:30
*** davidlenwell has quit IRC03:34
*** davidlenwell has joined #tripleo03:43
*** ChanServ sets mode: +v davidlenwell03:43
*** links has joined #tripleo04:01
*** tiswanso has joined #tripleo04:06
*** tiswanso has quit IRC04:11
*** bfournie has quit IRC04:15
openstackgerritSteve Baker proposed openstack/os-collect-config: WIP ec2 collector attempt config-drive before metadata server  https://review.openstack.org/36973104:32
*** radeks has quit IRC04:34
*** mbozhenko has joined #tripleo04:35
*** davidlenwell has quit IRC04:35
*** mbozhenko has quit IRC04:40
*** bfournie has joined #tripleo05:02
*** ayoung has quit IRC05:18
*** ayoung has joined #tripleo05:20
*** ayoung has quit IRC05:22
*** kjw3 has joined #tripleo05:28
*** cwolferh has quit IRC05:36
*** adam_g` has quit IRC05:38
*** d0ugal has quit IRC05:40
openstackgerritafazekas proposed openstack/python-tripleoclient: Remove stack_owner_role from the deployer input  https://review.openstack.org/38482005:40
*** d0ugal has joined #tripleo05:42
*** hjensas_ has quit IRC05:46
*** mbozhenko has joined #tripleo06:02
*** rcernin has joined #tripleo06:07
*** radeks has joined #tripleo06:12
*** adam_g has joined #tripleo06:18
*** adam_g has quit IRC06:18
*** adam_g has joined #tripleo06:18
*** jprovazn has joined #tripleo06:19
*** rasca has joined #tripleo06:30
*** mbozhenko has quit IRC06:35
*** pcaruana has joined #tripleo06:36
*** cwolferh has joined #tripleo06:40
*** dsariel has quit IRC06:49
*** davidlenwell has joined #tripleo06:51
*** ChanServ sets mode: +v davidlenwell06:51
*** sshnaidm|away has quit IRC06:54
*** mcornea has joined #tripleo06:54
*** davidlenwell has quit IRC06:56
*** hjensas has joined #tripleo06:58
*** ebalduf has quit IRC07:02
*** jaosorior has joined #tripleo07:03
*** dciabrin has quit IRC07:05
openstackgerritafazekas proposed openstack/python-tripleoclient: Remove heat_stack_user from the deployer input  https://review.openstack.org/38482007:05
openstackgerritChristian Schwede proposed openstack/puppet-tripleo: Add versioned_writes to Swift proxy config  https://review.openstack.org/38470407:06
*** cylopez has joined #tripleo07:06
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates: Enable object versioning in Swift proxy  https://review.openstack.org/38461607:07
*** dciabrin has joined #tripleo07:08
*** pgadiya has joined #tripleo07:09
*** pgadiya has quit IRC07:09
*** ccamacho has joined #tripleo07:10
*** tremble has joined #tripleo07:12
*** tremble has joined #tripleo07:12
*** aufi has joined #tripleo07:15
*** davidlenwell has joined #tripleo07:16
*** ChanServ sets mode: +v davidlenwell07:16
*** zoli_gone-proxy is now known as zoliXXL07:17
*** dbecker has joined #tripleo07:18
*** hewbrocca-afk is now known as hewbrocca07:23
*** flaper87 has quit IRC07:24
ccamachobandini morning!! related to "[Bug 1622360] [NEW] Deployment of overcloud fails at step4"07:24
openstackbug 1622360 in tripleo "Deployment of overcloud fails at step4" [Undecided,New] https://launchpad.net/bugs/162236007:24
ccamachoJust thinking, what about doing the same upstream and creating instead of a "puppet-pacemaker.yaml" environment file create a file like "puppet-nonpacemaker", by default the deployments will use but if you dont want to use it.. just -e environments/puppet-nonpacemaker.yaml ??07:24
ccamachoI mean to be consistent upstream and downstream07:24
openstackgerritafazekas proposed openstack/python-tripleoclient: Remove heat_stack_user from the deployer input  https://review.openstack.org/38482007:26
*** rawanh has joined #tripleo07:26
*** abregman has joined #tripleo07:26
bandiniccamacho: absolutely agreed. Actually I would love to kill the non HA stuff completely. With Newton the use for nonHA is minimal (https://review.openstack.org/359060)07:27
bandinimorning *07:27
rawanhHi, is there a way to update the overcloud kernel parameters before deploying the overcloud from the undercloud node ?07:28
*** abregman has quit IRC07:28
ccamachoI asked like 1 month before and some folks on IRC told me that there are still users that needs non ha deployments07:29
*** ebarrera has joined #tripleo07:29
ccamachoso thats why both downstream and upstream behavior is not the same07:29
bandiniis it me or CI is broken due to missing eth1 http://logs.openstack.org/67/384667/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f70000c/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-10-11_06_52_15_00007:29
bandiniccamacho: I think with newton the need has become close to 0. What would be the use case?07:29
*** zoliXXL is now known as zoli_gone-proxy07:29
ccamachommmm not remember correctly but I think was related to deployments with network isolation07:30
*** dmacpher has quit IRC07:31
bandiniccamacho: netiso works correctly with HA, so not sure that is a valid point ;)07:31
bandinimaybe there are, I can't think of any atm07:31
jaosoriorbandini: why don't you try switching the default to be the pacemaker case (and the CI for that matter)07:33
ccamachobandini agreed on that lets try to default to  HA  and if you dont need it, then use the non-ha env. file.. for the sake of simplicity07:33
*** jlinkes has joined #tripleo07:35
bandinijaosorior: I have a review for the default (need to update it). Am not sure where to poke to switch the CI to it07:36
bandiniI'd need a few hints in the right direction to do that07:37
bandinijaosorior: btw. ever seen this one http://logs.openstack.org/67/384667/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f70000c/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-10-11_06_52_15_000 ?07:37
*** zoli_gone-proxy is now known as zoliXXL07:38
jaosoriorbandini: haven't seen it, but currently my overcloud deployment is broken due to os-net-config too07:39
jaosoriorso it might be that some recent commit there broke us07:39
*** amoralej|off is now known as amoralej07:39
*** jpena|off is now known as jpena07:39
bandiniI see, will fetch some moarrrr coffee and then poke at it07:39
*** davidlenwell has quit IRC07:44
*** b00tcat has joined #tripleo07:45
*** davidlenwell has joined #tripleo07:46
*** ChanServ sets mode: +v davidlenwell07:46
*** hogepodge has quit IRC07:51
*** egafford has joined #tripleo07:52
*** panda|Zz is now known as panda07:57
*** hogepodge has joined #tripleo07:58
*** dsneddon has quit IRC07:58
jaosoriorpanda: hey dude, how's it going?07:59
jaosoriorpanda: noticed that the nonha job passed from your patch. at least the master version.07:59
*** cwolferh has quit IRC07:59
jaosoriorpanda: it seems that liberty and mitaka failed because of package issues :/08:00
ccamachobandini jaosorior, quick question, is there any guideline for creating python tests for our projects? I mean, there are cases in tripleo-common in which you have one test per method, and you have cases also there with a lot of tests without following any naming convention...08:00
pandajaosorior: yes, I have a patch for that too08:00
jaosoriorpanda: the next step, I think, is to figure out how to separate this SSL stuff from the nonha multinode. Since we can't run netiso on that.08:00
jaosoriorpanda: nice! did you get that working already?08:01
jaosoriorccamacho: no guidelines unfortunately08:01
pandajaosorior: I still don't understand how my certificate where wrong, has is something to do with country/CN informations08:01
pandajaosorior: liberty, yes, mitaka has another error08:01
*** tzumainn has quit IRC08:01
ccamachoI see :P08:02
jaosoriorpanda: so, I believe you created the certificate the right way. So that was alright. But for some reason there were two self-signed certificates. Did you make several attempts at creating the certs? It might be that you just copied the wrong one in one of the fields.08:02
pandajaosorior: first attempt, yes. THe second attempt I started from an empty dir08:03
jaosoriorccamacho: if we could get some guidelines (such as trying to follow behaviour driven tests or something like that) it would be cool, but it would require a bunch of refactoring for the tests.08:03
jaosoriorpanda: maybe you missed updating one of the files? not sure dude. If you have a local deployment you could attempt setting up SSL there.08:05
pandajaosorior: to separate multinode I need to refactor the entire case block, *OR* change the name of the multinode job08:06
ccamachojaosorior, yeah that's what im seeing, Im starting to write some tests and I wanted to follow a pattern, but not able to find it.. would be cool to start following a pattern and then step by step do some refactoring there..08:06
jaosoriorpanda: so which do you think is the best idea? refactoring or changing the name?08:08
openstackgerritMartin André proposed openstack/tripleo-common: Create new docker command hook.  https://review.openstack.org/31272308:09
openstackgerritMartin André proposed openstack/tripleo-common: Install configuration files for all downloaded packages  https://review.openstack.org/34741208:09
openstackgerritMartin André proposed openstack/tripleo-common: Clean up configure_containers.sh script  https://review.openstack.org/38486508:09
openstackgerritMartin André proposed openstack/tripleo-common: Allow building heat-agents image from master  https://review.openstack.org/38486608:09
pandajaosorior: changing the name is certainly the quickest, we'll just have to add nonha configuration to the multinode case and it's done08:09
*** ohamada has joined #tripleo08:09
pandajaosorior: that is until we have a multinode-HA job08:10
jaosoriorpanda: any idea when we would be able to do something like that?08:10
jaosoriorI guess right now the limitation is zuul, right?08:10
pandajaosorior: yes08:10
*** cwolferh has joined #tripleo08:11
*** dsneddon has joined #tripleo08:11
pandajaosorior: the other thing is the objection from bnemec, he doesn't want to lose coverage on nonha without netiso08:13
jaosoriorpanda: hey, so you mentioned you had a patch to fix the package issues in the liberty and mitaka gates. How's that one going?08:13
pandajaosorior: https://review.openstack.org/38462808:13
jaosoriorpanda: and we wouldn't if you change the name of the multinode-nonha. That is still without net-iso. And we could still do SSL there.08:14
*** davidlenwell has quit IRC08:14
*** athomas has joined #tripleo08:15
*** davidlenwell has joined #tripleo08:18
*** ChanServ sets mode: +v davidlenwell08:18
openstackgerritafazekas proposed openstack/python-tripleoclient: Disable encrypted volume in the tempest-deployer-input.conf  https://review.openstack.org/38486808:24
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo: Ensure presence of pacemaker restart directory.  https://review.openstack.org/38487008:27
*** dsneddon has quit IRC08:28
*** dsneddon has joined #tripleo08:28
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo: Ensure presence of pacemaker restart directory.  https://review.openstack.org/38487008:29
*** abregman has joined #tripleo08:33
*** limao_ has joined #tripleo08:33
*** derekh has joined #tripleo08:34
*** Slower has joined #tripleo08:36
*** soc_off has joined #tripleo08:36
*** another_larsks has joined #tripleo08:39
*** amoralej_ has joined #tripleo08:40
*** jraju has joined #tripleo08:40
*** dsneddon has quit IRC08:41
*** davidlenwell has quit IRC08:41
*** jlinkes has quit IRC08:41
*** dbecker has quit IRC08:41
*** adam_g has quit IRC08:41
*** links has quit IRC08:41
*** limao has quit IRC08:41
*** rbrady-afk has quit IRC08:41
*** amoralej has quit IRC08:41
*** soc_off_ has quit IRC08:41
*** Slower_ has quit IRC08:41
*** rhefner has quit IRC08:41
*** mwhahaha has quit IRC08:41
*** hrybacki has quit IRC08:41
*** gregwork has quit IRC08:41
*** fungi has quit IRC08:41
*** mgagne has quit IRC08:41
*** honza_ has quit IRC08:41
*** larsks has quit IRC08:41
*** onovy has quit IRC08:41
*** amoralej_ is now known as amoralej08:41
*** dbecker has joined #tripleo08:44
*** dsneddon has joined #tripleo08:49
*** davidlenwell has joined #tripleo08:49
*** jlinkes has joined #tripleo08:49
*** adam_g has joined #tripleo08:49
*** rbrady-afk has joined #tripleo08:49
*** mwhahaha has joined #tripleo08:49
*** hrybacki has joined #tripleo08:49
*** gregwork has joined #tripleo08:49
*** fungi has joined #tripleo08:49
*** mgagne has joined #tripleo08:49
*** honza_ has joined #tripleo08:49
*** onovy has joined #tripleo08:49
*** tepper.freenode.net sets mode: +v davidlenwell08:49
*** gregwork has quit IRC08:50
*** gregwork has joined #tripleo08:52
*** gfidente has joined #tripleo08:52
*** mwhahaha has quit IRC08:52
*** mwhahaha has joined #tripleo08:53
*** r-mibu has quit IRC09:00
pandaCI is on fire. derekh: do you remember if our test always needed two nics to be present on the undercloud ?09:05
*** akrivoka has joined #tripleo09:05
*** adam_g has quit IRC09:06
*** adam_g has joined #tripleo09:08
*** adam_g has quit IRC09:08
*** adam_g has joined #tripleo09:08
*** r-mibu has joined #tripleo09:15
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489209:16
*** hjensas has quit IRC09:20
derekhpanda: yes it does, here is the initial problem http://logs.openstack.org/65/384865/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/1e45fa9/console.html#_2016-10-11_08_15_44_64878509:21
derekhpanda: ovb testenv creation is failing, (that also adds the second nic)09:21
derekhpanda: looking into it now09:24
*** limao_ has quit IRC09:24
*** davidlenwell has quit IRC09:24
pandaderekh: just to understand, that is the error reported from gearman, because something on our ovb is failing to add the second nic ?09:27
pandaderekh: maybe we should stop the build if testenv get an error in env creation ?09:27
*** ubijtsa has joined #tripleo09:27
*** ubijtsa has quit IRC09:28
*** ubijtsa has joined #tripleo09:28
*** ubijtsa is now known as assassin09:28
derekhpanda: we did stop on fail at one stage, something must have changed09:28
*** paramite has joined #tripleo09:28
*** zoliXXL is now known as zoli|wfh09:28
*** openstack has joined #tripleo09:29
*** onovy has quit IRC09:30
pandaderekh: we are stopping testenv when the error message is "Couldn't retrieve env". Adding this error too to the error condition09:33
*** onovy has joined #tripleo09:38
derekhpanda: I've restarted rabbit, it was using a lot of RAM, lets see if that helps09:38
*** jaosorior has quit IRC09:39
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add "Failed creating OVB stack" to failure conditions in testenv-client  https://review.openstack.org/38490609:39
*** jaosorior has joined #tripleo09:39
pandaderekh: noo, 1 minute and I could test if this last change was working ... :)09:41
pandaderekh: ok, let's see what happens09:41
*** jpich has joined #tripleo09:41
derekhpanda: brb09:44
pandaderekh: 2016-10-11 09:46:29.271225 | 2016-10-11 09:46:29,270 - testenv-client - INFO - Received job : Failed creating OVB stack09:46
*** shardy has joined #tripleo09:47
pandaand my change didn't stop it ..09:47
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add "Failed creating OVB stack" to failure conditions in testenv-client  https://review.openstack.org/38490609:50
*** tosky has joined #tripleo09:50
*** zoli|wfh is now known as zoli|lunch09:55
pandaderekh: OVB continues to fail, but my change is stopping the jobs now.09:57
panda2016-10-11 09:56:52.225675 | 2016-10-11 09:56:52,225 - testenv-client - INFO - Received job : Failed creating OVB stack09:57
panda2016-10-11 09:56:52.225761 | 2016-10-11 09:56:52,225 - testenv-client - ERROR - Failed creating OVB stack09:57
*** chem has quit IRC10:00
pandaso we get a quick feedback when we get that error again10:00
pandaI noticed that testenv returns the error very quickly.10:01
shardyd0ugal: Hey, question - are there plans for an openstack overcloud delete command?10:03
shardyI thought I saw a patch at one point but may have imagined it10:03
shardyit'd be nice to have a clean way to remove the stack, mistral env and swift container10:03
*** pblaho has joined #tripleo10:04
d0ugalshardy: there is a plan delete, it wont allow you to do it if there is a stack - but maybe we could add a flag?10:04
d0ugalshardy: There are no plans that I am aware of, but I am not doing much CLI "design" work now :)10:04
shardyd0ugal: Yeah I mean openstack overcloud delete <overcloud name>10:04
d0ugaljpich: Has this came up with your lot? ^10:04
shardywhich would be equivalent to openstack stack delete overcloud && openstack overcloud plan delete overcloud10:05
shardyit's not a huge deal, but would be marginally more convenient in some cases10:05
shardybash alias provides a fine workaround tho :)10:05
d0ugalshardy: right, I follow. I just wonder if it will be confusing having a overcloud delete and a plan delete.10:05
jpichd0ugal: Do you mean have folks requested this as well? Not that I'm aware of10:06
d0ugaljpich: requested and/or discussed :)10:06
shardyjpich: I've been involved with a hackfest this week, and it's proven a minor source of confusion10:06
shardynot a big deal, just something to think about I guess :)10:06
*** milan has joined #tripleo10:06
d0ugalYeah, an O-1 bug might make sense for it10:06
shardyack, I'll raise one and we can discuss further in the bug if needed10:07
*** abregman is now known as abregman|afk10:07
d0ugalHow do I make sense of this failure? http://logs.openstack.org/94/383694/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/9ecf26d/console.html#_2016-10-11_07_53_34_02025510:07
jpichshardy: Interesting! Eventually/soon we want to decouple 1 plan == 1 stack though will it still be as relevant then?10:07
d0ugaljpich: good point!10:07
d0ugaloh, it looks like CI is all red now?10:08
d0ugalall ovb jobs anyway10:08
shardyjpich: I think there will still be times where you want a clean way to iterate on deleting all-the-things, but if you're saying we're going to deprecate overcloud deploy then you could be right10:08
derekhpanda: here is the error I'm seeing while creating the ovb testenv10:08
derekh2016-10-11 09:59:13.029 33652 INFO heat.engine.stack [-] Stack CREATE FAILED (baremetal_10372): Resource CREATE failed: Forbidden: resources.public_network: (rule:create_network and rule:create_network:shared) on {'router:external': False, 'availability_zone_hints': [], 'description': '', 'provider:physical_network': <object object at 0x7f487d4f0140>, u'admin_state_up': True, 'tenant_id': u'b79291658f384b7ebbc9019b6349e5c9', 'segments': <object object at10:08
derekh0x7f487d4f0140>, 'dns_domain': '', 'provider:network_type': <object object at 0x7f487d4f0140>, 'is_default': False, 'qos_policy_id': None, 'port_security_enabled': True, u'shared': True, 'provider:segmentation_id': <object object at 0x7f487d4f0140>, u'name': u'public-10372'} by {'domain': None, 'project_name': u'openstack-nodepool', 'tenant_name': u'openstack-nodepool', 'project_domain': None, 'timestamp': '2016-10-11 09:59:12.856425', 'auth_token': 'c63110:08
derekh6e0559684f2a8f0f7ea48fb889d9', 'resource_uuid': None, 'is_admin': False, 'user': u'ba119eef29ce49f5b8697f4d63948e3c', 'tenant': u'b79291658f384b7ebbc9019b6349e5c9', 'read_only': False, 'project_id': u'b79291658f384b7ebbc9019b6349e5c9', 'user_id': u'ba119eef29ce49f5b8697f4d63948e3c', 'show_deleted': False, 'roles': [u'_member_'], 'user_identity': 'ba119eef29ce49f5b8697f4d63948e3c b79291658f384b7ebbc9019b6349e5c9 - - -', 'tenant_id': u'b79291658f384b7ebbc9010:08
shardyAFAIK we've not quite agreed on that point yet tho, definitely worth some wider discussion tho10:08
derekh19b6349e5c9', 'request_id': 'req-3fda0674-6a57-4aed-8068-5c03bcc6e32c', 'user_domain': None, 'user_name': u'openstack-nodepool'} disallowed by policy10:08
derekhpanda:  Resource CREATE failed: Forbidden: resources.public_network:10:08
jpichshardy: I wasn't, but good to know that's being discussed :-)10:09
* derekh is wondering if anybody changed anything overnight10:09
jpichshardy, d0ugal: What I've noticed with my last few failed deployments if that there's no way to delete a stack from the UI (that I know about? jtomasek?) so that'll probably come up before deleting a plan + stack10:09
jpichderekh: Noticed https://bugs.launchpad.net/tripleo/+bug/1632245 pass by this morning, could it be related or something else going on?10:10
openstackLaunchpad bug 1632245 in tripleo "All CI jobs failing on missing eth1" [Undecided,New]10:10
jtomasekjpich: yeah, we need to implement stack undeploy, but I think we don't want to delete the plan10:11
pandajpich: that was my first analysis, but it's the OVB stack that is failing10:11
pandajpich: and derekh just pasted the error10:11
jtomasekjpich this is where CLI and GUI diverges a bit afaik. as CLI still uses plan as a side effect (afaik)10:11
pandajpich: on ovb side10:11
jpichjtomasek: Yeah, just wondering if we might want the option to also delete plan + stack(s) from the plan management side at some point, but it's not as high priority as "undeploying" indeed10:12
jpichpanda: I'll stop randomly pasting bug numbers then, thank you for the additional information :)10:12
jtomasekjpich: ah, right10:13
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step  https://review.openstack.org/38491210:13
jtomasekjpich: the GUI workflow is if deployment fails, undeploy, tweak environments selection/update parameters of the plan or update the plan by uploading modified/updated files, run validations and try deploy again10:13
derekhjpich: panda I've pasted the error into the bug, still not sure why its happening10:13
d0ugaljpich: I think deleting both at the same time makes less sense in the GUI10:13
shardyjpich, d0ugal: https://bugs.launchpad.net/tripleo/+bug/163227110:14
openstackLaunchpad bug 1632271 in tripleo "No overcloud delete command" [Low,Triaged]10:14
shardyOne thing I'm not that clear on yet is does deprecating the overcloud deploy stuff fix the issues we've had with resolving files not in the plan?10:14
shardyor do we still have that issue when doing plan create?10:14
* shardy needs to do more testing with plan create && plan deploy10:14
*** jraju has quit IRC10:15
d0ugalshardy: I don't think it will yet, as there are some missing features I think10:15
jpichjtomasek: Yeah, that makes sense to me. So only the undeploy is currently missing, right? (side-note: I got a successful  deploy this morning, but it's still showing in red with a big cross as step 4 in the UI :-o)10:15
mariosbandini: did the autopick at https://review.openstack.org/#/c/384912/110:15
shardyd0ugal: Ok, I was just wondering if we can introduce e.g the --extra-files-dir option we discussed to plan create and deprecate the deploy command10:15
jaosoriorshardy: hey dude, I'm getting some really strange issue... where overcloud deploy starts getting stalled in the network deployment if I modify the overcloud-full.qcow2 image. Basically only thing I do to it is an an additional package.10:15
jtomasekjpich: hmm, nice!, I am planning to test it today. It seems that the condition for stack state is incorrect then10:16
shardyjaosorior: what happens when you run the network configuration manually?10:16
jaosoriorshardy: I can't even ssh to the node :/10:16
jtomasekjpich: what is you'r stack state in openstack stack list?10:16
jpichjtomasek: Also if you have some cycles/ideas for https://bugs.launchpad.net/tripleo/+bug/1632007 , since RC3 isn't officially cut yet it'd be great if we could get this into it, AFAICT all the deployments fail until that's manually set :( (Maybe prio should be critical? I find these hard to set)10:17
openstackLaunchpad bug 1632007 in tripleo "Failed deployment due to Hiera resolution of keystone::wsgi::apache::workers" [High,Triaged]10:17
d0ugalshardy: Yeah, that is the direction I was headed when I raised the idea before10:17
d0ugalshardy: but I am not totally against keeping the current deploy command if we can remove the mess from it :)10:17
jpichjtomasek: Status is "CREATE_COMPLETE"10:17
bandinimarios: thanks ;) (forgot a cherry-pick -x?)10:17
jpichjtomasek: I'll try to track it down tomorrow or on Thu otherwise and we can backport it asap after that10:18
jtomasekjpich: thanks, I'll fix that problem with incorrect status10:18
jpichjtomasek: Though if you don't meet it in your own deploys maybe it's ok then :) Both dtrainor and myself saw it10:18
jpichjtomasek: Cheers! Want me to file a bug?10:19
jtomasekjpich: I was hitting the EC2 param one. Did not get further yet10:19
jtomasekjpich: would be great, thanks10:19
jtomasekjpich: The bug seems to me like a templates problem. I am not sure what could cause it from GUI pow10:19
jpichjtomasek: Hm, EC2 param doesn't ring a bell. I got the KeystoneCredential one, Snmpd one, and that keystone one. I got my CREATE_COMPLETE after that10:20
*** dtantsur|afk is now known as dtantsur10:20
shardyjaosorior: did you try using the primary console and a backdoor user/password like I suggested yesterday?10:20
jpichjtomasek: That's quite likely, it's still breaking us though :)10:20
jpichjtomasek: I'll have a look after these meetings tomorrow/Thu10:21
jpichIt doesn't look like folks are encountering on the CLI so there may be something not sync'd again10:21
mariosbandini: no the autopick didn't add it sec (not sure why sometimes it includes it and sometimes not... i think it may be cos they share the change id10:21
jtomasekjpich: yes, probably10:21
jaosoriorshardy: I did try to set up a password for the root user. but haven't been able to access any sort of consolde :/10:21
mariosbandini: but i will add it anyway sec10:21
jtomasekjpich: btw. what environments did you use for your deployment?10:22
jaosoriorshardy: not from virsh, not from virt-manager, not from nova10:22
jtomasekjpich: just the preselected one?10:22
shardyjaosorior: what do you mean - even if the networking is totally broken you can access the primary console?10:22
shardyunless the node isn't even booting?10:22
jpichjtomasek: Yeah, the only settings I set where the ones breaking the deployment really10:22
jaosoriorshardy: that might be a thing10:22
jtomasekjpich: ok10:22
jpichjtomasek: I think I also did not profile matching, just clicked deploy directly (but I've tested so many things yesterday I could be confusing myself)10:23
shardyjaosorior: FWIW I just use virt-manager and access the console for the VM when this happens10:23
shardythat wil also show you if it's hung trying to boot10:23
jaosoriorshardy: right, so that's what I'm trying to do and the console just looks black10:23
shardyjaosorior: So the image got broken then I guess10:23
shardywhat package did you add, and how?10:24
jaosoriorvirt-customize -a overcloud-full.qcow2 --install ipa-client10:24
jaosoriorshardy: and then I just did openstack overcloud image upload --update-existing10:25
jtomasekjpich: this is the parameter from the bug https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/keystone.yaml#L8310:26
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step  https://review.openstack.org/38491210:26
jtomasekjpich: it is type: string10:26
jtomasekjpich: so probably removing the "" should do10:26
jpichjtomasek: Filed https://bugs.launchpad.net/tripleo/+bug/1632278 fwiw10:28
openstackLaunchpad bug 1632278 in tripleo "UI shows successful deployment like an error message" [Medium,Triaged]10:28
jpichjtomasek: Thanks for looking that up! Want to add it to the bug?10:28
jtomasekjpich: I'll investigate a bit more and update the bug10:28
jpichjtomasek: It's weird that no one appears to have seen the issue on the CLI10:29
jtomasekjpich: if I could see the mistral environment of the deployment which fails on this bug, it would be helpful10:30
jtomasekI'll try to reproduce it once I work around the EC2 parameter problem10:30
jpichjtomasek: I used the workaround to fix it, I can also try to undo this and reproduce the breakage later in the afternoon (demoing after lunch!)10:32
jtomasekjpich: ok, enjoy!:)10:32
jpichjtomasek: Thanks :-) Maybe add your ec2 error to the workarounds at https://etherpad.openstack.org/p/tripleo-ui-newton-rc once you've solved it, in case people hit it too this afternoon :-010:33
jtomasekjpich: ack10:34
jpichjtomasek: Cheers10:34
derekhbnemec: by any chance did you re enable the refresh cron job on the te-broker last night?10:36
*** xuao has quit IRC10:42
derekhpanda: see comment 3, will know soon if it works https://bugs.launchpad.net/tripleo/+bug/163224510:46
openstackLaunchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Triaged]10:46
*** fultonj has joined #tripleo10:47
pandaderekh: whoa, it's like a murder cold case. Past and present intertwine. I think we could make a TV show out of CI failures.10:51
jaosoriorccamacho: hey dude, have you had some time to try out the stuff from the blog?10:53
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Enable showing multiple locations when using Ceph  https://review.openstack.org/38492610:53
ccamachojaosorior, Yeahp I started yesterday evening, but messed the environment nad now finishing to re installing it, Im working from today till Thursday on your patches :)10:54
ccamachoand in the mean time some python tests10:55
derekhpanda: things now appear to be working again10:55
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Enable showing multiple locations when using Ceph  https://review.openstack.org/38492610:55
jaosoriorccamacho: fuck yeah :D10:56
ccamachoI have only those 2 thing for this week to see if we can finish to land it :)10:56
*** hjensas has joined #tripleo10:57
pandaderekh: \o/10:57
derekhpanda: cold case closed10:58
rawanhHi, is there a UI for overcloud RHEL OSP9 after installing undercloud ? how can we access it ?11:00
openstackgerritMartin André proposed openstack/tripleo-heat-templates: WIP: Containerized Services for Composable Roles  https://review.openstack.org/33065911:01
jaosoriorccamacho: that would be brilliant11:01
*** thrash|g0ne is now known as thrash11:01
*** zoli|lunch is now known as zoli|wfh11:02
*** rhallisey has joined #tripleo11:06
*** links has joined #tripleo11:09
*** ooolpbot has joined #tripleo11:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Triaged]11:10
pandasorry, bug just closed now.11:12
*** egafford1 has joined #tripleo11:21
*** egafford1 has left #tripleo11:21
*** egafford has quit IRC11:22
*** yolanda has quit IRC11:25
*** egafford has joined #tripleo11:25
*** radeks has quit IRC11:30
*** abregman|afk is now known as abregman11:31
*** abregman is now known as abregman|afk11:33
*** pkovar has joined #tripleo11:33
openstackgerritMerged openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step  https://review.openstack.org/38466711:38
pandashardy: do you remember if during this https://review.openstack.org/362904 you discussed about overcloud nodes logs, and if it was acceptable to lose the logs because thery were gathered after deletion ?11:40
openstackgerritafazekas proposed openstack/python-tripleoclient: Disable encrypted volume in the tempest-deployer-input.conf  https://review.openstack.org/38486811:40
*** yolanda has joined #tripleo11:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-puppet-elements: Add ipa-client package to overcloud-controller image  https://review.openstack.org/38494711:42
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient: Downloads templates from swift before processing update  https://review.openstack.org/38189911:42
*** jpena is now known as jpena|lunch11:43
*** adam_g has quit IRC11:43
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-puppet-elements: Add ipa-client package to overcloud-controller image  https://review.openstack.org/38494711:44
*** ohamada_ has joined #tripleo11:45
*** ohamada has quit IRC11:45
jaosoriorccamacho: this would remove the virt-customize step to install ipa-client https://review.openstack.org/38494711:46
ccamachojaosorior, thanks :)11:47
jaosoriorccamacho: yeah, I'm trying to reduce the number of steps, that blog post is pretty damn big haha11:47
*** dprince has joined #tripleo11:47
*** lucas-afk is now known as lucasagomes11:48
*** openstackgerrit has quit IRC11:48
*** dsneddon has quit IRC11:48
*** openstackgerrit has joined #tripleo11:48
*** dsneddon has joined #tripleo11:49
jaosoriordprince, dsneddon: hey dude. So apparently the stuff I was seeing wasn't really an issue with os-net-config... it just got stuck there. But it seems like the image I was using was actually corrupted. And that part is pretty strange, since only modification I did to it was to install a package using virt-customize11:50
jaosorioronly thing I did was virt-customize -a overcloud-full.qcow2 --install ipa-client11:51
dprincejaosorior: interesting11:52
dprincejaosorior: is it reproducable11:52
jaosoriordprince: pretty much every time I try to install a package like that I get that behavior11:52
jaosoriorso I do the virt-customize, then I do openstack overcloud image upload --update-existing11:52
*** egafford1 has joined #tripleo11:53
*** egafford has quit IRC11:54
openstackgerritafazekas proposed openstack/tripleo-heat-templates: Allow versioned containers  https://review.openstack.org/38495511:54
*** egafford1 is now known as egafford11:55
openstackgerritDan Prince proposed openstack/tripleo-common: WIP: test the new hiera-heat-hook  https://review.openstack.org/38475111:55
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Split out hosts config deployment  https://review.openstack.org/38475611:56
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Hiera optimization: use a new hiera hook  https://review.openstack.org/38475711:56
hewbroccajaosorior: ping Pino Toscano (pinotree)11:56
hewbroccasee if he has any idea what virt-customize might be doing11:56
jaosoriorhewbrocca: thanks dude11:57
*** flaper87 has joined #tripleo11:57
dprinceccamacho: hi, so I actually figured out a pattern that would accomidate the nova workers issue quite nicely I think: https://review.openstack.org/#/c/384757/2/puppet/services/nova-base.yaml11:58
*** flaper87 is now known as Guest973811:58
dprinceccamacho: I was working on the hiera hook... and needed this. But I could use it to work around the nova workers breakage too11:58
dprinceccamacho: although, I may still push on the nova fix for this too11:58
dprinceccamacho: it is the line at the bottom: starting at 12111:59
*** Guest9738 has quit IRC12:01
jaosoriordprince: tripleo-common is present in the overcloud images, right?12:03
ccamachodprince, awesome! I just added me to the reviewers list, Im testing some TLS submissions, then Ill give it a look locally in my dev env :)12:04
*** honza_ is now known as honza12:07
*** honza is now known as Guest6095912:08
pandaEmilienM: do you remember if during this https://review.openstack.org/362904 you discussed about overcloud nodes logs, and if it was acceptable to lose the logs because thery were gathered after deletion ? I'm trying to understand if we want overcloud nodes logs anyway.12:09
jaosoriorjistr: ping12:12
EmilienMpanda: yes, we want overcloud logs anyway12:13
EmilienMpanda: and no, I don't recall this discussion12:13
pandaEmilienM: ok, thanks12:14
*** trown|outtypewww is now known as trown12:15
*** pradk has joined #tripleo12:15
jaosoriormwhahaha: ping12:18
*** openstackgerrit has quit IRC12:18
ccamachodprince the only thing that worries me is that pushing the fix in nova might take a while12:18
*** ccamacho is now known as ccamacho|lunch12:18
*** openstackgerrit has joined #tripleo12:18
*** maticue has joined #tripleo12:20
jistrjaosorior: pong12:20
jaosoriorjistr: hey dude, regarding the TLS patch. I'm trying to address that TODO for the post-save command. All I need is to put the script in some repo that both the overcloud and the undercloud can take it from. I was initially thinking of using tripleo-common, but apparently that's not in the overcloud images :/, got any ideas?12:23
*** amoralej is now known as amoralej|lunch12:23
*** jayg|g0n3 is now known as jayg12:23
*** kjw3 has quit IRC12:25
*** jpich has quit IRC12:28
mwhahahajaosorior: whats up?12:32
openstackgerritMerged openstack/tripleo-quickstart: Update libvirt role to identify flavor correctly  https://review.openstack.org/38260112:34
EmilienMall ovb jobs seem red, anyone know what happens?12:34
*** dsariel has joined #tripleo12:34
hewbroccaEmilienM: 2 nics12:35
hewbrocca1 OVB12:35
hewbroccaor so I'm told12:35
hewbroccafrom this morning at 9:2912:36
hewbrocca<bandini> is it me or CI is broken due to missing eth112:36
EmilienMis anyone working on it?12:37
*** jpena|lunch is now known as jpena12:37
EmilienMderekh: ^ have you seen http://logs.openstack.org/06/384906/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/20e63c0/console.html#_2016-10-11_09_56_37_666186 ?12:38
pandaEmilienM: the bug should be solved now12:39
pandaEmilienM: https://bugs.launchpad.net/tripleo/+bug/163224512:39
openstackLaunchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Fix released]12:39
EmilienMpanda, derekh: thanks!12:40
*** dmsimard|pto is now known as dmsimard12:40
EmilienMI'm still catching up from pto :)12:40
pandaEmilienM: one thing left to do is https://review.openstack.org/38490612:41
EmilienMccamacho|lunch: do you plan to write tests for https://review.openstack.org/#/c/384321/ ? or should we approve it now?12:42
*** rbrady-afk is now known as rbrady12:42
pandasigh .. s/catched/caught/12:42
EmilienMpanda: multinode now fails12:43
*** jcoufal has joined #tripleo12:44
*** rbowen has joined #tripleo12:45
ccamacho|lunchEmilienM yeahp Im trying to write some tests for all the validations from  https://review.openstack.org/#/c/384321/ let me finish them and Ill let you know when they are working fine (when its the new deadline for rc3?)12:46
EmilienMccamacho|lunch: ack12:46
pandammm, epel should have been removed even there ...12:48
EmilienMthrash: do we need to backport https://review.openstack.org/#/c/383708/ ?12:49
mrungeis there a way to skip tripleo-ui installation in undercloud?12:49
*** rlandy has joined #tripleo12:49
mrungetripleo-quickstart install fails currently due to failed dependencies for openstack-tripleo-ui12:50
*** lblanchard has joined #tripleo12:50
EmilienMpanda: it seems transient12:50
slaglemrunge: did you check in undercloud.conf? there is an option: enable_ui12:50
*** flaper87 has joined #tripleo12:51
*** flaper87 has joined #tripleo12:51
mrungeslagle, I looked at it, but did not see the option12:51
mrungeslagle, thanks, will check again12:51
slaglemaybe quickstart strips the default options?12:51
slagleanyway, check the sample in instack-undercloud12:51
mrungewill do12:52
*** limao has joined #tripleo12:52
*** pkovar has quit IRC12:53
pandamaybe it's just the outdated undercloud.conf template in quickstart12:53
jistrjaosorior: hmm as for immediate solution, i don't know to be honest... Though in the future i think we'd like to deploy the undercloud with t-h-t as well. So on one hand, it feels like such script could go to t-h-t, but on the other hand it probably depends how complex/big it is. Can you provide an idea of what you want to do in the script?12:53
*** pkovar has joined #tripleo12:55
trownya... that template is not maintainable... need to open a bug to remove that template in favor of just copying from /usr/share and using ini_file or the like to modify the conf if needed12:57
jistrjaosorior: for my own server i hacked together a tool which has a "<cert-name>.d" directory for each certificate, and one can drop in files that get executed when the cert changes (yea i'm afraid i basically rewrote a single-purpose ACME-protocol certmonger with hooks, using cron + bash...)12:57
*** jpich has joined #tripleo12:58
*** limao_ has joined #tripleo12:59
*** dsavineau has joined #tripleo13:01
*** limao has quit IRC13:01
jistrjaosorior: or maybe i'd start with a simpler question, given i'm not very familiar with the topic -- why can't the postsave_cmd be just "systemctl reload haproxy"? Don't we use that key/cert for haproxy exclusively?13:02
*** chem has joined #tripleo13:03
*** flepied has quit IRC13:03
jaosoriorjistr: so the issue is that haproxy doesn't just take the cert and key as separate files13:03
jaosoriorjistr: those files nee to be appended in a single PEM file13:04
jistrahhh you need to pre-chew it into a single one13:04
jistrgot it13:04
jaosoriorwhy? cause haproxy is weird13:04
*** fultonj_ has joined #tripleo13:04
*** Guest60959 is now known as honza13:04
*** chandankumar has joined #tripleo13:04
openstackgerritMerged openstack/tripleo-quickstart: Add centosci configs for cloudsig-stable pipeline  https://review.openstack.org/38451213:05
*** limao_ has quit IRC13:06
*** tzumainn has joined #tripleo13:07
*** Goneri has joined #tripleo13:07
jaosoriorjistr: so you think it should go in t-h-t instead?13:09
*** tiswanso has joined #tripleo13:10
jaosoriormwhahaha: I was just gonna ask you about a bug ID in a commit. But I figured which bug it was13:11
*** cdearborn has joined #tripleo13:11
jistrjaosorior: so on my server i need to do this for ZNC bouncer, because it does the single-pem thing too13:13
jistrhere's the code, it's a two liner https://paste.fedoraproject.org/448598/61915731/raw/13:13
*** jeckersb is now known as jeckersb_gone13:14
jistrso if we're talking something similar, and we need to notify just haproxy and not a bunch of other services too, and all the file locations are well known, then i think we could probably inline it? wdyt?13:15
jaosoriorjistr: so, we already have that script in instack-undercloud https://github.com/openstack/instack-undercloud/blob/master/scripts/instack-haproxy-cert-update13:16
jaosoriorjistr: so the script is not an issue. The issue is that we need that available for the both the undercloud and overcloud, and I don't know what repo to put that on :/13:16
*** flepied has joined #tripleo13:17
matbuwhere i can find the tripleo-ci periodic jobs ?13:17
jaosoriorjistr: for other services such as apache stuff is more straight forward, since they use certs and keys just as they come from certmonger. So only thing I do for those is do a reload13:17
pandamatbu: what are you looking for ?13:18
pandamatbu: definitions ? configurations ? results ?13:18
jistrjaosorior: ok i see, you added a bunch of safety checks etc. Hmm personally i'd probably still go with putting the script into t-h-t at this point. Currently we do many things twice, because uc/oc aren't using the same installation method. If we use t-h-t for undercloud in the future too, things will become DRY (not only w/r/t certs, also service configuration etc.).13:19
jaosoriorjistr: haha yeah, the script is a bit overkill13:20
matbupanda: i want to see the status of the mitaka periodic jobs13:20
matbupanda: if they are consistent13:21
matbupanda: cause i can't have a consistent mitaka deployment, which is really weird13:21
jaosoriorjistr: anyway, for the reference, this is what I'm giving for httpd-based services https://review.openstack.org/#/c/327029/51/manifests/certmonger/httpd.pp13:21
*** adam_g has joined #tripleo13:22
*** adam_g has quit IRC13:22
*** adam_g has joined #tripleo13:22
pandamatbu: http://status-tripleoci.rhcloud.com/#periodic-tripleo-ci-centos-7-ovb-ha-mitaka13:22
*** morazi has joined #tripleo13:23
matbupanda: k thanks, and who is monitoring that ?13:23
matbufor example the undercloud install fail (which is what i'm hitting)13:24
matbuthis job is broken since september 3013:24
mariosmatbu: can you update when you get a chance pls? https://bugs.launchpad.net/tripleo/+bug/163233013:24
openstackLaunchpad bug 1632330 in tripleo "Undercloud upgrade fails on the yum update we run before undercloud is re installed" [High,Triaged] - Assigned to Marios Andreou (marios-b)13:24
matbumarios: yep sure13:24
jistrjaosorior: yea i think we could either use the script with checks and do get_file in t-h-t, or we could inline the `cat ... > ...` and `systemctl reload haproxy` directly into the template, similarly as you inlined the httpd reload into puppet13:25
*** b3nt_pin is now known as beagles13:25
*** ccamacho|lunch is now known as ccamacho13:26
jaosoriorjistr: alright, sounds like a plan. I'll try the first option and see how that goes. Thanks dude13:26
*** beagles is now known as Guest4892713:26
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Fix deployment status identification  https://review.openstack.org/38499513:27
jtomasekjpich: ^, would be great if you could test this on succeeding deployment13:27
pandamatbu: me and sagi, but I was focusing on master and newton until last week. There is on change out to fix mitaka and liberty jobs now.13:28
jpichjtomasek: I was about to delete it! Will do now, thank you for looking into it13:28
lucasagomeshi all, can I get some eyes on https://review.openstack.org/#/c/370773/ ? It's a 1L change and is already +2'd. Thanks13:28
jtomasekjpich: :) cool, thanks13:28
jpichjtomasek: Sounds like something we'll want to backport too13:28
jtomasekjpich: btw. how old is your undercloud setup?13:29
jtomasekjpich: yep13:29
matbupanda: k, and i see in the log, it use the "consistent" tag to dpeloy mitaka13:29
matbupanda: do you how this tag is "promoted" ?13:29
matbupanda: maybe i can help, but i don't know how tbh right now13:30
*** morazi has quit IRC13:30
dciabrinbandini, jaosorior ping13:30
jaosoriordciabrin: hey13:30
d0ugalslagle: Hey13:30
bandinidciabrin: wasn't me13:31
pandamatbu: you mean http://trunk.rdoproject.org/centos7/consistent ?13:31
matbupanda: yes13:31
slagled0ugal: hi13:31
d0ugalslagle: I am not a fan of https://review.openstack.org/#/c/383694/ either :)13:31
pandamatbu: I don't know the specifics, better ask in #rdo13:31
d0ugalslagle: but it was the best we got after some discussion in here13:31
matbupanda: this is the rdo pipeline which create this tag ?13:32
jpichjtomasek: That one is a bit of a mixed environment, looks like the oldest bits were installed on Sept 29th but I also updated some mistral / common / UI / etc packages right up to yesterday. Think this may be because my heat is too old?13:32
d0ugalslagle: but maybe we could just have instack create a mistral environment (different from heat's env) and add it there. I just don't think that is in any way secure.13:32
d0ugalslagle: however, since we already store other passwords in mistral, that point is probably mute13:33
*** amoralej|lunch is now known as amoralej13:33
dciabrinjaosorior, hey, looking at https://bugs.launchpad.net/tripleo/+bug/1628521. do i get that right that t-h-t in newton now passes the node_names to the galera ras as "before", i.e. overcloud-controller-0 rather than overcloud-controller-0.internalapi.localdomain?13:33
openstackLaunchpad bug 1628521 in tripleo "nodename list disregards network" [High,Fix released] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)13:33
jtomasekjpich: not this one. I am wondering about the keystone workers one. I didn't hit that, I checked git blame and last time it was changed like this https://github.com/openstack/tripleo-heat-templates/commit/da419573fe3519c0d3119ab6f0a27217f0e39a6113:34
slagled0ugal: if it's not any less secure than what we already have, i think i'd prefer to see a mistral environment13:34
jtomasekjpich: which seems like what we're seeing now13:34
jaosoriordciabrin: yes, that's what we're doing currently. Though it would be nice to remove that workaround.13:34
jtomasekjpich: but the patch is relatively old13:34
d0ugalslagle: I don't think it will be, I'll do that.13:34
d0ugalslagle: it is also much easier, I think13:34
jtomasekjpich: I'll install latest setup tomorrow morning and try to reproduce it again13:34
dciabrinjaosorior, yeah that's what i understood. i'm asking because someone on RDO might have a similar issue, so i was wondering whether rdo needed to align to upstream or something13:35
*** jeckersb_gone is now known as jeckersb13:35
jaosoriordciabrin: they probably do.13:35
rawanhis there a way to trigger the hiera configurations to be loaded as a first boot script ?13:36
pradkcan i get some reviews on https://review.openstack.org/#/c/384481/ .. pretty simple fix13:36
matbupanda: grazie btw13:36
jpichjtomasek: Oh, the keystone / trusted_cas issue was on a brand new setup last week, I think it was the same for dtrainor as well... I wonder. The demo is behind me so I'll try to remove my manual workaround and see if I can reproduce the error. You wanted to see a mistral env to help debug it, right?13:36
jtomasekjpich: can you check your undercloud templates how that parameter is defined?13:36
jaosoriordciabrin: well, actually, the workaround is done in puppet-tripleo13:37
jaosoriordciabrin: so t-h-t at the moment passes both the shortnames and the fqdns13:37
jtomasekjpich: ok, yep, checking mistral environment to see if that parameter is defined there could help, but I suspect it is not, in which case GUI has nothing to do with that bug13:38
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375013:38
jpichjtomasek: It's set to '"%{::processorcount}"' in my default templates13:40
dtrainori have an osp10 environment that keeps hitting this error jpich jtomasek if either ofyou want to jump in to it13:40
dtrainorer, a newton environment that generates this error...13:41
jtomasekjpich: ook, that ruins my theory :D13:41
mcorneajaosorior: regarding the issue dciabrin mentioned I've got an environment where it shows up. Do I need any patch in puppet-tripleo to workaround it?13:41
jpichdtrainor: Oooh is it failed on this now? I think jtomasek has some ideas on data we should look at to debug further13:41
*** eggmaster has joined #tripleo13:42
dtrainorit is yes, i have not changed the state of that environment since i ran in to the error yesterday13:42
*** morazi has joined #tripleo13:43
jtomasekdtrainor: can you paste the 'mistral environment-get overcloud', please?13:43
jistrjaosorior: just thinking now -- when using get_file, you'd be inlining the content into postsave_cmd too. Can postsave_cmd be a bash script? (I don't mean if it can call a bash script, but rather if the thing itself gets evaluated as bash.)13:43
jaosoriormcornea: https://review.openstack.org/#/q/I4ef7786474c229d5212a0deb2ca02ee992b030d813:43
jaosoriorjistr: yep13:43
jaosoriorjistr: AFAIK13:43
dtrainorjtomasek, http://paste.openstack.org/show/585324/13:43
jtomasekjistr: we're hitting https://bugs.launchpad.net/tripleo/+bug/1632007 on https://github.com/openstack/tripleo-heat-templates/commit/da419573fe3519c0d3119ab6f0a27217f0e39a6113:44
openstackLaunchpad bug 1632007 in tripleo "Failed deployment due to Hiera resolution of keystone::wsgi::apache::workers" [High,Triaged]13:44
jtomasekjistr: any ideas welcome:)13:44
jistrjaosorior: ok, cool. Also to make any sort of param substitution in it, we'd probably have to use str_replace or yaql or something of that nature13:44
jtomasekdtrainor: ok, I see it, it is there.... that is probably the problem...13:44
jtomasekdtrainor: did you explicitly set this parameter on this setup?13:45
jtomasekdtrainor: "KeystoneWorkers"13:45
mcorneajaosorior: thanks! trying that on my env13:45
dtrainornot in this setup no, but previously i had explicitly set KeystoneWorkers to 11 and the deployment worked13:45
jaosoriormcornea: I assume you have this available too https://review.openstack.org/#/q/I754c4a41d8a294a4c7c18bd282ae014efd4b9b1613:46
jtomasekjistr: sorry, I think I see where the problem is coming from now...13:46
jistrjtomasek: hmm looks like we're losing quotes along the way13:46
*** dprince has quit IRC13:47
mcorneajaosorior: yes, I think the problem is that I have that but missing the puppet-tripleo change13:47
jistrjtomasek: here it's double quoted, so that the inner quotes get preserved when we translate from Heat into Hiera https://github.com/openstack/tripleo-heat-templates/blob/22decf4276c56b58b41ab6523c9a8da41ddeeab6/puppet/services/keystone.yaml#L8613:47
*** dprince has joined #tripleo13:47
jtomasekdtrainor, jistr: yeah, GUI paraemters processing causes it13:47
dtrainoroh no kidding13:47
dtrainorok then, that's good to know13:47
jaosoriormcornea: if you plan on deploying ceph you'll need this too https://review.openstack.org/#/c/382408/13:47
pandaEmilienM: I'll be able to join the meeting only on the second half.13:48
*** panda is now known as panda|afk13:48
jtomasekdtrainor: when GUI shows the parameter value, it displays it inside input and when this happens the string gets mangled, I'll take a look into it unless jpich wants to:)13:48
EmilienMpanda|afk: ack13:49
*** limao has joined #tripleo13:49
dtrainorwell that's excellent news, to know where it's getting butchered13:49
dtrainorthank you13:49
EmilienMweekly meeting in 10 min!13:50
jpichjtomasek: If the fix is in the GUI code I think I will stick to testing the fix once ready if that is ok? (If you're busy with other things and can give me a pointer as to where that's happening I can give it a shot though!!)  Great work on narrowing the problem down, thank you :)13:51
* social_ throws in https://review.openstack.org/#/c/381899/ for weekly meeting13:51
jtomasekjpich: I'll try to fix it. This might be a tough one because the value is basically a string within a string....13:52
jistrjtomasek: btw even the fact that we need to use nested quotes on Heat side to produce normal quotes on Hiera side is effectively a bug, as we're not preserving data types correctly, but it's a tough one to fix because for some uses we actually depend on it being broken...13:53
jistrit's been in tripleo since the dawn of time13:53
jpichjtomasek: Thank you!13:53
*** rawanh has quit IRC13:53
jpichjistr: Ouch13:53
openstackgerritMarios Andreou proposed openstack/python-tripleoclient: Stop services before yum update for undercloud upgrade  https://review.openstack.org/38501213:54
*** adam_g has quit IRC13:54
*** jcoufal_ has joined #tripleo13:54
jaosoriorrhallisey: hey, is it OK to ping you about SELinux stuff?13:56
rhalliseyjaosorior, what issue are you having?13:56
*** jcoufal has quit IRC13:57
*** kjw3 has joined #tripleo13:59
*** Guest48927 is now known as beagles14:00
jaosoriorrhallisey: So, I'm having issues starting apache. This is what I got from the audit logs: http://paste.openstack.org/show/585328/14:00
*** yamahata has joined #tripleo14:00
jaosoriorrhallisey: seems to be having some issue with the gnocchi port14:00
*** absubram has joined #tripleo14:01
rhalliseyi'll have a loo14:03
*** absubram_ has joined #tripleo14:03
jaosoriorrhallisey: thanks dude14:04
*** jprovazn has quit IRC14:04
*** mrunge is now known as mrunge__14:05
*** absubram has quit IRC14:05
*** absubram_ is now known as absubram14:05
*** rodrigods has quit IRC14:06
*** rodrigods has joined #tripleo14:06
*** rcernin has quit IRC14:07
*** dsariel has quit IRC14:09
rhalliseyjaosorior, what os are you running?14:09
*** limao has quit IRC14:10
jaosoriorrhallisey: CentOS Linux release 7.2.1511 (Core)14:10
rhalliseyjaosorior, update to 7.3. This policy is fixed there in base rhel14:10
jaosoriorrhallisey: good to know. unfortunately that's the base that quickstart is downloading by default.14:11
jaosoriortrown: ^^14:11
rhalliseyoh :(14:11
*** limao has joined #tripleo14:11
trownis there a 7.3 for centos?14:12
trownhttp://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 is what we point to, so it should always be latest14:13
jaosoriorrhallisey: ^^14:13
trownbut, if 7.3 is really recent, we have not had a promotion as the whole RDO CI infra was down for hardware move until this morning14:13
rhalliseyit's in rhel 7.314:14
rhalliseyso I'm assuming a centos 7.3 should be around soon if it isn't14:14
trownnot sure how centos release cycle works...14:14
*** egafford has quit IRC14:17
*** panda|afk is now known as panda14:17
*** egafford has joined #tripleo14:18
jaosoriorshardy: hey dude, have you encountered a heat error like this one? http://paste.openstack.org/show/585332/14:19
shardyjaosorior: No but it looks like a heat bug and/or the DB is very broken14:21
rookdtantsur: heya14:23
dtantsurrook, hi, I have 7 minutes before the next meeting, so shoot :)14:23
jaosoriorshardy: so I stumble upon that when trying to re-deploy an overcloud14:23
rookdtantsur lol, ill catch with you later then, no worries.14:23
bnemecderekh: Crap, I did.  Sorry about that.14:24
jaosoriorshardy: however, I'm using templates from my home directory, and environment files in another folder of my home dir14:24
*** dtrainor_ has joined #tripleo14:24
*** dtrainor has quit IRC14:26
*** dtrainor_ is now known as dtrainor14:26
*** tosky has quit IRC14:28
*** mrunge has joined #tripleo14:28
*** limao has quit IRC14:28
derekhbnemec: no prob, you switched it to the way it should have been, it shouldn't have been left like that for so long14:29
*** limao has joined #tripleo14:29
shardyjaosorior: ack, could be related but I still wouldn't expect heat to fail like that14:29
*** tosky has joined #tripleo14:29
shardyif you can reproduce please do raise a heat bug with details14:29
bnemecderekh: Yeah, I figured we just forgot to turn it back on the last time we were debugging.14:30
*** anshul has joined #tripleo14:30
*** dtrainor_ has joined #tripleo14:30
derekhbnemec: yup, unfortunatly it had a adverse side effect https://bugs.launchpad.net/tripleo/+bug/163224514:31
openstackLaunchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Fix released]14:31
derekhbnemec:  but we're all good again now, I havn't figured out how to change our deploy scripts to represent the change though14:31
*** mhenkel has joined #tripleo14:32
*** dtrainor has quit IRC14:33
*** dtrainor_ is now known as dtrainor14:33
*** jeckersb is now known as jeckersb_gone14:35
*** jeckersb_gone is now known as jeckersb14:35
*** mah has joined #tripleo14:36
mahHi all, I have a problem when I use jclouds with tripleo to create an instance, anybody could help in that ?14:36
*** dhill_ has joined #tripleo14:39
*** rajinir has joined #tripleo14:46
*** limao has quit IRC14:50
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Make the snmpd_readonly_user_password available to Mistral  https://review.openstack.org/38369414:50
*** limao has joined #tripleo14:50
*** egafford1 has joined #tripleo14:51
*** egafford has quit IRC14:51
pradkhmm looks like redis is having issues .. i cant connect to the vip or individual redis ips -> https://bugs.launchpad.net/tripleo/+bug/163235614:51
openstackLaunchpad bug 1632356 in tripleo "unable to connect to redis" [Critical,New]14:51
pradkjistr, dint you already fix this? ^^ looks like its back again?14:53
jistrhmm yea i've fixed one issue related to redis/vip, essentially that we weren't creating the redis VIP at all14:54
* jistr looking at the bug link14:54
pradki just noticed this in today's fresh deployment14:55
jistrpradk: is it pcmk or non-pcmk deployment?14:55
pradkha deploy.. so yea pcmk14:55
*** egafford1 is now known as egafford14:56
pradksee my latest comment in the bug14:56
jistrpradk: so that's likely a different issue than before. The previous issue was with keepalived non-pcmk14:56
pradkredis is having issues coming up14:56
pradk7910:M 10 Oct 13:43:02.535 # Opening Unix socket: bind: Permission denied14:56
pradksee the same in ci jobs14:56
weshaymatbu, marios getting questions about newton-> newton updates.. can you point me at jobs that are covering that14:57
openstackgerritMerged openstack/python-tripleoclient: Add ReNo support  https://review.openstack.org/38204614:57
pradkjistr, that error is new..14:57
*** akuznetsov has joined #tripleo14:57
pradk7910:M 10 Oct 13:43:02.534 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.14:57
jistrpradk: hmm wondering why does it want such a huge amount of file descriptors open14:58
jistrpradk: i guess the main problem is the subsequent "Opening Unix socket: bind: Permission denied", which i guess could be prevented by the FD problem too14:58
matbuweshay: since the rdo tab is still with master14:58
matbuweshay: it should be this one : https://ci.centos.org/view/rdo/view/tripleo-periodic/job/tripleo-quickstart-upgrade-minor-master-to-master/14:58
pradkjistr, yea seems like it14:59
jistrpradk: err s/prevented/caused/14:59
matbuweshay: but since 1 week it does a master to master and not a newton to newton14:59
rhalliseyEmilienM, https://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads15:00
matbuweshay: i create a newton to master btw15:00
pradkjistr, this seems pretty recent as this was working in my previous deploy a week ago15:00
EmilienMrhallisey: thx15:00
rhalliseyEmilienM, I'll update the schedule in there. I added the skeleton yesterday15:00
jistrpradk: btw i'm not sure actually about the bind being related to the FD limit15:01
weshaymatbu, we need newton upgrade jobs created..15:01
EmilienMrhallisey: excellent thanks15:01
EmilienMrhallisey: I'll update it when we'll decide what session at what slot15:01
EmilienMrhallisey: please give me your constraints in PM15:01
rhalliseyya sure15:01
rhalliseyEmilienM, gotcha i thought the times were in the etherpad15:02
pradkjistr, selinux?15:02
jistrpradk: it may not be that the limit has been used up, just trying to set it high, and maybe it fails in the sense of "i'm setting it to lower than you wanted but everything's fine in general, carry on" and then the bind error could be unrelated -- i'm just guessing but don't want us to focus on a red herring15:02
pradkjistr, wonder if selinux is not permitting bind15:02
EmilienMdprince, slagle, trown, flaper87 ^ you too please15:02
pradklemme check audit log15:02
trownEmilienM: I have no constraints, I plan to be at all tripleo sessions15:02
jistrpradk: yea it definitely could be one cause, another one that comes up to mind would be a port collision15:02
pradkEmilienM, btw this redis issue is a newton blocker i think ttps://bugs.launchpad.net/tripleo/+bug/163235615:02
pradkEmilienM, i just ran into this in my env and see it happen in ci jobs as well .. http://logs.openstack.org/81/384481/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/94cf8c0/logs/overcloud-controller-0/var/log/redis/redis.txt.gz15:03
EmilienMpradk: please add the tag then15:03
EmilienMtrown: ack15:03
pradkhmm nothing obvious in audit log15:04
*** jtomasek is now known as jtomasek|bbl15:05
*** limao has quit IRC15:06
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Make the snmpd_readonly_user_password available to Mistral  https://review.openstack.org/38369415:06
*** jlinkes has quit IRC15:12
*** tremble has quit IRC15:13
social_matbu: marios: so https://review.openstack.org/#/c/381899/ +1 from me15:13
matbusocial_: ack15:13
social_just finished successfull update15:13
*** [1]cdearborn has joined #tripleo15:16
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Make the snmpd_readonly_user_password available to Mistral  https://review.openstack.org/38369415:17
*** panda is now known as panda|afk15:20
*** ebarrera has quit IRC15:20
*** aufi has quit IRC15:20
openstackgerritDougal Matthews proposed openstack/tripleo-common: Fetch the snmpd_readonly_user_password from the Mistral environment  https://review.openstack.org/38369615:21
pradkjistr, EmilienM, seems like selinux issue.. if i set to permissive, i can start redis15:21
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Use Password Generation provided by the Mistral workflows  https://review.openstack.org/37554415:21
jaosoriorhey guys, can someone check this commits https://review.openstack.org/#/c/376255/ https://review.openstack.org/#/c/376267/  ??15:22
openstackgerritMerged openstack/python-tripleoclient: Allow referencing rendered yaml files in resource_registry  https://review.openstack.org/38382915:22
jaosoriorpradk: I stumbled upon a similar issue with Gnocchi, apparently this is fixed in RHEL 7.3, so we need to wait till that gets to CentOS. Maybe you gotta check if this is also the case for redis15:22
pradkjaosorior, interesting.. do you know where/what the fix was?15:23
jaosoriorpradk: not really15:24
jaosoriorrhallisey: seems pradk is also stumbling with selinux issues.15:24
rhalliseypradk, can you send me your the AVCs you're seeing15:25
*** jistr is now known as jistr|biab15:25
*** fultonj_ has quit IRC15:25
rhalliseysend me the AVCs you're seeing*15:25
pradkrhallisey, so that would be in audit.log?15:25
rhalliseypradk, yes15:25
*** dmacpher has joined #tripleo15:26
rhalliseypradk, grep for AVC if you don't mind.  It would make it easier on me15:26
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Remove the nova-metadata.yaml service  https://review.openstack.org/38505815:26
pradkrhallisey, k added to the bug https://bugs.launchpad.net/tripleo/+bug/163235615:27
openstackLaunchpad bug 1632356 in tripleo "unable to connect to redis" [Critical,New]15:27
dprincejaosorior: ^^^ removing the nova-metadata.yaml okay with you?15:27
jaosoriordprince: I responded in the commit.15:28
jaosoriordprince: so, this was the original bug that was fixed by the addition of that role https://bugs.launchpad.net/tripleo/+bug/162554315:29
openstackLaunchpad bug 1625543 in tripleo "Haproxy config for nova metadata uses ctlplane ip addresses instead of internal_api when using network isolation" [Undecided,Fix released] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)15:29
dprincejaosorior: there are *no* dependencies on this file. How would it matter?15:29
jaosoriordprince: cause that messes up the yaql magic that generates the hieradata for nova_metadata, like I mentioned in the comment.15:30
pradkrhallisey, does that info help?15:30
*** cdearborn has quit IRC15:31
*** pcaruana has quit IRC15:31
rhalliseypradk, perfect thank you15:31
jaosoriordprince: if you remove that service profile, we will not have nova_metadata_network nor nova_metadata_node_names, which is generated in yaql based on the list of enabled services.15:32
dprincejaosorior: at the very least I think nova-api should require nova-metadata if we do actually require this?15:32
*** lucasagomes is now known as lucas-hungry15:32
*** flepied1 has joined #tripleo15:32
dprincejaosorior: it is the duplicate settings, and associated logic that I'm going to have to add I'm trying to avoid here15:32
shardyThe problem is we've got no way to express dependencies between services (yet)15:32
jaosoriordprince: I understand what you're trying to fix. but stuff breaks if we don't have it.15:32
shardyI raised a heat spec which would allow it, but it's not been implemented15:33
dprinceshardy: we sort of do though, I can include the nova-metadata resource in the nova-api. And then remove the hiera settings there.15:33
dprinceshardy: both files can be included, and we only maintain the settings in a single place15:33
jaosoriordprince: trust me, I'm not too happy about having a service profile that's quite unnecessary. But we need those keys from heira15:33
shardydprince: Yeah, but then as jaosorior says that breaks the hieradata generation, which uses the service_name15:33
shardydprince: another option would be to allow service_name to be a list15:33
jaosoriordprince: so if you wanna get rid of it, I'm but, but we need a solution that doesn't break thigns.15:34
shardyor have some other means to remap the hieradata that's needed I guess15:34
jaosorior*I'm in, but15:34
shardymaybe we can just juggle the hieradata in puppet-tripleo?15:34
dprincejaosorior: at the very least I think I should remove the duplicate keys from nova-api.yaml right?15:35
jaosoriorshardy: one things could be to stop using nova_metadata_* and switch everything to use nova_api_*15:35
*** flepied has quit IRC15:35
dprincejaosorior: right now you are probably winning because nova-metadata comes after nova-api in the service list15:35
dprincejaosorior: and map_merge takes the latter15:35
dprinceI'm fine to go the other way now that you've pointed out the valid use case we have15:36
jaosoriordprince, shardy: So, if we're OK with that, we can switch the references to use nova_api_*15:36
dprincejaosorior: The simplest solution is to simply remove the keys from nova-api.yaml. I'd be entirely happy with that15:37
jaosoriordprince: which keys again?15:38
panda|afkany ideas what this could be ? http://logs.openstack.org/28/384628/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/688da1d/console.html#_2016-10-11_12_29_42_88873315:39
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Remove duplicate metadata keys from nova-api.yaml  https://review.openstack.org/38505815:39
dprincejaosorior: those ^^15:39
jaosoriordprince: I'm fine with that15:40
dprinceshardy: FWIW, I think the other needed fix here is service_names does need to be a list15:40
dprinceshardy: we'll certainly need that. We already "extend" one service via another. The use case for this is coming, and we could already make use of it here I think too15:41
dprinceshardy: if you agree with that I'd be willing to push it15:42
mariosthanks social_15:42
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Include ceilometer in swift proxy pipeline  https://review.openstack.org/37195015:42
dprinceshardy: it would require a unit test update in t-h-t to enforce though15:42
*** egafford1 has joined #tripleo15:42
*** egafford has quit IRC15:43
jaosoriordprince: so you want to aggregate the list of services to take into account the sub-services too? Say, you would also generate apache for services that run over it?15:44
rhalliseypradk, can you do `ls -lZ /var/run/redis`15:44
rhalliseypradk, I think you have some mislabled files15:45
pradkrhallisey, sure one sec15:45
pradkrhallisey, http://paste.openstack.org/show/585348/15:47
rhalliseypradk, `sudo restorecon -Rv /var/run/redis` should fix the issue15:47
pradkrhallisey, so this happened out of the box install (even in ci) .. so is there something we need to change during deploy?15:48
rhalliseypradk, ya this restorecon will need to be run after /var/run/redis is created15:50
rhalliseypradk, so maybe in puppet15:50
pradkunderstood, i'll look into it.. it is weird though that we only ran into this now15:52
shardydprince: ack, yeah provided it's optional I think that should be fine15:52
*** egafford1 has quit IRC15:52
shardyit'd be kinda cleaner if we could stick to one service per template, but it doesn't look like that's going to be possible15:52
*** links has quit IRC15:54
thrashrbrady: marios d0ugal this has the deprecation language now: https://review.openstack.org/#/c/381899/15:54
thrashand is passing CI15:54
jaosoriorany +A's for this https://review.openstack.org/#/c/360366/ ?15:55
*** egafford has joined #tripleo15:55
EmilienMpradk, chem, rhallisey: you should now be part of tripleo launchpad group15:56
rhalliseyEmilienM, thx15:56
EmilienMif anyone is not part of the group please let me know15:56
thrashjaosorior: +A15:59
jaosoriorthrash: thanks dude16:00
*** ebarrera has joined #tripleo16:02
*** b00tcat has quit IRC16:02
*** yamahata has quit IRC16:03
*** akuznetsov has quit IRC16:03
*** milan is now known as milan|afk16:07
*** milan|afk has quit IRC16:07
*** rasca has quit IRC16:07
*** jistr|biab is now known as jistr16:07
*** ebarrera has quit IRC16:09
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Only set NovaWorkers in the non-default case  https://review.openstack.org/38507816:12
dprinceccamacho: ^^16:12
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates: Switch Nova workers to '' by default  https://review.openstack.org/38347816:12
jaosoriormwhahaha: you might want to sync with dprince ^^16:13
dsneddondprince, shardy: Do you guys have a suggestion for who might be able to implement this NTP on Undercloud spec in Newton? I think it should be pretty easy. https://review.openstack.org/#/c/35072916:13
*** kjw3 has quit IRC16:15
gfidentedsneddon heh looks like you just raised another case where we'd benefit from having composable roles in the undercloud16:16
gfidenteit's not going to happen in newton though16:16
gfidentejust saying we have two use cases already I think16:16
dsneddongfidente, Yeah, I can see the benefit.16:16
dsneddongfidente, What's the other use case?16:16
gfidentedsneddon basically to have pluggability on the undercloud16:17
gfidenteto have optional services16:17
gfidentemonitoring for example16:17
dsneddongfidente, Right. Or the UI.16:17
dsneddongfidente, Speaking of that, what do you think about this blueprint? I helped Dan Trainor put it together. https://blueprints.launchpad.net/tripleo/+spec/enable-communication-ui-undercloud16:18
gfidentewe'd need to reprise the conversation about the undercloud template dprince worked on16:18
dprincemwhahaha: I hadn't realized you were working on this (bug was assigned to me FWIW)16:18
dsneddongfidente, The idea is to have an API proxy on the Undercloud to allow the UI to connect to the various APIs on the same IP as the UI is hosted.16:18
gfidentedsneddon ok16:19
dsneddongfidente, I couldn't think of a better or more secure way to do it, since the UI contacts the APIs directly, rather than there being some sort of AJAX gateway.16:19
gfidentewhat is the problem with going directly to the APIs?16:19
dsneddongfidente, They live on the Control Plane, which is not neccesarily available remotely.16:20
gfidentewhile the UI could be elsewhere?16:20
gfidenteI mean not hosted on the undercloud?16:20
dsneddongfidente, The UI runs on the external uplink on the Undercloud16:20
gfidenteso this sounds like yet another case where we could re-use the haproxy role16:20
jaosoriordsneddon: well, right now haproxy runs on the undercloud, and can bind to another interface (other than ctlplane) if specified. Wouldn't that be enough?16:20
gfidenteon the undercloud?16:20
dsneddongfidente, Yeah, the UI can be anywhere, there is a list of API endpoints provided to the UI client16:21
dsneddonjaosorior, Yeah, but we want to require SSL for remote connections.16:21
jaosoriordsneddon: sure, that wouldn't really be an issue16:21
*** egafford has quit IRC16:21
dsneddonjaosorior, I mean, that's basically exactly what I'm saying, we would add endpoints in HAProxy on the second IP.16:22
openstackgerritCarlos Camacho proposed openstack/tripleo-common: Default the J2 excludes files to safe values.  https://review.openstack.org/38432116:22
*** zoli|wfh is now known as zoli|gone16:22
ccamachodprince ack will check it soon16:23
* gfidente back in a while16:23
dsneddonjaosorior, gfidente: I'll add you guys to the spec when I write it, if you have any feedback.16:23
jaosoriordsneddon: so, for the SSL case, we already have SSL for haproxy, and one can already bind it to any given interface (needs to be configured in undercloud.conf), so... basically seems that that issue is already fixed, by requiring SSL for the undercloud, and just binding haproxy elsewhere16:23
jaosorior* elsewhere than the ctlplane16:23
jaosoriordsneddon: which one can already do with the wrongly named "undercloud_public_vip", which isn't necessarily just a vip (I actually use fqdns there)16:24
dsneddonjaosorior, Yes, you just described what I planned to put into the spec. The only work around it is adding the necessary params to undercloud.conf and modifying the HAProxy config appropriately.16:24
*** zoli|gone is now known as zoli_gone-proxy16:24
jaosoriordsneddon: I think I'm misunderstanding something, cause that can already be done16:24
jaosoriorlike, now16:24
EmilienMjistr, marios: why did you guys abandon https://review.openstack.org/#/c/319264/ ?16:25
dsneddonjaosorior, Yes, they can do that, but then the Keystone catalog gets updated with the external IP, which forces an extra hop if the Undercloud isn't the default gateway for the Control Plane.16:25
*** athomas has quit IRC16:25
jaosoriordsneddon: I see what you mean now16:25
jaosoriordsneddon: though... I don't really see an issue with having an extra hop16:25
*** egafford has joined #tripleo16:26
dsneddonjaosorior, Although, considering that the Undercloud catalog is really only used by the Undercloud itself...16:26
dsneddonjaosorior, Maybe you are right.16:26
jistrEmilienM: that approach i took there was -2'd (not on the spec but on the code part)16:26
jistrEmilienM: and i got pulled into downstream work at the same time16:26
EmilienMjistr: is here any open spec about upgrades right now?16:26
dsneddonjaosorior, I'll do some testing, and I wouldn't mind at all if you just made that spec redundant. Thanks for the ideas.16:27
jaosoriordsneddon: didn't mean to bring any ideas down. Just thought it would be easier if we used what's there already.16:27
*** jpich has quit IRC16:27
dsneddonjaosorior, Ha, I love it when I find out that work is already done or not needed. It's not like I'm having trouble keeping busy.16:28
jistrmarios, EmilienM: hmm not from my side. We had a couple of other PoCs for new upgrades but i don't think any of them matched requirements like tight integration with the Heat templates for composable services. (AFAIK)16:28
jaosoriordsneddon: haha well, hopefully that works for this case. Had a talk about just this with dtrainor a bit ago.16:28
*** chandankumar has quit IRC16:28
mariosEmilienM: i think mainly as there was pushback to the initial approach taken with ansible... but as noted on the https://etherpad.openstack.org/p/ocata-tripleo-upgrades we can restore and edit it dependign on how the discussion goes16:29
dtrainor(testing this right now)16:29
mariosjistr: ^16:29
*** dmacpher is now known as dmacpher-afk16:29
*** paramite has quit IRC16:29
dtrainordarn meetings are getting in the way though, it'll be a few16:29
EmilienMmarios: I think we should first split bash scripts into chunks and run them separately depending on the services16:30
*** hewbrocca is now known as hewbrocca-afk16:30
EmilienMmarios, jistr: and next iteration for the fiture is to investigate ansible16:30
EmilienMbut I don't think we'll switch upgrades to ansible during ocata cycle16:30
EmilienMnot enough time and no concensus has been reached so far16:30
*** dtantsur is now known as dtantsur|afk16:31
EmilienMmarios, jistr: it would be great to have a spec for that.16:31
mariosEmilienM: /me going in a minute, but i think the idea was to investigate integrating the ansible into the heat templates per service (upgrade snippets)... for ocata the main thing is 'composable service upgrades'... whtether we use ansible or any other approach is still open at the moment16:31
mariosjistr: ^16:31
mariosEmilienM: but delivering 'composable service upgrades' implies a complete change in the current upgrades workflow which is done per role/node16:32
mariosEmilienM: and given the short cycle this becomes even greater challenge16:32
*** jaosorior has quit IRC16:33
EmilienMthat's why adding ansible is a bad idea for such a short cycle16:33
EmilienMI would rather just split the bash scripts16:33
EmilienMso have a script by service16:33
EmilienMmarios: ^16:33
*** akuznetsov has joined #tripleo16:34
mariosEmilienM: sure i am pretty sure we considered that when we started doing that ansible stuff... we can definitely investigate it ... if we are going that way we can even consider puppet manifests so we can get dependencies for example16:34
*** lucas-hungry is now known as lucasagomes16:34
EmilienMmarios: we need to iterate here, and shell scripts split is the easiest solution for now16:35
mariosEmilienM: this makes sense ... and precisely the kind of thing we need to finalise at summit16:37
mariosEmilienM: needs some more thought but adding note for now thanks16:37
jistrmarios, EmilienM: re integrating Ansible -- it also depends how. E.g. we previously discussed 2 different ways, 1) as a normal SoftwareConfig/SoftwareDeployment, where Ansible only sees a single machine, or 2) somehow more classic execution, where ansible sees the whole deployment16:37
jistri think all we prototyped previously was 2), but that brings some challenges on the integration itself, and that was the reason for the pushback16:38
jistr1) would be probably easier to integrate, but it also doesn't bring that many benefits over plain bash scripts...16:38
EmilienMI would rather defer ansible to after Ocata and focus now on composability by splitting the bash scripts.16:39
chembandini: I'm hitting the nova upgrade problem on compute node upgrade upstream, did you have a patch ready ?16:42
mariosEmilienM: so no pushback to the notion of iterating with bash for the first pass especially given the short cycle... thanks for bringing it up ... /me ed16:43
mwhahahadprince: yea sorry i had typoed the bug, so i had a review for tripleo specifically and not the nova thing. I think it's a larger issue than just how nova is handling that value and we might want to rethink those worker values. I'm ok abandoning my review but I thought I'd propose a possible tripleo specific solution16:45
bandinichem: yes, sec16:45
bandinichem: https://review.openstack.org/#/q/I811ff19d7b44a935b2ec5c5e66e5b5191b259eb3,n,z16:46
dprincemwhahaha: sure, I filed the bug under both Nova and Tripleo16:46
dprincemwhahaha: I've got the tripleo fix here: https://review.openstack.org/#/c/385078/16:46
*** rwsu has quit IRC16:46
*** derekh has quit IRC16:46
chembandini: ah, oki that's the one.  Thanks.16:46
dprincemwhahaha: my take is I'd rather leave this parameter an integer in Tripleo. Because is should be...16:46
mwhahahadprince: sure i'll abandon mine it was last week16:46
mwhahahadprince: peronsally i don't agree with the use of 0 making it to the config file for any of the workers unless it specifically triggers a special case in the service, 0 or -1 should trigger the 'default' handling16:47
mwhahahadprince: puppet has already started using os_worker instead of processor count to improve the default number of workers16:48
dprincemwhahaha: ack, sorry for the confusion. We agree there.16:48
dprincemwhahaha: I think this patch to nova will land FWIW, but my t-h-t fix keeps it as an integer and allows us to move on now. No puppet fixes required ATM I think16:48
EmilienMmarios, jistr: so I would like our design sessions focused on that16:49
mwhahahadprince: yea unfortunately my need for consistency across all things really doesn't like the novaworker only case but we should at least create a bug to fix the others16:49
mariosEmilienM: ack i will revisit the spec and that etherpad ... i had a first pass but will revise before summit (not sure when ;) )16:49
dprincemwhahaha: all of them should be integers I think16:49
EmilienMmarios, jistr: the group to discuss about our problems to solve to make it happen16:49
EmilienMmarios: thanks for taking your time on it, it's really important to have a productive session.16:50
mwhahahadprince: yea they should all be integers, but we should handle the default cause of use # processors or whatever be consistently applied all workers16:50
mwhahahadprince: so we should come up with either 0 or -1 where it's the default and it falls back to puppet/service defaults for these numbers.16:51
mwhahahadprince: there by allowing people to tweak the workers if need be but having the defaults ship16:51
mwhahahadprince: it was my understanding that was what the 0 was supposed to do16:51
mwhahahadprince: basically your tht logic needs to be applied to *all* workers16:52
mwhahahanot just nova16:52
mwhahahaso that should get captured in a bug for later16:52
dprincemwhahaha: lets see if people by the Nova logic patch. If they do then I'd like to make them all integers perhaps using a similar mechanism16:52
dprincemwhahaha: FWIW we've got other flags that would benefit from stronger typeing in Heat. The 'Debug' flags are mostly strings I think and should actually be booleans. The conditionals would help there too16:53
*** rbrady is now known as rbrady-run16:54
dprinceI said flags. I mean 'parameters'16:54
mwhahahayea i'm not against typing, it would be nice to have a null or somthing for types tho16:54
mwhahahawhere it's an integer or null16:54
*** rbowen has quit IRC16:54
*** akuznetsov has quit IRC16:58
*** bana_k has joined #tripleo17:02
*** kjw3 has joined #tripleo17:02
*** ayoung has joined #tripleo17:10
*** ayoung has quit IRC17:12
*** ayoung has joined #tripleo17:13
*** ayoung has quit IRC17:14
openstackgerritMerged openstack/instack-undercloud: Make HAProxy terminate OpenStack internal/admin endpoints  https://review.openstack.org/36036617:15
openstackgerritMerged openstack/tripleo-heat-templates: Populate vnc_api_lib.ini on compute nodes with OpenContrail  https://review.openstack.org/38344517:16
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Make UpdateManager use a passed-in stack setup  https://review.openstack.org/38509617:17
*** ayoung has joined #tripleo17:17
pradkrhallisey, is  there a way to fix the context issue directly in openstack-selinux module?17:23
pradkrhallisey, the issue is we rely on a 3rd party puppet module for redis.. we can try submitting a patch to run exec , but chances are they wont accept17:23
*** fultonj has quit IRC17:24
pradkrhallisey, i'm curious why we are running into this now.. this has been the case with redis for a while now.. never had this issue17:24
rhalliseypradk, the problem has to do with when/how /var/run/redis is created17:25
*** fultonj has joined #tripleo17:25
rhalliseycould we provide them instructions to fix it?17:25
rhalliseythey just need to run the restorecon17:26
pradkrhallisey, so why would the context be wrong to being with?17:27
*** pkovar has quit IRC17:28
*** ohamada_ has quit IRC17:28
pradkrhallisey, we could perhaps run restorecon in the %post of redis rpm spec17:28
pradkthats another option17:28
rhalliseypradk, it depends when it is installed.  Puppet or bash script will run a restore con after17:28
rhalliseypradk, i don't think it should be in the redis rpm17:29
rhalliseypradk, when in the process does this occur?17:29
*** tosky has quit IRC17:29
rhalliseydeployment? undercloud setup?17:29
pradkrhallisey, deployment.. the overcloud deploy finishes fine, but if we check pcs status .. redis is down17:30
rhalliseypradk, having a restorecon in openstack-selinux might work17:33
rhalliseyit might not17:33
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Hiera optimization: use a new hiera hook  https://review.openstack.org/38475717:33
rhalliseybut we can try it17:33
rhalliseypradk, would you be able to test it if I can get you a new rpm?17:33
pradkrhallisey, yep i definitely can17:34
*** trown is now known as trown|lunch17:34
dtrainorWhy is /etc/hosts modified to point the fqdn of the Undercloud host to
bnemecdtrainor: Because if the fqdn doesn't resolve properly either puppet or rabbit (or both) explodes.17:36
slagleit's only done if the fqdn isn't already in /etc/hosts17:37
dtrainorbnemec++ slagle++ thank you17:37
slagleso you can prevent it from happening by setting it differently17:37
dtrainori'll make a habit out of modifying /etc/hosts before undercloud install17:37
*** shardy has quit IRC17:38
*** pblaho has quit IRC17:39
*** ccamacho has quit IRC17:39
*** cylopez has quit IRC17:42
*** kjw3 has quit IRC17:46
*** jpena is now known as jpena|off17:49
rhalliseypradk, I have a build for you17:51
rhalliseypradk, try that out and let me know17:51
pradkrhallisey, cool17:52
*** mcornea has quit IRC17:53
*** kjw3 has joined #tripleo17:54
*** rhefner has joined #tripleo17:58
*** ebarrera has joined #tripleo17:59
*** amoralej is now known as amoralej|off17:59
*** pblaho has joined #tripleo18:01
*** rbowen has joined #tripleo18:03
*** ebarrera has quit IRC18:03
*** abregman|afk has quit IRC18:04
openstackgerritMerged openstack-infra/tripleo-ci: Move qemu-img installation after repo setup  https://review.openstack.org/38462818:09
*** tiswanso has quit IRC18:09
openstackgerritMerged openstack/tripleo-heat-templates: Populate vnc_api_lib.ini on compute nodes with OpenContrail  https://review.openstack.org/38344618:09
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Try temprevert of keystone domain change on mitaka  https://review.openstack.org/38511418:13
jdennisis tripleo.org down? it's not responding.18:17
bnemecjdennis: It looks like it.18:19
bnemecdprince: ^18:19
dprincebnemec: checking18:20
dprincebnemec: I can't get in via ssh either. First time this has ever happened. Let me reboot my precious VM18:21
bnemecdprince: Cool, thanks18:21
*** chandankumar has joined #tripleo18:24
*** kjw3 has quit IRC18:27
*** tiswanso has joined #tripleo18:29
*** tiswanso has quit IRC18:29
*** tiswanso has joined #tripleo18:29
*** kjw3 has joined #tripleo18:45
*** trown|lunch is now known as trown18:46
*** chandankumar has quit IRC18:53
*** dsneddon_ has joined #tripleo19:03
*** dsneddon_ has quit IRC19:04
EmilienMslagle: http://logs.openstack.org/96/385096/1/check/gate-tripleo-ci-centos-7-nonha-multinode/7172c29/console.html#_2016-10-11_17_37_13_05027119:11
EmilienMslagle: wtf? lol19:11
EmilienMsomething is trying to use epel here19:11
EmilienMon the subnode219:12
EmilienMRepo-id      : epel/x86_6419:12
EmilienMsee http://logs.openstack.org/96/385096/1/check/gate-tripleo-ci-centos-7-nonha-multinode/7172c29/logs/subnode-2/var/log/host_info.txt.gz19:12
slagleEmilienM: is epel-release installed?19:13
slaglewe should remove that if it is present19:13
EmilienMyes !19:13
EmilienMit is installed19:13
EmilienMslagle: I thought we removed the element for that19:14
EmilienMslagle: maybe it's in the image19:14
slagleEmilienM: we dont use our images for multinode jobs19:15
slaglethat's why we remove epel-release in repo-setup19:15
EmilienMthe package is not on undercloud though19:15
slaglesubnode2 wouldnt be the undercloud19:16
EmilienMright, the package is on overcloud19:17
slaglewe havent removed epel from the image-yaml in tripleo-common19:18
EmilienMnice catch19:18
EmilienMslagle: you want to submit the patch?19:19
slagleb/c there already is one19:19
slaglejust hanging out waiting for reviews :)19:19
EmilienMno more19:19
slagleour surface area for errors just decreased19:22
trownis that the last remnant of EPEL?19:22
slaglei think so19:22
mwhahahaif anyone wants to review a trivial puppet typo fix, https://review.openstack.org/#/c/383934/19:25
openstackgerritMerged openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step  https://review.openstack.org/38491219:28
*** kjw3 has quit IRC19:35
thrashneed a final +2:19:36
openstackgerritCarlos Camacho proposed openstack/tripleo-common: Default the J2 excludes files to safe values.  https://review.openstack.org/38432119:36
thrashfor some reason I can't copy/paste. :P19:36
*** yamahata has joined #tripleo19:44
*** rbrady-run is now known as rbrady19:46
*** dprince has quit IRC19:48
*** Goneri has quit IRC19:58
gfidentestevebaker if only it wasn't 21.58 here!19:59
stevebakergfidente: :)19:59
panda|afkany ideas what this could be http://logs.openstack.org/28/384628/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/688da1d/console.html#_2016-10-11_12_29_42_88873320:01
*** mhenkel has quit IRC20:02
rooklucasagomes: ping any chance you are still around?20:05
*** another_larsks is now known as larsks20:05
lucasagomesrook, I was about to call it aday20:06
lucasagomesrook, but what's up ?20:06
rooklucasagomes: naw man, go for it.20:06
rooklucasagomes:  i wanted to see where i could insert some of my logic in my setups20:06
lucasagomesrook, no worries, if it's quick I can help out20:06
rookie when we are in a deploy and things go from build->active20:06
rookwhere can i find that in code, so I can look to add something for checking status.20:07
lucasagomesrook, so ironic status goes like: deploying -> deploy wait [-> deploying] -> active20:07
lucasagomesops I mean20:07
rookbuilding right?20:08
lucasagomesrook, that's in nova20:08
lucasagomesbut underneath ironic is going to have those status ^20:08
lucasagomesrook, so the first deploying is just setting up the environemnt to boot the ramdisk, in the code you will see funciotions like prepare_ramdisk()20:09
rooki think the handoff is where we could do some better checking.. Ironic says this node is done building, handing it off from there, but then nothing happens because something went fubar with that node.20:09
lucasagomesrook, oh, like not booting the ramdisk ?20:09
rookso, it boots the ramdisk, installs, but then doesn't reboot.20:09
rooksomething went fubar -- so the node is still sitting in the PXE20:09
rookbut ironic already determined this node is done.20:10
lucasagomesrook, so ironic the node still have the deploy ramdisk booted in it ? Or it rebooted and then nothing happened ?20:11
rooknothing happening. however w/ a nova list/ ironic node-list i see active in nova and active in ironic (no more wait-call-back)20:12
rooklucasagomes: this is in regard to the email i sent this morning20:13
lucasagomesrook, I haven't seem the email, sorry20:13
lucasagomesrook, so yeah, Ironic kinda assumes the node was deployed correct after that stage20:13
rooklucasagomes: so it is a hand off issue.20:13
lucasagomesit doesn't actually perform any check whether it has booted correctly or not20:14
lucasagomesit will happen after the "reboot_to_instance()" method is called20:14
* lucasagomes finds in code20:14
rookso, after the node is deployed/installed, no one checks to see if the desired state is met, ie booted into a OS20:14
lucasagomesrook, https://github.com/openstack/ironic/blob/659f951d72e96f39bb967455a6855682e517ca43/ironic/drivers/modules/agent.py#L31220:14
lucasagomesrook, nop, cause I don't think we have any means to actually do that20:14
rooklucasagomes well we have all the information to do that.20:15
lucasagomesspecially with network isolation20:15
rookeach should be in the provisioning network...20:15
lucasagomesrook, right yeah so we probably need to add some checks to it, what do you have in mind ?20:15
rooklucasagomes the check IMHO is a simple ping, after a desired timeout (something that dtantsur|afk and I have been emailing about)... if no connectivity after the timeout, we kick the host -- and if this fails, we re-schedule the host.20:16
*** jayg is now known as jayg|g0n320:17
rookthis will add a ton of resilience20:17
lucasagomesrook, yeah we probably can think of something like that, would be better to have it performed prior to marking the node as active20:18
lucasagomesright after the reboot we could check if the node is up20:18
lucasagomesand if so, reboot20:18
lucasagomesI mean20:18
lucasagomesmark as active20:18
* lucasagomes brain's slow20:18
rookright, no it is late. no problem20:18
rookI just think this will make the babysitting issue of OOO much better.20:18
rookmuch less handholding when you are doing a ton of nodes.20:19
leifmadsenplease! :)_20:19
rookhush leifmadsen20:19
leifmadsenrook: pfffft20:19
rooklucasagomes head out man, thanks for sticking around to discuss.20:19
lucasagomesrook, would be good to add an RFE for it20:19
rooklucasagomes: i have a email titled Suggestions -- give it a look in your freetime. sent to openstack-dev20:19
rookOOO Suggestions i think20:20
lucasagomesrook, totally+ I will add to my todo list to look at it tomorrow morning20:20
lucasagomesrook, thanks for it!20:20
lucasagomesrook, alright! I'm heading now20:20
lucasagomessee ya20:21
*** lucasagomes is now known as lucas-afk20:21
openstackgerritDan Sneddon proposed openstack/os-net-config: Add NIC Mapping Reporting Feature  https://review.openstack.org/38351620:26
*** gfidente has quit IRC20:32
*** panda|afk is now known as panda20:33
*** bana_k has quit IRC20:34
*** bana_k has joined #tripleo20:35
bnemecWow, epel is having a really bad day.20:36
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud: Enable telemetry services by default on undercloud  https://review.openstack.org/38514620:43
pradkbnemec, ^^20:43
remix_tjhello, i see that in newton's tripleo there still isn't neutron-lbaas. Is there any support plan? can be configured in some way?20:48
*** trown is now known as trown|outtypewww20:51
*** dsavineau has quit IRC20:52
pradkcan i get a +A on this please -> https://review.openstack.org/#/c/384481/20:53
*** jcoufal_ has quit IRC20:56
*** rbrady is now known as rbrady-afk20:59
*** egafford has quit IRC21:02
*** kbyrne has quit IRC21:04
*** akrivoka has quit IRC21:08
*** kbyrne has joined #tripleo21:10
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient: Downloads templates from swift before processing update  https://review.openstack.org/38515121:13
*** rlandy has quit IRC21:17
*** yamahata has quit IRC21:26
*** jeckersb is now known as jeckersb_gone21:27
*** dbecker has quit IRC21:28
*** tdasilva has quit IRC21:30
*** tdasilva has joined #tripleo21:33
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Default $STABLE_RELEASE in bootstrap-overcloud-full.sh  https://review.openstack.org/38515921:34
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Add --setup-nodepool-files option  https://review.openstack.org/38516021:34
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Create dummy overcloud-full in --boostrap-subnodes  https://review.openstack.org/38516121:34
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Add configurable mtu to --mulitnode-setup  https://review.openstack.org/38516221:34
*** yamahata has joined #tripleo21:36
*** fultonj has quit IRC21:47
bnemecslagle: probably just killed rh1 :-P21:49
openstackgerritMerged openstack/tripleo-common: Make UpdateManager use a passed-in stack setup  https://review.openstack.org/38509621:55
*** zigo has quit IRC22:02
*** zigo has joined #tripleo22:05
*** zigo is now known as Guest5063022:05
jristEmilienM: are you already PTL?22:09
jristEmilienM: :)22:09
*** bana_k has quit IRC22:09
jristEmilienM: I wonder if we can get some extra time/space at the PTG for TripleO-UI too?22:10
EmilienMjrist: well, I'll be for the Summit22:11
EmilienMjrist: of course22:11
jristEmilienM: do you think I should request that separately?22:12
EmilienMjrist: do you pay in popcorn?22:12
jristonly the finest22:12
EmilienMno need, you let me know that's enough22:12
EmilienMI'll ask for enough space22:12
EmilienMdepending of the amount of popcorn you'll pay22:12
jristwhat flavor?22:12
bkeroSweet popcorn? Kettle corn?22:13
*** panda is now known as panda|zZ22:14
jristthat's wide open, bkero22:16
jristsweet sweet popcorn22:16
EmilienMbnemec: nice https://review.openstack.org/#/c/385179/ I think it can fix the overcloud pingtest on mitaka Ci22:18
bnemecEmilienM: Yeah, that's hopefully the last blocker to having working stable branch jobs again. :-)22:19
EmilienMbnemec: did you check if rspec tests need an update?22:19
bnemecEmilienM: Ah shoot, I forgot.  It probably does.22:19
thrashbnemec: have you tried the update job again against https://review.openstack.org/#/c/381899/22:23
thrashdoes reverify actually work? Or does one use recheck if the verify fails?22:25
bnemecthrash: Not recently.  I might just merge https://review.openstack.org/#/c/374406/ and then you can try it by just running check experimental.22:25
bnemecIt's only failing on known issues with the stable branches, and doesn't actually affect any of those jobs anyway.22:25
thrashbnemec: is that non-voting?22:26
thrashI suppose it would be. :P22:26
bnemecthrash: Sort of.  The updates job is currently voting experimental.22:26
thrashI thought that did something different. :)22:26
bnemecthrash: Okay, I pushed the button on the updates job fix.  Once that merges you should be able to test any patch for updates using check experimental.22:28
bnemecOnce we get it working again we can put it back in the regular queue.22:29
thrashbnemec: awesome.22:29
*** kjw3 has joined #tripleo22:50
*** tiswanso has quit IRC22:58
*** tiswanso has joined #tripleo22:59
*** bana_k has joined #tripleo23:00
*** bana_k has quit IRC23:02
*** rhallisey has quit IRC23:03
*** anshul has quit IRC23:07
openstackgerritSteve Baker proposed openstack/os-collect-config: WIP ec2 collector attempt config-drive before metadata server  https://review.openstack.org/36973123:18
openstackgerritSteve Baker proposed openstack/os-collect-config: Treat ec2 collector data as immutable  https://review.openstack.org/38471223:18
openstackgerritSteve Baker proposed openstack/os-collect-config: Use TEST-NET-1 for unit tests, not  https://review.openstack.org/38519623:18
*** rajinir has quit IRC23:26
*** morazi has quit IRC23:32
*** pmannidi|brb is now known as pmannidi23:35
*** tiswanso has quit IRC23:38
*** bana_k has joined #tripleo23:38
openstackgerritEmilien Macchi proposed openstack/tripleo-specs: Propose TripleO Squads policy  https://review.openstack.org/38520123:43
*** hrybacki has quit IRC23:54
*** hrybacki has joined #tripleo23:55
*** bana_k has quit IRC23:56
*** bana_k has joined #tripleo23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!