Tuesday, 2016-10-11

openstackgerritMatt Fischer proposed openstack/puppet-keystone: set 0600 permissions on fernet keys & folder  https://review.openstack.org/38213600:04
*** ducttape_ has quit IRC00:14
*** agrebennikov has quit IRC00:21
*** markvoelker has joined #puppet-openstack00:36
openstackgerritKeith Schincke proposed openstack/puppet-openstack-integration: Introduce scenario004  https://review.openstack.org/33910600:37
*** markvoelker has quit IRC00:41
*** ducttape_ has joined #puppet-openstack01:02
*** ducttape_ has quit IRC01:07
*** ducttape_ has joined #puppet-openstack01:08
*** xiaohua has joined #puppet-openstack01:09
*** markvoelker has joined #puppet-openstack01:12
openstackgerritKeith Schincke proposed openstack/puppet-openstack-integration: Introduce scenario004  https://review.openstack.org/33910601:20
*** ducttape_ has quit IRC01:32
*** xiaohua has quit IRC01:34
*** xiaohua has joined #puppet-openstack01:35
*** xiaohua has quit IRC01:45
*** ilbot3 has quit IRC01:45
*** xiaohua has joined #puppet-openstack01:46
*** ilbot3 has joined #puppet-openstack01:47
*** xiaohua has quit IRC01:55
*** xiaohua has joined #puppet-openstack02:07
*** markvoelker has quit IRC02:10
*** markvoelker has joined #puppet-openstack02:10
*** caiobo has quit IRC02:12
*** xiaohua has quit IRC02:41
*** xiaohua has joined #puppet-openstack02:42
-openstackstatus- NOTICE: Jobs running on osic nodes are failing due to network issues with the mirror. We are temporarily disabling the cloud.02:49
*** xiaohua has quit IRC02:49
*** pradk has quit IRC02:59
*** ducttape_ has joined #puppet-openstack03:02
*** xiaohua has joined #puppet-openstack03:11
*** ducttape_ has quit IRC03:15
*** dh64 has quit IRC03:33
*** vern has quit IRC03:57
*** xiaohua has quit IRC04:00
*** xiaohua has joined #puppet-openstack04:01
*** xiaohua has quit IRC04:10
openstackgerritKeith Schincke proposed openstack/puppet-openstack-integration: Introduce scenario004  https://review.openstack.org/33910604:11
*** ducttape_ has joined #puppet-openstack04:15
*** vern has joined #puppet-openstack04:17
*** ducttape_ has quit IRC04:20
*** vern has quit IRC04:24
*** xiaohua has joined #puppet-openstack05:08
*** markvoelker has quit IRC05:13
*** xiaohua has quit IRC05:13
*** markvoelker has joined #puppet-openstack05:15
*** markvoelker has quit IRC05:16
*** ducttape_ has joined #puppet-openstack05:17
*** markvoelker has joined #puppet-openstack05:18
*** markvoelker has quit IRC05:18
*** ayoung has quit IRC05:18
*** markvoelker has joined #puppet-openstack05:19
*** ayoung has joined #puppet-openstack05:20
*** markvoelker has quit IRC05:20
*** ducttape_ has quit IRC05:22
*** ayoung has quit IRC05:22
*** markvoelker_ has joined #puppet-openstack05:25
*** markvoelker_ has quit IRC05:25
*** markvoelker has joined #puppet-openstack05:27
*** markvoelker has quit IRC05:28
*** markvoelker has joined #puppet-openstack05:28
*** richm has quit IRC05:40
*** markvoelker has quit IRC05:44
*** markvoelker has joined #puppet-openstack05:45
*** hjensas_ has quit IRC05:46
*** rcernin has joined #puppet-openstack06:07
*** xiaohua has joined #puppet-openstack06:14
*** vern has joined #puppet-openstack06:18
*** ducttape_ has joined #puppet-openstack06:18
*** ducttape_ has quit IRC06:22
*** xiaohua has quit IRC06:25
*** vern has quit IRC06:26
*** xiaohua has joined #puppet-openstack06:35
*** vern has joined #puppet-openstack06:53
*** dgurtner has joined #puppet-openstack06:56
*** hjensas has joined #puppet-openstack06:58
*** tdb has quit IRC06:59
*** tdb has joined #puppet-openstack07:00
*** ebalduf has quit IRC07:02
*** Vinsh has quit IRC07:02
*** jaosorior has joined #puppet-openstack07:03
*** Vinsh has joined #puppet-openstack07:03
openstackgerritChristian Schwede proposed openstack/puppet-tripleo: Add versioned_writes to Swift proxy config  https://review.openstack.org/38470407:06
*** ccamacho has joined #puppet-openstack07:10
*** dgurtner has quit IRC07:25
*** amoralej|off is now known as amoralej07:39
*** jpena|off is now known as jpena07:39
*** dgurtner has joined #puppet-openstack07:50
*** hogepodge has quit IRC07:51
*** Vinsh has quit IRC07:52
*** jesusaur has quit IRC07:52
*** aarefiev has quit IRC07:55
*** aarefiev has joined #puppet-openstack07:56
*** markvoelker has quit IRC07:57
*** hogepodge has joined #puppet-openstack07:58
*** jesusaur has joined #puppet-openstack08:05
*** _degorenko|afk is now known as degorenko08:23
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo: Ensure presence of pacemaker restart directory.  https://review.openstack.org/38487008:27
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-tripleo: Ensure presence of pacemaker restart directory.  https://review.openstack.org/38487008:29
*** xarses_ has joined #puppet-openstack08:32
*** pester has joined #puppet-openstack08:32
*** derekh has joined #puppet-openstack08:34
openstackgerritJavier Peña proposed openstack/puppet-openstacklib: [TEST DO NOT MERGE] Test for Packstack debugging on Puppet 4  https://review.openstack.org/38463708:35
*** llua` has joined #puppet-openstack08:39
*** amoralej_ has joined #puppet-openstack08:40
*** mjblack- has joined #puppet-openstack08:40
*** dgurtner has quit IRC08:41
*** vern has quit IRC08:41
*** xarses has quit IRC08:41
*** guimaluf has quit IRC08:41
*** khappone has quit IRC08:41
*** davidmichaelkarr has quit IRC08:41
*** mjblack has quit IRC08:41
*** amoralej has quit IRC08:41
*** llua has quit IRC08:41
*** cgfbee has quit IRC08:41
*** fxpester has quit IRC08:41
*** holser_ has quit IRC08:41
*** mwhahaha has quit IRC08:41
*** mgagne has quit IRC08:41
*** mjb` has quit IRC08:41
*** mjblack- is now known as mjblack08:41
*** amoralej_ is now known as amoralej08:41
*** dgurtner has joined #puppet-openstack08:44
openstackgerritMerged openstack/puppet-ec2api: Fix documentation for log_dir parameter  https://review.openstack.org/38459308:44
openstackgerritMerged openstack/puppet-murano: Fix documentation for log_dir parameter  https://review.openstack.org/38457508:44
openstackgerritMerged openstack/puppet-vitrage: Change worker defaults to ::os_workers  https://review.openstack.org/38395908:45
*** cgfbee has joined #puppet-openstack08:45
*** davidmichaelkarr has joined #puppet-openstack08:49
*** vern has joined #puppet-openstack08:49
*** guimaluf has joined #puppet-openstack08:49
*** khappone has joined #puppet-openstack08:49
*** holser_ has joined #puppet-openstack08:49
*** mwhahaha has joined #puppet-openstack08:49
*** mgagne has joined #puppet-openstack08:49
*** mjb` has joined #puppet-openstack08:49
*** davidmichaelkarr has quit IRC08:50
*** davidmichaelkarr has joined #puppet-openstack08:50
openstackgerritMerged openstack/puppet-octavia: Fix documentation for log_dir parameter  https://review.openstack.org/38459108:50
*** gfidente has joined #puppet-openstack08:52
*** mwhahaha has quit IRC08:52
*** mwhahaha has joined #puppet-openstack08:53
*** markvoelker has joined #puppet-openstack08:57
openstackgerritMerged openstack/puppet-watcher: Fix documentation for log_dir parameter  https://review.openstack.org/38456008:59
*** tdb has quit IRC08:59
*** tdb has joined #puppet-openstack09:00
*** markvoelker has quit IRC09:03
openstackgerritMerged openstack/puppet-panko: Fix documentation for log_dir parameter  https://review.openstack.org/38450209:14
*** hjensas has quit IRC09:20
*** iberezovskiy|off is now known as iberezovskiy09:20
*** vern has quit IRC09:24
*** paramite has joined #puppet-openstack09:28
*** openstack has joined #puppet-openstack09:31
*** jaosorior has quit IRC09:39
*** jaosorior has joined #puppet-openstack09:40
*** jpich has joined #puppet-openstack09:41
*** shardy has joined #puppet-openstack09:47
*** markvoelker has joined #puppet-openstack09:59
jaosorioriberezovskiy, degorenko: are you around?09:59
*** chem has quit IRC10:00
degorenkojaosorior, yes10:00
iberezovskiy+10:00
jaosoriordegorenko, iberezovskiy: So, I've put out this commit https://review.openstack.org/#/c/384371/ but haven't quite figured out why it's failing the unit tests http://logs.openstack.org/71/384371/2/check/gate-puppet-aodh-puppet-unit-3.8-centos-7/545dc8d/console.html10:02
jaosoriorSystemStackError: stack level too deep10:02
jaosoriorwhat's that about?10:02
degorenkostrange error10:03
degorenkohaven't seen this before10:04
*** markvoelker has quit IRC10:04
*** Vinsh has joined #puppet-openstack10:04
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-aodh: Add option to enable Aodh's SSL middleware  https://review.openstack.org/38437110:05
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-gnocchi: Add option to enable Gnocchi's SSL middleware  https://review.openstack.org/38435810:08
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-ceilometer: Add option to enable Ceilometer's SSL middleware  https://review.openstack.org/38436610:11
jaosoriordegorenko: I think I figured it out10:11
jaosoriorcame from a copy/paste error10:11
degorenkojaosorior, it happens :)10:12
*** richm has joined #puppet-openstack10:13
*** dtantsur|afk is now known as dtantsur10:20
openstackgerritMaksym Yatsenko proposed openstack/puppet-manila: Change worker defaults to ::os_workers  https://review.openstack.org/38462010:21
*** ducttape_ has joined #puppet-openstack10:23
*** ducttape_ has quit IRC10:27
*** ntpttr has quit IRC10:30
*** ntpttr has joined #puppet-openstack10:31
openstackgerritMerged openstack/puppet-congress: Fix documentation for log_dir parameter  https://review.openstack.org/38444910:44
*** fultonj has joined #puppet-openstack10:47
*** hjensas has joined #puppet-openstack10:57
*** markvoelker has joined #puppet-openstack10:59
*** markvoelker has quit IRC11:04
*** xiaohua has quit IRC11:07
*** xiaohua has joined #puppet-openstack11:08
*** basilAB has quit IRC11:09
*** dgurtner has quit IRC11:10
*** basilAB has joined #puppet-openstack11:11
*** xiaohua has quit IRC11:16
*** ducttape_ has joined #puppet-openstack11:24
*** ducttape_ has quit IRC11:28
*** xiaohua has joined #puppet-openstack11:36
*** xiaohua has quit IRC11:36
*** xiaohua has joined #puppet-openstack11:37
*** johnmilton has joined #puppet-openstack11:41
openstackgerritJavier Peña proposed openstack/puppet-nova: [TEST DO NOT MERGE] Test for Packstack debugging on Puppet 4  https://review.openstack.org/38494611:42
*** jpena is now known as jpena|lunch11:43
*** xiaohua has quit IRC11:45
*** EduNeto has quit IRC11:46
*** dprince has joined #puppet-openstack11:47
*** lucas-afk is now known as lucasagomes11:48
*** openstackgerrit has quit IRC11:48
ansiwenEmilienM: regarding neutron_vpnaas_available, why did you set it to false, and do you have any further information how to make it work?11:48
*** openstackgerrit has joined #puppet-openstack11:49
ansiwenEmilienM: enabling results in this error: http://logs.openstack.org/42/356442/29/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/bef0e08/logs/tempest.txt.gz#_2016-10-10_14_55_07_90011:49
ansiwenEmilienM: but the ec2-api tempest tests include VPN tests... :-/11:49
*** xiaohua has joined #puppet-openstack11:55
*** EduNeto has joined #puppet-openstack11:59
*** markvoelker has joined #puppet-openstack12:00
*** markvoelker has quit IRC12:05
EmilienMhello12:06
mjblackhello12:07
EmilienMansiwen: I haven't made it work12:14
*** trown|outtypewww is now known as trown12:15
*** pradk has joined #puppet-openstack12:15
*** ducttape_ has joined #puppet-openstack12:17
*** openstackgerrit has quit IRC12:18
*** ccamacho is now known as ccamacho|lunch12:18
*** openstackgerrit has joined #puppet-openstack12:18
*** dgurtner has joined #puppet-openstack12:19
*** dgurtner has joined #puppet-openstack12:19
*** amoralej is now known as amoralej|lunch12:23
*** jayg|g0n3 is now known as jayg12:23
*** jpich has quit IRC12:28
*** ducttape_ has quit IRC12:30
*** ducttape_ has joined #puppet-openstack12:30
*** dgurtner_ has joined #puppet-openstack12:32
*** dgurtner_ has quit IRC12:32
*** dgurtner_ has joined #puppet-openstack12:32
*** dgurtner has quit IRC12:33
*** ducttape_ has quit IRC12:37
*** jpena|lunch is now known as jpena12:37
dmsimard|ptomwhahaha: re: puppet-designate & monasca: https://bugs.launchpad.net/designate/+bug/1630666 :)12:38
openstackLaunchpad bug 1630666 in Designate "monasca-statsd is a non-optional import even when disabled" [Low,Triaged]12:38
dtantsurhey folks, do you have work session for the summit planned already? I've just realized I have a window at 11:50 on Thu, and there is a work session there.12:39
*** dmsimard|pto is now known as dmsimard12:40
*** markvoelker has joined #puppet-openstack12:43
*** xiaohua has quit IRC12:44
*** iberezovskiy is now known as iberezovskiy|off12:50
*** moravec has quit IRC12:53
*** jpich has joined #puppet-openstack12:58
*** chem has joined #puppet-openstack13:03
*** fultonj_ has joined #puppet-openstack13:04
*** jtriley has joined #puppet-openstack13:05
*** basilAB has quit IRC13:11
*** xiaohua has joined #puppet-openstack13:12
*** basilAB has joined #puppet-openstack13:14
*** jperry has joined #puppet-openstack13:14
*** morazi has joined #puppet-openstack13:23
*** StraubTW has joined #puppet-openstack13:24
*** b3nt_pin is now known as beagles13:25
*** ccamacho|lunch is now known as ccamacho13:26
*** beagles is now known as Guest4892713:26
*** morazi has quit IRC13:30
jaosoriorHey guys, are there currently issues with gate-puppet-ceilometer-puppet-beaker-rspec-ubuntu-xenial or gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial ?13:30
mwhahahaugh, why does rdo use *-dist.conf files :/13:30
mwhahahajaosorior: link?13:30
openstackgerritAlexey Deryugin proposed openstack/puppet-trove: Make swift_service_type configurable in guestagent.conf  https://review.openstack.org/38500013:31
jaosoriormwhahaha: https://review.openstack.org/#/c/384371/  http://logs.openstack.org/71/384371/3/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial/1111f14/13:31
mwhahahahttp://logs.openstack.org/71/384371/3/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial/1111f14/console.html#_2016-10-11_11_47_37_87273513:32
mwhahahalooks like connection issues with the apt configuration13:32
jaosoriorfunky13:32
*** amoralej|lunch is now known as amoralej13:33
jaosoriormwhahaha: and http://logs.openstack.org/66/384366/2/check/gate-puppet-ceilometer-puppet-beaker-rspec-ubuntu-xenial/a9d1f78/console.html13:33
mwhahahasame deal13:34
mwhahahalooks like it hung at the same spot doing the same thing13:34
jaosoriormwhahaha: I'll recheck them then. Thanks13:35
EmilienMjaosorior: yeah http://logs.openstack.org/71/384371/3/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial/1111f14/console.html#_2016-10-11_11_47_37_88031713:38
EmilienMit sounds like a transient issue to reach ceph repo13:38
*** morazi has joined #puppet-openstack13:43
EmilienMI see a lot of ssl issues13:44
EmilienMhttp://logs.openstack.org/63/384463/1/check/gate-puppet-ceilometer-puppet-beaker-rspec-ubuntu-xenial/8b271ea/console.html#_2016-10-11_10_32_28_25377613:44
*** StraubTW_ has joined #puppet-openstack13:46
EmilienMhttps://download.ceph.com/keys/release.asc looks down13:46
*** dprince has quit IRC13:47
EmilienMwell, http://download.ceph.com/ looks down13:47
*** dprince has joined #puppet-openstack13:47
EmilienMwhich causes all CI jobs running on Xenial to fail13:49
*** StraubTW has quit IRC13:50
dmsimardmwhahaha: what's the problem with -dist.conf files ?13:52
EmilienMdmsimard: have you seen the thread "[openstack-dev] [rpm-packaging][chef][puppet][salt][openstack-ansible][HA] Schema proposal for config file handling for services" ?13:53
mwhahahadmsimard: our assumptions around using os_service_default is broken if it's defined in a dist file13:53
mwhahahaI didn't know the dist file was a thing and I am sad13:53
EmilienMimho, it's too opinionated to add parameters in a dist file13:53
EmilienMwe should rely on what upstream provides13:53
EmilienMand fix upstream if something is not correct13:53
EmilienMok http://ceph.com/ is down too13:55
*** agrebennikov has joined #puppet-openstack13:56
dmsimardmwhahaha, EmilienM: not aware of the historical reasons of providing those, I'm not closed to the idea of getting them out13:57
dmsimardin most cases, the dist conf files don't really make sense anyway13:57
dmsimardEmilienM: no, I have not13:57
mwhahahathey are the 'defaults'13:57
claytonmwhahaha: I don't think it's really invalid, it's just different13:57
mwhahahathey actually define like db connections and stuff13:57
*** moravec has joined #puppet-openstack13:57
mwhahahaso they map to usually what are the required params anyway13:57
mwhahahawhich is probably why we've never noticed13:57
*** moravec has quit IRC13:58
*** jperry has quit IRC13:58
claytonanyone that doens't want that behavior can remove the -dist.conf files13:58
mwhahahaclayton: by default our puppet stance has been to fall back to the python defaults13:58
mwhahahawe shouldn't have to clean out packager's files to get that13:59
mwhahahait's already an annoyance when we have to do it for the apache files or deal with autostarting crap from deb13:59
mwhahahanow for redhat we also need to remember to nuke some random dist.conf?13:59
mwhahahameh13:59
claytonI think if an operator is using a distribution that does that, then they've implicitly accepted that the vendor knows better than upstream13:59
*** medberry is now known as med_13:59
*** dfisher has joined #puppet-openstack13:59
mwhahahayea that's totally not the case14:00
*** Guest48927 is now known as beagles14:00
mwhahahai don't think anyone asctually knows better14:00
mwhahahanot even upstream14:00
*** moravec has joined #puppet-openstack14:00
claytonmwhahaha: the real point of os_service_default was to get puppet out of the business of hard coding defaults14:00
claytonthe other parts of it was a side-effect14:00
mwhahaharight but packaging is doing that14:00
mwhahahaand they aren't doing it smartly either14:00
claytonsure, but that's a different issue, I don't see how the puppet modules can solve that issue14:00
*** ducttape_ has joined #puppet-openstack14:01
mwhahahai'm not trying to solve it with puppet modules14:01
mwhahahai'm trying to prevent packagers from doign this14:01
mwhahahajust fyi we can't clear out log_dir14:01
mwhahahabecause it's in the dist file14:01
claytonI agree with you that they shouldn't, but fundamentally it's kind of their and the operators problem14:01
claytonwhy would removing log_dir and defaulting to the value in the dist file cause an issue?14:02
mwhahahathe default value in dist is /var/log/heat14:02
mwhahahafor example in heat-dist.conf14:02
claytonwe already have a similar issue on ubuntu because it hard codes the log file paths in the startup scripts14:02
mwhahahaso to not log to a dir at all you need to completely remove it14:02
mwhahaharight this is just one example as to why these files are not a good idea14:03
claytonis that something people actually want to do, or just an example?14:03
mwhahahapeople actually want to do it (and it's an example)14:03
claytonso, personally I think the dist files are an improvement over the vendor shipping a huge config file full of settings, even if I have to override it14:04
claytonI'd rather the file that puppet manages only contain things that it put there14:04
mwhahahai think my issue is ordering around this14:04
claytonif I were deploying a distribution that did put dist files in place, I'd probably want a way to disable reading the dist files, but I'd leave them in place for reference.14:04
mwhahahawe've operated under the assumption that puppet is the source of truth for configuration14:04
mwhahahabut it's not when distros drop files in14:04
claytonI don't think that's true14:04
claytonThe way I think of it is that puppet is only managing things that the operator has explicitly asked it to, directly or indirectly.14:05
claytonthe purge config items pushes the puppet modules more in the direction you said, but it isn't on by default14:06
mwhahahawell here's the case where an operator has told it to configure something and it doesn't take because of the dist file14:06
mwhahahanot a complete purge but we are purging some values14:06
mwhahahavia os_service_default14:06
claytonso using your  example, how does an operator tell a module to remove log_dir from the config?14:07
mwhahahawhich breaks when the dist files actually contain it14:07
mwhahahasets it to os_service_default14:07
mwhahahaie remove it completely from the config files14:07
*** rcernin has quit IRC14:07
claytonI think that's a bad example, because log_dir isn't set to os_service_default14:07
mwhahahait used to be set it to false but that got broken a while ago14:07
mwhahahait is14:07
claytonhttps://github.com/openstack/puppet-nova/blob/e8a4765c624cc8b69ad9b0a7bd11202c76df4c34/manifests/logging.pp#L9914:08
mwhahahait's not by default but if a user wants to remove it they can't14:08
mwhahahaah you're right we define it, but we tell the user to use os_service_default to disbale it14:08
claytonI guess a user could set it to os_service_default14:08
mwhahahaso if they do explictly tell it to do something, it wouldn't work as designed14:08
mwhahahahttps://github.com/openstack/puppet-nova/blob/e8a4765c624cc8b69ad9b0a7bd11202c76df4c34/manifests/logging.pp#L2514:09
claytonI don't know if it's still the case, it used to be explicitly set to not os_service_default because services required it14:09
claytonI assume that's changed.14:09
mwhahahawe used to say false but oslo broke that at somepoint so false gets interpreted as a file or dir name14:09
mwhahahait's not required anymore as far as i know14:09
mwhahahaanwyay14:09
claytonmy take on this is that the dist files are roughly equivalent to the packager changing the defaults in the python code, just slightly cleaner14:10
claytonit's them saying "we know better than upstream, you need this set to something different"14:10
mwhahahamy concern is that feature and settings will creep into the dist files14:11
mwhahahaif you're relying on a configuration management tool, you'd expect it would do that for you14:11
mwhahahayou're not relying on the packaging for that info14:11
claytonmaybe so, but packagers pretty regularly do dumb things, at least in my experience.14:11
mwhahahawhich is even more of a reason why i don't want them playing in the config files14:11
mwhahahai'm not against a smart schema to allow people to do better things like a conf.d directory14:11
mwhahahai'm against packagers injecting configuration defaults unecessarily14:12
claytonsure, I could see where people not using the puppet modules might find that useful14:12
mwhahahait would be better to fall back to the dist file if no service.conf file exists14:12
claytonsure, I just don't see how much can be done for it.14:12
mwhahahaotherwise if say heat.conf exists don't use heat-dist.conf14:13
*** iurygregory_ has joined #puppet-openstack14:22
openstackgerritJavier Peña proposed openstack/puppet-openstacklib: [TEST DO NOT MERGE] Test for Packstack debugging on Puppet 4  https://review.openstack.org/38463714:36
*** moravec has quit IRC14:38
*** xiaohua has quit IRC14:41
*** xiaohua has joined #puppet-openstack14:41
*** xiaohua has quit IRC14:45
iurygregory_meeting in 15 minutes14:46
iurygregory_:D14:46
Vinshwill there be food served?14:46
iurygregory_Vinsh, only in Barcelona =P14:48
Vinsh-214:48
iurygregory_:O14:48
*** jperry has joined #puppet-openstack14:51
*** iberezovskiy|off is now known as iberezovskiy14:53
*** llua` is now known as llua14:59
*** llua has joined #puppet-openstack14:59
mwhahahashall we do a quick meeting or should we skip it15:01
mwhahahano real topics, just some past stuff15:01
iurygregory_i was wondering this XD15:01
iurygregory_nothing to discuss about summit or PTG?15:01
mwhahahai don't think so15:02
mwhahahawe'll just do a quick meeting15:02
openstackgerritMaksym Yatsenko proposed openstack/puppet-magnum: Change worker defaults to ::os_workers  https://review.openstack.org/38461915:04
*** jtomasek is now known as jtomasek|bbl15:05
openstackgerritIury Gregory Melo Ferreira proposed openstack/puppet-openstack-guide: Meeting #93  https://review.openstack.org/38505415:15
*** moravec has joined #puppet-openstack15:19
EmilienMhttp://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-east-1-cluster-service-disruption/15:24
EmilienMso this is why ceph.com is down15:25
*** jistr is now known as jistr|biab15:25
EmilienMI propose we disable ceph/ceph repos this afternoon if no change15:25
EmilienMit's blocking our CI15:25
*** fultonj_ has quit IRC15:25
EmilienMor make xenial jobs non voting15:25
*** _ducttape_ has joined #puppet-openstack15:25
Vinshso, is ceph.com down because ceph that it's hosted on is down!? :-015:27
EmilienMmost likely15:28
*** ducttape_ has quit IRC15:29
*** lucasagomes is now known as lucas-hungry15:32
jpenaanyone wants to help me debug the most awkward puppet issue I've ever seen? :)15:33
jpena(spoiler alert: it's puppet4-related)15:34
mwhahahajpena: what's the error?15:34
jpenamwhahaha, if I understood it correctly, the puppet-nova nova_config provider is trying to use openstackconfig's ini_setting as parent, instead of ruby as it should according to https://github.com/openstack/puppet-nova/blob/master/lib/puppet/provider/nova_config/openstackconfig.rb#L315:35
jpenawhat's worse, it only happens in the gate, I've seen 3 different people trying to reproduce it locally without success15:36
jpenaI can prepare an etherpad with a long explanation if it helps15:36
mwhahahayea that'd help alternatively do you have the raw logs somewhere?15:37
jpenamwhahaha, http://logs.openstack.org/87/383787/12/check/gate-packstack-integration-scenario001-tempest-centos-7/5a09c54/logs/packstack/latest/manifests/15.184.65.66_controller.pp.log.txt.gz is the main log15:37
jpenasearch for Nova_config[DEFAULT/debug], and you'll see a debug marker I added as a test with https://review.openstack.org/38463715:38
mwhahahanot sure how does nova_config have anything to do with the nova_flavor not working?15:40
jpenamwhahaha, that's because nova.conf is incorrectly configured. Nova fails to start, then nova_flavor can't contact any API server15:41
jpenahttp://logs.openstack.org/87/383787/12/check/gate-packstack-integration-scenario001-tempest-centos-7/5a09c54/logs/etc/nova/nova.conf.txt.gz is the generated nova.conf file, see how all configured options looks like arrays15:41
mwhahahathat's a new one15:41
EmilienMthe question is, how does it work in p-o-i and not in packstack?15:41
jpenaI'd love to know that15:42
jpenaalso, why does puppet-neutron (which does exactly the same as puppet-nova) work fine?15:42
*** moravec has quit IRC15:42
*** moravec has joined #puppet-openstack15:43
mwhahahawhat version of ini_file is it using?15:43
mjblackEmilienM,mwhahaha,jpena: was the nova_config updated recently for multiline?15:44
mwhahahawonder if puppet-nova needs https://github.com/openstack/puppet-neutron/blob/master/lib/puppet/provider/neutron_config/openstackconfig.rb#L10-L1315:44
mwhahahamjblack: no it's been there for a while15:45
mjblackwell15:45
jpenamjblack, the change merged late July15:45
mjblackhmmm15:45
mjblackbtw who is the maintainer of packstack?15:46
mwhahahajpena: what verison of inifile are you using in packstack?15:46
jpenamwhahaha, it's using inifile from master15:46
jpenamjblack, I'm one of the packstack maintainers15:46
jpenaimcsk8 (say hi!) is another one15:46
imcsk8hi!15:47
mjblack:D15:47
openstackgerritAlexey Deryugin proposed openstack/puppet-murano: Make Murano application provider use prefetch  https://review.openstack.org/37874115:47
mjblackso with the arrays, if the provider wasnt written right it will do that15:47
mwhahahajpena: we're using 1.6.015:47
jpenamwhahaha, the weirdest thing is that nova_config shouldn't ever use inifile, it should use the ruby provider of openstack_config15:47
mjblackor rather the type15:48
jpenamwhahaha, I can test inifile 1.6.0, let me try15:48
mwhahahajust to rule out inifile changes15:48
jpenalet's give it 20-30 mins, brb15:50
mwhahahathough i'm not seeing much changes bettwen 1.6.0 and master which would affect functionality15:50
EmilienMjpena: we are using latest tag of dependencies15:50
EmilienMjpena: our CI automatically bump Puppetfile to latest tags15:50
EmilienMhttps://github.com/puppetlabs/puppetlabs-inifile/releases15:50
mwhahahajpena: https://github.com/openstack/puppet-openstacklib/blob/master/lib/puppet/provider/openstack_config/ruby.rb#L59-L66 it might be using the right provider15:54
*** michapma_alt has quit IRC15:57
*** acabot has quit IRC15:58
*** moravec has quit IRC15:59
mjblackEmilienM: want a good laugh? http://logs.openstack.org/79/382379/4/check/gate-puppet-openstack-integration-4-scenario002-tempest-ubuntu-xenial/e4c68e4/logs/barbican/main.txt.gz#_2016-10-10_18_28_57_41615:59
EmilienMcan I cry?15:59
mjblackdepends16:00
mjblackhave you found the issue on that line? :P16:00
mjblackonce you do, you'll understand why it fails the volume encryption16:01
mjblack00000000-0000-0000-0000-000000000000 appears to be returned by barbican on error16:01
mjblackrequest: 404 Not Found - GET http://127.0.0.1:9311/v1/secrets/00000000-0000-0000-0000-00000000000016:02
*** jistr|biab is now known as jistr16:07
*** _ducttape_ has quit IRC16:09
*** ducttape_ has joined #puppet-openstack16:10
jpenamwhahaha, I don't get it. That bit is ensuring we're passing an array, but https://github.com/openstack/puppet-nova/blob/master/lib/puppet/type/nova_config.rb#L10 also sets the value to be an array. Or am I getting it wrong?16:10
*** _ducttape_ has joined #puppet-openstack16:21
*** ducttape_ has quit IRC16:24
*** jpich has quit IRC16:27
*** paramite has quit IRC16:29
*** dtantsur is now known as dtantsur|afk16:31
*** degorenko is now known as degorenko|afk16:32
*** jaosorior has quit IRC16:33
*** lucas-hungry is now known as lucasagomes16:34
*** dgurtner_ has quit IRC16:36
*** rwsu has quit IRC16:46
*** xarses_ has quit IRC16:46
*** derekh has quit IRC16:46
mwhahahajpena: that might be it but like you said neutron supports that and it worked :D16:56
jpenamwhahaha, yeah well... :). BTW, inifile 1.6.0 didn't do the trick. I'm checking the log files, but the issue stays16:58
mwhahahaso weird16:58
jpenais there any good way I could trace the puppet provider, so I can see where it comes from? Something like traceback in python16:59
mwhahahano idea, chem would probably know better16:59
*** moravec has joined #puppet-openstack17:00
mwhahahajpena: packstack captures all the modules right?17:01
mwhahahain the ci logs17:01
jpenamwhahaha, yes17:01
mwhahahai wonder if you have two nova_configs17:01
mwhahahaand the older one is the one loading17:01
chemjpena: do you mean interactive debugging session or something else ?17:01
mwhahahai really need to write something to pull down all the logs from openstack ci so you can just grep locally17:02
jpenachem; I mean a full backtrace17:02
EmilienMmwhahaha: like a single tarball?17:02
mwhahahaEmilienM: that would be ideal, but if not something you can just paste a url and it pulls all the files down17:03
EmilienMwell, a tarball can be implemented in one line17:03
EmilienMin the copy logs scripts17:03
mwhahahaEmilienM: to keep storage down, rather than duplicating the stuff in both exploded and unexploded forms17:03
mwhahahait's handy to be able to pass urls17:03
mwhahahabut sometimes I just want to grep all the files17:03
mwhahahaalso across projects :D17:04
chemjpena: hum ... I don't see the use case.  Do you have a example where you would like the provider to dump a full stacktrace ?17:04
mwhahahachem: he's trying to figure out why nova_config appears to be writing out using the ini_setting rather than the ruby provider17:05
jpenathat ^^ :)17:05
mwhahahasince the options in the config file are being written out as arrays17:05
mwhahahawhich makes it look like it's using the ini_settings rather than ruby17:05
chemjpena: ah, I see, you want the code path.  But show me the problem, I have some familiarity with this ini/ruby provider.  Just need a refresher.17:06
jpenachem: https://etherpad.openstack.org/p/Packstack-Puppet-4 is a summary17:07
chemjpena: (the only option here is to add Puppet.info("blah ${var}" inside the provider)17:07
jpenachem: ok, that's more or less what I'm trying :)17:07
*** xarses_ has joined #puppet-openstack17:07
chemjpena: you can add 'caller.join("\n")' to have the stacktrace inside your message, but I doubt it will be useful17:09
chemjpena: so you have a array output while you would like a string output, right ?17:10
*** iurygregory_ has quit IRC17:10
*** ayoung has joined #puppet-openstack17:10
jpenachem: I've just tried that. No, it shows ruby functions, but not the provider calls. Something like https://paste.fedoraproject.org/448689/20587014/17:11
*** ayoung has quit IRC17:12
chemjpena: the best way is to use 'gem "pry"' to the Gemfile, bundle install, and add dynamic breakpoint inside the provider with "binding.pry"17:13
*** ayoung has joined #puppet-openstack17:13
chemjpena: but I think I can sort this out ...17:13
chemjpena: I've got this problem ... a long time ago, I believe17:13
jpenachem: if you do, I'll owe you a few beers in Barcelona. I've been scratching my head for two days17:14
chemjpena: well for the trace, that what I though, it's kind of useless as it's so tangled with puppet that it's undecipherable.17:14
*** ayoung has quit IRC17:14
chemjpena: give me 1017:15
*** iurygregory has quit IRC17:16
*** ayoung has joined #puppet-openstack17:17
chemjpena: so well, I did something on this : https://review.openstack.org/#/c/354094/1/README.md17:20
chemjpena: I try to associate that with your current problem17:21
chemjpena: (note the binding.pry stuff is not doable at the gate)17:21
*** dgurtner has joined #puppet-openstack17:23
*** fultonj has quit IRC17:24
*** fultonj has joined #puppet-openstack17:25
*** AnarchyAo has joined #puppet-openstack17:27
*** iurygregory has joined #puppet-openstack17:29
*** mkoderer has quit IRC17:32
*** trown is now known as trown|lunch17:34
*** shardy has quit IRC17:38
*** ccamacho has quit IRC17:39
jpenachem: gotta go now, thanks for your help and I'll keep on troubleshooting ;)17:47
chemjpena: this is an interesting problem :)  I'm still spending some time on it, keep you updated17:48
jpenagreat, thanks!17:48
*** jpena is now known as jpena|off17:49
*** _ducttape_ has quit IRC17:51
*** ducttape_ has joined #puppet-openstack17:51
*** ducttape_ has quit IRC17:52
*** ducttape_ has joined #puppet-openstack17:52
*** iberezovskiy is now known as iberezovskiy|off17:56
*** amoralej is now known as amoralej|off17:59
*** dgurtner has quit IRC18:10
EmilienMthis think with ceph.com makes me think we should stop relying on ceph.com by default18:17
EmilienMand maybe use openstack mirrors or officials repos in ubuntu18:17
iurygregorywe have thoose options?18:18
*** ducttape_ has quit IRC18:21
*** ducttape_ has joined #puppet-openstack18:21
*** ducttape_ has quit IRC18:25
EmilienMiurygregory: not yet ;-) but I'm on it18:27
iurygregoryEmilienM, you rock o/18:27
dmsimardWhat is ::neutron::memcache_servers for ? Looks like overlap with keystone authtoken ? https://github.com/openstack/puppet-neutron/blob/master/manifests/init.pp18:32
dmsimardForgot to deprecate it ?18:33
dmsimardThe one from ::nova was properly deprecated: https://github.com/openstack/puppet-nova/blob/master/manifests/init.pp#L42818:34
dmsimardiurygregory: ^18:34
iurygregorydmsimard, looking18:35
* dmsimard working on upgrading from mitaka to newton18:36
dmsimardso far so good18:36
iurygregorydmsimard,  memcache_servers vs memcached_servers18:36
iurygregorythey should do something different?18:37
dmsimardiurygregory: https://github.com/openstack/puppet-neutron/blob/master/manifests/init.pp#L57818:37
dmsimardparameter is memcache_servers, config is memcached_servers :)18:37
iurygregorydamm it LOL18:37
dmsimardiurygregory: worth fixing and backporting that to stable/newton I think18:38
iurygregorydmsimard, yep we missed in the review https://review.openstack.org/#/c/351272/18:38
iurygregorydoing now18:39
iurygregoryif you don't mind =)18:39
EmilienMdmsimard: no18:40
EmilienMdmsimard: it's for servicegroup18:40
EmilienMlike in Nova18:40
EmilienMit's used to store the status of Neutron services18:40
dmsimardEmilienM: I don't understand18:40
iurygregoryhumm in nova the memcached_servers was used in keystone_authtoken not in DEFAULT18:42
iurygregorythat's why we deprecated18:42
iurygregoryhttps://github.com/openstack/puppet-nova/commit/94a6343b57d9f6ddc4f580a781449857983e9441#diff-60ae41fd0a31977447947f59940ee9a4L64618:42
iurygregoryso this option in neutron is used for something else like dmsimard18:42
dmsimardbut we can agree that if DEFAULT/memcached_servers is set, it'll be used as default if the one in keystone_authtoken is not yet, right ?18:42
EmilienMI'm not sure18:43
dmsimardlet me look at the neutron config18:43
EmilienMlet me find in code18:43
* dmsimard cloning neutron to generate sample config file T_T18:43
*** ducttape_ has joined #puppet-openstack18:43
EmilienMit seems like the option is not used anymore18:44
iurygregoryhttp://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/config-reference/source/samples/neutron/neutron.conf.sample18:45
*** trown|lunch is now known as trown18:46
iurygregoryso maybe we should deprecate =D18:46
dmsimardEmilienM, iurygregory: up to date sample http://paste.openstack.org/show/585381/18:46
dmsimardonly see it in keystone_authtoken18:47
iurygregoryyep18:47
iurygregoryand the code in init is for DEFAULT18:47
EmilienMlet's deprecate it!18:48
dmsimardsorry :P18:48
iurygregorywe will remove after Ocata?18:57
EmilienMyes18:57
iurygregoryack18:57
openstackgerritEmilien Macchi proposed openstack/puppet-ceph: repo: deploy ceph.com apt key only when required  https://review.openstack.org/38512419:03
iurygregorywaiting for local tests before push to gerrit =)19:08
*** ducttape_ has quit IRC19:13
*** xarses_ is now known as xarses19:14
*** johnmilton has quit IRC19:15
xarsesI thought ensure_packages() was supposed to avoid http://logs.openstack.org/36/384736/1/check/gate-puppet-openstack-integration-4-scenario002-tempest-ubuntu-xenial/eb478a3/console.html#_2016-10-10_23_19_16_950531 regarding https://review.openstack.org/#/c/384736/1/manifests/backend/swift.pp19:16
xarsesI guess I need to make it so they have the same name then?19:16
iurygregoryxarses, maybe use the same name and tags19:20
xarsesI need to add the new tag19:20
iurygregoryyou can add19:20
iurygregorybut also use openstack i think19:20
xarsesit has openstack already19:20
xarseshttps://github.com/openstack/puppet-swift/blob/master/manifests/client.pp19:21
xarsesif it's declared in swift19:21
iurygregorybut you are in glance no? so maybe  tag ['openstack', 'glance-support-package']19:22
*** jperry has quit IRC19:23
openstackgerritEmilien Macchi proposed openstack/puppet-ceph: repo: deploy ceph.com apt key only when required  https://review.openstack.org/38512419:24
*** moravec has quit IRC19:25
*** moravec has joined #puppet-openstack19:35
*** johnmilton has joined #puppet-openstack19:40
*** moravec has quit IRC19:43
*** johnmilton has quit IRC19:45
*** johnmilton has joined #puppet-openstack19:45
*** moravec has joined #puppet-openstack19:48
*** dprince has quit IRC19:48
openstackgerritTim Rozet proposed openstack/puppet-neutron: Changes OpenDaylight OVS 'provider_mappings' to allow an array type  https://review.openstack.org/38255119:52
openstackgerritIury Gregory Melo Ferreira proposed openstack/puppet-neutron: Deprecate memcache_servers  https://review.openstack.org/38513519:53
*** jperry has joined #puppet-openstack19:54
openstackgerritAndrew Woodward proposed openstack/puppet-glance: Ensure that swiftclient is installed  https://review.openstack.org/38473619:56
openstackgerritEmilien Macchi proposed openstack/puppet-ceph: repo: deploy ceph.com apt key only when required  https://review.openstack.org/38512419:58
openstackgerritTim Rozet proposed openstack/puppet-neutron: Changes OpenDaylight OVS 'provider_mappings' to allow an array type  https://review.openstack.org/38255119:59
dmsimardUpgraded mitaka puppet recipes to successfully deploy newton with minimal effort, gj :)20:11
*** jayg is now known as jayg|g0n320:17
*** moravec has quit IRC20:18
mwhahahayay?20:20
mwhahaha:D20:20
*** lucasagomes is now known as lucas-afk20:21
dmsimardThere was only one deprecation that was brutal20:25
dmsimardthe one with the ec2 removal from puppet-nova20:25
dmsimardunless mistaken that was outright removed without deprecation20:25
dmsimardI forgot to check though20:25
dmsimardor maybe it was something else, I forget now20:28
*** gfidente has quit IRC20:32
mwhahahayea we kinda missed the window on that one i think20:33
mwhahahakinda got everyone at the last minute20:33
xarsesanything we can do to reduce the cloning errors?20:36
xarseshttp://logs.openstack.org/36/384736/2/check/gate-puppet-glance-puppet-beaker-rspec-centos-7/71c4441/console.html#_2016-10-11_20_03_51_59373220:36
mwhahahanot really20:36
dmsimardyeah ultimately even if we carried them in /opt/git cache in the images, we need to fetch and that'd fail20:37
dmsimardand I don't know if r10k even supports fetching from git cache20:38
*** jtriley has quit IRC20:43
*** trown is now known as trown|outtypewww20:51
*** ducttape_ has joined #puppet-openstack20:54
*** michapma_alt has joined #puppet-openstack20:59
*** jperry has quit IRC21:15
*** ducttape_ has quit IRC21:16
openstackgerritDavid Moreau Simard proposed openstack/puppet-tempest: Fix config parameter name for volume-feature-enabled  https://review.openstack.org/38515221:17
dmsimardcolonwq: wanna rebase your patches with a Depends-On: ^ ? :)21:17
dmsimardor I can do it, hang on21:18
colonwqdmsimard, I can do that for all 3.21:18
openstackgerritDavid Moreau Simard proposed openstack/puppet-openstack-integration: Add cinder backup to scenario002  https://review.openstack.org/38108321:19
*** moravec has joined #puppet-openstack21:19
openstackgerritDavid Moreau Simard proposed openstack/puppet-openstack-integration: Add cinder::backup test to scenario0001  https://review.openstack.org/37715821:19
colonwqdmsimard, I am building a truth table for 004 to see what minimum conbination of p-o-i::tempest options will generate a working tepest config.21:20
dmsimardcolonwq: ^21:20
colonwqdmsimard, thanks.21:20
dmsimardcolonwq: all 3 ? did you also have that issue with scenario004 ?21:20
colonwqdmsimard, for 004, there is a combo of p-o-i::tempest arguments which will update part of the tempest packaging to a point where it runs.21:21
colonwqIt is a diff tempeset problem.21:21
dmsimardcolonwq: ok haven't had the chance to look at 004 yet.. got the low hanging cinder fruit :)21:23
dmsimardpretty confident that'll fix it21:23
colonwqdmsimard, thanks. I am wash/rinse/repeat my test with diff settings. I'll update the patch with a minimal set soon.21:25
dmsimardcolonwq: looks like all the centos7 errors in 004 are due to endpoint not found on cinder (???)21:27
dmsimardtempest::cinder_available is set to true in fixture but you don't deploy it21:27
dmsimardthe rest seems like it works relatively ok ?21:29
dmsimardI left a comment21:29
colonwqdmsimard, thanks for the comment. I have other config options where the tempest run fails. That run ran with a bunch of extra stuff turned on.21:31
dmsimardAh, I don't have historical context - my comment is on the latest patchset of course :)21:32
*** ducttape_ has joined #puppet-openstack21:32
colonwqI'll be back in a few hours. I have a boy scout meeting to take care of.21:32
dmsimardsee you later21:32
dmsimardmwhahaha: I'm really itching to refactor puppet-tempest, wish I had more time.. I'll probably do it bit by bit throughout ocata ..21:37
mwhahahak21:38
dmsimarddo you typically write specs for significant refactors ?21:39
mwhahahanot in puppet21:44
mwhahahaunless you're really going to hose backwards compatibility21:44
*** fultonj has quit IRC21:47
dmsimardok21:50
dmsimardman the mirror issues today all over the place :(21:51
openstackgerritEmilien Macchi proposed openstack/puppet-ceph: repo: deploy ceph.com apt key only when required  https://review.openstack.org/38512421:53
EmilienMCI looks broken http://logs.openstack.org/24/385124/3/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/a5855db/console.html#_2016-10-11_20_07_56_01895721:56
EmilienMdmsimard: ^21:56
dmsimardThere's a ton of network errors on OSIC nodes recently21:57
dmsimard#openstack-infra21:57
EmilienMdmsimard: ack, thanks21:58
*** agrebennikov has quit IRC22:01
*** zigo has quit IRC22:02
*** zigo has joined #puppet-openstack22:05
*** zigo is now known as Guest5063022:05
*** StraubTW_ has quit IRC22:09
*** johnmilton has quit IRC22:13
openstackgerritEmilien Macchi proposed openstack/puppet-ceph: repo: deploy ceph.com apt key only when required  https://review.openstack.org/38512422:14
* EmilienM still trying to bring our CI back alive22:14
dmsimardEmilienM: clarkb is working on isolating the issue22:16
EmilienMdmsimard: I'm dealing with ceph.com down22:16
EmilienMceph.com has been down for 12 hours22:16
EmilienMand of course, they host bug tracker, website, repo on the same node22:17
openstackgerritBen Nemec proposed openstack/puppet-heat: Make default domain id match actual default  https://review.openstack.org/38517922:17
*** ducttape_ has quit IRC22:36
openstackgerritBen Nemec proposed openstack/puppet-heat: Make default domain id match actual default  https://review.openstack.org/38517922:38
*** iurygregory_ has joined #puppet-openstack22:55
iurygregory_CI still red right?23:08
*** xarses has quit IRC23:10
EmilienMiurygregory_: yes23:14
EmilienMiurygregory_: updated ^23:14
openstackgerritEmilien Macchi proposed openstack/puppet-ceph: repo: deploy ceph.com apt key only when required  https://review.openstack.org/38512423:14
*** AnarchyAo has quit IRC23:17
*** iurygregory_ has quit IRC23:30
*** morazi has quit IRC23:32
*** xarses has joined #puppet-openstack23:34
*** iurygregory_ has joined #puppet-openstack23:41
openstackgerritEmilien Macchi proposed openstack/puppet-ceph: repo: deploy ceph.com apt key only when required  https://review.openstack.org/38512423:52
*** ducttape_ has joined #puppet-openstack23:56
*** jtriley has joined #puppet-openstack23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!