Friday, 2016-04-22

*** schwicht has quit IRC00:07
*** jamielennox is now known as jamielennox|away00:07
*** pjm6 has joined #openstack-ansible00:08
*** weezS has quit IRC00:10
openstackgerritSteve Lewis (stevelle) proposed openstack/openstack-ansible-os_gnocchi: Initial convergence testing  https://review.openstack.org/30488700:13
openstackgerritSteve Lewis (stevelle) proposed openstack/openstack-ansible-os_gnocchi: Initial convergence testing  https://review.openstack.org/30488700:14
*** markvoelker has joined #openstack-ansible00:19
*** asettle has quit IRC00:27
*** asettle has joined #openstack-ansible00:27
openstackgerritMerged openstack/openstack-ansible: Decrease MariaDB wait timeout  https://review.openstack.org/30782600:27
openstackgerritMerged openstack/openstack-ansible: DOCS: Update aio docs for Mitaka  https://review.openstack.org/30839700:28
openstackgerritMerged openstack/openstack-ansible: Docs: Add pip configuration removal to AIO re-deployment process  https://review.openstack.org/30873700:32
*** bbmbx has quit IRC00:36
*** pjm6 has quit IRC00:36
*** itlinux has quit IRC00:43
*** itlinux has joined #openstack-ansible00:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Update Ironic playbook inclusion for tftp network  https://review.openstack.org/30365900:58
*** iceyao has joined #openstack-ansible01:01
*** schwicht has joined #openstack-ansible01:03
*** can8dnSix has quit IRC01:06
*** schwicht has quit IRC01:08
*** itlinux has quit IRC01:13
*** iceyao_ has joined #openstack-ansible01:14
*** iceyao__ has joined #openstack-ansible01:16
*** iceyao has quit IRC01:17
*** iceyao_ has quit IRC01:18
*** sdake has quit IRC01:20
*** itlinux has joined #openstack-ansible01:25
*** itlinux has quit IRC01:28
*** asettle has quit IRC01:39
*** asettle has joined #openstack-ansible01:39
*** jamielennox|away is now known as jamielennox01:44
*** schwicht has joined #openstack-ansible01:48
*** weshay_food has quit IRC02:11
*** sdake has joined #openstack-ansible02:16
*** sdake has quit IRC02:20
*** sdake has joined #openstack-ansible02:20
*** itlinux has joined #openstack-ansible02:20
*** sdake_ has joined #openstack-ansible02:26
*** sdake has quit IRC02:29
*** jamielennox is now known as jamielennox|away02:32
*** markvoelker has quit IRC02:35
*** fishcried has joined #openstack-ansible02:37
*** fishcried has quit IRC02:43
*** kiranv_ has joined #openstack-ansible02:48
itlinuxhello all02:52
itlinuxI get this error can someone help out?02:52
itlinuxhttp://paste.openstack.org/show/495076/02:53
itlinuxwhat could the issue here? Thanks02:53
*** schwicht has quit IRC02:59
*** persia has quit IRC03:02
itlinuxanyone!!03:19
itlinuxthe only thing is that all of us are on the same boat here.. .. what goes around comes around I guess!03:21
*** iceyao has joined #openstack-ansible03:28
cloudnullitlinux: most folks are AFK right now and may be traveling seeing that the summit is next week03:29
cloudnulldo you still have the error handy?03:29
prometheanfirecloudnull: lost backlog?03:29
itlinuxyes I do cloudnull03:29
cloudnulli have the link03:29
cloudnullbut the error is above that03:29
prometheanfireah03:29
itlinuxyes03:29
cloudnullalso I'm assuming you're deploying an all in one?03:30
itlinuxI thought it was related to cpus so I added them..03:30
itlinuxyes aio03:30
*** iceyao__ has quit IRC03:31
*** persia has joined #openstack-ansible03:32
cloudnullthat print out is just a set of diagnostic information on just about everything we could think to look at.03:32
itlinuxI am not sure why it fails then..03:33
itlinuxlet me add more data to that doc.. one se03:33
cloudnullthere should be something above that03:33
itlinuxhttp://paste.openstack.org/show/495078/03:35
cloudnullcan you ssh to the keystone container (or any container for that matter? )03:35
itlinuxhere is the major output03:35
*** markvoelker has joined #openstack-ansible03:36
itlinuxI do not see to have docker anything03:36
cloudnullno docker,03:37
cloudnulllxc containers03:37
itlinuxaccording to the docs just 3 command really to do the installation so I have an ubuntu box with all updates ..03:37
itlinuxok one sec03:37
cloudnullis it ubuntu 14.04 ?03:37
itlinuxthere is no lxc03:37
cloudnulldo you have the output above "FATAL: all hosts have already failed -- aborting" ?03:38
itlinuxinstalling lxc now.. shall see03:38
itlinuxyes one sec03:38
itlinuxhere is one 172.29.236.100> ESTABLISH CONNECTION FOR USER: root03:39
itlinux<172.29.236.100> REMOTE_MODULE modprobe name=scsi_dh03:39
itlinux<172.29.236.100> EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=120 172.29.236.100 /bin/sh -c 'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python'03:39
itlinuxfailed: [aio1] => (item=scsi_dh) => {"changed": false, "failed": true, "item": "scsi_dh", "name": "scsi_dh", "params": "", "state": "present"}03:39
itlinuxmsg: modprobe: FATAL: Module scsi_dh not found in directory /lib/modules/4.4.0-21-generic03:39
itlinuxor I can put in paste if you wnat..03:39
cloudnullah.03:39
cloudnullthat works03:39
itlinuxlooks like modprobe: FATAL: Module scsi_dh not found in directory /lib/modules/4.4.0-21-generic03:40
cloudnullscsi_dh is not found when using kernel > 4.403:40
itlinuxis the other error..03:40
cloudnullwhat release are you deploying ?03:40
itlinuxubuntu 16 and mitaka03:40
*** asettle has quit IRC03:40
*** markvoelker has quit IRC03:41
cloudnullwe dont support ubuntu 16.04 at this time03:41
itlinuxok.. I will try the 14 version thanks.. I and i will install lxc03:41
itlinuxshall see.. thanks03:41
itlinuxI let you know cloudnull03:41
cloudnullthe systemd init system is incompatible in addition to package names/kernel modules have been removed / renamed03:42
cloudnull1604 support is coming though03:42
cloudnullits just not there yet03:42
itlinuxk03:42
itlinuxthe problem I have seen the same issue on the 14.0403:42
cloudnullas for installing lxc the deployment bits will take care of that03:42
*** asettle has joined #openstack-ansible03:42
cloudnullhttps://github.com/openstack/openstack-ansible-openstack_hosts/commit/4fe07928889e203b15bea908711f647a74284bff03:42
itlinuxso i wonder what it could be..03:42
cloudnullthat patch fixes that kernel issue03:42
cloudnullbut its in master right now03:43
cloudnulli dont think the fix has been backported to mitaka03:43
itlinuxok03:43
cloudnullscsi_dh was removed in 4.403:43
itlinuxso should I try this patch with the 4.4 or just go back to 14.04?03:44
cloudnullyou can pull that in and try again03:44
itlinuxhere is the issue I had with the 14.04 let me see if I can find the paste agin...03:44
cloudnullif youre just kicking the tires, I'd recommend deploying ubuntu 14.04 w/ kernel 4.2 using stable/mitaka03:45
cloudnullif you're wanting to test our Master (newton) you can deploy ubuntu 14.04 w/ any kernel you like03:45
cloudnullmy general workflow is to kick a 14.04 vm, install git and tmux, start tmux, clone openstack ansible and run the ./scripts/gate-check-commit.sh03:47
cloudnullthat will deploy a full AIO w/ images and networks built for it as well as test the environment03:47
itlinuxI have a 14.04 version shell see..03:49
itlinuxrunning that now03:49
itlinuxlet see if that works..03:50
itlinuxcool03:50
itlinuxhere is where it fails..03:51
itlinuxlet me put this on paste03:51
itlinuxhttp://paste.openstack.org/show/495079/03:52
itlinuxstill failing on the same part this is ubuntu 14.0403:52
itlinuxthe lxc-ls03:53
itlinuxroot@ubuntu:/opt/openstack-ansible#03:53
itlinuxis empty03:53
coolj_itlinux: looks like you have a syntax error04:01
coolj_ERROR: Syntax Error while loading YAML script, /etc/openstack_deploy/user_conf_files.yml04:01
itlinuxok let me see if I can copy that there..04:01
coolj_i think it's complaining that you have something before the --- at the top04:02
itlinuxI just removed it.. let me try again..04:03
*** ozialiendoze has quit IRC04:05
itlinuxstill dies at the same place..04:06
itlinux- [ XEN Server Information ] -----------------------------------------04:06
itlinux----------------------------------------------------------------------04:06
itlinuxNo xenstore Information04:06
itlinux++ info_block 'Error Info - 265' 004:06
itlinux++ echo ----------------------------------------------------------------------04:06
itlinux----------------------------------------------------------------------04:06
itlinux++ print_info 'Error Info - 265' 004:06
itlinux++ PROC_NAME='- [ Error Info - 265 0 ] -'04:06
itlinux++ printf '\n%s%s\n' '- [ Error Info - 265 0 ] -' --------------------------------------------04:06
itlinux- [ Error Info - 265 0 ]04:06
*** ozialien has joined #openstack-ansible04:06
*** thorst has quit IRC04:08
*** thorst has joined #openstack-ansible04:08
*** yatin has joined #openstack-ansible04:09
*** ozialien has quit IRC04:10
*** ozialien10 has joined #openstack-ansible04:11
*** yatin_ has joined #openstack-ansible04:12
*** b3rnard0 is now known as b3rnard0_away04:12
*** yatin has quit IRC04:15
*** thorst has quit IRC04:16
coolj_itlinux: i am not familiar with deploying osa on xen and i gotta run, good luck on your adventure!04:17
itlinuxit's on a virtualbox04:17
itlinuxso no osa or xen..04:17
itlinux:)04:17
*** ChrisBenson has quit IRC04:25
*** ChrisBenson has joined #openstack-ansible04:25
itlinuxso after i snapshot back and run the bootstrap ansible after that I got  ./scripts/gate-check-commit.sh this and it's running the installation now04:28
itlinuxit went further than before04:28
itlinuxI spoke too soon..04:28
itlinuxfailed agian same place..04:28
*** ozialien10 has quit IRC04:28
*** weezS has joined #openstack-ansible04:31
*** jayc_ has joined #openstack-ansible04:36
*** markvoelker has joined #openstack-ansible04:37
*** asettle has quit IRC04:41
*** markvoelker has quit IRC04:42
*** asettle has joined #openstack-ansible04:51
*** asettle has quit IRC04:55
*** saneax_AFK is now known as saneax04:56
*** jayc_ is now known as jayc04:58
*** fishcried has joined #openstack-ansible05:00
itlinuxhere is one more error05:02
itlinuxstderr:05:02
itlinuxFile /etc/apparmor.d/lxc-openstack not found, skipping...05:02
itlinuxfile (/sys/class/net/lxcbr0/bridge/bridge_id) is absent, cannot continue05:02
itlinuxthis is on 14.0405:02
itlinuxTASK: [lxc_hosts | Download lxc cache(s)]05:02
itlinuxshall see..05:02
*** shausy has joined #openstack-ansible05:06
*** hybridpollo has quit IRC05:08
*** wangqwsh has quit IRC05:10
*** chhavi_ has joined #openstack-ansible05:10
*** thorst has joined #openstack-ansible05:15
*** john51 has joined #openstack-ansible05:16
itlinuxyahoo! I passed that poin!05:16
*** john51_ has quit IRC05:18
*** thorst has quit IRC05:22
*** yatin has joined #openstack-ansible05:22
*** admin0 has joined #openstack-ansible05:25
*** yatin_ has quit IRC05:25
*** chhavi_ has quit IRC05:26
*** chhavi_ has joined #openstack-ansible05:26
*** admin0 has quit IRC05:28
*** yatin has quit IRC05:33
*** bbmbx has joined #openstack-ansible05:50
*** persia has quit IRC06:00
*** tlbr has quit IRC06:05
*** tlbr has joined #openstack-ansible06:05
*** sdake_ is now known as sdake06:08
*** intr1nsic has quit IRC06:10
*** persia has joined #openstack-ansible06:11
*** intr1nsic has joined #openstack-ansible06:13
*** kiranv_ has quit IRC06:15
*** weezS has quit IRC06:18
itlinuxhi I see this now  lxc-info --name aio1_horizon_container-8ec2db6506:19
itlinuxName:           aio1_horizon_container-8ec2db6506:19
itlinuxState:          RUNNING06:19
itlinuxPID:            2916706:19
itlinuxIP:             10.255.255.206:19
itlinuxIP:             172.29.237.6306:19
itlinuxCPU use:        3.59 seconds06:19
itlinuxBlkIO use:      12.79 MiB06:19
itlinuxMemory use:     17.62 MiB06:19
itlinuxKMem use:       0 bytes06:19
itlinuxLink:           8ec2db65_eth006:19
itlinux TX bytes:      4.92 KiB06:19
itlinux RX bytes:      49.67 KiB06:19
itlinux Total bytes:   54.59 KiB06:19
itlinuxLink:           8ec2db65_eth106:19
itlinux TX bytes:      40.15 KiB06:19
itlinux RX bytes:      151.52 KiB06:19
itlinux Total bytes:   191.67 KiB06:19
itlinuxbut the curl to either ip does not work..06:19
itlinuxConnection refused06:19
itlinuxsuggestions?06:19
itlinuxthanks06:19
*** bbmbx has quit IRC06:20
*** thorst has joined #openstack-ansible06:20
*** thorst has quit IRC06:27
*** javeriak has joined #openstack-ansible06:29
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Update sources-branch-updater to handle release note copying  https://review.openstack.org/30917906:31
*** mikelk has joined #openstack-ansible06:33
*** sdake has quit IRC06:35
*** sdake has joined #openstack-ansible06:42
*** itlinux has quit IRC06:42
*** sdake has quit IRC06:46
*** intr1nsic has quit IRC06:47
*** intr1nsic has joined #openstack-ansible06:49
*** yatin has joined #openstack-ansible06:54
*** yatin_ has joined #openstack-ansible06:55
*** yatin has quit IRC06:59
*** intr1nsic has quit IRC06:59
*** yatin has joined #openstack-ansible06:59
*** yatin_ has quit IRC07:00
*** sacharya has quit IRC07:01
*** admin0 has joined #openstack-ansible07:03
*** karimb has joined #openstack-ansible07:05
admin0good morning all07:10
*** intr1nsic has joined #openstack-ansible07:10
*** thorst has joined #openstack-ansible07:24
*** Andrew_jedi has joined #openstack-ansible07:25
Andrew_jediHello folks, keystone has v3 admin endpoint and v2 internal and public endpoint? Is this right?07:26
-openstackstatus- NOTICE: Logs are failing to be uploaded causing jobs to be marked as UNSTABLE. We are working on repairing the log filesystem and will update when ready. Please do not recheck before then.07:27
*** ChanServ changes topic to "Logs are failing to be uploaded causing jobs to be marked as UNSTABLE. We are working on repairing the log filesystem and will update when ready. Please do not recheck before then."07:27
*** yatin has quit IRC07:30
*** Oku_OS-away is now known as Oku_OS07:31
*** thorst has quit IRC07:31
*** furlongm_ has joined #openstack-ansible07:32
*** furlongm has quit IRC07:33
*** admin0_ has joined #openstack-ansible07:38
*** admin0 has quit IRC07:39
*** admin0_ is now known as admin007:39
*** jayc has quit IRC07:42
*** Andrew_jedi has quit IRC07:44
evrardjpgood morning07:53
admin0good morning07:57
*** fishcried has quit IRC07:57
admin0evrardjp:  i am going to try the ssl patch again .. are these settings enough? https://gist.github.com/a1git/a9a0a0c7db8f40c9302705a0f22047de07:58
admin0expected output: horizon + all public endpoints behind SSL07:58
evrardjpI'm currently testing with and without keystone_public_endpoint: "{{ keystone_service_publicuri }}"07:58
admin0ok07:58
admin0then i will wait for your test results :)07:58
*** iceyao has quit IRC07:58
evrardjpI think without should be fine07:58
evrardjpbut I had some glitches with clients07:59
evrardjpand I had some redirections error with07:59
evrardjpbut I'm the only one to have that apparently07:59
evrardjpis your certificate a perfectly valid certificate?07:59
admin0yep08:00
admin0a wildcard cert that is valid for the next 2 years08:00
evrardjpbeurk wildcard08:00
*** iceyao has joined #openstack-ansible08:00
evrardjp:p08:00
admin0what is that word :D ?08:00
evrardjpnothing :p08:01
*** sacharya has joined #openstack-ansible08:01
admin0say like cloud101.admin0.com .. and cloud101int.admin0.com .. where the int is mapped to a private ip and cloud is mapped to the public one08:01
admin0plus with one card, you can do c101 c101int .. <insert anything in there> :D08:02
evrardjpI understand what it is :D08:02
*** yatin has joined #openstack-ansible08:02
admin0:D08:03
admin0yeah .. silly me08:03
*** sacharya has quit IRC08:06
*** javeriak has quit IRC08:09
*** Andrew_jedi has joined #openstack-ansible08:22
*** yatin has quit IRC08:22
*** thorst has joined #openstack-ansible08:29
*** yatin has joined #openstack-ansible08:30
*** Andrew_jedi has quit IRC08:32
*** Andrew_jedi has joined #openstack-ansible08:33
*** thorst has quit IRC08:36
*** fishcried has joined #openstack-ansible08:39
joker_odyssey4me : I am getting following error on galera container. It can not install mysql, memcache and pycrypto.  https://gist.github.com/celikmustafa89/b9ef6599364ef223887cebd996c8c54708:46
admin0evrardjp: done done ?08:46
admin0which config should i try ?08:46
admin0with or without the endpoint ?08:46
evrardjpit's still building08:46
evrardjpit takes time08:46
admin0ok08:46
evrardjpit just finished the creation of containers right now08:47
evrardjpthe rest should be fast08:47
evrardjpexcept the repo ofc08:47
*** persia has quit IRC08:48
joker_odyssey4me : I installed mysql,memcache and pycrypto using easy_install by attaching the galera container.  It passes the following error and but now getting this one https://gist.github.com/celikmustafa89/5c219e7a1abca1dc95e50f0dcf10fd2a . It cannot restart the mysql. Can you help me? thanx08:49
*** fishcried has quit IRC08:53
*** persia has joined #openstack-ansible08:53
odyssey4meAndrew_jedi nope - all endpoints are v3 in the catalog, but v2 is available in the container, it's just not advertised in the catalog08:53
odyssey4mejoker_ the first issue you hit is the most important - working around it the way you've done doesn't help matters08:54
odyssey4mejoker_ the http 404 errors while collecting the python packages tells me that your LB is directing requests to a dead repo server08:55
odyssey4mejoker_ what release are you trying to install?08:55
odyssey4mebranch/tag?08:55
Andrew_jediodyssey4me: Ahhh, so this is correct? http://paste.openstack.org/show/495096/08:56
odyssey4meAndrew_jedi for a new environment, no08:57
admin0Andrew_jedi: this is what I get for a new one — https://gist.github.com/a1git/a56c850f9a9141c44807b666f90b2b3008:57
admin0Andrew_jedi: https://gist.githubusercontent.com/a1git/a56c850f9a9141c44807b666f90b2b30/raw/9700a77260bd0a6a1be8c59901f26b7f8dd696f8/gistfile1.txt08:57
odyssey4mefor an upgraded environment the endpoints are not changed/edited - so it could be anything you like and you'd have to update them manually08:57
Andrew_jediodyssey4me: All i did was adding 3 more compute nodes to a kilo setup08:57
odyssey4meAndrew_jedi do you happen to be using Kilo, or RPC?08:58
admin0kilo ^^08:58
*** persia has quit IRC08:58
joker_odyssey4me mitaka08:59
odyssey4meI vaguely recall that we might have done that mix back in Kilo - but it's been a long time since I have installed Kilo (over a year now I think)08:59
Andrew_jediadmin0 odyssey4me : # git branch * kilo08:59
joker_odyssey4me stable/mitaka08:59
odyssey4mejoker_ ok, if you want to test Mitaka today, then the best way to achieve the right results is to checkout the head of the branch, not the 13.0.0 tag - 13.0.0 was a pre-release based on RC code, and the current head of the branch will be the 13.0.1 tag and our proper Mitaka release09:00
odyssey4mejoker_ so if you checkout stable/mitaka you shouldn't be hitting any issues except those that are related to your locale09:01
*** tiagogomes has joined #openstack-ansible09:01
odyssey4meAndrew_jedi yeah, actually the admin endpoint being v3 makes sense for Kilo09:03
odyssey4mein Liberty we shifted to all v3 endpoints09:03
odyssey4mebut Kilo was still a mix09:03
odyssey4meOpenStack didn't support v3 endpoints for everything in Kilo - some services broke.09:03
Andrew_jediodyssey4me: Ahh, that's the case...thanks!09:04
*** electrofelix has joined #openstack-ansible09:05
odyssey4memattt hughsaunders would you mind reviewing these doc backports? https://review.openstack.org/309224 / https://review.openstack.org/30922309:05
Andrew_jediodyssey4me: I was having a flapping agents/services issue, i thought it could be related to different endpoint versions09:05
odyssey4meAndrew_jedi nope, that's more likely due to time being off, or the rabbitmq cluster having a wobble09:06
Andrew_jediodyssey4me: Yeah, i already investigated rabbitmq, but i haven't checked time, good point09:08
matttodyssey4me: i'll review once gate jobs are operational again09:10
odyssey4megate jobs are broken?09:10
matttlog issues which are causing jobs to be marked UNSTABLE09:11
mattthaven't seen an update to say it's fixed :(09:11
*** persia has joined #openstack-ansible09:12
matttactually i can hit logs on a job now, maybe it's back09:12
odyssey4memattt ah, I see the -infra backscroll... corrupt disks again :/09:14
odyssey4meit's amazing how often this happens in a Cloud :p09:14
*** yatin has quit IRC09:15
*** javeriak has joined #openstack-ansible09:22
*** xek has joined #openstack-ansible09:23
*** openstackstatus has quit IRC09:24
*** openstackstatus has joined #openstack-ansible09:24
*** ChanServ sets mode: +v openstackstatus09:24
*** furlongm_ has quit IRC09:28
*** furlongm has joined #openstack-ansible09:29
*** tiagogomes has quit IRC09:30
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible: Docs: Cleanup page to configure to docs standards  https://review.openstack.org/30931609:31
*** thorst has joined #openstack-ansible09:36
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: [WIP] Use tempest for testing  https://review.openstack.org/30156609:39
*** thorst has quit IRC09:42
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: [WIP] Use tempest for testing  https://review.openstack.org/30156609:43
*** spotz_zzz is now known as spotz09:43
*** tiagogomes has joined #openstack-ansible09:43
*** karimb has quit IRC09:47
openstackgerritMerged openstack/openstack-ansible-openstack_hosts: Adding Vagrantfile for local developer testing  https://review.openstack.org/30913009:50
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible: Docs: Removed multiple warnings appearing on first make html  https://review.openstack.org/30932709:52
spotzpeeks in before heading back to bed for an hour09:56
*** zhangjn has quit IRC10:02
*** sacharya has joined #openstack-ansible10:03
*** intr1nsic has quit IRC10:04
*** zhangjn has joined #openstack-ansible10:07
*** sacharya has quit IRC10:07
*** pjm6 has joined #openstack-ansible10:09
pjm6mornings10:10
*** karimb has joined #openstack-ansible10:15
*** krotscheck_dcm is now known as krotscheck10:19
*** intr1nsic has joined #openstack-ansible10:24
*** yatin has joined #openstack-ansible10:27
*** yatin has quit IRC10:31
admin0\o10:32
*** Andrew_jedi has quit IRC10:33
*** Andrew_jedi has joined #openstack-ansible10:34
*** zhangjn has quit IRC10:36
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: [WIP] Use tempest for testing  https://review.openstack.org/30156610:39
*** thorst has joined #openstack-ansible10:40
*** iceyao has quit IRC10:42
*** zhangjn has joined #openstack-ansible10:42
*** thorst has quit IRC10:46
*** thorst has joined #openstack-ansible10:46
-openstackstatus- NOTICE: Log server has been repaired and jobs are stable again. If necessary please recheck changes that have 'UNSTABLE' results.10:50
*** mikelk has quit IRC10:51
*** Andrew_jedi has quit IRC10:53
*** Andrew_jedi has joined #openstack-ansible10:57
*** johnmilton has joined #openstack-ansible11:00
*** jiteka has quit IRC11:09
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: [WIP] Use tempest for testing  https://review.openstack.org/30156611:15
*** thorst has quit IRC11:21
odyssey4megit-harry FYI https://review.openstack.org/30917011:22
*** thorst has joined #openstack-ansible11:22
git-harryodyssey4me: does Liberty introduce any new containers?11:23
odyssey4megit-harry I don't think so.11:24
odyssey4meIt's been a while since I looked though, so I can't be sure.11:24
git-harryodyssey4me: okay, that should be fine then. I assume that patch is to cater for Juno -> Kilo upgrades that were done prior to a similar patch being added to Kilo.11:25
git-harryThat patch didn't work because it was run before the new containers had been created.11:25
git-harryWhich I had to fix in https://review.openstack.org/#/c/302333/11:26
odyssey4megit-harry yep, I expect so - I just wanted to make sure you were aware of it as it's an upgrade patch which hasn't been included in the testing until now11:29
odyssey4meand the last one that did that broke things11:29
*** yatin has joined #openstack-ansible11:29
*** thorst has quit IRC11:30
*** yatin_ has joined #openstack-ansible11:31
git-harryOkay, ta. Thankfully our Kilo to Liberty testing should pick this up if it causes a problem when we next bump the SHA.11:32
*** yatin has quit IRC11:33
*** yatin_ has quit IRC11:37
*** markvoelker has joined #openstack-ansible11:42
*** Oku_OS is now known as Oku_OS-away11:43
odyssey4megit-harry automated testing for the win!11:46
*** jayc has joined #openstack-ansible11:46
*** thorst has joined #openstack-ansible11:47
openstackgerritMerged openstack/openstack-ansible: Remove old MariaDB repos during upgrades  https://review.openstack.org/30917011:47
*** markvoelker has quit IRC11:47
*** retreved has joined #openstack-ansible11:49
*** javeriak has quit IRC11:56
*** weshay has joined #openstack-ansible11:57
*** jayc has quit IRC11:57
openstackgerritMerged openstack/openstack-ansible-os_gnocchi: Initial convergence testing  https://review.openstack.org/30488711:58
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: Use tempest for testing  https://review.openstack.org/30156612:00
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: Use tempest for testing  https://review.openstack.org/30156612:01
matttfinally got the tempest functional test working12:01
*** pjm6 has quit IRC12:05
*** flwang has quit IRC12:07
*** spotz is now known as spotz_zzz12:07
*** pjm6 has joined #openstack-ansible12:07
*** psilvad has joined #openstack-ansible12:10
*** chhavi_ has quit IRC12:10
*** asettle has joined #openstack-ansible12:12
*** asettle has quit IRC12:13
*** pjm6 has quit IRC12:16
*** pjm6 has joined #openstack-ansible12:18
odyssey4memattt w00t!12:19
matttodyssey4me: was the default infra iptables rules that were tripping me up :P12:20
odyssey4memattt ah yes, forgot about those12:20
*** psilvad has quit IRC12:21
*** psilvad_ has joined #openstack-ansible12:21
*** psilvad_ is now known as psilvad12:21
*** chhavi_ has joined #openstack-ansible12:23
*** karimb has quit IRC12:23
*** psilvad has quit IRC12:26
*** karimb has joined #openstack-ansible12:28
*** markvoelker has joined #openstack-ansible12:28
mhaydenbuenos dias12:29
*** Oku_OS-away is now known as Oku_OS12:30
*** intr1nsic has quit IRC12:30
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible: Change keystone admin/internal insecure flags  https://review.openstack.org/30915512:32
*** pjm6 has quit IRC12:33
admin0cloudnull:  around ?12:35
*** flwang has joined #openstack-ansible12:35
*** Andrew_jedi has quit IRC12:37
admin0evrardjp:  odyssey4me : https://gist.github.com/a1git/f3f1c684d1c8e62b1e9575fbf69e1955  — where am I going wrong ?12:38
evrardjpadmin0 when setting keystone_public_endpoint: "{{ keystone_service_publicuri }}" that's the issue I got yesterday12:40
admin0evrardjp: i will retry with removing that variable, and then running haproxy and keystone playbooks and horuzin playbooks12:40
evrardjpyou could12:41
evrardjpyou'll have other issues I think12:41
admin0then i will wait for a solution :D12:42
admin0no idea what that solution is ..12:42
admin0so its either keystone fail or horizon fail for SSL .. not setting that fails keystone ..12:43
*** Andrew_jedi has joined #openstack-ansible12:43
evrardjpthe issue is mainly in the client when this variable is not set, I think12:44
evrardjpbut I'd like to have time to focus on the issue12:44
evrardjpand really find what's causing this12:44
*** mkrish004c has joined #openstack-ansible12:45
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: Use tempest for testing  https://review.openstack.org/30156612:47
*** psilvad has joined #openstack-ansible12:47
evrardjpadmin0 don't wait12:47
mkrish004cHi Guys i am trying to set ceph via OSA. I am not able to integrarte with ceilometer, is there any module for it ? ceilometer.compute.pollsters.disk libvirtError: internal error: missing storage backend for network files using rbd protocol.12:47
evrardjptry again without the variable and tell me what you hit12:47
evrardjp :D12:47
evrardjpcloudnull has its system working perfectly fine12:47
evrardjpthere is something we need to reproduce12:48
mkrish004c ERROR ceilometer.compute.virt.inspector [-] Unable to load the hypervisor inspector: No module named config12:48
mkrish004cafter i upgrade ceilometer client, got this above error12:48
admin0evrardjp: will try:)12:51
* admin0 is on it12:51
evrardjpthanks admin012:53
mhaydenmattt: would you want to backport this? https://review.openstack.org/#/c/304096/12:57
*** fxpester has joined #openstack-ansible12:58
*** busterswt has joined #openstack-ansible12:58
matttmhayden: sure12:58
mhaydeni mashed the easy button in gerrit but it looks like there are issues merging it with mitaka12:58
matttmhayden: did you test it?  bjoern left some feedback in that review but i didn't quite understand it12:59
mhaydeni haven't yet -- but i was confused by the feedback too12:59
matttdo you want ot validate it before we backport?12:59
*** mkrish004c has quit IRC12:59
odyssey4memkrish004c I don't know - you may have to verify that it's even possible in general. If it is and you can find out how, then we can help you configure it to make it work. :)13:00
*** Andrew_jedi has quit IRC13:00
odyssey4memhayden you're likely missing another patch that needs backporting first13:00
odyssey4memhayden welcome to why stable branches should be stable ;)13:01
odyssey4mebackporting is a PITA13:01
odyssey4meI would guess that with the security role you'll probably want to pro-actively backport absolutely everything that merges - but you'll have to figure out a backport policy that you're comfortable with.13:02
*** Andrew_jedi has joined #openstack-ansible13:02
mhaydenmakes sense13:02
mhaydeni'm also trying to figure out the next goal there13:03
mhaydenUbuntu 16.04 support is key13:03
*** karimb has quit IRC13:03
mhaydenbut the RHEL 7 stig is available now, and it would be good to rebase on that (and add centos 7 support along the way)13:03
odyssey4mePersonally I'm going for a rather general notion of backporting proactively is ok until milestone 1 - after that the backports are on request only. Once the next release is issued, the n-2 release goes into at the very least a critical patch only phase13:03
odyssey4memhayden time for you to recruit some help13:04
odyssey4memaybe you can find someone who wants to work on the role with you and who wants to use the role for RHEL systems?13:04
mhaydenhopefully so!13:05
mhaydenperhaps my talk next week will bring out some folks13:05
odyssey4methe project in general can assist with the Ubuntu 16.04 support because we all care about that for all integrated roles - it's a key deliverable for this cycle13:05
*** shausy has quit IRC13:05
odyssey4memhayden maybe pabelanger can rustle up a contact for us13:06
odyssey4memhayden check out the bottom of https://etherpad.openstack.org/p/openstack-ansible-multi-os-support and perhaps reach out to a few folks from there13:07
mhaydenOOOOOOOOORLY13:07
fxpester"Failed to connect to git.openstack.org port 443: Connection timed out" - openstack failed ? openstack-ansible can`t get apt_package_pinning13:07
odyssey4memhayden I'm thinking that perhaps permalac and oneswig might be good people to bring in - the security role is nice and easy to work with so it'll be a good introductory place to work with OSA13:08
mhaydenalrighty13:08
mhaydenodyssey4me: i'll add that to my list --today is devoted to slides :|13:08
odyssey4mefxpester oh dear, give it a few mins and retry - I would guess that OpenStack-Infra's having a bad day13:08
odyssey4memhayden oh yes, and ggillies_ was keen to help - as I recall he's in Aus13:09
*** iceyao has joined #openstack-ansible13:10
fxpesterworld is too fast novadays... within this 5 mins ansible can become some oldfashion tool, like Perl :D13:11
*** jayc has joined #openstack-ansible13:12
pabelangerodyssey4me: mhayden: when is the talk?13:12
openstackgerritBjoern Teipel proposed openstack/openstack-ansible-os_swift: Remove XFS filesystem from the daily mlocate cron job  https://review.openstack.org/30803613:12
mhaydenTuesday @ 11:1513:12
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_glance: Ansible 2.x - Address deprecation warning of bare variables  https://review.openstack.org/30915013:13
odyssey4memhayden a schedule link or it won't be attended :p13:18
*** klamath has joined #openstack-ansible13:23
*** mkrish1712 has joined #openstack-ansible13:26
*** Andrew_jedi has quit IRC13:27
*** galstrom_zzz is now known as galstrom13:27
odyssey4memattt I see that tempest is as flaky as always :/ is that tempest run happening in serial? that would likely affect the outcome fairly dramatically - we have had to do serial for the integrated gate13:27
mhaydenpabelanger / odyssey4me: https://www.openstack.org/summit/austin-2016/summit-schedule/events/704313:27
odyssey4memattt I noticed a while back that devstack does integrated tests with a concurrency of 413:28
*** woodard has joined #openstack-ansible13:32
*** woodard has quit IRC13:33
*** pjm6 has joined #openstack-ansible13:34
openstackgerritMerged openstack/openstack-ansible: Update sources-branch-updater to handle release note copying  https://review.openstack.org/30917913:36
*** woodard_ has joined #openstack-ansible13:37
*** joker_ has quit IRC13:38
*** Mudpuppy has joined #openstack-ansible13:39
*** pjm6 has quit IRC13:40
fxpestertrying to setup AIO, but have not so much free space, how to reduce it ? - 80GB free disk space on the root partition13:40
odyssey4mefxpester hmm, our wording is a bit wrong - you ned around 60GB free space13:41
odyssey4meas a minimum13:41
fxpesterI have 4Gb13:41
odyssey4meah I see that we said the absolute minimum is 60gb free13:41
fxpesteralways ok with devstack13:42
odyssey4mefxpester heh, that's definitely not enough13:42
odyssey4mefxpester the AIO does a mini production like environment, including a fully HA configuration and clusters for rabbit, mariadb, keystone, etc13:42
odyssey4medevstack does none of that13:42
odyssey4mefxpester so if you can I'd advise adding a secondary disk as is covered by the AIO quickstart13:43
matttodyssey4me: i assumed gate-check-commit.sh used serial by default, but perhaps that's not correct13:43
odyssey4memattt run-tempest does, so gate-check-commit inherits it: https://github.com/openstack/openstack-ansible/blob/master/scripts/run-tempest.sh#L2613:44
odyssey4mebut you're executing the tempest script yourself, so you'll likely need to add --serial as an option to the command13:45
fxpesterhm, yes... maybe it`s one fat component ? is it possible to disable it completely to save space ? or maybe old release like Juno can solve it ?13:45
*** mkrish1712 has quit IRC13:46
odyssey4mefxpester heh, I wouldn't even bother trying Juno - it's long EOL13:46
odyssey4mefxpester there are ways to cut it down - you'd have to cut the containers out so that you only use venvs, you'll have to not deploy horizon because if the same host runs keystone and horizon they will both fight for ownership of the apache config, you'll also have to ensure that the AIO's affinities are all the default of 1 instead of some being 3 - and you may wish to not deploy some of the services to also reduce13:48
odyssey4me space13:48
odyssey4meeven then, not sure you'll get down to 4GB13:48
odyssey4mefxpester what do you actually need as a minimal setup?13:49
fxpesterI was expecting "you can set ceilometer=OFF" and you`re fine :D13:49
admin0evrardjp: ran haproxy and horizin playbooks .. no changes13:49
admin0so reverting the change13:49
evrardjp503?13:50
admin0yes13:50
fxpesterI trying to figure out is openstack-ansible project is sane and should I use it in production or not :)13:50
evrardjpadmin0 this is maybe normal13:51
odyssey4mefxpester OSA is made for production environments first, and 4GB disks are not useful for production13:51
odyssey4meexcept perhaps for a RAMdisk :p13:51
admin0well, expectation is both should work :D13:51
admin0sorry  normal is both must work13:51
fxpesterodyssey4me: yes, but LAB first, and my lab is intel nuc with single ssd :( everyday problems with space13:52
odyssey4mefxpester add an external drive to it via USB13:52
*** rromans has joined #openstack-ansible13:52
odyssey4mefxpester you can even get a 64GB USB thumb drive and try that - although I think we make quite a few assumptions centred around a real disk being used13:53
fxpesterodyssey4me: in background I started setup via cloud-init... it now reporting me with - serial8250: too much work for irq413:54
fxpesterubuntu14.0413:54
fxpesteron x86.trystack.org13:55
odyssey4meI've never seen that error. Maybe someone else has thoughts?13:55
odyssey4memany of us use cloud instances for testing on a very regular basis, and all our code tests use the same, so I don't know what's going on in your instance to cause that13:56
odyssey4meyou may wish to rather just build a starting instance and ssh into it, then create a tmux/screen session and do the steps from the AIO quickstart13:57
fxpesternot error... message from kernel :) all looks ok and going for 1 hour now.13:57
odyssey4mefxpester which release are you building? a specific tag or branch?13:58
fxpesterI`m a git expert, so just did git checkout stable/mitaka && git checkout 13.0.0 (cause git checkout 13.0.1 failed)13:59
fxpester"i`m NOT a git expert"14:00
odyssey4mefxpester also, if you have a decent host with enough disk space around you may wish to try using https://gist.github.com/cloudnull/f71e3078f9a0018017c3 for a far better test environment that is more production-like14:00
evrardjpcloudnull ping?14:00
odyssey4mefxpester ah, we're about to tag 13.0.1 - so stable/mitaka would have been the same thing as 13.0.1 - I'm just waiting for the release request to be processed14:00
odyssey4me13.0.0 is, in fact, a bit broken - it was based on pre-release OpenStack, but we had to tag it at the time... 13.0.1 is our actual Mitaka release14:01
mhaydenodyssey4me: openstack-ansible at the red hat summit! https://rh2016.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=7567514:02
odyssey4memhayden w00t!14:02
fxpesterI see, thx, will know what really exploded if it will =)14:03
*** b3rnard0_away is now known as b3rnard014:03
*** sigmavirus24_awa is now known as sigmavirus2414:03
*** sacharya has joined #openstack-ansible14:04
odyssey4mefxpester once it completes, assuming it does, you can use http://docs.openstack.org/developer/openstack-ansible/install-guide/app-minorupgrade.html to upgrade to the latest version... just use 'stable/mitaka' instead of 13.0.114:04
*** pjm6 has joined #openstack-ansible14:05
fxpesterodyssey4me: cobbler-osa-multi-aio.sh - looks very sexy, but what is this ? it works only in Rackspace cloud ?14:07
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_keystone: Ansible 2.x - Address deprecation warning of bare variables  https://review.openstack.org/30942514:07
odyssey4mefxpester it should work on any Ubuntu host with enough disk space, cpu's and memory to handle the number of VM's it builds14:08
*** sdake has joined #openstack-ansible14:09
*** sacharya has quit IRC14:09
fxpesterodyssey4me: but no one knows how much resources it needs ? :D have you built it ? what box do you use ?14:10
*** sdake_ has joined #openstack-ansible14:10
fxpesterI mean hdd and memory14:10
odyssey4mefxpester once cloudnull gets in he can tell you more14:11
*** sdake has quit IRC14:13
Mudpuppyfxpester: There are some host specs listed here https://github.com/cloudnull/osa-multi-node-aio14:15
Mudpuppy20 cpu cores, 124GB memory 1.3TB disk14:15
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_magnum: Ansible 2.x - Address deprecation warning of bare variables  https://review.openstack.org/30942914:16
odyssey4mefxpester Mitaka 13.0.1 has been released :)14:16
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_magnum: Remove unneeded with_items usage  https://review.openstack.org/30943014:18
odyssey4me#success OpenStack-Ansible Mitaka 13.0.1 has been released!14:18
openstackstatusodyssey4me: Added success to Success page14:18
*** galstrom is now known as galstrom_zzz14:24
openstackgerritMajor Hayden proposed openstack/openstack-ansible: Add docs for HAProxy ping checks  https://review.openstack.org/30943314:24
mhaydenodyssey4me: great success14:25
MudpuppyYou will get cake14:26
*** spotz_zzz is now known as spotz14:32
admin0cloudnull:  ping (again) !14:33
*** galstrom_zzz is now known as galstrom14:33
admin0evrardjp: you and I both stuck @ the same thing ?14:33
openstackgerritMerged openstack/openstack-ansible: Add missing line number report, fix coverage dep  https://review.openstack.org/30920514:33
admin0or is your case different from mine14:33
spotzdid someone say cake?14:34
* admin0 gives a big cake to spotz 14:34
evrardjpI'm just reviewing cloudnull's change14:34
evrardjpit really is disruptive if the value is set14:34
spotzadmin0 thanks14:34
evrardjphopefully it's not set per default14:34
admin0in my case, if that value is set, it works for all (except horizon)14:34
evrardjpso it should be part of the release note14:35
admin0without that, my keystone was not working14:35
evrardjpwhen you don't set it, everything works internally, and externally it seems to work with curl14:35
admin0not with SSL14:35
openstackgerritMerged openstack/openstack-ansible-os_designate: Remove bind9 and add functional tests  https://review.openstack.org/30718914:35
admin0not as a client14:35
evrardjpadmin0 the summary: you shouldn't set the variable and rebootstrap everything14:37
evrardjpit seems to work for cloudnull and for me14:37
admin0i have time and resources .. let me just do that and test :)14:37
evrardjpthe small issue would be the client14:37
evrardjpopenstack client seems to be a pain14:37
admin0but that is what all cleints will use (majority ) as opposed to horizon14:38
admin0evrardjp: you mean if i rebootstrap, horizon will be fixed, but the client will fail ?14:38
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_tempest: [WIP] Install tempest plugins  https://review.openstack.org/30903414:38
evrardjpadmin0 that's what I think14:38
mattt^^^ odyssey4me not sure what the best way to capture the plugins is, that way won't be picked up by repo server but means we can add a ton of plugins and the installation should be straightforward without having to explicitly install each one14:39
admin0evrardjp: then i would not fix it ..   €10/month client will use horizon ..   €1000/month cleints will use the openstack cli client ..  :)14:39
admin0so better the cli works and i fix horizon manually than the other way around14:39
evrardjpfeel free to fix it :D14:39
odyssey4memattt, why not rather have the plugins be a list of dicts with all the bits needed as you added in https://review.openstack.org/#/c/309034/3/defaults/main.yml14:41
admin0evrardjp: i will attempt :)14:41
odyssey4mebut have the tempest role have an empty list by default, and have each role give it the data14:41
*** toan has quit IRC14:42
*** bbmbx has joined #openstack-ansible14:42
odyssey4methen perhaps the test list should also be part of that dict structure, and the tempest_gate script can be fed the regex from the calling role14:43
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_nova: Use tempest for testing  https://review.openstack.org/30156614:43
matttodyssey4me: yeah, that'd work ... good idea14:44
*** phalmos has joined #openstack-ansible14:46
*** ChrisBenson has quit IRC14:46
*** ChrisBenson has joined #openstack-ansible14:46
*** sdake_ has quit IRC14:50
odyssey4memattt it may also be worthwhile to actually simply the enabled services in a similar way - basically expect a list and if the service 'glance' is not in the list then it's not enabled14:51
odyssey4meit'll cut down the number of vars and should be pretty straightforward14:52
odyssey4mebut that's certainly not essential to your need right now14:52
matttodyssey4me: yeah it is quite verbose at the moment and only going to get worse14:54
odyssey4memattt yep, we might have to do an interim step which allows both the current convention and a new one to work - then we deprecate the old convention for Newton14:55
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_tempest: [WIP] Install tempest plugins  https://review.openstack.org/30903414:55
matttodyssey4me: yeah, i think i will do a separate patch for taht14:55
odyssey4meyep, good thinking14:55
*** sacharya has joined #openstack-ansible14:56
mattti also want to rewrite openstack_tempest_gate.sh14:56
cloudnullmornings14:56
mattti hate this script :)14:56
mattthowdy cloudnull14:56
cloudnullhows it ?14:56
spotzodyssey4me: Thanks for the edits and getting that patch through14:56
evrardjplbragstad did you got time to look at the keystone usability issue?14:56
*** itlinux has joined #openstack-ansible14:57
*** pjm6 has quit IRC14:57
lbragstadevrardjp I do have less meetings today so I should have a chance to get around to it14:57
lbragstadevrardjp do you have that link again?14:57
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_keystone: Ansible 2.x - Address deprecation warning of bare variables  https://review.openstack.org/30942514:57
evrardjplbragstad I will make a new paste :D14:58
*** javeriak has joined #openstack-ansible14:58
*** pjm6 has joined #openstack-ansible14:58
-openstackstatus- NOTICE: Log server was repaired as of 10:50 UTC and jobs have been stable since. If necessary, please recheck changes that have 'UNSTABLE' results.14:58
*** itlinux has quit IRC14:59
*** Brew has joined #openstack-ansible14:59
openstackgerritMajor Hayden proposed openstack/openstack-ansible: Add docs for HAProxy ping checks  https://review.openstack.org/30943314:59
evrardjplbragstad http://paste.openstack.org/show/495161/14:59
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_hosts: Changed for lxc-host setup/build for multi-distro  https://review.openstack.org/30785615:01
*** saneax is now known as saneax_AFK15:01
*** pjm6_ has joined #openstack-ansible15:02
*** pjm6 has quit IRC15:03
evrardjpmhayden reviewed15:04
evrardjpand assigned the bug to you15:05
*** pellaeon has joined #openstack-ansible15:06
openstackgerritMajor Hayden proposed openstack/openstack-ansible: Add docs for HAProxy ping checks  https://review.openstack.org/30943315:06
mhaydenevrardjp: thanks -- revised it a little15:09
lbragstadevrardjp is this the ssl everywhere patch?15:09
evrardjplbragstad yes15:10
evrardjphttps://review.openstack.org/#/c/277199/15:10
pellaeonHi, I have a openstack-ansible kilo deployment, with haproxy. I found that there are lots of connections to galera (over 400, but the mariadb's max_connections limit is set to 400), but haproxy is set to use one galera node as active and others as backup, and there are no connections limit set on haproxy, so in the end haproxy forwards all mysql connection to a single node, exceeding 400 connec15:10
pellaeontions, then the node reject further connection15:10
*** sdake has joined #openstack-ansible15:10
evrardjppellaeon 400 connections is much15:11
evrardjpbut yes that's the intended behaviour15:11
evrardjpwant to raise it?15:11
pellaeonis 400 connections normal ? I see a lot of connections to nova and neutron database15:12
pellaeonno, I think setting a connection limit per node on haproxy should be the better way to solve it15:12
pellaeonie. connections still by default goes to a single node, but if a single node has more than 400 connections, haproxy will start forwarding new connections to other nodes15:13
cloudnullpellaeon:  its likely that you will run into DB deadlocks if you have more than one node accepting writes .15:14
evrardjpthe way we designed the cluster was to hit all on the same... I'm not sure about the multiple writes on multiple servers... while it's technically possible I don't know the reasons why it was done that way15:14
evrardjphere they are15:15
evrardjpcloudnull I thought this was also the purpose of galera cluster to be honest15:15
evrardjpbut yes whatever15:15
*** ChanServ changes topic to ":Austin Design Summit Schedule: https://goo.gl/WSRblf || Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs"15:15
*** iceyao has quit IRC15:15
cloudnullgalera will take care of most of it however it's not been very stable in my testing15:16
pellaeonHave other people hit this issue? I'm running a relatively small installation, 3 infra nodes, 1 compute node, about 30 VMs, it doesn't make sense that installation this small still hits connection limit15:17
evrardjpoh small installation it shoudn't happen15:18
*** javeriak_ has joined #openstack-ansible15:18
*** weezS has joined #openstack-ansible15:19
*** sdake_ has joined #openstack-ansible15:19
*** sdake has quit IRC15:19
pellaeonOK, I will investigate further regarding the issue on my installation, but in the mean time, we could discuss how this could be solved15:20
cloudnullit may happen, its all about the allowed connection and pool workers in each service. I'd raise the connection limits in mariadb or reduce the workers on services initiating too many connections.15:20
cloudnullpellaeon:  options like https://github.com/openstack/openstack-ansible-os_nova/blob/master/defaults/main.yml#L69-L81 may help15:20
evrardjpcloudnull it's weird for a small cluster anyway15:21
*** javeriak has quit IRC15:21
*** admin0 has quit IRC15:21
*** admin0 has joined #openstack-ansible15:21
cloudnullneutron like so https://github.com/openstack/openstack-ansible-os_neutron/blob/master/defaults/main.yml#L88-L9015:21
evrardjps/cloudnull/pellaeon15:21
lbragstadcloudnull evrardjp this is the change you're seeing issues with - https://review.openstack.org/#/c/277199/ ?15:22
odyssey4mepellaeon we merged this recently and ported it all the way back to liberty: https://review.openstack.org/#/c/307231/15:22
odyssey4methat may be pertinent15:22
lbragstadcloudnull evrardjp and by issues you mean - http://paste.openstack.org/show/495161/15:22
evrardjplbragstad yes15:22
evrardjpcloudnull doesn't see any issue15:22
evrardjpbut other ppl do15:22
odyssey4mepellaeon you may wish to plan an upgrade to Liberty ASAP too, Kilo's going EOL in a month15:22
cloudnullif i set the public url endpoint.15:23
evrardjpcloudnull mattt just set the public url endpoint and got other issues15:23
evrardjpthe one I got too15:23
cloudnullwhat other issues?15:23
evrardjpabout cinder not being able to internally speak with keystone15:23
evrardjpduring playbooks run or later15:24
pellaeonI see, thanks for all the hints!15:24
* palendae shakes fists at trying to test things that rely on order with ordinary dicts15:24
cloudnulli've not tested cinder externally but it is being exercised in our gate test15:24
evrardjpcloudnull I think the best is not to set the public url endpoint15:24
evrardjpbecause it works perfectly fine internally15:24
*** jaypipes is now known as leakypipes15:24
*** ChanServ changes topic to "Austin Design Summit Schedule: https://goo.gl/WSRblf || Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs"15:25
matttcloudnull: my deploy failed, it coudln't even install cinder15:25
evrardjpdoesn't needlessly go to public endpoint15:25
odyssey4meyes, ideally we don't want to have to hard set the public endpoint15:25
cloudnullit does. tempest tests the public endpoints when it does the dashboard scenario tests15:25
admin0cloudnull:  in my case, everything works except horizon  ..15:25
*** openstack has joined #openstack-ansible15:28
cloudnullright.15:29
evrardjptrue15:29
cloudnullmy argument isnt that there is some client bug15:29
*** pjm6 has joined #openstack-ansible15:29
cloudnullmy argument is that the public endpoints work as expected15:29
evrardjpcloudnull but you are right15:29
odyssey4mehttps://github.com/openstack/openstack-ansible-os_horizon/blob/master/defaults/main.yml#L7415:29
evrardjppublic endpoints work when you set this variable15:29
evrardjpit's internal that start to fail15:29
evrardjpwhich is not good15:30
*** fxpester has quit IRC15:30
evrardjpor admin15:30
evrardjpI don't remember15:30
odyssey4mecloudnull and my point is that the openstack public endpoints are not exercised at all in the gate15:30
evrardjpbut let's forget this variable15:30
evrardjpand focus on the other obne15:30
evrardjpthe one without setting the variable15:30
odyssey4methe haproxy public load balancer address is, but not the openstack endpoints... only horizon is accessed via the public address15:30
evrardjp(the default)15:30
*** johnmilton has quit IRC15:31
openstackgerritNolan Brubaker proposed openstack/openstack-ansible: Check for two IP addresses assigned to same host  https://review.openstack.org/30805215:31
*** pjm6_ has joined #openstack-ansible15:32
*** pjm6 has quit IRC15:33
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_keystone: Ansible 2.x - Address deprecation warning of bare variables  https://review.openstack.org/30942515:35
lbragstadevrardjp you're seeing issues getting the service catalog when using the clients?15:36
evrardjpyes15:36
lbragstadevrardjp according to your paste, curl seems to be working fine against keystone15:36
evrardjpyes15:36
evrardjpmaybe my paste isn't perfect15:37
evrardjpbut you get the idea :D15:38
*** ChrisBenson has quit IRC15:38
palendaeodyssey4me: Were you planning on getting to https://etherpad.openstack.org/p/openstack-ansible-newton-ubuntu16-04 before summit?15:38
cloudnullodyssey4me: http://cdn.pasteraw.com/faswq0ekv9t2lyupq248j8b7hm0uci215:38
cloudnullthe public urls work fine15:38
cloudnullfrom within the env15:38
cloudnulland that var not set15:38
odyssey4mepalendae yes :)15:38
odyssey4mefeel free to add anything you like right now15:39
cloudnulland evrardjp mattt cinder seems to be working too15:39
palendaeodyssey4me: Ok :) Was trying to review reading beforehand15:39
palendaeI don't have much to add but someone was asking and reminded me15:39
cloudnullif you want access to my env i'd be happy to provide it15:39
*** ChrisBenson has joined #openstack-ansible15:39
cloudnullthis is likely an issue with forward handling within the clients15:39
odyssey4methe floor is open, I've not managed to turn https://etherpad.openstack.org/p/openstack-ansible-multi-os-support into something useful to facilitate that discussion yet15:39
palendaeodyssey4me: Yeah, I don't have many well-formed thoughts around it, just wanted to check in15:40
evrardjpcloudnull the issue I and matt have had on the cinder deploy is barely consistent. I managed to deploy yesterday without issue15:41
evrardjpwith the variable set15:41
mhaydencloudnull: i just ran into those rabbitmq shenanigans on mitaka15:41
mhaydenor something similar15:41
evrardjpcloudnull I can give you access to my environment where the variable is set if you want15:41
cloudnulleither way.15:42
evrardjplet's do this15:42
mhaydencloudnull: how do i make it past these rabbitmq hostname shenanigans?15:45
*** fxpester has joined #openstack-ansible15:45
cloudnulldo you have all of the rfc changes ?15:45
mhaydeni doubt it :(15:46
stevellepalendae, re: 16.04 etherpad -- it's no longer a blank page. your turn15:46
mhaydencloudnull: is that in the docs or release notes?15:46
palendaestevelle: I was just looking for stuff to read, so I'll read :p15:47
stevellepalendae: just be sure to put your name in15:48
palendaegrr etherpad forgot it again15:48
cloudnullmhayden: http://docs.openstack.org/releasenotes/openstack-ansible/mitaka.html#new-features15:48
cloudnulland http://docs.openstack.org/releasenotes/openstack-ansible/mitaka.html#upgrade-notes15:49
odyssey4memhayden are you using master or mitaka?15:49
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_neutron: Ansible 2.x - Address deprecation warning of bare variables  https://review.openstack.org/30947415:49
mhaydenodyssey4me: i'm on stable/mitaka15:49
*** ccesario has joined #openstack-ansible15:49
openstackgerritAla Raddaoui proposed openstack/openstack-ansible: Check for IP addresses assigned to multiple hosts  https://review.openstack.org/30900115:49
odyssey4memhayden ok, that includes everything needed as far as I know15:50
mhaydeni was on 13.0.0 and going to 13.0.1 now15:50
*** pjm6_ has quit IRC15:52
lbragstadevrardjp based on your environment - it seems like the clients are making an assumption about the keystone endpoint15:52
*** johnmilton has joined #openstack-ansible15:52
mhaydencloudnull: i guess i'm confused on what action i need to take on an existing environment15:52
mhaydendo i need to destroy those containers and start over?15:52
evrardjplbragstad yes indeed15:52
evrardjplbragstad would that be a behavior when they do not get appropriate info?15:53
cloudnullno you should not have to15:53
cloudnullare you upgrading from liberty to mitaka ?15:53
lbragstadactually - from your trace, it's in keystoneauth115:53
odyssey4memhayden just run the minor upgrade process15:53
odyssey4mehttp://docs.openstack.org/developer/openstack-ansible/mitaka/install-guide/app-minorupgrade.html15:54
openstackgerritMerged openstack/openstack-ansible: Add docs for HAProxy ping checks  https://review.openstack.org/30943315:54
mhaydenodyssey4me: that's that i'm doing :)  the setup-infrastructure playbook failed during the rabbitmq part15:54
*** pjm6 has joined #openstack-ansible15:55
odyssey4memhayden hmm, http://docs.openstack.org/developer/openstack-ansible/13.0.1/install-guide/app-minorupgrade.html :)15:55
*** fxpester has quit IRC15:55
*** fxpester has joined #openstack-ansible15:55
odyssey4meit looks like the branch head docs haven't been publiched again after the tag reference changed15:55
evrardjplbragstad that's what I think too15:56
evrardjpI don't know how to properly report this to keystone ppl15:56
odyssey4memhayden can you backport https://review.openstack.org/309433 to stable/mitaka please?15:56
openstackgerritMerged openstack/openstack-ansible: Docs: Split Network Services section into multiple files  https://review.openstack.org/30764015:56
mhaydenodyssey4me: done sir15:56
mhaydenodyssey4me: still working on that other thing15:57
odyssey4meta15:57
lbragstadevrardjp drop into the keystone channel15:57
evrardjpthat's what I tried yesterday :p15:57
lbragstadevrardjp did anyone respond?15:57
evrardjpI got better results with you today :p15:57
evrardjpcloudnull just gave me an idea15:58
evrardjpabout header issues15:58
evrardjpI'll analyse this too15:58
lbragstadwell - the tokens aren't 8K ;)15:58
* lbragstad celebrates15:58
evrardjp?16:00
lbragstadevrardjp I was making a joke about PKI tokens16:00
mhaydenodyssey4me / cloudnull: i had to do a lxc-stop, and then run lxc-containers-create.yml -- it looks like things are okay now16:00
mhaydenor on the way to being okay16:00
mhaydeni did that only on the rabbit containers16:01
evrardjplbragstad too early for token jokes for me16:01
cloudnullyes the lxc-container create play need to be run which will do the RFC changes.16:01
cloudnullyou shouldnt need to stop/start16:02
cloudnullthe host name bits happen here https://github.com/openstack/openstack-ansible/blob/master/playbooks/lxc-containers-create.yml#L33-L5516:02
sigmavirus24evrardjp: what versions of OSA/osc/ksa are you using?16:05
evrardjpksa?16:06
evrardjposa master16:06
*** sdake_ has quit IRC16:07
odyssey4meksa=keystoneauth116:07
lbragstadpython-keystoneclient (2.3.1)16:07
lbragstadevrardjp that's the version you have on the keystone container16:07
odyssey4meit might be time for us to update master SHA's again16:07
*** zhangjn has quit IRC16:12
*** pjm6 has quit IRC16:13
*** pjm6 has joined #openstack-ansible16:14
cloudnulllbragstad:  fixed in > versions ?16:15
*** Oku_OS is now known as Oku_OS-away16:16
*** pjm6_ has joined #openstack-ansible16:16
*** Oku_OS-away is now known as Oku_OS16:17
*** Bofu2U has quit IRC16:17
*** elo has quit IRC16:18
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ceilometer: Update paste, policy and rootwrap configurations 2016-04-22  https://review.openstack.org/30949016:18
evrardjppython-openstackclient==2.3.0, python-keystoneclient==2.3.1,keystoneauth1==2.4.0,keystonemiddleware==4.4.016:18
*** elo has joined #openstack-ansible16:18
*** pjm6 has quit IRC16:18
evrardjpsorry for the delay16:18
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder: Update paste, policy and rootwrap configurations 2016-04-22  https://review.openstack.org/30949216:19
*** zhangjn has joined #openstack-ansible16:19
lbragstadcloudnull checking - I thought we did a release recently16:19
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_heat: Update paste, policy and rootwrap configurations 2016-04-22  https://review.openstack.org/30949316:19
lbragstadcloudnull we released 2.6.0 on april 13th16:20
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone: Update paste, policy and rootwrap configurations 2016-04-22  https://review.openstack.org/30949416:20
evrardjplbragstad 2.6 for keystone auth?16:21
lbragstadevrardjp yeah16:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_neutron: Update paste, policy and rootwrap configurations 2016-04-22  https://review.openstack.org/30949616:23
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova: Update paste, policy and rootwrap configurations 2016-04-22  https://review.openstack.org/30949716:23
evrardjpthat's quite a bump16:24
evrardjpfor osa16:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Update Newton SHA's 2016-04-22  https://review.openstack.org/30949816:25
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-os_keystone: Ansible 2.x - Address deprecation warning of bare variables  https://review.openstack.org/30942516:26
lbragstadevrardjp what's the header thing you were going to try?16:26
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Update Newton SHA's 2016-04-22  https://review.openstack.org/30949816:26
*** admin0 has joined #openstack-ansible16:26
evrardjpcloudnull tried on its cloud16:26
evrardjpso I don't need to try16:26
evrardjpcapturing what's happening16:26
cloudnullevrardjp: it looks like a hap issue16:27
evrardjpreverse engineering for understanding how this work16:27
cloudnullif you go to root@aio1-utility-container-be92fd95:~#16:27
cloudnulland use openrc.publi16:27
cloudnullyou can do things16:27
cloudnulli've pulled back the hap config to a bare minimum16:27
cloudnulland it works16:27
cloudnullso i think hap is filtering16:28
cloudnulland or not passing the headers correctly16:28
cloudnullits about to be broken again fyi16:28
evrardjpwhat do you mean?16:28
evrardjp"it's about to be broken again"16:29
evrardjpcloudnull do you mind if we shut down the aio you're on?16:29
cloudnullhttp://cdn.pasteraw.com/1fsi4dlhrxoe56x83sda22vsvsv7zyb16:29
evrardjpsolving two different issues of ssl at the same time is difficult16:29
cloudnullim using your node as my external check16:30
cloudnull:)16:30
evrardjpok16:30
*** jthorne has joined #openstack-ansible16:30
evrardjpcontinue then!16:30
evrardjp:D16:30
cloudnullsee the last paste16:30
cloudnullworks16:30
evrardjpso we are on the last page16:30
evrardjpcool16:30
*** Oku_OS is now known as Oku_OS-away16:30
evrardjpdid you remove the header part of the haproxy config?16:30
evrardjpare also edited to add verify none after ssl ?16:31
cloudnullif i take these lines out16:31
cloudnullhttp://cdn.pasteraw.com/a8z8g8tkjus1krrxo2n8wq2wsrdzzh216:31
cloudnullit works16:31
cloudnullyou have the openrc.public file to verify on that server16:31
cloudnullconfirmed remove those lines all is well16:33
*** pjm6_ has quit IRC16:33
evrardjpok will test right away16:34
cloudnullseems adding the header and or forcing it to be part of the backend breaks the clients.16:34
evrardjpthat's the line I thought could be problematic16:34
*** pjm6 has joined #openstack-ansible16:34
lbragstadcloudnull evrardjp so have we narrowed this down to hap/16:34
cloudnullyup16:35
cloudnullit would seem so16:35
cloudnullsorry lbragstad16:35
*** tlbr has quit IRC16:35
evrardjpit looked like a client behavior16:35
evrardjpsorry16:35
lbragstadcloudnull not at all - just curious if there way anything that needed to be done on the keystone side16:35
*** admin0 has quit IRC16:35
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible: Enable SSL termination for all services  https://review.openstack.org/27719916:36
lbragstadevrardjp I started looking at it and immediately went to the client libraries because that's exactly what it looked like16:36
*** tlbr has joined #openstack-ansible16:36
cloudnullevrardjp/admin0:  ^16:36
cloudnullgoing to eat. bbl16:37
cloudnullevrardjp: let me know if that solves the things for you16:37
evrardjpI'm still doing your fix one sec :p16:37
evrardjpI'll review afterwards16:38
evrardjpand then week-end!16:38
*** pjm6 has quit IRC16:39
evrardjpok it seems to fix stuff16:40
evrardjpso sorry again lbragstad I'll think differently next time. But I learned a lot, so really thank you.16:41
*** pjm6 has joined #openstack-ansible16:41
evrardjpI should have checked that16:43
*** johnmilton has quit IRC16:45
*** johnmilton has joined #openstack-ansible16:45
*** pjm6 has quit IRC16:45
odyssey4methat's not how this works evrardjp - you're supposed to blame cloudnull :p16:49
evrardjpoh no I'll let you have this pleasure16:49
*** pjm6 has joined #openstack-ansible16:50
*** winggundamth_ has joined #openstack-ansible16:50
evrardjpI'm off16:50
evrardjphave a nice summit16:51
*** winggundamth_ has quit IRC16:51
odyssey4mehave a great weekend evrardjp, and enjoy the quiet week next week16:51
*** pjm6 has quit IRC16:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Updates all SHAs for 12.0.12  https://review.openstack.org/30951116:57
openstackgerritgit-harry proposed openstack/openstack-ansible: Fix MariaDB cleanup for Kilo to Liberty upgrades  https://review.openstack.org/30951317:01
spotzmhayden - https://etherpad.openstack.org/p/openstack-ansible-newton-role-docs17:02
openstackgerritMerged openstack/openstack-ansible-os_neutron: removed duplicate key  https://review.openstack.org/29976117:04
*** weezS has quit IRC17:04
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-lxc_container_create: Template /etc/environment rather than use lineinfile  https://review.openstack.org/30951717:06
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-lxc_container_create: Template /etc/environment rather than use lineinfile  https://review.openstack.org/30951717:06
spotzodyssey4me cloudnull Any reviews you specically need us to hit today?17:07
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Updates all SHAs for 12.0.12  https://review.openstack.org/30951117:08
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-lxc_container_create: Template /etc/environment rather than use lineinfile  https://review.openstack.org/30951717:08
automagicallyApologies for the spam ^17:08
odyssey4mespotz no urgency on anything - Mitaka 13.0.1 is released, and that's the final thing - so we can all relax, do summit things and on Sat I'll be joining y'all to lay waste to Austin17:08
spotzodyssey4me: Sweet. bring shorts you're going to be warm. We're going to try to check in early on Sunday after I go ride in the morning I think17:09
automagically88 on Monday!17:10
odyssey4mespotz ride? horse, bicycle, or motorbike?17:10
spotzhorse/pony17:10
automagicallyAnd Tuesday17:10
odyssey4meheh, that's awesome :) I always enjoyed riding a horse, regardless of the fact that they somehow seem to enjoy throwing me off. :p17:11
automagicallyspotz: I’ll have to talk horses with you a bit in Austin. I recently purchased 2 off the track thoroughbreds for retraining/resale as sporthorses and my partner is an eventing rider17:11
*** rromans has quit IRC17:11
odyssey4meautomagically oh nice!17:11
*** rromans has joined #openstack-ansible17:11
spotzautomagically: SWEET we will so talk:) I'm currently more dressage but used to do hunters and eventing17:11
MudpuppyI feed carrots17:12
*** tiagogomes has quit IRC17:13
odyssey4meMudpuppy I wasn't aware that carrots could be fed?17:13
automagicallyThey don’t eat much17:13
spotzHe rides my old AA hunter, 17.1+ hand dutch WB17:13
automagicallyBut you can try17:13
odyssey4melol, now that is a funny picture17:13
spotzI've got a 14.1 connemara17:13
*** mattoliverau has quit IRC17:13
*** finchd has quit IRC17:13
automagicallyA shorty17:13
Mudpuppyhttp://cdn01.cdn.justjaredjr.com/wp-content/uploads/headlines/2009/02/miley-cyrus-carrot-top.jpg17:14
*** matt6434 has joined #openstack-ansible17:14
odyssey4meoh dear17:14
MudpuppyI"m horrible I know.17:14
spotz<- 5'2" on a good day17:14
automagicallyEverybody loves a pony!17:14
automagicallymy partner is Irish so she’ll appreciate hearing you’ve got a connemara17:15
spotzMy last connemara was 15 which was a good height lost him to colic last summer:(17:15
spotzerr 15 hands17:15
spotzautomagically: There's Dover not too too far from downtown:)17:16
automagicallySorry to hear that. Colic is no good17:16
spotzautomagically: No:(17:16
spotzHis dad died last year in his 30s, I used to event him17:16
automagicallyI’ve been spending plenty of time in my local Dover17:16
spotzheheh17:17
automagicallyWow, this transitioned to #horses really quick like ;)17:17
Mudpuppyhahah17:17
spotzheheh17:18
spotzautomagically: Ok summit... You gonna try the mechanical bull? Mudpuppy won't let me something about not spending his summit in the ER:)17:18
*** admin0 has joined #openstack-ansible17:18
palendaeThere's a mechanical bull?17:18
automagicallyThat wouldn’t end well for me17:18
palendaeWait of course there is17:18
palendaeTexas17:19
electrofelixif anyone is on ansible 1.9 you may want to backport the ssh connection plugin from https://github.com/ansible/ansible/issues/15436 to fix a rather annoying loss of data17:19
automagicallyIts TX, there are many mechanical (and real) bulls17:19
palendaeI guess the only question will be is: is it in the convention center?17:19
palendaeMy assumption is yes, probably the Mirantis booth17:19
*** finchd has joined #openstack-ansible17:19
palendaeTheir Mechastack theme in Tokyo was neat17:19
spotzpalendae: Did you not study the Stackcity stuff?:) No it's on Rainey street17:19
palendaespotz: No, I did not17:19
spotzI'm a planner, though for summit it doesn't help without a few clones17:20
palendaeI'm planning my sessions right now, but I've not looked up extracirrucular stuff17:20
palendaeI did way more for Tokyo17:20
*** keedya has quit IRC17:20
spotzI think the only marriot thing I might get to is the one I'm helping with. Too far to run between17:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Updates all SHAs for 11.2.15  https://review.openstack.org/30952617:22
*** keedya has joined #openstack-ansible17:22
spotzNeed to try to hit a few of the Heat sessions besides ours17:23
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Updates all SHAs for 11.2.15  https://review.openstack.org/30952617:25
spotzodyssey4me: You still ending that one?17:28
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Updates all SHAs for 12.0.12  https://review.openstack.org/30951117:28
odyssey4meelectrofelix interesting - we've not seen that failure, but we've seen failures, which is why we submitted a patch which is in 2.0 which automatically retries the connection better... but yeah, we're still on 1.94 at this point17:31
odyssey4mespotz ending which one?17:31
odyssey4meAnd by the way, the only Connamara I know is a rather nice whiskey. :)17:32
spotzodyssey4me: ha they're differnt ones was looking at the title not the numbers:)17:32
electrofelixodyssey4me: we've seen it periodically during CI runs, I suspect that sometimes it occurs and people just don't realise it17:33
electrofelixretry, don't see it again and forget17:34
odyssey4mespotz so many branches active right now, it's a bit much17:36
odyssey4meelectrofelix yeah, I suspect that it's masked by an automated retry we use in the gate17:37
odyssey4meunfortunately that retry also masks other issues I suspect17:37
odyssey4mebut we'll be shifting to development on 2.1 pretty soon17:37
openstackgerritTravis Truman (automagically) proposed openstack/openstack-ansible-lxc_container_create: Adding Vagrantfile for local developer testing  https://review.openstack.org/30953217:37
*** brad[] has quit IRC17:38
*** brad[] has joined #openstack-ansible17:40
odyssey4meelectrofelix are you going to propose a backport fix to 1.9 ?17:42
*** itlinux has joined #openstack-ansible17:43
*** itlinux has quit IRC17:44
cloudnullevrardjp: ping17:51
cloudnullRE: https://review.openstack.org/#/c/277199/33/playbooks/roles/haproxy_server/templates/service.j217:51
*** v1k0d3n has joined #openstack-ansible17:52
*** falanx has quit IRC17:53
v1k0d3nhey guys...is there some better/additional documentation for this project: https://github.com/openstack/openstack-ansible-os_magnum ?17:53
cloudnullsigmavirus24: -cc ^17:53
v1k0d3ni would like to integrate this...i've been trying this: https://github.com/sigmavirus24/openstack-ansible-magnum17:54
v1k0d3nbut ran into issues, which i explained here: https://github.com/sigmavirus24/openstack-ansible-magnum/issues/117:54
cloudnullv1k0d3n: there is this http://docs.openstack.org/developer/openstack-ansible-os_magnum/17:54
*** falanx has joined #openstack-ansible17:54
openstackgerritMerged openstack/openstack-ansible-lxc_container_create: Adding Vagrantfile for local developer testing  https://review.openstack.org/30953217:55
*** sdake has joined #openstack-ansible17:56
v1k0d3ncloudnull: does this plug right into OSAD fairly easily?17:56
sigmavirus24v1k0d3n: I don't think the SHA in the role has been updated at all17:57
v1k0d3nsigmavirus24's seem to work pretty well, i just started running into a bit of a dependency nightmare...and it seems like i can't "find" the pip package for "magnum"...17:57
cloudnullv1k0d3n: it should however i've not done it myself17:57
sigmavirus24You will want to try to adjust the SHA of magnum you're using as global-requirements.txt shifts and OSA makes this a fragile relationship17:57
sigmavirus24v1k0d3n: this role was merely a Proof of Concept17:57
sigmavirus24It was imported before it was close to being mature or finished though17:58
v1k0d3nhmmm. that's kind of magnum in general right now though, eh? lol17:58
*** mkrish004c has joined #openstack-ansible17:58
sigmavirus24v1k0d3n: no joke17:58
v1k0d3ni haven't been keeping up with magnum much lately. i guess it's mature in mitaka now?17:58
sigmavirus24That's why I stopped working on it17:58
electrofelixodyssey4me: it's already proposed by someone that hit it and put up a fix around the same time I was digging into it17:58
sigmavirus24v1k0d3n: lol you'd hope that17:59
sigmavirus24but not really17:59
v1k0d3nyeah...dude..don't blame you on that one! ^^ (support)17:59
v1k0d3nyeah, that sucks.17:59
v1k0d3nso this whole kubernetes thing...it's a great movement, right? but really fragmented when you try to get it to work within openstack.17:59
sigmavirus24v1k0d3n: they have a lot of "top priorities" right now that they're sorting out for newton and a new PTL so I doubt they're going to get very far with anything17:59
sigmavirus24v1k0d3n: not even kubernetes' fault.17:59
v1k0d3nthey seem to support all other public clouds just great...but openstack as a provider isn't documented well at all.18:00
sigmavirus24v1k0d3n: you could deploy kubernetes on top of an openstack cloud now with some playbooks18:00
v1k0d3ni would really, really like to clear up some of the confusion for our folks here.18:00
v1k0d3nwell, i do...18:00
sigmavirus24v1k0d3n: well magnum is one way to get kubernetes on openstack within a tenant18:00
odyssey4mev1k0d3n sigmavirus24 hmm, that role still needs plenty of work - v1k0d3n you're welcome to inspect the other roles to see how we setup the ability for it to build and test on its own, it'd be really great if that could be done... that'll also make it work a lot more easily in an integrated test which is an eventual goal18:00
odyssey4mebut it does need people interested in making that a reality to join up with us18:00
mkrish004cHi Guys, i am installing Ceph via OSA, is there any module to integrateto ceilometer or it does automatically, cuz i am not getting any metrics of ceph.18:00
v1k0d3nwell, i mean...count me in18:01
sigmavirus24odyssey4me: if it were a priority on my end, I would have done that but I get reassigned to something new fairly frequently18:01
v1k0d3ni'm highly interested in making this work18:01
v1k0d3nthe unfortunate thing is what works...is outside of openstack.18:01
v1k0d3nand i guess that's fine...but what the community sees....that's really fractured.18:01
odyssey4memkrish004c I answered you this morning - you'll need to find out from the Ceph/Ceilometer guys if that's even a possibility. If you can find out how to do it, assuming it is possible, then we'd be happy to help you make that happen.18:01
sigmavirus24metral: v1k0d3n you two should chat about deploying kubernetes on top of an openstack cloud18:02
*** pjm6 has joined #openstack-ansible18:02
v1k0d3nhahahaha metral...and here we are full circle. :)18:02
metral:) hey there18:02
v1k0d3nwe've chatted. hey man...long time no talk, right? :D18:02
metrallol exactly18:03
v1k0d3ndude, laco would get a kick out of this, i swear...18:03
v1k0d3nlol18:03
* sigmavirus24 should have known that metral knows everyone interested in kubernetes on top of OS already18:03
mkrish004cohh sorry  odyssey4me i might have missed your reply....i ll try to get them...18:03
sigmavirus24v1k0d3n: I can summon him for you18:03
metral@sigmavirus24 haha not quite, but close ;)18:03
v1k0d3nno. it's all good18:04
*** pjm6 has quit IRC18:04
v1k0d3nso metral...i'll spare you from having to read up too much.18:04
v1k0d3ni was talking about magnum integration with the PoC sigmavirus24 was working on....18:05
v1k0d3nstill trying to get a good working ingress working for 1.2 k8s.18:05
v1k0d3nthat's where the magic is....18:05
v1k0d3nbut ran into issues...because, well....it's magnum and there are so many moving parts.18:05
v1k0d3ni thought about deploying your solution with weave, but then that gets pretty darn opinionated, and i'm trying to find something that's not very opinionated.18:06
v1k0d3n^^ and there is the issue ^^18:06
metralwhat is your overarching goal you're trying to accomplish?18:06
cloudnullCLOUD!18:07
* cloudnull goes away18:07
v1k0d3nlol! ^^18:07
v1k0d3nvapor18:07
openstackgerritMajor Hayden proposed openstack/openstack-ansible: Add docs for limiting access to services  https://review.openstack.org/30954118:07
v1k0d3ngoal is to have a simple PoC with ingress controllers, so I can test what i need to...security, CNI vs Docker plug-in points of interest, etc.18:08
odyssey4mev1k0d3n but security isn't cloudy18:09
v1k0d3nodyssey4me: very true...but i need to prove that to people who don't RTFM.18:09
odyssey4mejust make sure you refresh everything every few minutes in a rolling implementation, then skip the security :p18:09
v1k0d3ninstead of saying...i want to do.18:09
v1k0d3nhahahahaha18:10
v1k0d3nyour suggestion is hilarious...i mentioned that at one point.18:10
v1k0d3n"yes, just roll those ephemeral containers every 90 seconds...that's secure, right?" :)18:10
odyssey4mecloud has no manuals, we're all just a bunch of cowboys making it up as we go along18:10
odyssey4mepssh manuals18:10
odyssey4methose container guys, well now... they're like crack addicts or something18:11
metral;)18:11
v1k0d3nit's true...we're a special kind of special.18:11
odyssey4meI swear I've never seen a bunch of people get so excited about a patch release of any piece of software.18:11
odyssey4meit's amazing18:12
v1k0d3nhahaha18:12
v1k0d3n12FACTOR!!!!18:12
odyssey4me*must have my hotfix*18:12
cloudnullsuch wow!18:12
metralif you're interested in just testing those features out and magnum is giving you issues, why not test it via one of the corekube deployments?18:12
v1k0d3nwell, that's where i'm ultimately heading....18:13
metraldidn't you get one stood up the other day we chatted?18:13
odyssey4meAll the devs continuously updating and getting excited about this new tweak and that new tweak... meanwhile back in Ops lands they're trying to figure out how to make sure that the devs never, ever touch anything in production, ever.18:13
v1k0d3nso the magnum stuff was a hail mary....to prove to our guys it's kind stitched together atm, and other solutions work better (now at least).18:13
metralah, gotcha - makes sense18:13
*** admin0 has quit IRC18:14
odyssey4mev1k0d3n well, challenge them to find a manual that works :p18:14
odyssey4melike, any kind of install guide that works, at all18:14
v1k0d3nwell, to your point odyssey4me...these are some guys around here who are "decision makers".18:14
metrallol18:14
v1k0d3nthey say..."well, we're an openstack shop, so we're using magnum".18:14
v1k0d3nand i start getting a little worried.18:15
*** pjm6 has joined #openstack-ansible18:15
*** weezS has joined #openstack-ansible18:15
odyssey4mev1k0d3n well, then it's probably wise to inform them that they'd best put aside some investment capital because they're hoping to use an alpha solution18:15
metralagreed18:15
metralthe openstack world != the container world18:16
v1k0d3nagain, talk doesn't mean anything. i want to prove things. and yeah odyssey4me we're in the same place because we're doers. :)18:16
v1k0d3nmetral: right ^^ agreed18:16
odyssey4meit's all well and good, but if you're wanting to make use of an immature project then you had best be ready to dedicate some devs to it18:16
metralthis ^18:16
v1k0d3nso here's how telcom's work...18:16
v1k0d3n*telecom18:17
v1k0d3nwe like to build rules and processes for up and coming things...18:17
v1k0d3nthing is...as you all know...these are being developed daily. landscape changes.18:17
odyssey4meof course, there's the other question - what're they planning on using containers for? like is there an actual use-case for it or is the intention to just jump on the bandwagon because of all the market noise18:17
v1k0d3nsecurity need to stay head and keep wild cowboys in check. i'm security. i just build, learn, fail...fail...fail...fail...learn...teach.18:18
*** phiche has joined #openstack-ansible18:18
v1k0d3nand even at this point, i'm having so much fun learning, i wonder if i haven't become a cowboy myself now.18:19
*** javeriak_ has quit IRC18:19
v1k0d3nmetral: you mentioned you're not using ingress right now? i guess carina is coming about with swarm because it's a bit easier than the k8s model?18:20
metralI'm not using ingress personally but it is available in k8s v1.2 so you can in fact use it, and it works18:23
v1k0d3nyeah, i ran into some priv mode issues with corekube that i am working through right now.18:23
metralswarm vs k8s on carina isn't about difficulty, it was just a good next logical step for containers coming from just the single docker engine POV18:23
v1k0d3ni think it's on the docker side...but from what i read both docker 1.10 and kubenetes since 1.1 are default priv escalation mode +.18:24
v1k0d3nso working through it, and exploring it.18:24
metralyea like we chatted about the other day, so long as you have 1.10 and the flags enabled in the apiserver & kubelet to allow privileged mode, you're fine - you just have to work through the corekube "details" to make them stick. you can easily just take down the apiserver & kubelet right now, add in the flags to their respective unit files and fire them back up and it'll work18:25
metralbut on reboots it'll use the units shipped w/ corekube by default18:26
v1k0d3noh, i remember the issue i had....18:26
v1k0d3n./rebuild caught error on line 5...18:26
v1k0d3ni have to look. i ran into that and had to leave...didn't have time to dig in at the time.18:26
*** pjm6_ has joined #openstack-ansible18:27
v1k0d3nbut i think that forking and making the mods should work well enough, as long as i keep the repos pointed correctly though, right?18:27
metralhmm well that line is in reference to the discovery IP address, which if your stack came up fine (public cloud nuances and all from time to time can cause weird issues here), should be working18:27
metralyes18:28
metrallots of other folks have done similar seeing how there are 38 forks of https://github.com/metral/corekube and i've gotten inquiries about similar efforts in the past18:28
*** pjm6 has quit IRC18:29
*** Brew has quit IRC18:30
*** pjm6_ has quit IRC18:35
v1k0d3nmetral: so the run was fine...k8s is install...all looks pretty good. this is my error: ./redeploy.sh: line 5: /run/discovery_ip_port: No such file or directory18:36
v1k0d3nso file doesn't exist.18:37
cloudnullim out18:37
cloudnulltalk to you all later18:37
odyssey4meyeah, me too18:37
automagicallylater cloudnull18:37
automagicallylater odyssey4me18:37
odyssey4metime for me to pack18:37
automagicallySee ya in Austin18:37
cloudnullcheers automagically18:37
cloudnullwhen you you all get into town ?18:37
metralsee you all next week18:37
automagicallySunday afternoon18:37
odyssey4mecloudnull Sat for me.18:38
cloudnullok, im driving up sunday in the AM18:38
cloudnullso maybe grab a bite18:38
odyssey4mecloudnull I'll ping you once I'm there and settled - we can take it from there.18:38
automagically++18:38
cloudnull++ cloudnull on twitter :)18:39
cloudnulltalk to you all soon.18:39
*** chhavi_ has quit IRC18:40
openstackgerritMajor Hayden proposed openstack/openstack-ansible: Add docs for limiting access to services  https://review.openstack.org/30954118:42
mhaydenodyssey4me: bumped that review18:43
*** admin0 has joined #openstack-ansible18:57
*** sdake_ has joined #openstack-ansible19:00
*** javeriak has joined #openstack-ansible19:02
*** sdake has quit IRC19:03
spotzSo for those of you coming in on Sat, if there's any interest in mudpuppy and I coming up for dinner let me know19:03
*** admin0 has quit IRC19:07
*** eil397 has joined #openstack-ansible19:09
automagicallymhayden: You inspired this: https://review.openstack.org/#/c/309557/ as a result of your recent review19:14
*** hybridpollo has joined #openstack-ansible19:14
mhaydenautomagically: nicely done, sir!19:15
spotzwatch out he's getting into security:)19:16
automagicallyMy first deep dive into an OSA role was the security role actually19:16
* mhayden starts to make tears19:16
mhayden:)19:16
automagicallyI had been wanting to play around with OpenSCAP for awhile and threw this together: https://github.com/trumant/openstack-ansible-security/commit/63055a9788a02767d89732ad3fff223b4f49f24819:17
automagicallyWas a fun reason to use two things I’d been wanting to try out for awhile during a 20% time day19:17
mhaydenah nice19:17
mhaydenSCAP's XML is kinda freaky19:18
automagicallyIts XML….19:18
automagicallyIts all freaky to me19:18
automagicallyYAML is bad enough19:18
mhaydeni'd considered writing a YAML to SCAP XML converter19:18
automagicallyI think the GovCloud folks have one19:18
mhaydenso people could describe tests and fixes in something more human readable19:18
Mudpuppyis it YAML or YML?19:19
mhaydenhttps://en.wikipedia.org/wiki/YAML19:19
*** weezS has quit IRC19:20
mhaydenYAML is a recursive acronym for "YAML Ain't Markup Language". Early in its development, YAML was said to mean "Yet Another Markup Language",[3] but it was then reinterpreted (backronyming the original acronym) to distinguish its purpose as data-oriented, rather than document markup.19:20
automagicallymhayden: This - https://github.com/GovReady/easyscap19:20
mhaydenBACKRONYMING -- impressive19:20
mhaydenautomagically: oh my, that's way better19:20
automagicallyLater all. Time for me to call it a day. Hope to see many of you in Austin19:29
spotzLater automagically19:32
*** phiche has quit IRC19:34
eil397automagically: thank for link.19:36
*** sdake_ is now known as sdake19:38
*** johnmilton has quit IRC19:48
*** jayc has quit IRC20:04
palendaeMudpuppy: The extensions yaml and yml are both valid20:04
*** abitha has joined #openstack-ansible20:04
*** dweaver has quit IRC20:08
*** phalmos has quit IRC20:09
*** jayc has joined #openstack-ansible20:17
*** mkrish004c has quit IRC20:22
*** falanx has quit IRC20:27
*** javeriak has quit IRC20:31
*** falanx has joined #openstack-ansible20:40
*** Mudpuppy has quit IRC20:45
*** admin0 has joined #openstack-ansible20:48
*** spotz is now known as spotz_zzz20:49
*** falanx has quit IRC20:50
*** retreved has quit IRC20:56
*** thorst has quit IRC21:03
*** weezS has joined #openstack-ansible21:05
*** thorst has joined #openstack-ansible21:07
*** weezS has quit IRC21:10
*** busterswt has quit IRC21:10
*** jthorne_ has joined #openstack-ansible21:11
*** jthorne has quit IRC21:11
*** jthorne_ has quit IRC21:12
*** thorst has quit IRC21:12
*** jthorne has joined #openstack-ansible21:13
*** psilvad has quit IRC21:16
*** jayc has quit IRC21:25
mhaydenhas anyone run into issues with neutron migrations lately? https://gist.github.com/major/53da659a71b7b9ee9ed4e78d5c7cd44821:27
mhaydenin mitaka21:27
*** BjoernT has joined #openstack-ansible21:29
mhaydenneutron seems to be functional without that expand run21:29
*** thorst has joined #openstack-ansible21:30
*** thorst has quit IRC21:35
*** thorst has joined #openstack-ansible21:37
*** jayc has joined #openstack-ansible21:37
sigmavirus24mhayden: looking at the security role, do any of the kernel tasks require a restart?21:42
*** jayc has quit IRC21:49
*** electrofelix has quit IRC21:54
*** mancdaz has quit IRC22:06
*** Oku_OS-away has quit IRC22:07
*** Oku_OS-away has joined #openstack-ansible22:07
*** mancdaz has joined #openstack-ansible22:08
cloudnullmhayden: yes22:12
*** psilvad has joined #openstack-ansible22:12
cloudnullthats an issue with a mighration not being in the source code that had happened previously22:12
cloudnullnormally that happens when the DB was updated to something like the head of newton and then you go to redeploy / upgrade to a later version of mitaka.22:13
*** sdake has quit IRC22:17
*** sigmavirus24 is now known as sigmavirus24_awa22:23
odyssey4memhayden that may be because you deployed 13.0.0 (rc-based openstack) then updated to 13.0.122:25
*** aludwar has quit IRC22:29
*** markvoelker has quit IRC22:39
*** sdake has joined #openstack-ansible22:42
*** admin0 has quit IRC22:55
*** markvoelker has joined #openstack-ansible23:09
*** woodard_ has quit IRC23:15
*** woodard has joined #openstack-ansible23:15
*** matt6434 is now known as mattoliverau23:19
*** psilvad has quit IRC23:21
*** galstrom is now known as galstrom_zzz23:30
*** dweaver has joined #openstack-ansible23:30
*** dweaver has quit IRC23:32
*** dweaver has joined #openstack-ansible23:32
*** BjoernT has quit IRC23:34
*** dweaver has quit IRC23:37
*** woodard has quit IRC23:43
*** eil397 has left #openstack-ansible23:49
*** dweaver has joined #openstack-ansible23:51
*** dweaver has quit IRC23:52
*** dweaver has joined #openstack-ansible23:52

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!