| *** jtriley has quit IRC | 00:09 | |
| *** sdake has joined #kolla | 00:23 | |
| *** sdake has quit IRC | 00:23 | |
| *** subscope has quit IRC | 00:27 | |
| *** achanda has quit IRC | 00:36 | |
| *** Jeffrey4l has quit IRC | 00:39 | |
| *** salv-orl_ has joined #kolla | 00:42 | |
| *** salv-orlando has quit IRC | 00:45 | |
| *** iceyao has joined #kolla | 01:00 | |
| *** salv-orl_ has quit IRC | 01:00 | |
| *** iceyao_ has joined #kolla | 01:10 | |
| *** iceyao__ has joined #kolla | 01:11 | |
| *** iceyao_ has quit IRC | 01:11 | |
| *** iceyao has quit IRC | 01:14 | |
| *** tfukushima has joined #kolla | 01:17 | |
| *** alisonh has quit IRC | 01:30 | |
| *** alisonh has joined #kolla | 01:35 | |
| *** iceyao__ has quit IRC | 01:36 | |
| *** achanda has joined #kolla | 01:37 | |
| *** iceyao has joined #kolla | 01:37 | |
| *** iceyao_ has joined #kolla | 01:42 | |
| *** achanda has quit IRC | 01:42 | |
| *** iceyao has quit IRC | 01:45 | |
| *** achanda has joined #kolla | 01:52 | |
| *** iceyao has joined #kolla | 02:03 | |
| *** iceyao_ has quit IRC | 02:06 | |
| *** Liuqing has joined #kolla | 02:06 | |
| *** salv-orlando has joined #kolla | 02:38 | |
| *** salv-orlando has quit IRC | 02:41 | |
| *** Marga_ has quit IRC | 02:51 | |
| *** salv-orlando has joined #kolla | 02:55 | |
| *** salv-orlando has quit IRC | 03:01 | |
| *** vhosakot has joined #kolla | 03:02 | |
| *** vhosakot has quit IRC | 03:05 | |
| *** Jeffrey4l has joined #kolla | 03:12 | |
| *** yuanying has quit IRC | 03:20 | |
| *** Marga_ has joined #kolla | 03:28 | |
| *** Marga_ has quit IRC | 03:29 | |
| *** Marga_ has joined #kolla | 03:29 | |
| *** klint has joined #kolla | 03:33 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Multiple hypervisor driver support for nova https://review.openstack.org/282943 | 03:38 |
|---|---|---|
| *** Marga_ has quit IRC | 03:44 | |
| *** Marga_ has joined #kolla | 03:45 | |
| *** Marga_ has quit IRC | 03:49 | |
| *** salv-orlando has joined #kolla | 03:57 | |
| *** chandankumar has joined #kolla | 04:01 | |
| *** Liuqing has quit IRC | 04:03 | |
| *** salv-orlando has quit IRC | 04:04 | |
| *** Marga_ has joined #kolla | 04:10 | |
| *** yuanying has joined #kolla | 04:10 | |
| *** iceyao_ has joined #kolla | 04:18 | |
| *** Marga_ has quit IRC | 04:20 | |
| *** iceyao has quit IRC | 04:22 | |
| *** dave-mcc_ has joined #kolla | 04:24 | |
| *** dave-mccowan has quit IRC | 04:25 | |
| *** dave-mcc_ has quit IRC | 04:28 | |
| *** tfukushima has quit IRC | 04:34 | |
| *** dave-mccowan has joined #kolla | 04:40 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Multiple hypervisor driver support for nova https://review.openstack.org/282943 | 04:50 |
| *** Marga_ has joined #kolla | 05:05 | |
| *** Allen_Gao has quit IRC | 05:17 | |
| *** chandankumar has quit IRC | 05:19 | |
| *** Allen_Gao has joined #kolla | 05:21 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Allow cinder-volume to be configured to use nfs https://review.openstack.org/279913 | 05:28 |
| *** Marga_ has quit IRC | 05:30 | |
| *** tfukushima has joined #kolla | 05:32 | |
| *** chandankumar has joined #kolla | 05:32 | |
| *** dave-mccowan has quit IRC | 05:37 | |
| *** iceyao_ has quit IRC | 05:40 | |
| *** iceyao has joined #kolla | 05:40 | |
| *** Liuqing has joined #kolla | 05:44 | |
| *** daneyon_ has quit IRC | 05:45 | |
| *** mbound has joined #kolla | 05:46 | |
| *** tpot has quit IRC | 05:49 | |
| *** Liuqing has quit IRC | 05:59 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Allow cinder-volume to be configured to use nfs https://review.openstack.org/279913 | 06:02 |
| *** mbound has quit IRC | 06:08 | |
| *** salv-orlando has joined #kolla | 06:11 | |
| *** salv-orlando has quit IRC | 06:14 | |
| *** Marga_ has joined #kolla | 06:22 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Multiple hypervisor driver support for nova https://review.openstack.org/282943 | 06:28 |
| *** alger has joined #kolla | 06:29 | |
| *** Liuqing has joined #kolla | 06:38 | |
| *** alger has quit IRC | 06:43 | |
| *** alger has joined #kolla | 06:43 | |
| *** Liuqing has quit IRC | 06:45 | |
| *** jasonsb has joined #kolla | 06:52 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: fix neutron-dhcp-agent start failure https://review.openstack.org/282971 | 06:52 |
| *** salv-orlando has joined #kolla | 06:54 | |
| *** salv-orlando has quit IRC | 07:02 | |
| *** alger has quit IRC | 07:28 | |
| *** jasonsb has quit IRC | 07:33 | |
| *** jasonsb has joined #kolla | 07:36 | |
| *** salv-orlando has joined #kolla | 07:40 | |
| *** mbound has joined #kolla | 07:40 | |
| *** alger has joined #kolla | 07:45 | |
| *** mbound has quit IRC | 07:46 | |
| *** subscope has joined #kolla | 07:51 | |
| *** subscope has quit IRC | 07:51 | |
| *** subscope has joined #kolla | 08:07 | |
| *** cui has joined #kolla | 08:11 | |
| *** akwasnie has joined #kolla | 08:14 | |
| *** macsz has joined #kolla | 08:15 | |
| *** mikelk has joined #kolla | 08:17 | |
| *** subscope has quit IRC | 08:17 | |
| cui | Hi, can i deploying nova-compute with kolla now? | 08:21 |
| *** jmccarthy has quit IRC | 08:24 | |
| *** alger has quit IRC | 08:24 | |
| *** salv-orlando has quit IRC | 08:25 | |
| *** subscope has joined #kolla | 08:26 | |
| *** iceyao_ has joined #kolla | 08:28 | |
| *** iceyao has quit IRC | 08:28 | |
| elemoine_ | cui: yes, kolla does support deploying nova-complute | 08:30 |
| ajafo | o/ | 08:31 |
| *** athomas has joined #kolla | 08:36 | |
| *** subscope has quit IRC | 08:40 | |
| cui | elemoine_:thanks! but as i know,iscsid can't work in the docker: https://bugzilla.redhat.com/show_bug.cgi?id=1102911 | 08:40 |
| openstack | bugzilla.redhat.com bug 1102911 in iscsi-initiator-utils "can't run iscsid in a docker container" [Unspecified,Closed: duplicate] - Assigned to cleech | 08:40 |
| cui | https://bugzilla.redhat.com/show_bug.cgi?id=1100000 | 08:41 |
| openstack | bugzilla.redhat.com bug 1100000 in iscsi "can't run iscsid in a docker container" [Unspecified,Assigned] - Assigned to cleech | 08:41 |
| *** alger has joined #kolla | 08:41 | |
| *** achanda has quit IRC | 08:42 | |
| *** jmccarthy has joined #kolla | 08:44 | |
| *** achanda has joined #kolla | 08:46 | |
| cui | elemoine_:How did kolla resoved this issue? | 08:49 |
| *** achanda has quit IRC | 08:58 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: [WIP] Add openvswitch support https://review.openstack.org/282384 | 09:00 |
| *** subscope has joined #kolla | 09:00 | |
| akwasnie | morning guys | 09:04 |
| nihilifer | hi akwasnie | 09:04 |
| *** iceyao_ has quit IRC | 09:04 | |
| *** iceyao has joined #kolla | 09:05 | |
| *** iceyao has quit IRC | 09:08 | |
| *** iceyao has joined #kolla | 09:08 | |
| mdnadeem | . | 09:08 |
| elemoine_ | morning | 09:11 |
| *** bmace has quit IRC | 09:15 | |
| *** alger has quit IRC | 09:17 | |
| *** tzn has joined #kolla | 09:23 | |
| *** subscope has quit IRC | 09:25 | |
| *** athomas has quit IRC | 09:27 | |
| *** subscope has joined #kolla | 09:29 | |
| *** pbourke has joined #kolla | 09:30 | |
| *** iceyao_ has joined #kolla | 09:38 | |
| *** iceyao has quit IRC | 09:38 | |
| openstackgerrit | MD NADEEM proposed openstack/kolla: Swift ubuntu binary container https://review.openstack.org/277294 | 09:39 |
| *** athomas has joined #kolla | 09:42 | |
| *** athomas has quit IRC | 09:48 | |
| *** athomas has joined #kolla | 09:51 | |
| *** cui has quit IRC | 10:02 | |
| *** iceyao_ has quit IRC | 10:11 | |
| *** iceyao has joined #kolla | 10:11 | |
| elemoine_ | hi akwasnie I'll open a Gerrit CR setting up Heka/ES integration this week | 10:17 |
| akwasnie | hi elemoine_:great, please let me know if you need any help | 10:22 |
| *** salv-orlando has joined #kolla | 10:26 | |
| elemoine_ | I hope we can collaborate on this | 10:28 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add openvswitch support https://review.openstack.org/282384 | 10:29 |
| *** salv-orlando has quit IRC | 10:30 | |
| *** bmace has joined #kolla | 10:39 | |
| akwasnie | elemoine_:yes sure, looking forward to | 10:40 |
| *** subscope has quit IRC | 10:57 | |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Run yum clean all in Heka Dockerfile https://review.openstack.org/283034 | 10:59 |
| *** iceyao has quit IRC | 10:59 | |
| *** akwasnie has quit IRC | 11:01 | |
| *** tzn has quit IRC | 11:10 | |
| *** tfukushima has quit IRC | 11:16 | |
| *** akwasnie has joined #kolla | 11:24 | |
| *** subscope has joined #kolla | 11:30 | |
| *** salv-orlando has joined #kolla | 11:33 | |
| *** gfidente has joined #kolla | 11:37 | |
| openstackgerrit | Merged openstack/kolla-mesos: Updated from global requirements https://review.openstack.org/282746 | 11:39 |
| *** elemoine_ has quit IRC | 11:40 | |
| *** elemoine_ has joined #kolla | 11:40 | |
| *** zigo has quit IRC | 11:40 | |
| *** zigo has joined #kolla | 11:41 | |
| *** salv-orlando has quit IRC | 11:43 | |
| *** subscope has quit IRC | 11:45 | |
| *** subscope has joined #kolla | 11:45 | |
| *** chandankumar has quit IRC | 12:00 | |
| *** limamauricio has joined #kolla | 12:03 | |
| *** liyi has joined #kolla | 12:05 | |
| liyi | after deploy, I try to login with admin:password, keep getting " Unauthorized. Please try logging in again." | 12:06 |
| liyi | anyone experiencing similar problem | 12:06 |
| limamauricio | i have this error when i try to build centos/binary http://pastebin.com/eRwUnu2V | 12:10 |
| *** tzn has joined #kolla | 12:10 | |
| *** tzn has quit IRC | 12:11 | |
| pbourke | limamauricio: there is a review open for that issue | 12:13 |
| limamauricio | :) | 12:13 |
| *** gfidente has quit IRC | 12:14 | |
| limamauricio | pbourke, can you send me the review? | 12:14 |
| pbourke | https://review.openstack.org/#/c/282906/ | 12:15 |
| limamauricio | Thanks | 12:15 |
| *** JoseMello has joined #kolla | 12:15 | |
| liyi | pbourke, this the change solve the problem in ubuntu as well? | 12:16 |
| pbourke | liyi: good question | 12:16 |
| pbourke | liyi: does ubuntu have the same problem? | 12:16 |
| akwasnie | pbourke: I checked and there is no problem with ubuntu | 12:16 |
| liyi | sorry guy, I mistake the topic | 12:18 |
| liyi | I thought you are talking about " Unauthorized. Please try logging in again." as i am experiencing | 12:18 |
| *** iceyao has joined #kolla | 12:19 | |
| liyi | @akwasnie, are you able to deploy and use openstack on ubuntu at the moment? | 12:20 |
| *** rhallisey has joined #kolla | 12:21 | |
| *** tzn has joined #kolla | 12:23 | |
| akwasnie | liyi: testing right now | 12:24 |
| limamauricio | akwasnie you use ubuntu? | 12:24 |
| liyi | @akwasnie Thanks, looking forward to your good news | 12:25 |
| *** stvnoyes has quit IRC | 12:35 | |
| *** chandankumar has joined #kolla | 12:36 | |
| *** stvnoyes has joined #kolla | 12:37 | |
| *** salv-orlando has joined #kolla | 12:40 | |
| *** salv-orlando has quit IRC | 12:44 | |
| *** chandankumar has quit IRC | 12:52 | |
| akwasnie | limamauricio: yes | 12:54 |
| limamauricio | ubuntu/binary or ubuntu/source? | 12:55 |
| akwasnie | ubuntu/source | 12:55 |
| limamauricio | você pode iniciar uma instância com sucesso? | 12:56 |
| limamauricio | can you launch a instance successfully?*** | 12:56 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make cleanup-containers remove bootstrap_heka https://review.openstack.org/283065 | 12:57 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make cleanup-containers remove Heka volumes https://review.openstack.org/283066 | 13:00 |
| akwasnie | limamauricio: I am rebuilding everything right now | 13:01 |
| limamauricio | ok | 13:01 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Change ubuntu base_image for vagrant to wily64 https://review.openstack.org/281968 | 13:03 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Harmonise vagrant provider defaults https://review.openstack.org/283068 | 13:03 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Fix vagrant bootstrapping for ubuntu https://review.openstack.org/283069 | 13:03 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Create "heka" group and add "heka" user to it https://review.openstack.org/283071 | 13:06 |
| *** kproskurin has joined #kolla | 13:11 | |
| *** chandankumar has joined #kolla | 13:12 | |
| *** dwalsh has joined #kolla | 13:14 | |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Fix vagrant bootstrapping for ubuntu https://review.openstack.org/283069 | 13:26 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Harmonise vagrant provider defaults https://review.openstack.org/283068 | 13:26 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Change ubuntu base_image for vagrant to wily64 https://review.openstack.org/281968 | 13:26 |
| *** cui has joined #kolla | 13:29 | |
| *** chandankumar has quit IRC | 13:32 | |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Remove duplicate code in nova extend_start.sh https://review.openstack.org/283082 | 13:43 |
| *** salv-orlando has joined #kolla | 13:45 | |
| *** salv-orlando has quit IRC | 13:52 | |
| *** v1k0d3n has joined #kolla | 13:53 | |
| *** subscope has quit IRC | 13:56 | |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Change Heka sudoers file mode bits https://review.openstack.org/283086 | 13:57 |
| *** klint has quit IRC | 13:58 | |
| *** fthiagogv has joined #kolla | 14:02 | |
| *** macsz has quit IRC | 14:03 | |
| *** chandankumar has joined #kolla | 14:04 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Add a precheck for MountFlags=shared https://review.openstack.org/281279 | 14:06 |
| *** subscope has joined #kolla | 14:09 | |
| *** dwalsh has quit IRC | 14:10 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla: Pin Mesos to 0.27.0 version https://review.openstack.org/283091 | 14:10 |
| *** wadeholler has joined #kolla | 14:11 | |
| *** dwalsh has joined #kolla | 14:12 | |
| wadeholler | hi all. brand new user from a koala perspective. trying to deploy and hit a early ansible error "cannot access /var/lib/mysql/mariadb.pid: No such file or directory" | 14:12 |
| wadeholler | ill put the message on pastbin now | 14:12 |
| *** dwalsh has quit IRC | 14:13 | |
| wadeholler | ansible error is @ http://pastebin.com/83jL0ege | 14:13 |
| wadeholler | anyone on that can help ? | 14:13 |
| *** dave-mccowan has joined #kolla | 14:14 | |
| *** dwalsh has joined #kolla | 14:14 | |
| *** openstackgerrit has quit IRC | 14:17 | |
| *** openstackgerrit has joined #kolla | 14:17 | |
| *** blahRus has joined #kolla | 14:19 | |
| elemoine_ | wadeholler: what does "docker logs mariadb" tell you? | 14:21 |
| *** sdake has joined #kolla | 14:21 | |
| sdake | yo | 14:21 |
| sdake | masters of orion out next week | 14:21 |
| * sdake taking the month off | 14:21 | |
| kproskurin | Oh it really depends it it will be the same MoO and not something like MoO3 | 14:26 |
| wadeholler | elemoine: - just looks like it is starting to me. maybe start is hung ? http://pastebin.com/nxuVkJC9 | 14:26 |
| elemoine_ | kproskurin: wrong channel? | 14:26 |
| kproskurin | In a way :-) | 14:27 |
| sdake | Jeffrey4l ping | 14:30 |
| elemoine_ | wadeholler: and when you do the operation manually: "docker exec mariadb ls /var/lib/mysql/mariadb.pid" ? | 14:30 |
| wadeholler | eleomine: ls: cannot access /var/lib/mysql/mariadb.pid: No such file or directory | 14:31 |
| rhallisey | sdake, what the heck is masters of orion | 14:32 |
| sdake | dude | 14:32 |
| sdake | just the best video game in the world | 14:32 |
| sdake | diablo like from the old days :) | 14:32 |
| sdake | build sapceships conquer the universe | 14:33 |
| sdake | apparentl ythe new one is online | 14:33 |
| sdake | ever play "tradwars" | 14:33 |
| sdake | its like a grphical version of that game | 14:33 |
| wadeholler | eleomine: other files are there - mysql.sock, gallera.cache, ..... | 14:33 |
| rhallisey | sdake, I skipped a lot of 90's video games | 14:34 |
| pbourke | wadeholler: ansible is waiting for mariadb to start by checking for the pid file 6 times | 14:34 |
| rhallisey | :) | 14:34 |
| elemoine_ | wadeholler: you use kolla master right? | 14:34 |
| sdake | probablybecause youwere 5 ;) | 14:34 |
| pbourke | wadeholler: I would check the mariadb logs to see why its failing to start | 14:34 |
| rhallisey | sdake, yes :). The first game I played like crazy on pc was warcraft 3 | 14:35 |
| sdake | mariadb failing to start because of the pdi issue? | 14:35 |
| wadeholler | elemoine: yes. cloned down per the quick start on saturday | 14:35 |
| sdake | pid issue? | 14:35 |
| pbourke | wadeholler: if the mariadb container is running you can exec into it and look under /var/log | 14:35 |
| elemoine_ | wadeholler: pbourke or look at /var/lib/docker/volumes/kolla_logs/_data/mariadb/mariadb.log | 14:35 |
| sdake | docker exec first_3_numbers_of_container_id ls /var/lib/docker/volumes/kolla_logs | 14:35 |
| pbourke | ah yes I forgot elemoine_ is our logging expert now :) | 14:35 |
| sdake | i'd just run docker logs on it though | 14:36 |
| sdake | to make sure its not the pid issue | 14:36 |
| sdake | been seeing alot of reports of that lately | 14:36 |
| elemoine_ | sdake: wadeholler already provided a pastebin with the output of docker logs mariadb | 14:36 |
| sdake | elemoine_ i was not here | 14:36 |
| sdake | was it the PID issue? | 14:36 |
| sdake | this happens over and over to people | 14:37 |
| sdake | i'm not sure what the resolution is | 14:37 |
| pbourke | docker logs is not reliable in this case | 14:37 |
| sdake | but we need a bug filed! | 14:37 |
| pbourke | actually no scratch that | 14:37 |
| pbourke | ignore me | 14:37 |
| wadeholler | end of mariadb.log http://pastebin.com/LhUztxb6 | 14:38 |
| openstackgerrit | Merged openstack/kolla: Create "heka" group and add "heka" user to it https://review.openstack.org/283071 | 14:39 |
| wadeholler | I have re-ran the ansible deploy, docker stop, docker rm all containers, docker rmi mariadb, tools/build.py mariadb, re-ran deploy - same result...... what next ? | 14:39 |
| pbourke | I wonder is it a case of a bad data vol left over from previous deploy | 14:41 |
| *** salv-orlando has joined #kolla | 14:41 | |
| elemoine_ | wadeholler: are you running aio or multinode? (just for me to know) | 14:42 |
| wadeholler | multinode | 14:42 |
| pbourke | wadeholler: try copying tools/cleanup-containers to your target node(s) | 14:42 |
| *** chandankumar has quit IRC | 14:42 | |
| pbourke | run that, then re-run ansible | 14:42 |
| wadeholler | will do. | 14:42 |
| sdake | pbourke he didn't remove the named volumes | 14:43 |
| sdake | but we should not have ot remove named volumes in order for a redeploy to work | 14:44 |
| pbourke | sdake: we shouldn't but I think in this case it might be the problem | 14:44 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Run yum clean all in Heka Dockerfile https://review.openstack.org/283034 | 14:44 |
| pbourke | sdake: maybe not, it's all I've got right now :) | 14:44 |
| *** jtriley has joined #kolla | 14:45 | |
| wadeholler | p.s. - I think it doesn't matter but for koala-toolbox I had to remove "mariadb-libs" from the list to get it to build. otherwise it faulted with a yum conflict on my.cnf | 14:45 |
| *** akwasnie has quit IRC | 14:45 | |
| pbourke | wadeholler: that's correct, there is a patch for that | 14:45 |
| wadeholler | and I will eventually fix the spellchecker for koala -> kolla :) | 14:46 |
| elemoine_ | wadeholler: yes that's a known issue that is being addressed by https://review.openstack.org/#/c/282906/ | 14:46 |
| sdake | pbourke link to patch please? | 14:46 |
| elemoine_ | sdake: https://review.openstack.org/#/c/282906/ | 14:46 |
| wadeholler | pbourke: it appears to be working, mariadb.pid exists and ansible is into the nova section now - THANKS! | 14:48 |
| sdake | wadeholler what did you do | 14:49 |
| pbourke | wadeholler: good to hear | 14:49 |
| sdake | ok pbourke lets not treat the symptom, lets fix the problem ;) | 14:49 |
| elemoine_ | wadeholler: was it your first deploy? | 14:49 |
| sdake | this happens over and over - like 7+ times on irc in the last week from different individuals | 14:50 |
| sdake | so lets understand what just worked, and why it worked | 14:50 |
| sdake | i'm out of the loop on what hte solution was | 14:50 |
| sdake | would someone mind offering that up? | 14:50 |
| sdake | pbourke ^^ | 14:50 |
| sdake | I always hear "clean up your environment" - that is a really bad answer to redeploy | 14:51 |
| pbourke | sdake: when you say it's happened 7+ times over the past week, it may not be the same issue | 14:51 |
| sdake | mariadb.pid problem right? | 14:51 |
| dave-mccowan | hello ansible gurus... is there an ansible module that will simply concatenate two files? I'm trying to avoid using "shell: cat file1 file2 > bigfile", but that is the exact functionality i want. | 14:52 |
| SamYaple | morning | 14:53 |
| pbourke | sdake: mariadb failed to start. the only way ansible can tell this is by the lack of pid, so it shows as a series of 'ls /var/lib/mysql/mariadb.pid file not found' errors | 14:53 |
| wadeholler | sdake: yes I think: stderr: "ls: cannot access /var/lib/mysql/mariadb.pid: No such file or directory" http://pastebin.com/83jL0ege | 14:53 |
| SamYaple | sdake: "cleanup you environemtn" is a great answer for _redeploy_ | 14:53 |
| pbourke | sdake: why did it fail to start? that's the hard part, but from experience its due to a previous failure in the deploy process, which leaves the data vol in a state where it can't be reclustered | 14:54 |
| pbourke | sdake: the failure that left it in that state could be anything | 14:54 |
| pbourke | sdake: I understand trashing the env is not a good solution to the problem, I just also want to help new users get running quickly | 14:55 |
| cui | Hi,all. Anybody know if kolla resolved this issue "iscsid can't work in docker container"? https://bugzilla.redhat.com/show_bug.cgi?id=1100000 | 14:55 |
| openstack | bugzilla.redhat.com bug 1100000 in iscsi "can't run iscsid in a docker container" [Unspecified,Assigned] - Assigned to cleech | 14:55 |
| cui | when deploying nova-compute | 14:55 |
| SamYaple | i mean for a full redploy you _do_ want a clean environment pbourke | 14:55 |
| sdake | in the field, do we expect people to onl yrun "deploy" once? | 14:56 |
| pbourke | also SamYaple was not around and I don't know much about mysql :( | 14:56 |
| SamYaple | automating a "cleaning" is a really bad idea (thats going to trash production environments" | 14:56 |
| pbourke | SamYaple: no one is advocating that | 14:56 |
| pbourke | unless I missed something | 14:56 |
| SamYaple | sdake has suggested we automatically fix those issues | 14:57 |
| sdake | pbourke i was not advocating deleting anything | 14:57 |
| sdake | ok lets come to a common definition of how "deploy" operation is supposed to work | 14:57 |
| sdake | in the past it has meant upgrade + reconfigure | 14:57 |
| sdake | since we are breaking those steps apart | 14:57 |
| sdake | do we expect then deploy to just happen once? | 14:57 |
| ccesario | is there planned any kind of UI (eg Kolla-UI) for manage/monitor the docker images/services!? | 14:58 |
| sdake | i am fine with deleting the environment on a failed deploy if it happens once | 14:58 |
| SamYaple | dave-mccowan: hmmm i know of no simple task to do that. you hve a few options, 3 tasks : read file1, read file2, write bigfile | 14:58 |
| SamYaple | dave-mccowan: option 2 is to create a simple module to do this | 14:58 |
| sdake | but if kolla-ansible deploy happens over and over (as is the case now) on the same box to upgrade/reconfgiure, then we have a problem | 14:59 |
| SamYaple | sdake: unless something has changes, we said back in liberty if a bootstrap action fails it is expected you either rekick or reset just that one service | 14:59 |
| SamYaple | bootstrap action being key | 14:59 |
| SamYaple | thats a one time special action that happens once per environment | 14:59 |
| sdake | i understand how bootstrap works | 14:59 |
| SamYaple | no one said you didnt | 14:59 |
| SamYaple | my point was _if_ that fails its quickest to have the user reset teh evnrionemtne | 14:59 |
| sdake | let me explain how I have been using deploy | 14:59 |
| sdake | kolla-ansible deploy | 15:00 |
| sdake | hey look my environment is up, let me reconfigure | 15:00 |
| SamYaple | technically they could fix the issue, but the path to quickest working is reset teh envrionemtn | 15:00 |
| sdake | kolla-ansible deploy | 15:00 |
| sdake | hey look I need to change something again | 15:00 |
| sdake | kolla-ansible deploy | 15:00 |
| SamYaple | the issue here is your premises is a working kolla-ansible deploy the first time | 15:00 |
| sdake | so we have this fixed with reconfigure | 15:00 |
| SamYaple | thats where you have you flaw in flow | 15:00 |
| SamYaple | the initial bootstrap is failing | 15:00 |
| sdake | well lets figure out what the flow should be and get it documented | 15:00 |
| sdake | also initial bootstrap should not be failing imo ;-) | 15:01 |
| SamYaple | if bootstrap fails, rekick | 15:01 |
| SamYaple | it fails for alot of reasons aoutside our control | 15:01 |
| SamYaple | including user error _alot_ | 15:01 |
| dave-mccowan | SamYaple as a code reviewer, which option would you prefer for https://review.openstack.org/#/c/282733/ ? For this use case, maybe "shell: cat" is OK? | 15:01 |
| SamYaple | dave-mccowan: any use of shell module will be met with -1 | 15:02 |
| sdake | I don't like shell module | 15:02 |
| SamYaple | its a vary dangeours one we avoid at all costs | 15:02 |
| sdake | security problem | 15:02 |
| SamYaple | there is always a way around it | 15:02 |
| sdake | if deploy is only meant to happen one time, we should not allow a second deploy to occur if the first one has proceeded in any way | 15:03 |
| sdake | but i'm not hearing any answers on if deploy is meant to only happen once or not | 15:03 |
| SamYaple | deploy is idempotent | 15:03 |
| sdake | it clearly is not idempotent | 15:03 |
| sdake | you mean the intent ? | 15:03 |
| SamYaple | it is if you look up what idempotent means | 15:04 |
| sdake | I know what idempotent means | 15:04 |
| SamYaple | but the issue is if it progresses to a certain point, it will think bootstrap is finished and not do it again | 15:04 |
| SamYaple | clearly not | 15:04 |
| SamYaple | deploy _is_ idempotent | 15:04 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Fix Swift logging https://review.openstack.org/283118 | 15:04 |
| sdake | doesn't deploy pull in new configs? | 15:05 |
| SamYaple | it will push new configs | 15:05 |
| SamYaple | it doesnt restart services | 15:05 |
| sdake | need a mullligan optoin ;) | 15:06 |
| sdake | ok I guess I misunderstood in thinking it restarted the services on config changes | 15:06 |
| SamYaple | if a bootstrap fails, theserivce needs to be cleaned up | 15:06 |
| SamYaple | easiest way to do that is a full rekick | 15:06 |
| SamYaple | its not a requiremetns | 15:06 |
| SamYaple | but its the fastest way to a working deploy | 15:06 |
| pbourke | thats kind of what I was getting at | 15:06 |
| SamYaple | ^^ | 15:06 |
| SamYaple | we talked about this at lenght before liberty | 15:06 |
| pbourke | ideally it should never fail but there are many reasons it can, many not our fault | 15:06 |
| sdake | ok, well now I understand the problem, it can be documented | 15:07 |
| sdake | the mariadb pid thing seems to havebeen failign alot recently | 15:07 |
| sdake | as i said 7+ in the last week | 15:07 |
| pbourke | there may be some common reasons its failing that we *can* address | 15:07 |
| SamYaple | sdake: thats because of partial deploys without full cleans | 15:07 |
| sdake | its not a dirty environment, its a failed deploy | 15:07 |
| SamYaple | it was because of a dirty environemnt | 15:07 |
| SamYaple | it always is | 15:07 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Fix Swift logging https://review.openstack.org/283118 | 15:08 |
| sdake | the dirty environment is a failed deploy ? | 15:08 |
| SamYaple | no it failed because of dirty environment | 15:08 |
| sdake | here is the order of operations | 15:08 |
| sdake | deploy | 15:08 |
| sdake | deploy fails | 15:08 |
| SamYaple | someone deployed, something broke or they cancelled it | 15:08 |
| sdake | deploy again | 15:08 |
| SamYaple | they redeploy and it fails | 15:08 |
| sdake | mariadb implodes | 15:08 |
| SamYaple | i have yet to see a non-user error problem cause this fyi | 15:08 |
| sdake | i'm not proposing to fix that | 15:08 |
| SamYaple | if the prechecks are run, then the deploy is run if it breaks it has always been user error | 15:09 |
| sdake | what user error | 15:09 |
| sdake | VIP=local IP? | 15:09 |
| *** inc0 has joined #kolla | 15:09 | |
| SamYaple | user had a bad configuration. user canceled the process | 15:09 |
| *** inc0_ has joined #kolla | 15:09 | |
| SamYaple | user did this user did that | 15:09 |
| SamYaple | not once have i found an issue with the process as it currently is (we had issue in the past, sure0 | 15:10 |
| sdake | here is how i would handle these problems in the future on irc | 15:10 |
| sdake | "You deployed once, and it failed in some way" | 15:10 |
| sdake | "Kolla does not delete data on a failed deploy" | 15:10 |
| sdake | "the fastest way to get running again is clean up your environment and deploy again" | 15:10 |
| sdake | "we find deploy errors are nearly almost always operator error" | 15:10 |
| sdake | explain WHY | 15:11 |
| SamYaple | i have indeed done this | 15:11 |
| SamYaple | i agree with your approach | 15:11 |
| pbourke | sounds good to me | 15:11 |
| sdake | because without the WHY they are left wondering if the code is solid enough to use in an eval | 15:11 |
| sdake | thanks pbourke ;) | 15:11 |
| sdake | SamYaple i msut have missed the explination, but its good to repeat each time | 15:11 |
| sdake | or even put in the documenation ;) | 15:11 |
| *** chandankumar has joined #kolla | 15:12 | |
| sdake | i'll write docs for that now | 15:13 |
| SamYaple | sdake: post a docs patch and ill approve! | 15:13 |
| SamYaple | yay | 15:13 |
| pbourke | sdake: I suspect the why is rooted in how the cluster is formed and how it doesn't recover well from being halted during that process | 15:13 |
| *** inc0 has quit IRC | 15:13 | |
| *** inc0_ has quit IRC | 15:13 | |
| pbourke | sdake: if you know anything about that process would definitely be great to get into doc form | 15:14 |
| *** inc0 has joined #kolla | 15:14 | |
| sdake | pbourke I don't know how that particular cluster forms | 15:14 |
| *** inc0_ has joined #kolla | 15:14 | |
| pbourke | didnt you write it? :) | 15:14 |
| sdake | pbourke I think we need to figure out if there is still a problem with mariadb | 15:14 |
| sdake | pbourke mariadb + galera? no | 15:14 |
| pbourke | ah yeah you were corosync | 15:14 |
| sdake | i had help with corosync | 15:15 |
| sdake | but most of totem I wrote myself | 15:15 |
| *** gfidente has joined #kolla | 15:15 | |
| *** spisarski has joined #kolla | 15:15 | |
| sdake | and tested on thousands of customers ;-) | 15:15 |
| *** jtriley has quit IRC | 15:15 | |
| sdake | btw our meetings are moved aroudn now | 15:16 |
| sdake | I'm not sure which is an odd week and which is an even week | 15:16 |
| sdake | but 23:00 is the new voted time | 15:16 |
| *** inc0___ has joined #kolla | 15:16 | |
| SamYaple | pbourke: I will tell you how! | 15:17 |
| pbourke | SamYaple: i should read myself and then come back with questions | 15:17 |
| SamYaple | pbourke: so in /var/lib/mysql is a file called "galera.dat" it contains information about the cluster | 15:17 |
| wadeholler | ok next prob. bootstrap_neutron failure http://pastebin.com/wdeLqp1R sorry to be so needy | 15:18 |
| pbourke | SamYaple: but I can do the lazy way if you have time :p | 15:18 |
| SamYaple | pbourke: you need to find the galera.dat file with the highst seq number, thats your new master | 15:18 |
| *** inc0 has quit IRC | 15:18 | |
| SamYaple | pbourke: once you have that you need to start _taht_ node first | 15:18 |
| *** inc0 has joined #kolla | 15:18 | |
| pbourke | wadeholler: looks possible like a bug in recent logging changes. elemoine_ ? | 15:19 |
| SamYaple | to start that node you need to pass '--wsrep-new-cluster' or set the 'gcomm://' list to empty in the mysql.cnf | 15:19 |
| *** inc0_ has quit IRC | 15:19 | |
| *** ayoung has joined #kolla | 15:19 | |
| SamYaple | elemoine_: wadeholler pbourke that is a problem, ill submit a patch right now | 15:19 |
| *** inc0 has quit IRC | 15:20 | |
| *** inc0___ has quit IRC | 15:20 | |
| elemoine_ | pbourke: looking | 15:20 |
| SamYaple | elemoine_: its a problem with the sudoers file | 15:21 |
| elemoine_ | pbourke: you're right, that's certainly related to the recent logging changes | 15:21 |
| SamYaple | i suppose this wasnt tested split node | 15:22 |
| SamYaple | elemoine_: ive got a patch brewing | 15:22 |
| elemoine_ | SamYaple: cool | 15:22 |
| *** inc0 has joined #kolla | 15:24 | |
| inc0 | hey good morning | 15:25 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Fix sudoers files issue for neutron from Heka https://review.openstack.org/283127 | 15:25 |
| SamYaple | mornign inc0 | 15:25 |
| openstackgerrit | Jeff Peeler proposed openstack/kolla: Playbook for glance upgrade https://review.openstack.org/278040 | 15:26 |
| elemoine_ | SamYaple: is tools/deploy_aio.sh used on the gates? anywhere else? | 15:26 |
| SamYaple | elemoine_: only in gate | 15:27 |
| *** iceyao has quit IRC | 15:27 | |
| sdake | what removes the named volumes? | 15:27 |
| sdake | do we have a script for that already? | 15:27 |
| elemoine_ | sdake: cleanup_containers | 15:27 |
| sdake | coool thanks | 15:27 |
| *** inc0 has quit IRC | 15:28 | |
| *** inc0 has joined #kolla | 15:28 | |
| *** tzn has quit IRC | 15:29 | |
| wadeholler | SamYaple: After adding your patch changes - do I just remove my failed bootstrap container and re-run deploy or should I just kick the whole thing with cleanup-containers ? | 15:29 |
| sdake | if your in eval, just do cleanup-containers | 15:29 |
| wadeholler | am in eval. will do | 15:30 |
| sdake | to kick the boottrap container you also need to remove the named volumes | 15:30 |
| sdake | (if there are any associated) | 15:30 |
| inc0 | so anyone here wants an ATC? I think we still need migration script from data_containers to thin volumes | 15:30 |
| openstackgerrit | Merged openstack/kolla: fix neutron-dhcp-agent start failure https://review.openstack.org/282971 | 15:32 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add docs about failures during evaluation https://review.openstack.org/283129 | 15:33 |
| *** limamauricio has quit IRC | 15:33 | |
| *** achanda has joined #kolla | 15:34 | |
| openstackgerrit | Merged openstack/kolla: Swift ubuntu binary container https://review.openstack.org/277294 | 15:35 |
| *** dwalsh has quit IRC | 15:35 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add docs about failures during evaluation https://review.openstack.org/283129 | 15:35 |
| elemoine_ | SamYaple: regarding https://review.openstack.org/#/c/283127, I don't get it. What code does "sudo chown neutron: /var/log/kolla/neutron"? | 15:35 |
| SamYaple | elemoine_: what do you mean? like what does chown do? | 15:37 |
| sdake | SamYaple thanks for your comments | 15:37 |
| sdake | i'll add them in in a bit - school rush atm | 15:37 |
| sdake | would you mind telling me, is there a kolla-ansible prechecks command? | 15:37 |
| elemoine_ | SamYaple: I mean, I don't see who is the user of that sudo you added | 15:38 |
| *** dwalsh has joined #kolla | 15:38 | |
| SamYaple | elemoine_: i dont understand what yo uare asking. it was failing due to not having permissions right? | 15:38 |
| SamYaple | fyi im rebuilding my images right now to test this patch | 15:38 |
| SamYaple | you may understand the issue better than me at this point, i just dont understand your question | 15:39 |
| sdake | elemoine_ remember the 400 chmod | 15:39 |
| sdake | !!! | 15:39 |
| openstack | sdake: Error: "!!" is not a valid command. | 15:39 |
| *** chandankumar has quit IRC | 15:39 | |
| sdake | if your sudo is failing it is because of the chmod 400 that is missing | 15:40 |
| elemoine_ | sdake: I created a patch for this already | 15:40 |
| elemoine_ | https://review.openstack.org/#/c/283086/ | 15:40 |
| sdake | are you running with it? | 15:40 |
| sdake | i missed your failure | 15:40 |
| elemoine_ | not my failure, someone else's | 15:41 |
| sdake | what was the failure | 15:41 |
| elemoine_ | SamYaple: you added "chown neutron: /var/log/kolla/neutron" to the neutron_sudoers file while I don't see any bash code in kolla that does "sudo chown neutron: /var/log/kolla/neutron" | 15:42 |
| elemoine_ | sdake: http://pastebin.com/wdeLqp1R | 15:43 |
| wadeholler | I just tried to test and got same failure. Was looking for the same thing - maybe that should be in the Dockerfile on the chown -R line ? | 15:43 |
| sdake | /var/log/kolla is a named volume | 15:44 |
| sdake | its chown should happen in the start file | 15:44 |
| *** achanda has quit IRC | 15:45 | |
| SamYaple | elemoine_: i think youre right, but i wont be able to look into it for a few minutes | 15:45 |
| SamYaple | please feel free to fix the issue the way you see fit | 15:46 |
| elemoine_ | SamYaple: yes | 15:46 |
| *** jtriley has joined #kolla | 15:47 | |
| *** inc0 has quit IRC | 15:49 | |
| elemoine_ | wadeholler: do you have the heka container running on that node? | 15:53 |
| *** inc0 has joined #kolla | 15:53 | |
| wadeholler | elemoine: yes | 15:54 |
| *** mikelk has quit IRC | 15:56 | |
| elemoine_ | are you able to run "docker inspect bootstrap_neutron"? | 15:56 |
| wadeholler | yep - will pastebin it- standby | 15:57 |
| *** inc0_ has joined #kolla | 15:57 | |
| elemoine_ | wadeholler: thanks | 15:58 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add docs about failures during evaluation https://review.openstack.org/283129 | 15:58 |
| *** inc0 has quit IRC | 15:59 | |
| wadeholler | elemoine_: http://pastebin.com/37uJyMf4 | 16:03 |
| elemoine_ | wadeholler: the problem is related to kolla_logs:/var/log/kolla not being mounted into the bootstrap_neutron container | 16:04 |
| elemoine_ | but it should be: https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/tasks/bootstrap.yml#L53 | 16:04 |
| openstackgerrit | Merged openstack/kolla: Playbook for glance upgrade https://review.openstack.org/278040 | 16:05 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add docs about failures during evaluation https://review.openstack.org/283129 | 16:06 |
| sdake | upgrades almost done !! | 16:07 |
| inc0_ | sdake, not yet, on my todo list we still have all of infra and docs | 16:08 |
| inc0_ | but I'll leave docs for post-M3 I guess | 16:08 |
| elemoine_ | wadeholler: can you check the content of your ansible/roles/neutron/tasks/bootstrap.yml file, I really don't get it… | 16:08 |
| inc0_ | or at least when we have rest done | 16:08 |
| *** chandankumar has joined #kolla | 16:10 | |
| sdake | inc0_ we can sort out docs in the rcs | 16:10 |
| wadeholler | elemoine: checking... | 16:10 |
| sdake | inc0_ I could live with just openstack upgrades for this cycle, but yup, lets try to get all the ugprades going ;) | 16:11 |
| dgonzalez | hi SamYaple, are you here? | 16:11 |
| SamYaple | dgonzalez: i am here | 16:11 |
| *** Allen_Gao has quit IRC | 16:12 | |
| inc0_ | sdake, let's try, if we will have problems we can focus on required infra upgrades, if there are any | 16:13 |
| wadeholler | elemoine_: http://pastebin.com/46vtsV9z | 16:13 |
| dgonzalez | hey, i am the author of https://review.openstack.org/#/c/283069. Just a short question: do you think i should also update the centos-bootstrap.sh script with [[]] instead of []? | 16:13 |
| elemoine_ | wadeholler: so it's correct, as expected | 16:14 |
| inc0_ | SamYaple, do you have your haproxy config changes that will change ports for air? | 16:14 |
| inc0_ | aio | 16:14 |
| elemoine_ | wadeholler: I guess that "docker volume ls" reports that the volume kolla_logs exists | 16:14 |
| SamYaple | inc0_: what do you mean? no changes are needed? | 16:14 |
| SamYaple | i run AIO with haproxy all the time... | 16:14 |
| inc0_ | well if you run it on vm VIPs won't work | 16:15 |
| inc0_ | on public cloud | 16:15 |
| SamYaple | dgonzalez: yes, always use [[ ]] | 16:15 |
| SamYaple | inc0_: if you run AIO vips still work... | 16:15 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Remove Rsyslog entirely https://review.openstack.org/283150 | 16:15 |
| SamYaple | the gate is a VM running on AIO in a public cloud | 16:15 |
| SamYaple | it uses haproxy | 16:15 |
| inc0_ | but I won't be able to connect to it, that's my point | 16:15 |
| inc0_ | I don't have second IP for it | 16:16 |
| SamYaple | but even running with haproxy and a dumb non-routable ip other address is still listening | 16:16 |
| SamYaple | im not in favor of having different oports like you asked for just now | 16:17 |
| wadeholler | [root@cpn00011 kolla]# docker volume ls DRIVER VOLUME NAME local kolla_logs | 16:17 |
| SamYaple | you are welcome to set that up yourself with static mappings (or just not use haporxy) | 16:17 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Fix vagrant bootstrapping for ubuntu https://review.openstack.org/283069 | 16:17 |
| inc0_ | SamYaple, it will be hart not to use haproxy in haproxy upgrades;) | 16:17 |
| dgonzalez | SamYaple: ok thanks, i just updated my commit | 16:17 |
| inc0_ | I don't use haproxy otherwise | 16:17 |
| elemoine_ | wadeholler: did you try "docker rm -f bootstrap_neutron" and redeploying? not saying that's a solution, just wondering… | 16:18 |
| wadeholler | i used clean_containers | 16:18 |
| SamYaple | inc0_: well if you dont have an evironment to test the upgrades properly, you probaly shouldnt be doing them inc0_ | 16:20 |
| elemoine_ | wadeholler: do you see errors in the Docker daemon logs? | 16:20 |
| wadeholler | totally new to this -probably obvious- looking now - | 16:21 |
| elemoine_ | wadeholler: or in /var/lib/docker/volumes/kolla_logs/_data/ansible.log? | 16:22 |
| inc0_ | I will test them properly soon, my env is coming in soon I hope | 16:22 |
| inc0_ | and I can test them properly, just let me make little patchiest | 16:22 |
| *** Allen_Gao has joined #kolla | 16:23 | |
| SamYaple | inc0_: hoenstly, i really dont think you need to make any changes to kolla to make this work.... | 16:24 |
| SamYaple | just use a 169.x.x.x address for the vip | 16:25 |
| SamYaple | you can still address it by your public address just like oyu owuld without hte vip | 16:25 |
| wadeholler | elemoine_: I don't understand why the time is all wrong in that (ansible.log) log (time on host is fine). I will clean-containers make sure the volumes and log of course is gone, and re-run. | 16:26 |
| SamYaple | wadeholler: time in container is UTC i believe | 16:26 |
| inc0_ | SamYaple, I can make new var "mariadb_external_port" | 16:26 |
| SamYaple | UTC time is the recommended time to run openstack on | 16:26 |
| inc0_ | and override these | 16:26 |
| SamYaple | inc0_: just but were not going to merge that... | 16:27 |
| SamYaple | why do all that extra work? | 16:27 |
| wadeholler | SamYaple: ah, thank you ... again, | 16:27 |
| inc0_ | SamYaple, why? oO | 16:27 |
| elemoine_ | wadeholler: after cleanup-containers also run docker volume rm heka_socket kolla_logs heka | 16:27 |
| inc0_ | what's your issue with that? | 16:28 |
| SamYaple | inc0_: what is the use case? its just extra cruft | 16:28 |
| inc0_ | configuring ports is cool | 16:28 |
| SamYaple | we already do that | 16:28 |
| elemoine_ | as cleanup-containers does not remove the Heka-related volumes at this point (https://review.openstack.org/#/c/283066/) | 16:28 |
| wadeholler | just noticed they were still there. will do. | 16:28 |
| inc0_ | you might want to have different ports at internal for whatever reason | 16:29 |
| inc0_ | and externally expose normal endpoints that people are used to | 16:29 |
| SamYaple | ill need to see a use case or request first then | 16:29 |
| inc0_ | I see no reason why not to have this | 16:29 |
| SamYaple | because as it stands i know no reason _to_ have this | 16:30 |
| inc0_ | I can't test aio properly without it | 16:30 |
| inc0_ | there you go, use case | 16:30 |
| SamYaple | you can too | 16:30 |
| SamYaple | i just told oyu how | 16:30 |
| inc0_ | ehh...whatever. you always fight things. | 16:30 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make cleanup-containers remove Heka volumes https://review.openstack.org/283066 | 16:31 |
| SamYaple | same to you buddy | 16:31 |
| SamYaple | but seriuosly, you can without changing the kolla code base, currently run AIO with haproxy with your setup | 16:31 |
| SamYaple | and do testing | 16:31 |
| inc0_ | not always | 16:31 |
| SamYaple | give me a sceanrio | 16:31 |
| inc0_ | ok, well, you want to run haproxy on controller nodes | 16:31 |
| SamYaple | this is about AIO right? | 16:32 |
| SamYaple | or are you changing the arguement? | 16:32 |
| inc0_ | real world example right | 16:32 |
| SamYaple | so changing the arguement, got it. | 16:32 |
| SamYaple | go | 16:32 |
| inc0_ | for aio I did show you example and you didn't accept it | 16:32 |
| SamYaple | i told you why you were wrong... | 16:33 |
| SamYaple | and how to set it up | 16:33 |
| SamYaple | so yes, i didnt accept it | 16:33 |
| SamYaple | the gate is AIO and does haproxy. you can too | 16:33 |
| inc0_ | so example - people have running openstack on different ports | 16:33 |
| inc0_ | (which people do sometimes) | 16:33 |
| inc0_ | and they want to keep same ports for endpoints in koala deploy | 16:33 |
| inc0_ | kolla | 16:33 |
| inc0_ | but they want to keep things openstackish inside | 16:34 |
| SamYaple | thats not a use case... | 16:34 |
| inc0_ | why? | 16:34 |
| SamYaple | becasue no one does that? | 16:34 |
| SamYaple | there are about a thousand scenarios where what you describe is a theoretical "use-case" | 16:34 |
| SamYaple | are we going to blow kolla up to cover all of those? | 16:34 |
| SamYaple | i see no real world use to this and no one has asked or requested this | 16:35 |
| inc0_ | I've seen deploys of people who do | 16:35 |
| inc0_ | so yes, people do use that | 16:35 |
| SamYaple | what it will do is change _alot_ of logic and alot of things | 16:35 |
| SamYaple | so if we are going to do it, plan it for newton please | 16:35 |
| *** dwalsh has quit IRC | 16:36 | |
| inc0_ | ah..I just noticed it won't be simple new variable | 16:36 |
| inc0_ | it this case I agree, I don't want to change logic | 16:36 |
| SamYaple | ok. want to add a bp for this so we can discuss for newton? | 16:37 |
| inc0_ | no | 16:38 |
| inc0_ | too big of a change | 16:38 |
| inc0_ | I thought it will be simpler | 16:38 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add docs about failures during evaluation https://review.openstack.org/283129 | 16:39 |
| inc0_ | with 2 vips tho | 16:39 |
| inc0_ | this will be simpler | 16:39 |
| inc0_ | but 2 vips is not merged yet | 16:39 |
| SamYaple | i agree with those statements | 16:39 |
| SamYaple | so discuss for newton? | 16:39 |
| inc0_ | with 2 vips I'll just probably submit a patch and review | 16:40 |
| inc0_ | nothing to discuss really | 16:40 |
| inc0_ | we'll have kolla_external_ip, we might have mariadb_external_port | 16:40 |
| inc0_ | or even no, because mariadb won't have external facing port | 16:40 |
| SamYaple | it stil lwont be simple | 16:40 |
| SamYaple | and then there are precheck changes and additions | 16:40 |
| sdake | mitaka is locked up fornew features | 16:41 |
| sdake | if you want something new, submit blueprint, we will get it in newton or later ;) | 16:41 |
| sdake | we have about 12 days left until mitaka-3 | 16:41 |
| wadeholler | eleomine_: that helped. now I made it to an error where the dhcp_agent containers are Created but the deploy failed http://pastebin.com/M72mjNqA | 16:43 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Run yum clean all in Heka Dockerfile https://review.openstack.org/283034 | 16:44 |
| wadeholler | docker logs neutron_dhcp_agent doesn't show anything - i.e. blank | 16:44 |
| SamYaple | wadeholler: how long ago did you rebuild your contaienrs? | 16:45 |
| SamYaple | that sounds like its hanging on waiting for a socket | 16:45 |
| elemoine_ | wadeholler: sounds like a different issue indeed | 16:46 |
| wadeholler | if by rebuild you mean tools/build.py - then last night. If you mean deploy then directly before this deploy I did a clean-containers && docker rm ... heka volumes and koala_logs, ... | 16:46 |
| openstackgerrit | Merged openstack/kolla: Add a precheck for MountFlags=shared https://review.openstack.org/281279 | 16:48 |
| sdake | if you eat 3 meals a day for 2 weeks straight, how many meals is that? | 16:48 |
| sdake | trust me, you will feel better if you do :) | 16:48 |
| SamYaple | i want to build a quote bot that just randomly quotes what sdake says | 16:50 |
| SamYaple | i mean sometimes _even_ with context.... | 16:51 |
| sdake | sounds like a waste of time, but whatever works :) | 16:51 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Fix Swift logging https://review.openstack.org/283118 | 16:53 |
| wadeholler | elemoine_: running away for lunch - sdake made me wicked hungry. will hop back on after lunch. Thanks for all the help thus far! | 16:53 |
| *** haplo37 has joined #kolla | 16:53 | |
| elemoine_ | wadeholler: thank you for reporting these issues | 16:53 |
| *** tzn has joined #kolla | 16:54 | |
| sdake | pbourke I'm not sure what happend hered | 17:03 |
| sdake | https://review.openstack.org/#/c/276855/ | 17:03 |
| sdake | one way to fix it is to resubmit the patch with the same changeid I think | 17:03 |
| sdake | let me try a recheck first tho | 17:03 |
| *** jtriley has quit IRC | 17:04 | |
| SamYaple | sdake: already done | 17:04 |
| SamYaple | pbourke: no reason to resubmit | 17:05 |
| SamYaple | gates been failing alot, thats on infra though | 17:05 |
| *** vhosakot has joined #kolla | 17:05 | |
| pbourke | sdake: SamYaple: ok thanks | 17:06 |
| elemoine_ | this gate thing does not sound easy to manage :) | 17:07 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: DO NOiT MERGE https://review.openstack.org/283170 | 17:07 |
| sdake | nihilifer can you rebase this please https://review.openstack.org/#/c/275798/ | 17:08 |
| sdake | nihilifer can you rebase htis please https://review.openstack.org/#/c/278045/ | 17:09 |
| sdake | core reviewers, please have a look at https://review.openstack.org/#/c/281380/1 | 17:18 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Mistral logs https://review.openstack.org/283177 | 17:19 |
| *** daneyon has joined #kolla | 17:21 | |
| *** xionglingfeng has joined #kolla | 17:21 | |
| *** daneyon_ has joined #kolla | 17:21 | |
| *** jtriley has joined #kolla | 17:24 | |
| *** daneyon has quit IRC | 17:25 | |
| *** tzn has quit IRC | 17:29 | |
| openstackgerrit | Merged openstack/kolla: Make cleanup-containers remove Heka volumes https://review.openstack.org/283066 | 17:29 |
| openstackgerrit | Merged openstack/kolla: Run yum clean all in Heka Dockerfile https://review.openstack.org/283034 | 17:30 |
| *** jasonsb has quit IRC | 17:30 | |
| *** unicell1 has quit IRC | 17:44 | |
| *** subscope has quit IRC | 17:47 | |
| *** thumpba has joined #kolla | 17:51 | |
| sdake | pbourke please review https://review.openstack.org/#/c/283068/2 | 17:52 |
| pbourke | sdake: change looks good but I no longer use vagrant... | 17:53 |
| pbourke | so am not the approver you're looking for! | 17:53 |
| sdake | if you use vagrant | 17:53 |
| sdake | in the past | 17:53 |
| sdake | that i sgood enough for me | 17:53 |
| openstackgerrit | Merged openstack/kolla: Change ubuntu base_image for vagrant to wily64 https://review.openstack.org/281968 | 17:58 |
| xionglingfeng | Please suggest how shall I do to improve https://review.openstack.org/#/c/282943/ | 17:59 |
| xionglingfeng | thank you everyone :-) | 17:59 |
| openstackgerrit | Merged openstack/kolla: Harmonise vagrant provider defaults https://review.openstack.org/283068 | 18:00 |
| openstackgerrit | Merged openstack/kolla: Make cleanup-containers remove bootstrap_heka https://review.openstack.org/283065 | 18:00 |
| openstackgerrit | Merged openstack/kolla: Change Heka sudoers file mode bits https://review.openstack.org/283086 | 18:00 |
| openstackgerrit | Merged openstack/kolla-mesos: Fix cleanup with empty volumes https://review.openstack.org/282693 | 18:02 |
| *** gtt116_ has joined #kolla | 18:04 | |
| *** gtt116 has quit IRC | 18:07 | |
| sdake | elemoine_ before you make a bajillion patches for all of the other services please read my comments here: https://review.openstack.org/#/c/283082/1 | 18:07 |
| *** dwalsh has joined #kolla | 18:12 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Use external address in novncproxy and spice https://review.openstack.org/283199 | 18:14 |
| *** unicell has joined #kolla | 18:17 | |
| openstackgerrit | Merged openstack/kolla: Upgrade task for Swift https://review.openstack.org/276855 | 18:18 |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Glance image path customization https://review.openstack.org/282840 | 18:20 |
| *** tzn has joined #kolla | 18:26 | |
| *** dwalsh has quit IRC | 18:27 | |
| *** dwalsh has joined #kolla | 18:31 | |
| *** dtturner has joined #kolla | 18:34 | |
| *** dwalsh has quit IRC | 18:36 | |
| *** salv-orl_ has joined #kolla | 18:42 | |
| *** salv-orlando has quit IRC | 18:44 | |
| *** Marga_ has quit IRC | 18:55 | |
| *** macsz has joined #kolla | 18:56 | |
| *** sdake has quit IRC | 18:58 | |
| *** dwalsh has joined #kolla | 18:58 | |
| elemoine_ | sdake, I will | 18:58 |
| *** tzn has quit IRC | 18:59 | |
| *** sdake has joined #kolla | 19:03 | |
| *** macsz has quit IRC | 19:03 | |
| *** subscope has joined #kolla | 19:04 | |
| openstackgerrit | Merged openstack/kolla: Use external address in novncproxy and spice https://review.openstack.org/283199 | 19:14 |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Multiple hypervisor driver support for nova https://review.openstack.org/282943 | 19:23 |
| *** chandankumar has quit IRC | 19:28 | |
| *** neophy has joined #kolla | 19:30 | |
| *** Jeffrey4l_ has joined #kolla | 19:32 | |
| openstackgerrit | william snow proposed openstack/kolla: Minor update add prechecks into workflow path Add kolla-ansible prechecks prior to deploy to make sure targets are ready for deploy TrivialFix https://review.openstack.org/283219 | 19:33 |
| *** Jeffrey4l has quit IRC | 19:34 | |
| openstackgerrit | william snow proposed openstack/kolla: Minor update add prechecks into workflow path TrivialFix https://review.openstack.org/283219 | 19:34 |
| openstackgerrit | william snow proposed openstack/kolla: Minor update add prechecks into workflow path https://review.openstack.org/283219 | 19:36 |
| *** xionglingfeng has quit IRC | 19:36 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Upgrade stub for elasticsearch https://review.openstack.org/283222 | 19:43 |
| inc0_ | can I get quick merge on this one plz? | 19:44 |
| SamYaple | w00t. i broke top 10 on reviews this cycle according to stackalytics! | 19:47 |
| SamYaple | i know its a pointless stat... but still it makes me happy :) | 19:47 |
| rhallisey | SamYaple, nice! | 19:49 |
| *** sdake_ has joined #kolla | 19:49 | |
| *** sdake has quit IRC | 19:50 | |
| *** tzn has joined #kolla | 19:55 | |
| sdake_ | SamYaple is this valid jinja2 syntax https://review.openstack.org/#/c/283118/3/ansible/roles/common/templates/heka.json.j2 | 19:56 |
| *** Marga_ has joined #kolla | 19:56 | |
| sdake_ | and if so, what does -%} do? | 19:56 |
| sdake_ | whitespace stripping? | 19:57 |
| kproskurin | sdake_, yes | 19:59 |
| SamYaple | sdake_: it is, but we dont need to use it there I dont think | 19:59 |
| SamYaple | either way its valid | 19:59 |
| *** fthiagogv has quit IRC | 20:00 | |
| *** inc0___ has joined #kolla | 20:15 | |
| *** inc0_ has quit IRC | 20:17 | |
| *** Guest40848 has quit IRC | 20:20 | |
| *** akscram has quit IRC | 20:20 | |
| *** mkoderer has quit IRC | 20:20 | |
| *** inc0_ has joined #kolla | 20:20 | |
| *** inc0_ has quit IRC | 20:21 | |
| *** inc0_ has joined #kolla | 20:21 | |
| *** akscram has joined #kolla | 20:21 | |
| *** mkoderer has joined #kolla | 20:21 | |
| *** Guest40848 has joined #kolla | 20:21 | |
| *** athomas has quit IRC | 20:22 | |
| openstackgerrit | Merged openstack/kolla: Minor update add prechecks into workflow path https://review.openstack.org/283219 | 20:22 |
| *** inc0___ has quit IRC | 20:23 | |
| *** tzn has quit IRC | 20:29 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add docs about failures during evaluation https://review.openstack.org/283129 | 20:33 |
| vhosakot | sdake_: SamYaple: I got my kolla up! :) after adding --registry "--registry 192.168.122.41:4000" to kolla-build | 20:34 |
| vhosakot | I can access horizon now! | 20:34 |
| *** athomas has joined #kolla | 20:34 | |
| sdake_ | huh | 20:34 |
| *** sdake_ is now known as sdake | 20:35 | |
| sdake | that sounds like a bug | 20:35 |
| vhosakot | well, I have a list of commands to update the quickstart | 20:35 |
| sdake | more like update build.py ;) | 20:35 |
| vhosakot | sdake: well, I checked with SamYaple and he said it is not a bug | 20:35 |
| SamYaple | vhosakot: awesome!! | 20:36 |
| sdake | SamYaple --registry "--registry xyz:port" is intended? | 20:36 |
| vhosakot | sdake: do you think kolla-build must force the user to use --registry ? SamYaple said --registry is not a mandatory flag and can be anything | 20:36 |
| SamYaple | sdake: i think vhosakot mistyped | 20:36 |
| SamYaple | "--registry 192.168.122.41:4000" | 20:36 |
| sdake | yes i mentioned that several times over the last few days | 20:37 |
| sdake | vhosakot did you mistype? | 20:37 |
| vhosakot | what mistype.. I added "-registry 192.168.122.41:4000" to kolla-build and it worked! | 20:37 |
| SamYaple | w00t! | 20:38 |
| sdake | [13:34:29] <vhosakot>sdake_: SamYaple: I got my kolla up! :) after adding --registry "--registry 192.168.122.41:4000" to kolla-build | 20:38 |
| vhosakot | I see all the images prefixed with "192.168.122.41:4000/kollaglue/" in the output of "docker images" | 20:38 |
| sdake | vhosakot yes I mentioned that severalt imes over the last few weeks ;) | 20:38 |
| SamYaple | you got it | 20:38 |
| sdake | last few days | 20:38 |
| SamYaple | glad its working | 20:38 |
| vhosakot | sdake: right, I know you mentioned it.. agreed 100% :) I'm wondering why you said "that sounds like a bug" ? :) | 20:39 |
| openstackgerrit | Merged openstack/kolla: Upgrade stub for elasticsearch https://review.openstack.org/283222 | 20:39 |
| wadeholler | hey all, can someone validate with me that I took the appropriate steps to avoid https://bugs.launchpad.net/kolla/+bug/1546798 | 20:40 |
| openstack | Launchpad bug 1546798 in kolla "neutron failed to deploy in ubuntu binary" [Low,Triaged] - Assigned to Sam Yaple (s8m) | 20:40 |
| sdake | I thought you meant you typed the command build.py --registry "--registry IP:port" | 20:40 |
| SamYaple | wadeholler: looking | 20:40 |
| sdake | ubuntu binary does not deploy afaik | 20:40 |
| wadeholler | I think that I am hitting that error and really thought I followed the right steps | 20:40 |
| wadeholler | os is centos | 20:40 |
| wadeholler | output / pastern - http://pastebin.com/9waBv6pV | 20:40 |
| SamYaple | wadeholler: no read the quickstart guide wadeholler | 20:40 |
| vhosakot | sdake: ah ok cool.. BTW, I will update the quickstart soon | 20:40 |
| SamYaple | for centos you need to set the flag in docker.service | 20:40 |
| sdake | sandbox error | 20:40 |
| wadeholler | I think that I did, and its in the paste bin and I restarted the service | 20:41 |
| wadeholler | I'm sure its something simple. sorry to be a pain | 20:41 |
| SamYaple | wadeholler: now remove the failed container and retry | 20:41 |
| SamYaple | wadeholler: if you are positive you restarted the service | 20:41 |
| SamYaple | docker rm -v -f neutron_dhcp_agent | 20:41 |
| wadeholler | ok | 20:41 |
| vhosakot | SamYaple: I think "r = requests.get(source['source'])" in kolla/kolla/cmd/build.py needs a timeout... that line hung/blocked for me many times... let me know what you think | 20:42 |
| vhosakot | SamYaple: I think kolla-build should not block/hang.. | 20:42 |
| SamYaple | vhosakot: i mean timeouts are fine. i would be curious as to why you time out but if you need one, sure | 20:42 |
| SamYaple | i agree | 20:42 |
| SamYaple | i would think fixing the cause of the hang would be prefered though | 20:42 |
| SamYaple | sdake: got my cloud going. complete with ssl certs | 20:43 |
| sdake | SamYaple nice ;) | 20:43 |
| vhosakot | SamYaple: intermittent network issues, and tarballs not reachable at http://tarballs.openstack.org/ (due to lot of threads/requests ?) | 20:43 |
| SamYaple | vhosakot: timeouts seem reasonable. please submit a patch :) | 20:44 |
| sdake | vhosakot bug first then patch plz | 20:44 |
| vhosakot | SamYaple: ok, what timeout should I specify, 60 secs, 120 secs, should we read it from globals ? | 20:44 |
| SamYaple | vhosakot: i think 120 is reasonable | 20:44 |
| vhosakot | sdake: bug first, then patch... yes sir | 20:45 |
| sdake | sir is for old people | 20:45 |
| vhosakot | SamYaple: cool, I will hardcode 120 then.. | 20:45 |
| sdake | call me sdake steak steve ;) | 20:45 |
| SamYaple | the tarballs are like 1mb in size so 2 minutes is fine | 20:45 |
| sdake | any of those work | 20:45 |
| SamYaple | sdake: im from the south, sir is proper for al | 20:45 |
| SamYaple | all* | 20:45 |
| vhosakot | sdake: hahaha :) | 20:45 |
| sdake | well us people in the west don't hanker to such formalities :) | 20:46 |
| SamYaple | heathens | 20:46 |
| *** openstackgerrit has quit IRC | 20:47 | |
| *** openstackgerrit has joined #kolla | 20:47 | |
| *** neophy has quit IRC | 20:48 | |
| vhosakot | After I change code, I need to run "tox" to run unit tests and test sanity... is there any other comman/script I need to run before pushing my patch set ? | 20:51 |
| SamYaple | vhosakot: nope | 20:52 |
| vhosakot | SamYaple: just "tox" ? | 20:53 |
| SamYaple | technically you dont _need_ to | 20:53 |
| SamYaple | but its a good thing to do | 20:53 |
| SamYaple | it can catch a gate failure earlier | 20:53 |
| vhosakot | SamYaple: what ? no tox before pushing ? | 20:53 |
| SamYaple | you dont have to, its a good idea though | 20:53 |
| vhosakot | SamYaple: wow, I dont want to fail at the gate | 20:53 |
| kproskurin | :-) Just do it, it's a good habbit | 20:53 |
| SamYaple | agreed | 20:53 |
| vhosakot | SamYaple: I will run tox.. cool | 20:54 |
| SamYaple | but, like, I am not going to do it if i change a docs patch, you know? | 20:54 |
| vhosakot | yeah yeah... only if we touch code | 20:54 |
| vhosakot | core code especially | 20:54 |
| SamYaple | for that matter, only python and bash scripts | 20:54 |
| SamYaple | there is no tox check for ansible stuff | 20:54 |
| vhosakot | SamYaple: how can tox make sense for an ansible playbook ? | 20:55 |
| SamYaple | ansible-lint, i just havent implemented it yet | 20:55 |
| vhosakot | SamYaple: ah, | 20:55 |
| vhosakot | SamYaple: cool | 20:55 |
| *** thumpba has quit IRC | 20:55 | |
| SamYaple | sdake: you around? | 20:56 |
| *** thumpba has joined #kolla | 20:56 | |
| sdake | shoot | 20:56 |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Allow cinder-volume to be configured to use nfs https://review.openstack.org/279913 | 20:56 |
| *** thumpba has quit IRC | 21:00 | |
| *** pbourke_ has joined #kolla | 21:03 | |
| *** ayoung has quit IRC | 21:06 | |
| *** unicell has quit IRC | 21:11 | |
| *** unicell has joined #kolla | 21:11 | |
| *** ayoung has joined #kolla | 21:12 | |
| sdake | rhallisey can you look at the last comment in https://review.openstack.org/#/c/279913/8 | 21:13 |
| rhallisey | ya | 21:25 |
| wadeholler | -- user error in that I did not "systemctl daemon-reload" before restarting docker. | 21:25 |
| wadeholler | deploy finished successfully. thank everyone. now the real fun begin! | 21:26 |
| wadeholler | exit | 21:26 |
| *** wadeholler has left #kolla | 21:26 | |
| sdake | user error ftw | 21:27 |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Multiple hypervisor driver support for nova https://review.openstack.org/282943 | 21:27 |
| sdake | d_code's patch should fix that up ;) | 21:27 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Multiple hypervisor driver support for nova https://review.openstack.org/282943 | 21:29 |
| *** gfidente has quit IRC | 21:30 | |
| openstackgerrit | Liz DiTucci proposed openstack/kolla: Minor updates to Tips and Tricks https://review.openstack.org/283267 | 21:36 |
| SamYaple | sdake: your -1 in not valid here https://review.openstack.org/#/c/282943 | 21:36 |
| SamYaple | i have replied to your comments | 21:36 |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Allow cinder-volume to be configured to use nfs https://review.openstack.org/279913 | 21:40 |
| *** tzn has joined #kolla | 21:44 | |
| sdake | that xen implementation thing needs to be reworked later | 21:44 |
| sdake | pbourke is right, and as I see it, it could turn into a real mess with multiple hypervisor implementations in the same container | 21:45 |
| sdake | time for a break, bbl | 21:46 |
| SamYaple | sdake: no the way the xen thing is now is fine | 21:46 |
| SamYaple | thats not what pbourke was talking about | 21:46 |
| SamYaple | i had him remove all the stuff pbourke was talking about as a condition of merging this xen stuff | 21:46 |
| SamYaple | (also had him remove the /dev:/dev binding as that change worried me right now) | 21:47 |
| SamYaple | the stuff that _is_ in that patch should be safe for now | 21:47 |
| SamYaple | no changes to non-xen nodes | 21:47 |
| openstackgerrit | Merged openstack/kolla: Fix Swift logging https://review.openstack.org/283118 | 21:49 |
| *** tpot has joined #kolla | 21:50 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Unpin elasticsearch https://review.openstack.org/283275 | 21:52 |
| *** sdake has quit IRC | 21:57 | |
| *** thumpba has joined #kolla | 22:03 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: HAProxy and Keepalived upgrade https://review.openstack.org/283278 | 22:03 |
| inc0_ | Jeffrey4l_, when you read it, could you please reassign bp for upgrade keepalived to me? I did both haproxy and keepalived in same patchiest | 22:05 |
| inc0_ | patchset | 22:06 |
| SamYaple | inc0_: that PS looks good | 22:06 |
| SamYaple | i like it :) | 22:07 |
| inc0_ | PS? | 22:07 |
| SamYaple | patchset | 22:07 |
| inc0_ | ahh | 22:07 |
| inc0_ | ok | 22:07 |
| inc0_ | well it's not done yet | 22:07 |
| inc0_ | but that's how I want to handle it | 22:07 |
| SamYaple | fighting over a compliment even! | 22:07 |
| SamYaple | im kidding :P | 22:07 |
| inc0_ | downtime should be minimal with this approach | 22:07 |
| *** pbourke_ has quit IRC | 22:07 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: HAProxy and Keepalived upgrade https://review.openstack.org/283278 | 22:08 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: HAProxy and Keepalived upgrade https://review.openstack.org/283278 | 22:10 |
| inc0_ | ok I'll finish it up tomorrow | 22:12 |
| inc0_ | SamYaple, so you are ok with this approach? if so, I'll just clean it up and test the hell out of it | 22:12 |
| *** sdake has joined #kolla | 22:13 | |
| *** thumpba has quit IRC | 22:13 | |
| SamYaple | looks good to me. one thing you will want it the -V option for keepalived | 22:13 |
| SamYaple | also it could be optomized if you want | 22:13 |
| inc0_ | ok, I'll ping you tomorrow about it | 22:13 |
| *** thumpba has joined #kolla | 22:13 | |
| SamYaple | the _lowest_ failover time would be to upgrade haproxy on the off nodes | 22:13 |
| sdake | SamYaple i am thinking beyond xen | 22:13 |
| SamYaple | kick the vip | 22:13 |
| sdake | that is why it requires rework | 22:14 |
| SamYaple | so doesnt time is the time it takes to reesstablish ssh | 22:14 |
| SamYaple | sdake: even beyond xen... what needs to change? | 22:14 |
| SamYaple | nothing I can think of | 22:14 |
| inc0_ | well kicking the vip will cause other stuff to happen as well | 22:14 |
| sdake | each hypervisor installs hypervisor specific software | 22:14 |
| inc0_ | and redeploy of container is fast enough | 22:14 |
| sdake | we dont wantto install hypervisor specific software in the libvirt container | 22:14 |
| SamYaple | sdake: not vmware | 22:14 |
| sdake | could get messy | 22:14 |
| inc0_ | so it's like a middle ground between lower impact | 22:14 |
| SamYaple | yea. absolutely we do. its just a few packages that sit idle. like a few MB | 22:15 |
| inc0_ | but name is misleading | 22:15 |
| SamYaple | besides at this point im pretty sure we can do them all suprisingly | 22:15 |
| SamYaple | inc0_: sure no worries. like i said get it working then optomize | 22:15 |
| inc0_ | ok | 22:15 |
| inc0_ | we can just rename it to nova-hypervisor | 22:15 |
| inc0_ | or whatever | 22:16 |
| sdake | xen-hypervisor | 22:16 |
| inc0_ | nah, we have nova-libvirt | 22:16 |
| sdake | i already chnaged the ittle | 22:16 |
| SamYaple | but thats not acurate | 22:16 |
| *** jtriley has quit IRC | 22:16 | |
| SamYaple | its nova-libvirt | 22:16 |
| SamYaple | nova-libvirt doesnt me kvm | 22:16 |
| SamYaple | it means libvirt | 22:16 |
| inc0_ | nova doesn't use libvirt for xen management tho right? | 22:16 |
| SamYaple | yes | 22:16 |
| SamYaple | thats my point | 22:17 |
| inc0_ | it does? then I'm completely fine with it;) | 22:17 |
| *** tpot has quit IRC | 22:17 | |
| SamYaple | for, say, vmware and hyper-v the nova-libvirt container isn't even needed | 22:17 |
| sdake | a more optimal approach would be nova-libvirt-xen | 22:17 |
| sdake | nova-libvirt-kvm | 22:17 |
| sdake | etc | 22:17 |
| SamYaple | i disagree | 22:17 |
| SamYaple | then kolla has to be aware of hypervisor | 22:17 |
| SamYaple | kolla-ansible* | 22:17 |
| inc0_ | also sdake keep in mind that these might change between hosts | 22:17 |
| inc0_ | unlikely, but possible | 22:18 |
| sdake | zomg | 22:18 |
| sdake | brain implode | 22:18 |
| inc0_ | I'd rather keep us out of it as much as we can | 22:18 |
| SamYaple | i like what we have | 22:18 |
| inc0_ | this way we're hypervisor agnostic and that's cool | 22:18 |
| SamYaple | we already overload the containers with ceph, even if you dont use ceph | 22:18 |
| sdake | well we aren't changing it before newton | 22:18 |
| sdake | so it doesn't matter | 22:18 |
| SamYaple | now nfs | 22:18 |
| *** tpot has joined #kolla | 22:18 | |
| inc0_ | anyway, I'm going home | 22:18 |
| SamYaple | i think a few MB of extra packages is fine | 22:18 |
| inc0_ | cya all tomorrow | 22:18 |
| sdake | later inc0_ | 22:18 |
| SamYaple | bye inc0_ | 22:18 |
| sdake | yes, we need to have a coordinated discussion in austin about how to handle these plugins | 22:19 |
| sdake | atm we have punted for two releases | 22:19 |
| SamYaple | i also disagree with that | 22:19 |
| SamYaple | i havent punted | 22:19 |
| SamYaple | i thought we agreed | 22:19 |
| SamYaple | a few MB is fine... | 22:19 |
| sdake | i dont mind mb | 22:20 |
| SamYaple | thats all this is though | 22:20 |
| sdake | as long as they dont conflict | 22:20 |
| SamYaple | they dont | 22:20 |
| sdake | i understand | 22:20 |
| sdake | that is why I approved the patch | 22:20 |
| sdake | and approved the blueprint | 22:20 |
| *** inc0_ has quit IRC | 22:22 | |
| *** thumpba has quit IRC | 22:23 | |
| sdake | samyaple elasticsearch appears to not be in the ubuntu repos | 22:26 |
| sdake | can you check into it? | 22:26 |
| sdake | http://logs.openstack.org/88/269688/14/check/gate-kolla-dsvm-build-ubuntu-source/ca21673/console.html.gz#_2016-02-19_18_30_45_641 | 22:26 |
| sdake | the centos gating work is being sorted out by jeffrey4l | 22:26 |
| sdake | perhaps you need a special thing in the gate - like you added for iirc mariadb | 22:27 |
| SamYaple | sdake: i do, its the gates fault | 22:27 |
| SamYaple | cost of violating DRY | 22:27 |
| sdake | the failure is persistent, is it fixable? | 22:28 |
| *** JoseMello has quit IRC | 22:28 | |
| *** thumpba has joined #kolla | 22:28 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add elasticsearch repo to gate https://review.openstack.org/283275 | 22:29 |
| sdake | i'd like the gate green asap ;) | 22:29 |
| sdake | thanks | 22:29 |
| SamYaple | it shouldnt have been merged initially | 22:29 |
| SamYaple | someone wasnt watching the gates | 22:29 |
| sdake | yup that happens | 22:29 |
| sdake | atleast we are getting a list of repos we need infra to mirror ;) | 22:31 |
| *** tzn has quit IRC | 22:37 | |
| *** sdake has quit IRC | 22:44 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Allow cinder-volume to be configured to use nfs https://review.openstack.org/279913 | 22:45 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Allow cinder-volume to be configured to use nfs on Centos https://review.openstack.org/283290 | 22:45 |
| *** haplo37 has quit IRC | 22:49 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Glance image path customization https://review.openstack.org/282840 | 22:51 |
| *** ayoung has quit IRC | 22:54 | |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Updated doc/quickstart.rst to help new contributors https://review.openstack.org/283298 | 22:56 |
| vhosakot | I've updated doc/quickstart.rst to help new contributors and submitted for review. Could folks please review it and let me know their comments! | 22:58 |
| *** dims has joined #kolla | 23:03 | |
| *** salv-orl_ has quit IRC | 23:08 | |
| *** sdake has joined #kolla | 23:10 | |
| *** salv-orlando has joined #kolla | 23:11 | |
| openstackgerrit | Merged openstack/kolla-mesos: Add openvswitch support https://review.openstack.org/282384 | 23:12 |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Updated doc/quickstart.rst to help new contributors https://review.openstack.org/283298 | 23:16 |
| *** cui has quit IRC | 23:20 | |
| asalkeld | sdake: you about? want to chat about https://blueprints.launchpad.net/kolla/+spec/custom-extend-start | 23:23 |
| *** spisarski has quit IRC | 23:24 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Multiple hypervisor driver support for nova https://review.openstack.org/282943 | 23:27 |
| *** vhosakot has quit IRC | 23:32 | |
| *** dwalsh has quit IRC | 23:32 | |
| sdake | asalkeld shoot | 23:32 |
| asalkeld | hangon just adding a comment to that bp | 23:33 |
| asalkeld | sdake: look now | 23:35 |
| asalkeld | basically we have to be the calling all actions | 23:35 |
| asalkeld | to keep track of dependencies | 23:35 |
| asalkeld | (you have ansible for that) | 23:35 |
| asalkeld | bootstrap is just one kind of command | 23:35 |
| asalkeld | and at the moment it is kind of hard baked into the image | 23:36 |
| asalkeld | we just want it not hard baked in | 23:36 |
| *** tzn has joined #kolla | 23:37 | |
| sdake | i'm not keen on having it outside of the container because then there could potentially be deviations | 23:37 |
| *** kzaitsev_mb has joined #kolla | 23:37 | |
| sdake | but controlling when seems ok | 23:38 |
| *** kzaitsev_ws has joined #kolla | 23:38 | |
| asalkeld | sdake: also see these service files more as ansible config tasks | 23:38 |
| asalkeld | (this stuff https://github.com/openstack/kolla-mesos/blob/master/services/mariadb/mariadb.yml.j2) | 23:38 |
| asalkeld | we need an equivelent | 23:38 |
| sdake | good lets make one abi to do it | 23:39 |
| sdake | rather then two | 23:39 |
| asalkeld | sdake: huh? | 23:39 |
| sdake | that is passed into the container to configure it by set_configs.py right? | 23:39 |
| sdake | the format of that file is the ABI | 23:39 |
| sdake | ansible has one mesos has one | 23:40 |
| asalkeld | sdake: that is nuts | 23:40 |
| sdake | they should be the same | 23:40 |
| asalkeld | noooo | 23:40 |
| asalkeld | that file has marathon stuff in it | 23:40 |
| asalkeld | like the ansible files have ansible stuff | 23:40 |
| asalkeld | this is just not a problem | 23:40 |
| sdake | which ansible stuff specifically? | 23:40 |
| asalkeld | well look at the ansible tasks | 23:41 |
| asalkeld | start/config etc.. | 23:41 |
| sdake | those are not parsed by set_configs.py | 23:41 |
| asalkeld | exactly | 23:41 |
| asalkeld | all we want is bootstrap task not to be automatically run | 23:41 |
| *** tzn has quit IRC | 23:41 | |
| asalkeld | (or a way to turn it off) | 23:41 |
| asalkeld | so we can run it, and keep track of the fact that it has run | 23:42 |
| *** tzn has joined #kolla | 23:42 | |
| sdake | asalkeld let me tell you what i'm after | 23:43 |
| sdake | and you can do as you please to make that happen ;) | 23:43 |
| *** tzn has quit IRC | 23:43 | |
| asalkeld | not if i disagree;) | 23:43 |
| sdake | where there is commonality, I don't' want deviation | 23:43 |
| sdake | (this was from midcycle ,not my own opinion) | 23:43 |
| sdake | where there must be deviation, deviation is fine | 23:44 |
| asalkeld | sdake: we are as common as we can can be | 23:44 |
| sdake | SamYaple seemed to think that was not the case | 23:44 |
| kzaitsev_mb | I've got a question. I'm working on murano and starting with L we have optional support for glance-glare repository as storage for the binary assets. This is done with a glare plugin (a stevedore plugin, which is in fact just a python-package), that needs to be installed in the env, where glare api operates. The catch is that plugin currently lives in murano codebase. This poses some challenges for pack | 23:44 |
| asalkeld | there is much different in the *way* things are deployed | 23:44 |
| kzaitsev_mb | agers. I was wondering if this would be a problem for you guys and are there any best practices on where we should keep such plugins (afair kolla is capable of deploying murano now) | 23:44 |
| sdake | there was duplciation | 23:44 |
| kzaitsev_mb | same might be true for, say searchlight plugins. | 23:45 |
| asalkeld | sdake: more in the actual templates | 23:45 |
| asalkeld | (which i think is a good thing) | 23:45 |
| asalkeld | as it allows use to make sure that the config of services is largely the same | 23:45 |
| sdake | asalkeld dont shoot messenger plz | 23:46 |
| sdake | perhaps i got message wrong | 23:46 |
| sdake | i tried to capture all the details, but only type 110wpm ;) | 23:46 |
| asalkeld | i'll chat to sam and see what he was referering to | 23:46 |
| sdake | people tlak faster then i can type | 23:46 |
| sdake | he was talking about the extend scripts | 23:47 |
| *** blahRus has quit IRC | 23:47 | |
| sdake | which are duplicated between projects | 23:47 |
| sdake | between repos i mean | 23:47 |
| asalkeld | sdake: that is because of this issue of the extend scripts | 23:47 |
| sdake | needlessly according to nihilifer | 23:47 |
| asalkeld | the automatic running of ti | 23:47 |
| sdake | well lets make it configurable and maek that part of the abi | 23:47 |
| asalkeld | kill that and things will get better | 23:47 |
| asalkeld | ok | 23:48 |
| sdake | i want a standard abi to the containers | 23:48 |
| kzaitsev_mb | should I come around and ask that question later? =) anyway pls ping me if anyone can shed some light on the situation | 23:48 |
| sdake | if that means the containers have to change to implement ansible and mesos, so be it | 23:48 |
| sdake | kzaitsev_mb can't context dump at this time i'll be right with you | 23:48 |
| sdake | asalkeld atm the abi is the BOOTSTRAP environment variable | 23:49 |
| sdake | and the yaml fiel that tells which command to run and which files to copy to where | 23:49 |
| sdake | i'd like that to stay intact between repos | 23:49 |
| *** subscope has quit IRC | 23:50 | |
| asalkeld | sdake: sure, but the extend start is still sourced | 23:50 |
| asalkeld | and we want to be the one to run that | 23:50 |
| sdake | yup and i'd like extend source to work for both repos not one | 23:50 |
| asalkeld | cool, i'll post something | 23:50 |
| sdake | so if the sourcing of the start script needs to be controlled | 23:50 |
| sdake | i'm totally open to that | 23:50 |
| sdake | that seems like a reasonable change | 23:50 |
| sdake | i just want one start file, one abi :) | 23:51 |
| sdake | btw when I say I, this was feelign of the midcycle | 23:51 |
| sdake | heka makes it more compelx as well because it introduces a second extension | 23:51 |
| asalkeld | when you say "start file", you mean extend_start.sh? | 23:51 |
| sdake | right | 23:51 |
| asalkeld | ok, totally fine with that | 23:51 |
| sdake | yes, i also want one start file (the stuff that runs set_configs.py) | 23:52 |
| sdake | so both of those should be common in the container | 23:52 |
| sdake | for whatever underlays we implement | 23:52 |
| sdake | I recognize it gets messy in one file | 23:52 |
| asalkeld | our start.py gets run by the base start file | 23:52 |
| sdake | but the other optoin, two sets of scripts is worse | 23:52 |
| sdake | it doubles the amount of maintenance that needs to be done | 23:52 |
| sdake | and knowledge of one repo doesn't transfer easily to the other | 23:53 |
| asalkeld | and it calls all the commands (one being the bootstrap) | 23:53 |
| sdake | really want to keep one community here, rather then fragment | 23:53 |
| asalkeld | sure | 23:53 |
| sdake | this is how we keep one community, one set of files ;) | 23:53 |
| sdake | one abi | 23:53 |
| asalkeld | tho' ansible is just different to microservices | 23:53 |
| asalkeld | shrug, doing what we can | 23:54 |
| sdake | i think if we work at it, we can make things common | 23:54 |
| sdake | asalkeld this is long term work, if it cant be done immediatey start thinking about it | 23:54 |
| sdake | i'm not acusing you of not doing what you can :) | 23:54 |
| sdake | i really dont know tbh if we have one abi or not | 23:54 |
| sdake | from that last file you inked, it appears not | 23:55 |
| sdake | because you have a cmd in there, that is matched by the yaml files used in ansible | 23:55 |
| sdake | service->daemon->command | 23:55 |
| asalkeld | sdake: so one reason we use set_configs very little is that we render the templates within the container | 23:55 |
| sdake | there is no good reason to duplicate that | 23:55 |
| asalkeld | *confused* | 23:55 |
| sdake | mem, cpus, i can understand | 23:56 |
| asalkeld | what is a duplication? | 23:56 |
| sdake | https://github.com/openstack/kolla-mesos/blob/master/services/mariadb/mariadb.yml.j2#L11 | 23:56 |
| sdake | moment | 23:56 |
| asalkeld | i am not going to learn how to parse anisble files | 23:56 |
| sdake | https://github.com/openstack/kolla/blob/master/ansible/roles/mariadb/templates/mariadb.json.j2#L3 | 23:57 |
| sdake | this is not an ansible file | 23:57 |
| sdake | this file is parsed by set_configs.py | 23:57 |
| sdake | ansible copies this into the container | 23:57 |
| sdake | (after jinja2ifying it) | 23:57 |
| asalkeld | except set_config does not track dependencies | 23:57 |
| asalkeld | so one option is we can somehow replace set_config.py | 23:58 |
| sdake | the second link could be expanded to merge the abi in the first mariadb.yml | 23:58 |
| sdake | you see how there are two mariadb.yml "config" loaders | 23:58 |
| sdake | that is the abi | 23:58 |
| sdake | i prefer one | 23:58 |
| asalkeld | there is a lot more going on | 23:59 |
| sdake | if you tell me it can't be done, i'll take your word for it | 23:59 |
| sdake | but I believe that was the objection raised at the midcycle | 23:59 |
| asalkeld | not sure, "can't be done", but not sure if it is worth the effort | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!