| *** aginwala has joined #kolla | 00:00 | |
| *** ssurana has joined #kolla | 00:05 | |
| *** jtriley has joined #kolla | 00:12 | |
| *** spisarski has quit IRC | 00:15 | |
| *** tzn has joined #kolla | 00:20 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Tidy up README.rst https://review.openstack.org/281591 | 00:21 |
|---|---|---|
| *** tzn has quit IRC | 00:26 | |
| asalkeld | any one ever seen docker fork errors during the image build? | 00:35 |
| asalkeld | also: "failed to register layer: Untar re-exec error: exit status 2: output: runtime/cgo: pthread_create failed: Resource temporarily unavailable" | 00:36 |
| asalkeld | Docker version 1.10.1, build 9e83765 | 00:37 |
| asalkeld | tried docker-py version 1.6 and 1.7 | 00:37 |
| *** RA__ has joined #kolla | 00:38 | |
| *** jtriley has quit IRC | 00:40 | |
| *** jasonsb has joined #kolla | 00:50 | |
| SamYaple | asalkeld: ... no | 00:51 |
| SamYaple | try fewer threads | 00:51 |
| SamYaple | whats your backend? | 00:51 |
| openstackgerrit | Merged openstack/kolla: Fix the mistral services name typo https://review.openstack.org/281199 | 00:51 |
| asalkeld | how do i figure that out? | 00:51 |
| SamYaple | docker info | 00:52 |
| asalkeld | aufs | 00:53 |
| SamYaple | nope | 00:53 |
| SamYaple | haent seen it | 00:53 |
| asalkeld | http://paste.openstack.org/show/487358/ | 00:53 |
| asalkeld | ok, don't worry | 00:53 |
| SamYaple | oh asalkeld i know | 00:53 |
| *** Jeffrey4l has joined #kolla | 00:54 | |
| SamYaple | i bet thats a similiar problem with rabbit | 00:54 |
| asalkeld | yeah? | 00:54 |
| SamYaple | rabbit runs out of process and you have to tell the kernel it can start more | 00:54 |
| SamYaple | some limit | 00:54 |
| SamYaple | plimit? | 00:54 |
| SamYaple | ulimit! | 00:54 |
| asalkeld | ok, looking | 00:55 |
| SamYaple | thats what i tweak with rabbitmq | 00:55 |
| SamYaple | whats ulimit -a have for open files? | 00:55 |
| SamYaple | `ulimit -a` | 00:55 |
| SamYaple | `ulimit -n 10000` sets it to 10000 open files | 00:56 |
| asalkeld | open files (-n) 1024 | 00:56 |
| * SamYaple points out sockets and handles are open files | 00:56 | |
| asalkeld | cool thx | 00:56 |
| openstackgerrit | Merged openstack/kolla: Fix missing 14.04 info https://review.openstack.org/281539 | 00:57 |
| *** iceyao has joined #kolla | 01:00 | |
| *** achanda has quit IRC | 01:03 | |
| *** sdake_ is now known as sdake | 01:05 | |
| *** iceyao_ has joined #kolla | 01:07 | |
| *** tpot has quit IRC | 01:07 | |
| *** iceyao has quit IRC | 01:10 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: also remove the volumes when cleaning up containers https://review.openstack.org/280722 | 01:13 |
| openstackgerrit | Merged openstack/kolla: Adds support for cleaning up bootstrap containers in the toos/cleanup-containers script. https://review.openstack.org/279961 | 01:14 |
| *** achanda has joined #kolla | 01:20 | |
| *** ssurana has quit IRC | 01:28 | |
| *** sdake has quit IRC | 01:34 | |
| openstackgerrit | Merged openstack/kolla: Tidy up README.rst https://review.openstack.org/281591 | 01:35 |
| *** diogogmt has joined #kolla | 01:42 | |
| *** diogogmt has quit IRC | 01:42 | |
| *** diogogmt has joined #kolla | 01:44 | |
| *** diogogmt has quit IRC | 01:49 | |
| *** sdake has joined #kolla | 02:03 | |
| *** sdake has quit IRC | 02:03 | |
| *** sdake has joined #kolla | 02:04 | |
| *** rhallisey has joined #kolla | 02:05 | |
| *** aginwala has quit IRC | 02:06 | |
| SamYaple | hey mandre | 02:08 |
| *** aginwala has joined #kolla | 02:09 | |
| *** aginwala has quit IRC | 02:09 | |
| *** aginwala has joined #kolla | 02:10 | |
| *** Marga__ has joined #kolla | 02:10 | |
| mandre | hey | 02:11 |
| mandre | SamYaple, doesn't check_failure() handle all cases? | 02:11 |
| SamYaple | all failure cases sure | 02:12 |
| SamYaple | we still want to copy the logs on success | 02:12 |
| *** Marga_ has quit IRC | 02:12 | |
| mandre | I was mistaken because of the trap, I though it was called when the script exited | 02:14 |
| SamYaple | oh right. yea it used to be | 02:14 |
| SamYaple | Jeffrey4l fixed that up | 02:14 |
| * mandre needs to revise the basics | 02:14 | |
| *** Marga__ has quit IRC | 02:14 | |
| *** vhosakot has joined #kolla | 02:15 | |
| *** d_code has quit IRC | 02:18 | |
| *** d_code has joined #kolla | 02:19 | |
| *** tzn has joined #kolla | 02:24 | |
| mandre | SamYaple, check_failure is also called when the script exits properly without error | 02:25 |
| mandre | am I missing something? | 02:25 |
| *** diogogmt has joined #kolla | 02:26 | |
| *** tfukushima has joined #kolla | 02:28 | |
| *** tzn has quit IRC | 02:29 | |
| SamYaple | mandre: no but i might be | 02:29 |
| SamYaple | i thought it only caught failures | 02:29 |
| SamYaple | but reading it again it definetely catches everything | 02:29 |
| SamYaple | i think it intially was only failures and i just never updated my head code | 02:29 |
| mandre | nah, initially it could be missed when there was a error in the script because of set -o errexit | 02:31 |
| mandre | now the trap makes sure it's always called when the script exits | 02:32 |
| SamYaple | well either way the way it works now you are right, its redundant | 02:32 |
| Jeffrey4l | with trap, the check_failure is called when script exit( including script exit with error). | 02:37 |
| *** alger has joined #kolla | 02:37 | |
| openstackgerrit | Martin André proposed openstack/kolla: Make Mistral deployable via ansible https://review.openstack.org/281610 | 02:39 |
| *** aginwala has quit IRC | 02:40 | |
| ayoung | SamYaple, Was able to run a python-keystoneclient functional test against Kolla...getting closer! | 02:41 |
| *** vhosakot has quit IRC | 02:43 | |
| SamYaple | w00t ayoung ! | 02:44 |
| SamYaple | i got in my new processor | 02:44 |
| SamYaple | that completes my cluster! | 02:44 |
| ayoung | SamYaple, Need to make a fix to Keystone. Link to deploying Keystone On Kolla out of a git repo? | 02:45 |
| ayoung | SamYaple, its one of those things that works when executed against SQLite, which does not enforce integrity constraints. Need to test against MySQL... | 02:45 |
| SamYaple | ayoung: did you generate a kolla-build.conf yet? | 02:45 |
| ayoung | nope | 02:46 |
| SamYaple | tox -e genconfig | 02:46 |
| SamYaple | ayoung: | 02:47 |
| SamYaple | # The location for source install (string value) | 02:47 |
| SamYaple | oops | 02:47 |
| *** tpot has joined #kolla | 02:47 | |
| SamYaple | ayoung: http://paste.openstack.org/show/487359/ | 02:47 |
| ayoung | looking | 02:47 |
| SamYaple | be warned, i did not implement this nor have I used it. But the oracle cats use it exlusively | 02:48 |
| ayoung | SamYaple, what about a local git repo? | 02:48 |
| SamYaple | I do not know the answer to that | 02:48 |
| ayoung | location = /opt/stack/keystone | 02:49 |
| SamYaple | it *might* work | 02:49 |
| SamYaple | people have talked about taht before | 02:49 |
| SamYaple | ayoung: looks like no. give me a second ill throw up a patch for you | 02:50 |
| ayoung | cool | 02:50 |
| dtturner | Hi - I'm hitting the following error starting rsyslog container during deployment: msg: AttributeError("'Client' object has no attribute 'create_host_config'",) | 02:55 |
| dtturner | On my deploy host I’m running docker version 1.9.1, and docker-py 1.7.0 | 02:55 |
| dtturner | I hit this building / deploying from trunk a couple of weeks back as well as with a build from trunk today. Any ideas? | 02:56 |
| SamYaple | dtturner: docker 1.10 is require for trunk | 02:57 |
| SamYaple | ayoung: got a patch, testing | 02:57 |
| SamYaple | ayoung: be warned, it will probably change before merging. people have opinions | 02:57 |
| dtturner | SamYaple: Eek. OK. I missed that. Thanks. | 02:57 |
| ayoung | SamYaple, not a problem. I'll give it a test run once its posted | 02:58 |
| *** tpot has quit IRC | 03:00 | |
| *** tpot has joined #kolla | 03:02 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: [WIP] local git repo build https://review.openstack.org/281615 | 03:02 |
| SamYaple | ayoung: tested ^^ | 03:02 |
| ayoung | SamYaple, thanks | 03:02 |
| SamYaple | ayoung: http://paste.openstack.org/show/487360/ | 03:03 |
| SamYaple | no reference tag yet, so it just tars up your git directory | 03:03 |
| SamYaple | mandre: im a bit overloaded, do you have time to take over https://review.openstack.org/281615 ? | 03:03 |
| ayoung | SamYaple, not file:/ ? | 03:03 |
| SamYaple | ayoung: nah this was quick and dirty | 03:04 |
| ayoung | I guess no need | 03:04 |
| SamYaple | long term it will be file:// i bet | 03:04 |
| SamYaple | no need for a "localgit" type | 03:04 |
| ayoung | SamYaple, did the non-root deploys merge already? | 03:04 |
| SamYaple | yea | 03:04 |
| *** salv-orl_ has quit IRC | 03:04 | |
| mandre | SamYaple, np | 03:05 |
| ayoung | also I have the edit | 03:05 |
| ayoung | --- a/requirements.txt | 03:05 |
| mandre | let me see what you've done... | 03:05 |
| ayoung | -docker-py>=1.4.0 # Apache-2.0 | 03:05 |
| ayoung | +docker-py<1.7 # Apache-2.0 | 03:05 |
| ayoung | can that go? | 03:05 |
| ayoung | guess I can git stash that one | 03:05 |
| SamYaple | ayoung: you _have_ to edit those reqs? | 03:05 |
| SamYaple | btw master should have docker-py 1.6.0 | 03:05 |
| ayoung | SamYaple, I don;t remember | 03:05 |
| ayoung | so that merged | 03:05 |
| SamYaple | yea | 03:06 |
| SamYaple | 1.6.0 is up | 03:06 |
| SamYaple | its in master kolla | 03:06 |
| *** iceyao_ has quit IRC | 03:09 | |
| *** iceyao has joined #kolla | 03:09 | |
| ayoung | So far so good | 03:14 |
| ayoung | SamYaple, does that work for libraries as well, or just the servers? | 03:15 |
| SamYaple | ? | 03:15 |
| SamYaple | oh you mean like a certain version of a certain library? | 03:15 |
| ayoung | SamYaple, python-keystoneclient specifically | 03:15 |
| ayoung | and keystoneauth | 03:15 |
| SamYaple | well we install based on constraints from master requirements repo | 03:16 |
| SamYaple | https://github.com/openstack/kolla/blob/master/docker/keystone/Dockerfile.j2#L55 | 03:16 |
| SamYaple | tell me what specifically you need and ill see what needs to happen to make that work | 03:16 |
| *** iceyao has quit IRC | 03:18 | |
| *** iceyao has joined #kolla | 03:18 | |
| *** unicell has quit IRC | 03:19 | |
| ayoung | SamYaple, I'll loop back around with you on that. Don;'t need it tonight | 03:19 |
| *** sdake has quit IRC | 03:20 | |
| SamYaple | awesome | 03:20 |
| *** yuanying_ has quit IRC | 03:21 | |
| *** tzn has joined #kolla | 03:25 | |
| openstackgerrit | Martin André proposed openstack/kolla: [WIP] Add local directory or file for source build https://review.openstack.org/281615 | 03:29 |
| *** tzn has quit IRC | 03:29 | |
| mandre | brb, lunchtime | 03:30 |
| *** dims_ has quit IRC | 03:30 | |
| *** dims has joined #kolla | 03:32 | |
| *** iceyao has quit IRC | 03:35 | |
| *** iceyao has joined #kolla | 03:35 | |
| *** jtriley has joined #kolla | 03:36 | |
| *** yuanying has joined #kolla | 03:39 | |
| *** inc0 has joined #kolla | 03:41 | |
| inc0 | hey | 03:41 |
| SamYaple | hey inc0 | 03:42 |
| SamYaple | youre on late | 03:42 |
| inc0 | well, happends | 03:42 |
| inc0 | I think we need second opinion on nova upgrades:P | 03:42 |
| inc0 | because we are in tie | 03:42 |
| inc0 | and need someone to break it | 03:42 |
| SamYaple | i stil ldont understand what you are complaining about | 03:42 |
| inc0 | let me put it this way | 03:42 |
| inc0 | if I would change something in start task for nova_conductor | 03:43 |
| SamYaple | if you want to do the work to split out the start tasks now to not violate DRY | 03:43 |
| SamYaple | thats fine | 03:43 |
| inc0 | currently I'd probably grep for start task | 03:43 |
| SamYaple | its a bit more work but it sfine | 03:43 |
| inc0 | we need to do this | 03:43 |
| inc0 | but not in this patchset | 03:43 |
| SamYaple | thats what i said | 03:43 |
| inc0 | and even then it wont help this case | 03:43 |
| SamYaple | it will | 03:44 |
| inc0 | because if we split it | 03:44 |
| SamYaple | only one place for nova-conductor to start | 03:44 |
| inc0 | we still need to change common start AND check image | 03:44 |
| *** iceyao has quit IRC | 03:44 | |
| inc0 | every upgrade play have duplicate start | 03:44 |
| *** iceyao has joined #kolla | 03:44 | |
| SamYaple | what are you talkign about | 03:44 |
| inc0 | so if you make a project-wide change of mounts or whatever | 03:44 |
| inc0 | I bet you will forget about this place | 03:45 |
| SamYaple | if yo uare talkign about the duplicated start nova-conductor tasks, then yes thats violating DRY | 03:45 |
| inc0 | I'm talking about copypaste from start.yml in yupgrade plays | 03:45 |
| SamYaple | we have discussed this | 03:45 |
| SamYaple | everyone should be aware of this | 03:45 |
| SamYaple | we said we would refactor after upgrades | 03:45 |
| SamYaple | we didnt want to slow or complicate upgrades | 03:45 |
| SamYaple | if you are saying that _HAS_ to be done now, im ok with that | 03:46 |
| inc0 | and if we refactor we probably will forget about this one | 03:46 |
| SamYaple | like i said, you want to do it now im not going to _block_ that | 03:46 |
| SamYaple | inc0: you pick the wierdest battles. I jsut don't want to violate DRY | 03:49 |
| SamYaple | (also the underscore naming is just bad practice here) | 03:49 |
| *** alger has quit IRC | 03:54 | |
| *** yuanying has quit IRC | 03:55 | |
| *** unicell has joined #kolla | 03:55 | |
| inc0 | SamYaple, well, DRY exists to avoid bugs caused by changing stuff in one place and forget to do it in other places | 03:57 |
| inc0 | there is little space for it in my code and this file changes raretly | 03:57 |
| inc0 | on the other hand if I follow by your decision, I introduce new, less obvious and more likely to change place for that | 03:58 |
| inc0 | and bugs there are harder to debug and much less obvious | 03:58 |
| inc0 | and yeah, we need to refactor dry in plays, but that's ansible problem and that's not an easy fix | 03:59 |
| inc0 | as for underscore names, this is cosmetics, I'm ok with change but I would actually refactor whole thing so each of this function is standalone | 04:00 |
| SamYaple | im sorry i dont think you understand | 04:01 |
| inc0 | one of us don't | 04:01 |
| SamYaple | you need to split up start.yml | 04:01 |
| SamYaple | then there is no duplicated start code | 04:01 |
| SamYaple | no DRY violation | 04:01 |
| inc0 | if you can't do simple import, there have to be | 04:02 |
| inc0 | and in this case I don't think you can | 04:02 |
| SamYaple | what are you talking about? | 04:02 |
| SamYaple | import? | 04:02 |
| inc0 | include start.yml in upgrade plays | 04:02 |
| SamYaple | no i said splut up start.yml | 04:03 |
| SamYaple | as in start_conductor.yml | 04:03 |
| SamYaple | in the start play you include start_conductor.yml | 04:03 |
| SamYaple | same with upgrade play | 04:03 |
| SamYaple | we talked about doign that _after_ the upgrade play so as not to slow down upgrades, but if you want to do it now im fine with that | 04:03 |
| inc0 | but that won't help with case at hand | 04:03 |
| SamYaple | it will absolutely | 04:04 |
| inc0 | because if you change mounts in start_conductor.yml | 04:04 |
| SamYaple | then there is no chance t odiffer | 04:04 |
| inc0 | it won't affect check task | 04:04 |
| SamYaple | ...... if you change the mounts in start_conductor.yml you are going to be starting a new container anyway | 04:04 |
| inc0 | so if you forget to make same change in check task | 04:04 |
| *** rhallisey has quit IRC | 04:04 | |
| inc0 | wonderful, but then if we forget to make this change there | 04:04 |
| SamYaple | there is no mounts in check task | 04:05 |
| inc0 | each time you run upgrade task it will stop | 04:05 |
| SamYaple | where are you getting this info | 04:05 |
| inc0 | not-idempotently | 04:05 |
| SamYaple | you are absolutely wrong | 04:05 |
| SamYaple | 100% | 04:05 |
| *** yuanying has joined #kolla | 04:05 | |
| *** alger has joined #kolla | 04:07 | |
| *** yuanying_ has joined #kolla | 04:07 | |
| SamYaple | oh man inc0 | 04:07 |
| SamYaple | im so sorry | 04:07 |
| SamYaple | oh man i feel so bad | 04:07 |
| inc0 | https://github.com/openstack/kolla/blob/master/ansible/library/kolla_docker.py#L246 we change env and forget to do it in check task == stop every time | 04:07 |
| SamYaple | i see what you mean | 04:07 |
| SamYaple | dude i was in the wrong way | 04:08 |
| SamYaple | jeez i get what you are saying | 04:08 |
| inc0 | yeah, I thought you might | 04:08 |
| SamYaple | i was focusing on the start | 04:08 |
| inc0 | I did start from check_container_differs | 04:08 |
| *** Marga_ has joined #kolla | 04:08 | |
| SamYaple | i was thinking you were talking about the starting of the container | 04:08 |
| SamYaple | i was so confused | 04:08 |
| inc0 | that's most obvious way, but it didnt work as intended | 04:08 |
| inc0 | separation start task is acutally good idea, I'll do that | 04:09 |
| SamYaple | ok so here is somethign we can actually do | 04:09 |
| inc0 | (tomorrow) | 04:09 |
| SamYaple | ive built kolla_docker to support the check mode | 04:09 |
| SamYaple | what we can do is set checkmode on the tasks and registry if it changes | 04:09 |
| *** yuanying has quit IRC | 04:09 | |
| *** Marga_ has quit IRC | 04:10 | |
| inc0 | how does it work? | 04:10 |
| SamYaple | check mode? | 04:10 |
| inc0 | yeah | 04:10 |
| inc0 | low level | 04:10 |
| SamYaple | it makes no changes | 04:10 |
| inc0 | because method for stopping is single flow | 04:10 |
| SamYaple | it just reports if there was achange | 04:10 |
| *** Marga_ has joined #kolla | 04:10 | |
| SamYaple | but lets table that | 04:11 |
| SamYaple | thats different work | 04:11 |
| SamYaple | lets not hold up upgrades for that work | 04:11 |
| inc0 | https://review.openstack.org/#/c/254395/16/ansible/library/kolla_docker.py line 508 | 04:11 |
| inc0 | how does it know which line is changing stuff | 04:11 |
| inc0 | ? | 04:11 |
| inc0 | or I have to write it in different way to support it? | 04:12 |
| SamYaple | inc0: the module needs a bit of tweaking for that thats what i said tabling it | 04:12 |
| inc0 | ok | 04:12 |
| inc0 | so for now we're good to keep logic like I written it? | 04:12 |
| SamYaple | i mean it just needs a check and exit call for each "changed = true" flag, but lets not do it now | 04:12 |
| inc0 | I can change underscore naming and separate start.yml to increase dry | 04:12 |
| SamYaple | i dont like the underscore renaming | 04:12 |
| SamYaple | but thats it | 04:13 |
| SamYaple | dude i feel so bad right now. | 04:13 |
| *** alger has quit IRC | 04:13 | |
| inc0 | ok, I'll figure it out tomorrow | 04:13 |
| SamYaple | i had thirty things going on in my head | 04:13 |
| inc0 | don't worry too much | 04:13 |
| SamYaple | i should have listened more | 04:13 |
| SamYaple | sorry man | 04:13 |
| *** alger has joined #kolla | 04:13 | |
| inc0 | I'm glad we got that cleared out, that's all | 04:13 |
| SamYaple | yea. well heres to moving forward | 04:14 |
| SamYaple | inc0: alternatively you could just add a "check_image_only" flag to check_contaier_differs | 04:16 |
| inc0 | that will complicate whole thing | 04:16 |
| SamYaple | that would be no DRY violation, and no function renaming | 04:16 |
| SamYaple | how so? | 04:16 |
| inc0 | I'd rather refactor all checks to either accept container_info or if not provided get container by themselves | 04:17 |
| inc0 | so in effect all of them will be standalone | 04:17 |
| SamYaple | yea but they wont all be useable | 04:17 |
| SamYaple | since you have to update the acceptable choices | 04:18 |
| inc0 | yeah, other will just be to create a standard | 04:18 |
| SamYaple | yea thats true | 04:18 |
| inc0 | I can do this with just an image | 04:18 |
| inc0 | might be actually good for claridy | 04:18 |
| inc0 | clarity | 04:18 |
| inc0 | since it is somewhat special | 04:19 |
| SamYaple | each function starts with 'container = container_info or self.check_container() | 04:19 |
| inc0 | yeah | 04:19 |
| SamYaple | and function accepts (self, container_info=None) | 04:19 |
| SamYaple | yea that works | 04:19 |
| *** iceyao has quit IRC | 04:19 | |
| SamYaple | i like that | 04:19 |
| inc0 | I wanted to refactor it this way anyway | 04:19 |
| inc0 | will do that tomorrow | 04:19 |
| SamYaple | cool | 04:19 |
| *** iceyao has joined #kolla | 04:19 | |
| inc0 | would you plz do me a favor and comment patchset so others will see that we agreed on this one? | 04:20 |
| inc0 | and add note for splitting start.yml plz so it wont skip my mind | 04:20 |
| SamYaple | LET THE RECORD SHOW MICHAL AND SAM HAVE REACHED AN AGREEMENT | 04:20 |
| inc0 | we should clad it in stone or something | 04:20 |
| SamYaple | lol we always come to an agreement eventually | 04:20 |
| inc0 | fund a bench in waterfall park to remember it | 04:20 |
| SamYaple | ill carve it at table rock | 04:21 |
| SamYaple | inc0: ok commented | 04:25 |
| *** tzn has joined #kolla | 04:26 | |
| inc0 | thanks | 04:26 |
| inc0 | these should be fairly easy | 04:26 |
| SamYaple | yup | 04:26 |
| inc0 | I'll ping you later to discuss best approach for keepalived upgrades | 04:28 |
| *** iceyao has quit IRC | 04:28 | |
| SamYaple | i think those need full stop too but i cant remember. i looked it up at one point | 04:28 |
| SamYaple | but vip can stick around, so no worries | 04:28 |
| inc0 | well if you stop keepalived container it will migrate | 04:29 |
| inc0 | can you force it not to? | 04:29 |
| SamYaple | nah you stop the non-active one | 04:29 |
| SamYaple | then you stop the active and VIP will stick around | 04:29 |
| SamYaple | well you need -V on the daemon i think... | 04:29 |
| inc0 | ahh...that might work | 04:29 |
| *** iceyao has joined #kolla | 04:29 | |
| SamYaple | but we can do a rolling way so it only changes at max once | 04:29 |
| inc0 | we need to figure out best way to find currently active | 04:29 |
| inc0 | might not be trivial | 04:30 |
| inc0 | I'd rather do full stop without change | 04:30 |
| SamYaple | i mean ansible knows | 04:30 |
| *** tzn has quit IRC | 04:30 | |
| SamYaple | it has all ip info | 04:30 |
| inc0 | hmm...yeah that might work, if we do it correctly | 04:30 |
| inc0 | I'll play around it | 04:31 |
| SamYaple | when [api_interface] == {{ kolla_internal_address }} | 04:31 |
| SamYaple | something like that | 04:31 |
| SamYaple | i mean obviously thats wrong, but you get it | 04:31 |
| inc0 | I get it, hacky but might work | 04:31 |
| inc0 | hacky is ok for this release | 04:31 |
| inc0 | then we'll see if it needs changing | 04:31 |
| SamYaple | idk seems pretty clean to me | 04:31 |
| SamYaple | but anyway | 04:31 |
| SamYaple | for tomorrow | 04:31 |
| inc0 | yeah | 04:31 |
| inc0 | I'm going off, cyas | 04:31 |
| SamYaple | bye | 04:32 |
| *** inc0 has quit IRC | 04:32 | |
| *** RA__ has quit IRC | 04:37 | |
| *** vhosakot has joined #kolla | 04:41 | |
| *** alger has quit IRC | 04:49 | |
| *** alger has joined #kolla | 04:52 | |
| *** dims_ has joined #kolla | 04:53 | |
| *** dims has quit IRC | 04:55 | |
| *** dims_ has quit IRC | 05:00 | |
| *** dims has joined #kolla | 05:02 | |
| *** dims has quit IRC | 05:02 | |
| mdnadeem | SamYaple, ping | 05:03 |
| *** ccesario has quit IRC | 05:05 | |
| mdnadeem | had any one face issue with TASK: [neutron | Starting neutron-dhcp-agent container] while deploying kolla on ubuntu os ? | 05:10 |
| mdnadeem | Please have a look on http://pastebin.com/bMgnMz04 | 05:10 |
| mdnadeem | ^^ jeffrey4l , nihilifer | 05:12 |
| *** mdnadeem_web has joined #kolla | 05:15 | |
| *** Allen_Gao has quit IRC | 05:16 | |
| *** ccesario has joined #kolla | 05:18 | |
| *** ccesario has quit IRC | 05:26 | |
| *** tzn has joined #kolla | 05:26 | |
| *** tzn has quit IRC | 05:31 | |
| openstackgerrit | Martin André proposed openstack/kolla: Set nova hypervisor to qemu in Vagrant https://review.openstack.org/281637 | 05:32 |
| *** Allen_Gao has joined #kolla | 05:33 | |
| *** dave-mccowan has quit IRC | 05:38 | |
| *** ccesario has joined #kolla | 05:39 | |
| *** iceyao has quit IRC | 05:45 | |
| *** iceyao has joined #kolla | 05:45 | |
| mdnadeem | oh now i see , i have hit this bug https://bugs.launchpad.net/kolla/+bug/1546798 | 05:54 |
| openstack | Launchpad bug 1546798 in kolla "neutron failed to deploy in ubuntu binary" [Low,Triaged] - Assigned to Sam Yaple (s8m) | 05:54 |
| *** Marga_ has quit IRC | 06:01 | |
| *** aginwala has joined #kolla | 06:24 | |
| *** Marga_ has joined #kolla | 06:25 | |
| *** tzn has joined #kolla | 06:27 | |
| *** tzn has quit IRC | 06:32 | |
| *** vhosakot has quit IRC | 06:34 | |
| *** iceyao has quit IRC | 06:38 | |
| *** iceyao has joined #kolla | 06:39 | |
| *** vhosakot has joined #kolla | 06:41 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: also remove the volumes when cleaning up containers https://review.openstack.org/280722 | 06:53 |
| *** Serlex has joined #kolla | 07:03 | |
| *** vhosakot has quit IRC | 07:14 | |
| *** achanda has quit IRC | 07:14 | |
| *** achanda has joined #kolla | 07:17 | |
| *** chandankumar has joined #kolla | 07:22 | |
| *** intr1nsic has quit IRC | 07:25 | |
| *** iceyao has quit IRC | 07:26 | |
| *** iceyao has joined #kolla | 07:27 | |
| *** alisonh has quit IRC | 07:27 | |
| *** intr1nsic has joined #kolla | 07:28 | |
| *** iceyao_ has joined #kolla | 07:28 | |
| *** tzn has joined #kolla | 07:28 | |
| *** iceyao has quit IRC | 07:31 | |
| *** tzn has quit IRC | 07:32 | |
| *** iceyao_ has quit IRC | 07:39 | |
| *** alisonh has joined #kolla | 07:42 | |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Fix typo in quickstart https://review.openstack.org/281681 | 07:43 |
| *** salv-orlando has joined #kolla | 07:44 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: also remove the volumes when cleaning up containers https://review.openstack.org/280722 | 07:45 |
| *** iceyao has joined #kolla | 07:48 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Ensuring the rsyslog service to be ready https://review.openstack.org/279064 | 07:50 |
| *** alisonh has quit IRC | 07:50 | |
| *** jtriley has quit IRC | 07:54 | |
| *** alisonh has joined #kolla | 07:56 | |
| *** alger has quit IRC | 07:56 | |
| *** alger has joined #kolla | 08:00 | |
| *** dtturner has quit IRC | 08:02 | |
| *** iceyao_ has joined #kolla | 08:02 | |
| Jeffrey4l | elemoine do you have any idea why the link is not create when using `sudo ln -sf /var/lib/kolla/rsyslog/log /dev/log` in Sam's PS,now. | 08:04 |
| *** iceyao has quit IRC | 08:05 | |
| *** jmccarthy1 has joined #kolla | 08:06 | |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Also remove the volumes when cleaning up containers https://review.openstack.org/280722 | 08:07 |
| *** Marga_ has quit IRC | 08:07 | |
| *** fgimenez has joined #kolla | 08:10 | |
| *** jtriley has joined #kolla | 08:12 | |
| Serlex | Anyone seen | 08:18 |
| Serlex | ERROR:kolla.cmd.build:Error: Status 502 trying to push repository lokolla/centos-binary-haproxy: | 08:18 |
| *** mdnadeem_web has quit IRC | 08:20 | |
| *** mikelk has joined #kolla | 08:21 | |
| *** tzn has joined #kolla | 08:29 | |
| *** mikelk has quit IRC | 08:30 | |
| *** mikelk has joined #kolla | 08:31 | |
| *** tzn has quit IRC | 08:33 | |
| *** Jeffrey4l has quit IRC | 08:45 | |
| *** alger has quit IRC | 08:46 | |
| *** alger has joined #kolla | 08:46 | |
| *** Jeffrey4l has joined #kolla | 08:58 | |
| *** akwasnie has joined #kolla | 09:02 | |
| *** alger has quit IRC | 09:02 | |
| *** alger has joined #kolla | 09:03 | |
| *** tzn has joined #kolla | 09:04 | |
| *** macsz has joined #kolla | 09:05 | |
| *** gfidente has joined #kolla | 09:10 | |
| *** tzn has quit IRC | 09:10 | |
| *** mbound has joined #kolla | 09:12 | |
| *** alger has quit IRC | 09:21 | |
| *** achanda has quit IRC | 09:25 | |
| *** Allen_Ga_ has joined #kolla | 09:28 | |
| *** achanda has joined #kolla | 09:28 | |
| *** achanda has quit IRC | 09:29 | |
| *** Allen_Ga_ has quit IRC | 09:29 | |
| *** aginwala has quit IRC | 09:33 | |
| *** alger has joined #kolla | 09:33 | |
| *** tpot has quit IRC | 09:35 | |
| *** achanda has joined #kolla | 09:46 | |
| akwasnie | elemoine: ping | 09:47 |
| akwasnie | elemoine, sdake, inc0 : https://www.elastic.co/blog/heya-elastic-stack-and-x-pack | 09:48 |
| *** tfukushima has quit IRC | 09:49 | |
| akwasnie | good to know that there are some changes coming soon, hope it won't be a problem for our logging solution | 09:50 |
| elemoine_ | akwasnie: hie | 09:50 |
| elemoine_ | thanks for the link | 09:52 |
| *** tfukushima has joined #kolla | 09:54 | |
| *** tfukushima has quit IRC | 09:54 | |
| *** pbourke has joined #kolla | 09:55 | |
| nihilifer | btw, can the other cores look at this? https://review.openstack.org/#/c/267714/ | 09:55 |
| nihilifer | it'd be nice to merge it soon | 09:55 |
| nihilifer | (to be honest, i don't get why we need "more eyes" and more than 2 cores on this patch, but ok... ;) ) | 09:58 |
| akwasnie | nihilifer hi:) | 10:00 |
| akwasnie | and thanks | 10:01 |
| nihilifer | np akwasnie ;) | 10:01 |
| *** fgimenez has quit IRC | 10:02 | |
| *** openstackgerrit has quit IRC | 10:02 | |
| akwasnie | nihilifer are you going to attend summit? | 10:03 |
| *** openstackgerrit has joined #kolla | 10:03 | |
| *** salv-orl_ has joined #kolla | 10:03 | |
| nihilifer | akwasnie: probably yes | 10:05 |
| *** salv-orlando has quit IRC | 10:06 | |
| akwasnie | great;) | 10:06 |
| *** fgimenez has joined #kolla | 10:06 | |
| *** pbourke has quit IRC | 10:10 | |
| *** pbourke has joined #kolla | 10:11 | |
| *** achanda has quit IRC | 10:12 | |
| *** achanda has joined #kolla | 10:14 | |
| *** achanda has quit IRC | 10:16 | |
| *** Jeffrey4l has quit IRC | 10:25 | |
| *** tzn has joined #kolla | 10:31 | |
| *** kproskurin has joined #kolla | 10:36 | |
| *** Jeffrey4l has joined #kolla | 10:42 | |
| *** Allen_ has joined #kolla | 10:54 | |
| *** Allen_ is now known as Allen_Gao_ | 10:54 | |
| *** Allen_Gao_ is now known as AllenGao | 10:56 | |
| *** dims has joined #kolla | 10:59 | |
| *** Allen_Gao has quit IRC | 10:59 | |
| *** asalkeld has quit IRC | 11:00 | |
| *** AllenGao is now known as Allen_Gao | 11:02 | |
| *** athomas has quit IRC | 11:09 | |
| *** tobe has joined #kolla | 11:11 | |
| *** tfukushima has joined #kolla | 11:11 | |
| *** athomas has joined #kolla | 11:14 | |
| *** athomas has quit IRC | 11:16 | |
| *** achanda has joined #kolla | 11:17 | |
| *** pbourke has quit IRC | 11:18 | |
| *** athomas has joined #kolla | 11:18 | |
| *** alger has quit IRC | 11:19 | |
| *** Jeffrey4l has quit IRC | 11:19 | |
| *** iceyao_ has quit IRC | 11:21 | |
| *** mdnadeem has quit IRC | 11:22 | |
| *** tfukushima has quit IRC | 11:23 | |
| *** achanda has quit IRC | 11:24 | |
| *** Jeffrey4l has joined #kolla | 11:29 | |
| *** dims_ has joined #kolla | 11:30 | |
| *** dims has quit IRC | 11:31 | |
| *** fgimenez has quit IRC | 11:32 | |
| *** fgimenez has joined #kolla | 11:32 | |
| limamauricio | TASK: [common | Starting rsyslog container] *********************************** failed: [localhost] => {"changed": true, "failed": true} msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/rsyslog/start',),) | 11:34 |
| limamauricio | Sometimes this error happens, but I do not know why | 11:34 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Add Heka log decoder for RabbitMQ https://review.openstack.org/276668 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Add Heka log decoder infrastructure https://review.openstack.org/276667 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Murano logs https://review.openstack.org/281328 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Magnum logs https://review.openstack.org/281329 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect RabbitMQ logs https://review.openstack.org/275844 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Add Heka log decoder for OpenStack https://review.openstack.org/275845 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Nova logs https://review.openstack.org/275846 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Add Heka log decoder for MariaDB https://review.openstack.org/275847 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Heat logs https://review.openstack.org/281327 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Cinder logs https://review.openstack.org/281259 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Add Heka log decoder for Keystone/Apache https://review.openstack.org/279192 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Add a Dockerfile for Heka https://review.openstack.org/275840 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Keystone logs https://review.openstack.org/279193 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Add Heka to common role https://review.openstack.org/275841 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect HAProxy and Keepalived logs https://review.openstack.org/275842 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Swift logs https://review.openstack.org/279490 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Heka collect Neutron logs https://review.openstack.org/279489 | 11:36 |
| openstackgerrit | Eric Lemoine proposed openstack/kolla: Make Ansible write its logs into "kolla_logs" https://review.openstack.org/279191 | 11:36 |
| *** Allen_Gao has quit IRC | 11:39 | |
| *** jmccarthy has quit IRC | 11:40 | |
| limamauricio | TASK: [common | Starting rsyslog container] *********************************** failed: [localhost] => {"changed": true, "failed": true} msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/rsyslog/start',),) | 11:41 |
| limamauricio | Sometimes this error happens, but I do not know why | 11:41 |
| *** jmccarthy has joined #kolla | 11:42 | |
| *** Allen_Gao has joined #kolla | 11:42 | |
| *** mdnadeem has joined #kolla | 11:43 | |
| *** pbourke has joined #kolla | 11:49 | |
| *** Jeffrey4l__ has joined #kolla | 11:50 | |
| *** Jeffrey4l has quit IRC | 11:52 | |
| *** bmace has quit IRC | 11:53 | |
| *** pbourke has quit IRC | 11:55 | |
| *** bmace has joined #kolla | 11:56 | |
| *** salv-orl_ has quit IRC | 12:03 | |
| *** Allen_Ga_ has joined #kolla | 12:04 | |
| *** Allen_Gao has quit IRC | 12:08 | |
| *** Jeffrey4l_ has quit IRC | 12:09 | |
| *** rhallisey has joined #kolla | 12:10 | |
| *** Allen_Ga_ has quit IRC | 12:18 | |
| *** Allen_Gao has joined #kolla | 12:21 | |
| *** salv-orlando has joined #kolla | 12:26 | |
| *** dave-mccowan has joined #kolla | 12:26 | |
| *** Allen_Gao has quit IRC | 12:26 | |
| *** Allen_Gao has joined #kolla | 12:27 | |
| *** JoseMello has joined #kolla | 12:28 | |
| *** Allen_Gao has quit IRC | 12:29 | |
| *** stvnoyes has quit IRC | 12:33 | |
| *** Allen_Gao has joined #kolla | 12:33 | |
| *** Allen_Gao has quit IRC | 12:33 | |
| *** stvnoyes has joined #kolla | 12:33 | |
| *** Allen_Gao has joined #kolla | 12:35 | |
| *** iceyao has joined #kolla | 12:37 | |
| *** fgimenez has quit IRC | 12:38 | |
| *** Allen_Gao has quit IRC | 12:39 | |
| *** fgimenez has joined #kolla | 12:41 | |
| *** fgimenez has quit IRC | 12:41 | |
| *** fgimenez has joined #kolla | 12:41 | |
| *** dwalsh has joined #kolla | 12:42 | |
| *** pbourke has joined #kolla | 12:43 | |
| *** Allen_Gao has joined #kolla | 12:46 | |
| *** pbourke_ has joined #kolla | 12:46 | |
| *** pbourke has quit IRC | 12:48 | |
| *** limamauricio has quit IRC | 13:02 | |
| *** limamauricio has joined #kolla | 13:02 | |
| *** liyi has joined #kolla | 13:12 | |
| liyi | anyone know if there is a "GOOD" commit that I could deploy multiple nodes? | 13:13 |
| *** thumpba has joined #kolla | 13:15 | |
| *** chandankumar has quit IRC | 13:16 | |
| pbourke_ | liyi: master | 13:17 |
| limamauricio | pbourke_ kolla not works with ansible 2.0.0 right? | 13:19 |
| pbourke_ | limamauricio: correct | 13:19 |
| limamauricio | ok | 13:19 |
| limamauricio | pbourke_ msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/rsyslog/start',),) | 13:20 |
| limamauricio | ansible 1.9.4 docker 1.10.1 | 13:20 |
| pbourke_ | docker-py version? | 13:21 |
| limamauricio | 1.7 | 13:21 |
| *** achanda has joined #kolla | 13:22 | |
| pbourke_ | perhaps check journalctl and see what docker is complaining about | 13:22 |
| liyi | i have the same problem like limamauricio, | 13:22 |
| liyi | on master brancher I mean | 13:22 |
| limamauricio | me too liyi | 13:22 |
| liyi | docker start fc0b7baaede2 Error response from daemon: Cannot start container fc0b7baaede2684c834e5997700cd0623b45d800f554d95e6c63996e53a3286e: Path /run/netns is mounted on /run but it is not a shared mount. Error: failed to start containers: fc0b7baaede2 | 13:23 |
| *** chandankumar has joined #kolla | 13:23 | |
| liyi | this is what I got from docker log if I try to manually start kollaglue/ubuntu-binary-neutron-dhcp-agent:2.0.0 | 13:24 |
| limamauricio | MountFlags=shared | 13:24 |
| limamauricio | set it | 13:24 |
| elemoine_ | pbourke_: are you sure kolla support ansible 2? | 13:25 |
| pbourke_ | elemoine_: kolla does *not* support ansible 2 | 13:25 |
| liyi | I have ubuntu, have tired: mount --make-shared /run | 13:25 |
| elemoine_ | I misread | 13:25 |
| elemoine_ | sorry | 13:25 |
| liyi | make no difference | 13:25 |
| limamauricio | :/ | 13:25 |
| pbourke_ | liyi: can you run any docker containers? (docker run -it ubuntu ls) | 13:26 |
| elemoine_ | limamauricio wrote "kolla not works with ansible 2.0.0" which I read as "kolla now works…" | 13:26 |
| pbourke_ | elemoine_: :) | 13:26 |
| limamauricio | how i check journalctl? | 13:27 |
| *** achanda has quit IRC | 13:27 | |
| liyi | yes, other container runs nicely | 13:27 |
| liyi | but when it try to start this particular container, it failed :( | 13:28 |
| pbourke_ | actually, this could be a problem with the new thin neutron containers | 13:28 |
| pbourke_ | so others start up, just not neutron? | 13:28 |
| pbourke_ | s/so/do | 13:28 |
| liyi | just not kollaglue/ubuntu-binary-neutron-dhcp-agent:2.0.0, others like kollaglue/ubuntu-binary-neutron-openvswitch-agent:2.0.0 and kollaglue/ubuntu-binary-neutron-server:2.0.0 works OK | 13:29 |
| limamauricio | pbourke_ i should set docker_insecure_registry: "True" and docker_pull_policy: "missing" in globals? | 13:30 |
| pbourke_ | limamauricio: no Im pretty sure this is a bug | 13:30 |
| pbourke_ | limamauricio: would you mind logging something in launchpad? till then you may like to try the stable/liberty branch | 13:31 |
| pbourke_ | SamYaple: when you're around if you see this ping me | 13:31 |
| limamauricio | i'm in master | 13:31 |
| limamauricio | master/centos/source | 13:31 |
| pbourke_ | limamauricio: git checkout stable/liberty | 13:31 |
| limamauricio | i will not do that because the master deploy was working yesterday | 13:32 |
| limamauricio | but i rebuild and now i have an error | 13:32 |
| limamauricio | :/ | 13:32 |
| pbourke_ | :/ | 13:33 |
| limamauricio | what I did differently was stop and disable firewalld.service | 13:33 |
| pbourke_ | will check logs and see if anything suspect merged since yesterday | 13:33 |
| pbourke_ | yeah it may be something changed on your host that the namespace propagation feature doesn't like | 13:34 |
| limamauricio | before I had not disabled it | 13:34 |
| limamauricio | but i have to disable firewalld.service rigth? | 13:35 |
| limamauricio | right* | 13:35 |
| *** Liuqing has joined #kolla | 13:38 | |
| openstackgerrit | Merged openstack/kolla-mesos: Add memcached dep to horizon https://review.openstack.org/281236 | 13:40 |
| openstackgerrit | Merged openstack/kolla: Fix typo in quickstart https://review.openstack.org/281681 | 13:41 |
| *** sdake has joined #kolla | 13:41 | |
| liyi | @pbourke, anything else could I try? | 13:41 |
| openstackgerrit | Merged openstack/kolla: Make Mistral deployable via ansible https://review.openstack.org/281610 | 13:42 |
| *** reddy has joined #kolla | 13:44 | |
| pbourke_ | liyi: unfortunately I dont have any other suggestions right now. I haven't looked at the thin neutron containers yet | 13:45 |
| pbourke_ | liyi: when SamYaple is online in an hour or two he will likely have suggestions | 13:45 |
| liyi | @thanks a lot Pbourke! | 13:46 |
| liyi | btw, I have this mount output on my node | 13:46 |
| liyi | sudo mount | grep run none on /run type tmpfs (rw,relatime) nsfs on /run/docker/netns/default type nsfs (rw) none on /run type tmpfs (rw,relatime) nsfs on /run/docker/netns/default type nsfs (rw) | 13:46 |
| liyi | if everyone else has a hint, please help | 13:47 |
| *** Allen_Gao has quit IRC | 13:47 | |
| Serlex | I thought the issue was resolved with shared mount | 13:48 |
| Serlex | make sure you've reloaded and restarted the service | 13:48 |
| limamauricio | Serlex, msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/rsyslog/start',),) | 13:48 |
| limamauricio | you known this error? | 13:48 |
| liyi | @Serlex, I did restart docker serivce, but not the OS. | 13:49 |
| pbourke_ | probably, its not related to your issue though | 13:49 |
| limamauricio | know* | 13:49 |
| pbourke_ | ignore above message, wrong window | 13:49 |
| Serlex | I was referring to neutron-agent problem | 13:49 |
| Serlex | my bad | 13:49 |
| pbourke_ | liyi: limamauricio: I was about to suggest something similar to serlex, perhaps try removing all running containers and then restarting docker | 13:50 |
| pbourke_ | bit dumb but it may reset the problematic mount | 13:50 |
| limamauricio | ok | 13:51 |
| liyi | thanks, I will give it a try. | 13:51 |
| fthiagogv | Guys, do someone know about this error msg? msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/kolla_toolbox/start',),) | 13:52 |
| liyi | will report back later. I have 3 controller nodes, will take a while for doing so | 13:52 |
| liyi | @fthiagogv it is hot issue now | 13:53 |
| liyi | if u have not set MountFlags, might give it a try first. hope u luckier than me | 13:53 |
| pbourke_ | fthiagogv: please check journalctl for docker error msgs | 13:54 |
| rhallisey | fthiagogv, have you updated you repo recently? | 13:58 |
| rhallisey | fthiagogv, there was a recent commit that fixed that | 13:59 |
| rhallisey | I would update your repo and rebuild kolla-toolbox | 13:59 |
| pbourke_ | liyi: are you 100% sure the mountflags are getting picked up by the daemon | 14:00 |
| fthiagogv | I think so, but I will update again and try. | 14:00 |
| limamauricio | rhallisey msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/rsyslog/start',),) | 14:01 |
| limamauricio | :/ | 14:01 |
| liyi | @pbourke i don't know how to verify it, do u know how? | 14:01 |
| limamauricio | yesterday was working, but today... | 14:01 |
| pbourke_ | liyi: trying to find that now. I just ask as I have found docker fiddly to configure | 14:02 |
| *** Liuqing has quit IRC | 14:02 | |
| rhallisey | liyi, systemctl daemon-reload | 14:02 |
| rhallisey | systemctl restart docker | 14:02 |
| rhallisey | limamauricio, I haven't run into that.. Will rebuild and deploy and get back to you | 14:02 |
| liyi | i run ubuntu, but did try something similar | 14:03 |
| rhallisey | are you in 14.04? | 14:03 |
| liyi | yes | 14:03 |
| rhallisey | look in /proc/self/mountinfo | 14:04 |
| rhallisey | I think you can grep for 'shared' in there | 14:04 |
| liyi | 349 335 0:17 / /run rw,relatime shared:1 - tmpfs none rw 285 349 0:4 / /run/docker/netns/default rw shared:2 - nsfs nsfs rw | 14:05 |
| *** Allen_Gao has joined #kolla | 14:05 | |
| liyi | here is what I got when grep shared in the file | 14:05 |
| Serlex | can't you do something like this sudo systemctl show docker | grep Mount | 14:06 |
| rhallisey | ubuntu 14.04 uses upstart | 14:06 |
| *** fgimenez has quit IRC | 14:07 | |
| pbourke_ | what file are you setting the mountflags liyi | 14:07 |
| Serlex | still upstart must have a way no showing what the service is running with | 14:07 |
| ajafo | in ubuntu dont set shared in file it need to make mount command on host | 14:07 |
| liyi | I run "mount --make-shared /run" as suggested in the doc | 14:08 |
| *** fgimenez has joined #kolla | 14:08 | |
| *** fgimenez has quit IRC | 14:08 | |
| *** fgimenez has joined #kolla | 14:08 | |
| ajafo | mount --make-shared /run | 14:08 |
| liyi | yep, there is what i did | 14:08 |
| liyi | no help | 14:08 |
| liyi | delete all containers and re-deploy | 14:09 |
| liyi | endup with errors | 14:09 |
| liyi | TASK: [mariadb | Waiting for MariaDB service to be ready] ********************* failed: [k-blade-1] => {"attempts": 6, "changed": false, "cmd": ["docker", "exec", "mariadb", "ls", "/var/lib/mysql/mariadb.pid"], "delta": "0:00:00.212851", "end": "2016-02-18 13:57:24.275956", "failed": true, "rc": 2, "start": "2016-02-18 13:57:24.063105", "stdout_lines": [], "warnings": []} | 14:09 |
| *** thumpba has quit IRC | 14:09 | |
| *** dave-mccowan has quit IRC | 14:09 | |
| rhallisey | I'm wondering if there needs to be some ordering or an addition step here | 14:09 |
| rhallisey | I don't run ubuntu. SamYaple would be able to answer this | 14:10 |
| *** dave-mccowan has joined #kolla | 14:10 | |
| liyi | OK, I wait for Sam then | 14:10 |
| sdake | SamYaple ping me when you get in plz | 14:11 |
| sdake | llyi we have seen that report 3 separate times | 14:11 |
| sdake | please file a bug | 14:11 |
| sdake | i think it has to do with selinux possibly | 14:11 |
| sdake | but i am not certain | 14:11 |
| sdake | liyi just file a bug based upon what you know | 14:12 |
| sdake | liyi that fact that people keep running into it is a problem, but bug reports over irc arenot helpful because they get lost ;) | 14:12 |
| *** pbourke_ has quit IRC | 14:13 | |
| *** pbourke has joined #kolla | 14:13 | |
| liyi | thanks for suggestion sdake. I now clean up all nodes and re-deploy from re-install os | 14:13 |
| liyi | will try to get a report after the new test | 14:13 |
| sdake | just report what you got ;) | 14:14 |
| sdake | if you clean up your environment it may never happen again | 14:14 |
| ajafo | limamauricio: I had this error until get https://review.openstack.org/#/c/279064/ | 14:14 |
| liyi | a little too late, already wipe off logs :) | 14:14 |
| sdake | i see | 14:14 |
| sdake | well we cna't fix things that aren't in bug tracker ;) | 14:15 |
| limamauricio | ajafo, sometimes i have this erro too msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/containers/kolla_toolbox/start',),) | 14:16 |
| ajafo | limamauricio: as I wrote I had this error before apply patch from link, but I see it's abandoned | 14:16 |
| *** britthou_ has joined #kolla | 14:17 | |
| limamauricio | ajafo | 14:18 |
| limamauricio | you use centos? | 14:18 |
| *** pbourke has quit IRC | 14:18 | |
| ajafo | I get this error on centos vagrant machine and I run there ubuntu images | 14:19 |
| *** britthouser has quit IRC | 14:19 | |
| limamauricio | ajafo, u disabled firewalld.service? | 14:20 |
| Allen_Gao | limamauricio: it's abandoned because the rsyslog is moved to named_volume https://review.openstack.org/#/c/280429 | 14:20 |
| limamauricio | I'm doing everything from scratch again | 14:20 |
| limamauricio | centos/source | 14:20 |
| *** pbourke has joined #kolla | 14:21 | |
| limamauricio | and i following this guide http://docs.openstack.org/developer/kolla/quickstart.html | 14:22 |
| limamauricio | step-by-step | 14:22 |
| limamauricio | :) | 14:22 |
| limamauricio | I saw several different ways to create a registry | 14:23 |
| Serlex | why don't you do binary? | 14:23 |
| limamauricio | It should work both, right? | 14:23 |
| limamauricio | ExecStart=/usr/bin/docker daemon -H fd:// $other_args or ExecStart=/usr/bin/docker daemon -H fd:// --insecure-registry localhost:4000 | 14:24 |
| limamauricio | I'll try binary later | 14:25 |
| *** jtriley has quit IRC | 14:25 | |
| *** diogogmt has quit IRC | 14:26 | |
| *** diogogmt has joined #kolla | 14:28 | |
| Allen_Gao | sdake: did you see this bug https://review.openstack.org/#/c/281159/ ? | 14:33 |
| *** jtriley has joined #kolla | 14:33 | |
| *** Marga_ has joined #kolla | 14:40 | |
| *** kproskurin has quit IRC | 14:41 | |
| *** kproskurin has joined #kolla | 14:42 | |
| *** pbourke has quit IRC | 14:44 | |
| *** Allen_Ga_ has joined #kolla | 14:45 | |
| *** pbourke has joined #kolla | 14:45 | |
| *** Marga__ has joined #kolla | 14:46 | |
| *** Allen_Gao has quit IRC | 14:46 | |
| *** Allen_Gao has joined #kolla | 14:46 | |
| *** Allen_Ga_ has quit IRC | 14:47 | |
| sdake | Allen_Gao comment left in bug tracker | 14:47 |
| sdake | i'll leave one in the review | 14:47 |
| *** dave-mccowan has quit IRC | 14:48 | |
| *** Marga_ has quit IRC | 14:49 | |
| Allen_Gao | sdake: I see your comment in traker, thank you for your quick response, then the "rpm -e " is the right fix? I can't understand why Sam -2 on that ? | 14:51 |
| *** tobe has quit IRC | 14:57 | |
| SamYaple | sdake: ping | 14:58 |
| SamYaple | pbourke: ping | 14:58 |
| pbourke | SamYaple: hey | 14:58 |
| SamYaple | whats up | 14:58 |
| pbourke | SamYaple: my connection has been on and off today so I may have missed some messages. but basically limamauricio and liyi are both having the same issue with thin neutron on ubuntu | 14:58 |
| *** diogogmt has quit IRC | 14:58 | |
| SamYaple | is /run a shared mountpoint? | 14:59 |
| pbourke | SamYaple: ill see if I can find the error, basically they have run the mount command as in docs but docker still complains | 14:59 |
| SamYaple | ill need more info than that. but just fyi our gate is ubuntu 14.04 and i run 15.10 so all the ubuntu bases are covered | 15:00 |
| pbourke | docker start fc0b7baaede2 Error response from daemon: Cannot start container fc0b7baaede2684c834e5997700cd0623b45d800f554d95e6c63996e53a3286e: Path /run/netns is mounted on /run but it is not a shared mount. Error: failed to start containers: fc0b7baaede2 | 15:00 |
| pbourke | just wondering if that rings any bells | 15:00 |
| SamYaple | it sounds like /run is not a shared mountpoint | 15:00 |
| pbourke | yeah | 15:00 |
| *** tzn has quit IRC | 15:01 | |
| pbourke | ok I dont have a whole pile more info, maybe the guys will reappear and let us know if its still an issue | 15:01 |
| SamYaple | if you do run this: | 15:01 |
| SamYaple | awk '$5 == "/run" {print $7}' /proc/self/mountinfo | 15:01 |
| SamYaple | if that doesnt say shared, its not shared | 15:02 |
| liyi | hi Sam, nice to see u here | 15:02 |
| *** dave-mccowan has joined #kolla | 15:02 | |
| liyi | I just post my errior into https://bugs.launchpad.net/kolla/+bug/1546798 | 15:02 |
| openstack | Launchpad bug 1546798 in kolla "neutron failed to deploy in ubuntu binary" [Low,Triaged] - Assigned to Sam Yaple (s8m) | 15:02 |
| openstackgerrit | Merged openstack/kolla: Also remove the volumes when cleaning up containers https://review.openstack.org/280722 | 15:03 |
| SamYaple | liyi: did you remove the neutron container after you got tht message? | 15:03 |
| *** tzn has joined #kolla | 15:03 | |
| liyi | if running your magic command, i got this: | 15:03 |
| liyi | awk '$5 == "/run" {print $7}' /proc/self/mountinfo - shared:1 | 15:03 |
| liyi | i actual wipe off everything, include OS, and re-deploy from scratch. | 15:04 |
| liyi | got that error | 15:04 |
| liyi | i mean the error I just reported in https://bugs.launchpad.net/kolla/+bug/1546798 | 15:05 |
| openstack | Launchpad bug 1546798 in kolla "neutron failed to deploy in ubuntu binary" [Low,Triaged] - Assigned to Sam Yaple (s8m) | 15:05 |
| *** sdake has quit IRC | 15:05 | |
| SamYaple | liyi: you ran that command on k-blade-1? or on the host where you run ansible? | 15:06 |
| liyi | you mean your magic command, I run it on k-blade-1 | 15:06 |
| liyi | liyi@k-blade-1:~$ awk '$5 == "/run" {print $7}' /proc/self/mountinfo - shared:1 | 15:07 |
| *** sdake has joined #kolla | 15:08 | |
| *** tzn has quit IRC | 15:08 | |
| limamauricio | i'm not using ubuntu | 15:08 |
| *** Allen_Gao has quit IRC | 15:08 | |
| SamYaple | so it looks like you aren't removing your neutron-dhcp-agent container between tests | 15:08 |
| SamYaple | if you get that errror, correct the mountpoint to shared | 15:08 |
| SamYaple | then docker rm -v -f neutron_dhcp_agent | 15:08 |
| SamYaple | and run the playbooks again | 15:08 |
| liyi | Sam, I wipe off everything my disk and re-install the system then test | 15:09 |
| liyi | i am very sure everything is gone before I deploy. | 15:09 |
| SamYaple | but did you remember to make the mountpoint shared before you ran the playbooks again | 15:10 |
| liyi | yes i do | 15:10 |
| SamYaple | im just not sure how to replicate your issue then. I haven't seen that. The gate doesnt see that. Cna you give me steps to replicate it? | 15:11 |
| SamYaple | the only time ive seen this is when someone forgot something | 15:11 |
| liyi | i make a simple playbook and run "sudo mount --make-shared /run && sudo service restart docker" | 15:11 |
| *** britthouser has joined #kolla | 15:12 | |
| liyi | after that, i run the deployment as usual | 15:12 |
| *** britthou_ has quit IRC | 15:12 | |
| openstackgerrit | Merged openstack/kolla: Fix the incorrectly removed dependent packages https://review.openstack.org/281159 | 15:13 |
| openstackgerrit | Merged openstack/kolla: Add double quotes in kolla_enable_sanity_checks https://review.openstack.org/281823 | 15:13 |
| liyi | Here is what I did: re-format my disk -> install new OS -> run "sudo mount --make-shared /run && sudo service restart docker" on each node -> run kolla playbook | 15:14 |
| SamYaple | liyi: you could attempt to create a seperrate mountpoint for /run/netns | 15:14 |
| SamYaple | its just tmpfs anyway | 15:14 |
| liyi | hmm, on my k-blade-1, I don't see /run/netns | 15:15 |
| liyi | I have this k-blade-1:~$ sudo mount | grep run none on /run type tmpfs (rw,relatime) nsfs on /run/docker/netns/default type nsfs (rw) none on /run type tmpfs (rw,relatime) nsfs on /run/docker/netns/default type nsfs (rw) | 15:15 |
| SamYaple | liyi: you wont see it until its used | 15:16 |
| liyi | none on /run type tmpfs (rw,relatime) | 15:16 |
| liyi | nsfs on /run/docker/netns/default type nsfs(rw) | 15:16 |
| liyi | OK | 15:16 |
| SamYaple | mkdir /run/netns && mount -o rw,relatime,shared -t tmpfs tmpfs /run/netns | 15:17 |
| liyi | mkdir: cannot create directory '/run/netns': File exists | 15:17 |
| liyi | sudo mkdir -p /run/netns && sudo mount -o rw,relatime,shared -t tmpfs tmpfs /run/netns mount: wrong fs type, bad option, bad superblock on tmpfs, missing codepage or helper program, or other error (for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program) In some cases useful info is found in syslog - try dmesg | tail or so | 15:18 |
| *** spisarski has joined #kolla | 15:18 | |
| SamYaple | you said you didnt see /run/netns | 15:18 |
| liyi | i don't see it when in the output of mount | 15:18 |
| SamYaple | ok docker just creates a folder there | 15:19 |
| liyi | when it is actually in the filesystem, strange | 15:19 |
| SamYaple | feel free to remove it | 15:19 |
| SamYaple | its not a mountpoint | 15:19 |
| SamYaple | im saying you can turn it into one | 15:19 |
| liyi | mount: wrong fs type, bad option, bad superblock on tmpfs, missing codepage or helper program, or other error (for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program) In some cases useful info is found in syslog - try dmesg | tail or so | 15:19 |
| liyi | when "sudo mkdir /run/netns && sudo mount -o rw,relatime,shared -t tmpfs tmpfs /run/netns" | 15:20 |
| liyi | I got that error, it seems faile to mount | 15:21 |
| *** iceyao has quit IRC | 15:21 | |
| SamYaple | liyi: can you post the output of `uname -a; cat /etc/issue` | 15:21 |
| liyi | it seem my system dose not support shared flag | 15:22 |
| liyi | after I remove the shared flag, it works | 15:22 |
| *** pbourke has quit IRC | 15:22 | |
| liyi | Linux k-blade-1 4.2.8-ubuntu #1 SMP Wed Feb 3 22:19:08 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | 15:24 |
| liyi | Ubuntu \n \l docker-sys: 172.18.42.1 eth0: 10.1.1.11 lo: 127.0.0.1 | 15:24 |
| *** inc0 has joined #kolla | 15:25 | |
| *** achanda has joined #kolla | 15:25 | |
| SamYaple | liyi: im not sure what to make of that. what distro and version is on k-blade-1? | 15:26 |
| inc0 | good morning | 15:26 |
| akwasnie | hi inc0 | 15:26 |
| inc0 | good morning Ala, long time no see:) | 15:26 |
| liyi | i build my old kernel and pull down ubuntu 14.04 as root fs | 15:27 |
| liyi | i build my own kernel and pull down ubuntu 14.04 as root fs | 15:27 |
| liyi | for some reason, the shared flag is not supported in my system. | 15:27 |
| akwasnie | inc0: yes, quite long...what's up in SA? Poland better?:D | 15:28 |
| inc0 | nope, I like these winters | 15:28 |
| liyi | I will try to have a look into. Thanks for the help Sam! | 15:28 |
| *** pbourke has joined #kolla | 15:28 | |
| inc0 | today is 25C | 15:28 |
| akwasnie | 'winters' | 15:28 |
| SamYaple | I would recommend testing a vanilla install of ubuntu liyi. see if the same issue occurs | 15:28 |
| SamYaple | '25C' | 15:28 |
| akwasnie | xD | 15:29 |
| inc0 | I should move to Kelvin degrees | 15:29 |
| SamYaple | Kelvin is more accurate | 15:30 |
| SamYaple | I would support a switch | 15:30 |
| *** achanda has quit IRC | 15:30 | |
| inc0 | it's just as acurate as celsous | 15:30 |
| inc0 | just with offset | 15:30 |
| akwasnie | in Poland we have 3 C...so it would be sth like 276 K? | 15:31 |
| *** tzn has joined #kolla | 15:31 | |
| akwasnie | in Poland I mean in my city...Poland is not so small that we have only one temperature... ;p | 15:32 |
| liyi | Sam, I was expect the different kernel should not be an issue. Maybe not the case as it turns out :( | 15:32 |
| SamYaple | inc0: F is just as accurate as C with a different offset! | 15:32 |
| inc0 | it is not SamYaple, 1 degree C != 1 degree F | 15:33 |
| inc0 | but truth be told, it's slightly more accurate | 15:33 |
| SamYaple | i got my hard drives in | 15:34 |
| inc0 | cool | 15:36 |
| *** chandankumar has quit IRC | 15:36 | |
| SamYaple | elemoine_: looking good, correct a small issue in common role patch and a full rebase and i think we are ready to merge! | 15:40 |
| *** tobe has joined #kolla | 15:40 | |
| elemoine_ | SamYaple: I had the detach: false change locally already :) | 15:41 |
| SamYaple | elemoine_: oh im suuuuure you did! | 15:41 |
| elemoine_ | SamYaple I did not test Swift yet | 15:41 |
| *** vhosakot has joined #kolla | 15:41 | |
| elemoine_ | I also want to add a commit that removes rsyslog | 15:42 |
| liyi | hi again Sam, what do u get in when run "sudo mkdir /run/netns && sudo mount -o rw,relatime,shared -t tmpfs tmpfs /run/netns" in your OS? I got the same error in both my home built OS and Ubuntu 14.04.3 LTS | 15:42 |
| SamYaple | elemoine_: I would prefer the Heka stuff land in before we remove rsyslog | 15:42 |
| SamYaple | so we can revert if needed | 15:42 |
| elemoine_ | SamYaple: fine with me | 15:43 |
| elemoine_ | for Swift, I am not sure how I can test (I use vagrant) | 15:43 |
| SamYaple | liyi: I iddnt actually run that command but it probably errors teh same way | 15:43 |
| SamYaple | liyi: mount binary is old on 14.04, may not recogized shared | 15:43 |
| SamYaple | but mount --make-shared /run is stil lvalid | 15:43 |
| SamYaple | even on 14.04 | 15:43 |
| elemoine_ | SamYaple: pushing now… | 15:43 |
| liyi | yep | 15:43 |
| liyi | that works for both in my case | 15:44 |
| liyi | but I still end up in problems. | 15:44 |
| SamYaple | are yo urunning on bare metal? in a vm? | 15:44 |
| liyi | in a VM | 15:44 |
| SamYaple | did you install vanilla ubuntu 14.04? | 15:45 |
| liyi | yes, but I have not tried to deploy on vailla one yet. because that take hours to install the OS | 15:45 |
| SamYaple | I dont know what to suggest other than that. If it works on vanilla there is likely some difference that is breaking things | 15:47 |
| liyi | my customized built ubuntu only take minutes to installed, so I have been happily using it instead of vanilla one, until i hit on this problem | 15:47 |
| SamYaple | i deploy vanilla in minutes, its all about that apt-cacher | 15:47 |
| liyi | will test on vanilla ubuntu later on when get time | 15:48 |
| liyi | yep, u r right. | 15:48 |
| *** thumpba has joined #kolla | 15:49 | |
| vhosakot | failed: [localhost] => {"changed": true, "failed": true} | 15:49 |
| vhosakot | msg: APIError(HTTPError(u'500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/images/create?tag=2.0.0&fromImage=192.168.122.45%3A4000%2Fkollaglue%2Fubuntu-source-rsyslog’,),)”. Is the conainter/image broken ? | 15:49 |
| vhosakot | I see this when I run "kolla-ansible deploy" | 15:50 |
| vhosakot | I filed the bug https://bugs.launchpad.net/bugs/1546863 last night | 15:50 |
| openstack | Launchpad bug 1546863 in kolla ""kolla-ansible deploy" throws HTTPError when starting rsyslog container" [Undecided,In progress] - Assigned to Vikram Hosakote (vhosakot) | 15:50 |
| vhosakot | any idea how to resolve this... can I ignore this error ? | 15:51 |
| pbourke | vhosakot: can you check the docker logs to see what error its throwing? | 15:54 |
| *** pbourke has quit IRC | 15:54 | |
| *** pbourke has joined #kolla | 15:56 | |
| *** diogogmt has joined #kolla | 15:57 | |
| *** Serlex has quit IRC | 15:59 | |
| fthiagogv | vhosakot what OS? Centos or Ubuntu? | 16:01 |
| vhosakot | Ubuntu 14.04 | 16:01 |
| vhosakot | pbourke: I cannot check logs as the container rsyslog is not running | 16:02 |
| pbourke | vhosakot: I mean docker daemon logs | 16:02 |
| pbourke | under /var/log | 16:02 |
| *** salv-orl_ has joined #kolla | 16:03 | |
| *** zigo has quit IRC | 16:03 | |
| fthiagogv | vhosakot do you set insecure-registry? | 16:03 |
| vhosakot | yes | 16:04 |
| vhosakot | ok | 16:04 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add docs to explain why we disabling selinux https://review.openstack.org/280995 | 16:04 |
| vhosakot | in /var/log ok | 16:04 |
| *** zigo has joined #kolla | 16:05 | |
| *** blahRus has joined #kolla | 16:05 | |
| vhosakot | I see this error for the "Starting rsyslog container" task when running "kolla-ansible deploy" | 16:05 |
| * vhosakot is checking /var/log | 16:05 | |
| fthiagogv | vhosakot are you doing single or multinode? | 16:06 |
| *** salv-orlando has quit IRC | 16:06 | |
| *** coolsvap|away has quit IRC | 16:06 | |
| SamYaple | vhosakot: /var/log/upstart/docker.log | 16:06 |
| vhosakot | fthiagogv: all-in-one single | 16:06 |
| SamYaple | can I get an ack on this patch? https://review.openstack.org/#/c/280738/ | 16:07 |
| vhosakot | wow | 16:07 |
| vhosakot | I see this | 16:07 |
| vhosakot | [37460] Handler for GET /v1.22/containers/kollaglue/ubuntu-source-rsyslog/json returned error: No such container: kollaglue/ubuntu-source-rsyslog | 16:07 |
| vhosakot | Is /var/log/upstart/docker.log docker daemon's logfile ? | 16:07 |
| SamYaple | vhosakot: did you set your registry correctly in gobals.yml? | 16:08 |
| SamYaple | globals* | 16:08 |
| SamYaple | alright I need another core to volunteer to go through all the Heka patches. | 16:08 |
| SamYaple | I have +2 them all after many patchsets | 16:08 |
| SamYaple | they are ready for merge, but they need to all my gone through again | 16:08 |
| *** chandankumar has joined #kolla | 16:09 | |
| SamYaple | starting here https://review.openstack.org/#/c/275840 | 16:09 |
| vhosakot | I have "docker_registry: "192.168.122.45:4000" in globals.yml | 16:09 |
| elemoine_ | thanks SamYaple, I'll try to test Swift in the mean time | 16:09 |
| vhosakot | I have this in /etc/init.d/docker | 16:09 |
| vhosakot | DOCKER_OPTS="-H fd:// --insecure-registry 192.168.122.45:4000" | 16:09 |
| vhosakot | I also did docker run -d -p 4000:5000 --restart=always --name registry registry:2 | 16:10 |
| pbourke | SamYaple: Ive started going through them, but swamped with other stuff. I want to get swift upgrade tested before I look at anything else | 16:10 |
| SamYaple | pbourke: i get that. this logging this is just a big change and likely full of bugs (no offense elemoine_) and I want to see it merged well before mitaka-3 | 16:11 |
| SamYaple | so if you can carve out time that would be greatly appreciated | 16:12 |
| pbourke | yeah i'll do my best. Id def like to test it before merge | 16:12 |
| elemoine_ | SamYaple: that makes totally sense to me (merging this well before mitaka-3) | 16:12 |
| *** inc0 has quit IRC | 16:12 | |
| pbourke | switching between liberty and master is a killer | 16:12 |
| fthiagogv | vhosakot try to push some image using your registry, "docker push 192.168.122.45:4000/kollaglue/..." When i had this error was because that registry was wrong. However was in Centos. | 16:12 |
| SamYaple | pbourke: I did not test all of the services, but the core ones are working fwiw | 16:13 |
| vhosakot | fthiagogv: cool, let me push | 16:14 |
| fthiagogv | Do someone know about this? fatal: [control] => One or more undefined variables: 'dict object' has no attribute 'stdout' | 16:15 |
| fthiagogv | task: neutron | reading json from variable | 16:15 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 16:17 |
| SamYaple | mandre: i think i figured it out | 16:17 |
| SamYaple | mandre: turns out some of the hypervisors have HVM enabled for nested KVM. who would have guessed! | 16:17 |
| *** coolsvap|away has joined #kolla | 16:20 | |
| *** reddy has quit IRC | 16:22 | |
| *** akwasnie has quit IRC | 16:23 | |
| *** daneyon has joined #kolla | 16:30 | |
| *** daneyon__ has joined #kolla | 16:31 | |
| *** macsz has quit IRC | 16:31 | |
| limamauricio | manila needs to use openvswitch | 16:32 |
| *** fgimenez has quit IRC | 16:33 | |
| limamauricio | SamYaple, manila needs openvswitch, but i can't create openvswitch dir on manila_share container | 16:34 |
| limamauricio | :/ | 16:34 |
| *** daneyon_ has quit IRC | 16:34 | |
| *** daneyon has quit IRC | 16:34 | |
| *** fgimenez has joined #kolla | 16:34 | |
| SamYaple | limamauricio: i dont follow | 16:35 |
| vhosakot | "docker push 192.168.122.45:4000/kollaglue/ubuntu-source-ceph-mon" returns | 16:35 |
| vhosakot | Repository does not exist: 192.168.122.45:4000/kollaglue/ubuntu-source-ceph-mon | 16:35 |
| vhosakot | "curl http://192.168.122.45:4000/v2/" returns | 16:36 |
| vhosakot | curl: (7) Failed to connect to 192.168.122.45 port 4000: No route to host | 16:36 |
| vhosakot | should I be able to ping the registry IP 192.168.122.45.. I can't ping | 16:36 |
| *** chandankumar has quit IRC | 16:38 | |
| SamYaple | vhosakot: i would recommend setting up your registry and attempting to push/pull images outside of kolla first | 16:39 |
| *** haplo37 has joined #kolla | 16:39 | |
| SamYaple | there are many documents online about how to do this | 16:39 |
| pbourke | vhosakot: are you running a registry | 16:39 |
| pbourke | on 192.168.122.45 | 16:39 |
| *** pbourke has quit IRC | 16:39 | |
| vhosakot | yep.. doing it... I think I need to tag first | 16:39 |
| vhosakot | yes | 16:39 |
| vhosakot | on 192.168.122.45 | 16:39 |
| vhosakot | on port 4000 | 16:40 |
| *** pbourke has joined #kolla | 16:41 | |
| fthiagogv | vhosakot, when you build, you use the option --push? And didn't show any error? When your registry are broke, build --push does not works good and many images named <none> appears. | 16:42 |
| vhosakot | fthiagogv: no, I did not use the option --push when building | 16:43 |
| *** prithiv has joined #kolla | 16:45 | |
| prithiv | I am having trouble deploy kolla on bare metal and facing issue | 16:46 |
| prithiv | can someone help me.. | 16:46 |
| g3ek | hello, is kolla have a way to test the installation of a container appart of ansible or should I use ansible unit test to test it? | 16:46 |
| rhallisey | prithiv, ya sure what's up? | 16:46 |
| prithiv | i am just tryin got build on a baremetal AIO node. i am getting error with memcached container | 16:48 |
| rhallisey | g3ek, not sure what you mean. Do you want to test if a service is working? | 16:48 |
| prithiv | 500 Server Error: Internal Server Error for url: http+docker://localunixsocket/v1.22/images/create?tag=2.0.0&fromImage=10.237.214.35%3A5000%2Fkollaglue%2Fubuntu-source-memcached' | 16:48 |
| rhallisey | g3ek, you can look at the containers status. Docker ps -a | 16:48 |
| *** mikelk has quit IRC | 16:49 | |
| prithiv | how would i find which one is memcached container | 16:49 |
| prithiv | i ddint try to look at providing names for the container when pulling | 16:50 |
| g3ek | rhallisey, I have to use kolla to deploy a whole openstack environment and some of our own service. I need to check if everything have been correctly installed and is working after the kolla-ansible deploy. I think the best way to check this will be with ansible, but I don't know if kolla have another way to make those check. | 16:50 |
| *** dims has joined #kolla | 16:50 | |
| *** aginwala has joined #kolla | 16:51 | |
| rhallisey | prithiv, docker logs memcached | 16:51 |
| rhallisey | need some more info | 16:51 |
| prithiv | i dont see any containers in that name | 16:51 |
| *** dims_ has quit IRC | 16:51 | |
| rhallisey | ok so it's just failing to show up hmm | 16:51 |
| g3ek | prithiv, maybe the container is down try with docker ps -a | 16:52 |
| rhallisey | g3ek, ./tools/kolla-ansible post-deploy | 16:52 |
| rhallisey | I think that's what you may be looking for | 16:52 |
| g3ek | rhallisey, this is not for the admin_openrc file? | 16:52 |
| prithiv | i tried with ps -a… | 16:53 |
| prithiv | the thing is i have container name auto-assigned by docker. i did not know if there is a way to specify container name when doing kolla-build | 16:54 |
| g3ek | rhallisey, ohhhh this run the file ./ansible/post-deploy.wml cool, thanks | 16:54 |
| rhallisey | g3ek, ya I think that's what your looking for | 16:55 |
| rhallisey | prithiv, there is. `kolla-build memcached <flags> | 16:55 |
| limamauricio | someone know how i do to install openvswitch in manila? | 16:57 |
| limamauricio | just add yum -y install openvswitch on manila-base dockerfile? | 16:58 |
| rhallisey | ya that would do it | 16:59 |
| SamYaple | limamauricio: so manila needs the ovs binaries is what you are saying? | 16:59 |
| limamauricio | yes | 17:00 |
| SamYaple | yea then thats the way to do it | 17:00 |
| SamYaple | just mount /run to get the correct socket too | 17:00 |
| limamauricio | openvswitch is required to manila | 17:00 |
| limamauricio | mount /run | 17:01 |
| limamauricio | hm... | 17:01 |
| limamauricio | ok | 17:01 |
| SamYaple | limamauricio: no install openvswitch and then rememeber to add /run:/run to the mounts for the container | 17:03 |
| SamYaple | so it can talk to openvswitch server | 17:03 |
| *** alisonh has quit IRC | 17:03 | |
| *** fgimenez has quit IRC | 17:04 | |
| limamauricio | So, i shouldn't add yum install openvswitch in manila-base dockerfile? | 17:04 |
| limamauricio | just add /run:run? | 17:04 |
| limamauricio | just add /run:/run? | 17:04 |
| *** tzn has quit IRC | 17:05 | |
| SamYaple | limamauricio: install openvswitch in manila-base (or whatever subset application needs it) | 17:07 |
| SamYaple | then add /run:/run | 17:07 |
| *** aginwala has quit IRC | 17:07 | |
| openstackgerrit | Merged openstack/kolla: Ceilometer ubuntu binary container https://review.openstack.org/277286 | 17:10 |
| *** tzn has joined #kolla | 17:11 | |
| *** dmsimard is now known as rdobot | 17:11 | |
| sdake | its a shame openvswitch sin't packaged into the cleint and sever as seprate packages | 17:12 |
| sdake | all that server bloat for no good reason | 17:12 |
| sdake | oh well containers ftw ;) | 17:12 |
| *** rdobot is now known as dmsimard | 17:13 | |
| *** tobe has quit IRC | 17:13 | |
| *** daneyon has joined #kolla | 17:16 | |
| *** daneyon_ has joined #kolla | 17:17 | |
| *** daneyon__ has quit IRC | 17:19 | |
| *** daneyon has quit IRC | 17:21 | |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Change ubuntu base_image for vagrant to wily64 https://review.openstack.org/281968 | 17:21 |
| *** chandankumar has joined #kolla | 17:22 | |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/kolla: Change ubuntu base_image for vagrant to wily64 https://review.openstack.org/281968 | 17:23 |
| SamYaple | sdake: there was openvswitch talk about seperate packaging back in 2.1 | 17:26 |
| SamYaple | i think its landing/landed in 2.5 | 17:26 |
| *** achanda has joined #kolla | 17:28 | |
| prithiv | nope it didnt help | 17:29 |
| prithiv | docker ps -a doesnt show any of the containers with openstack services name in it | 17:30 |
| *** achanda has quit IRC | 17:33 | |
| liyi | can anyone explains a little bit what is thin container about? I am lost after we introduce thin container with docker 1.10 | 17:35 |
| *** ssurana has joined #kolla | 17:39 | |
| *** unicell has quit IRC | 17:40 | |
| *** kproskurin has quit IRC | 17:40 | |
| SamYaple | liyi: "thin container" is a container running a single process | 17:41 |
| SamYaple | kolla has always been thin containers for most services | 17:41 |
| *** kproskurin has joined #kolla | 17:41 | |
| *** alisonh has joined #kolla | 17:46 | |
| *** jmccarthy1 has quit IRC | 17:48 | |
| liyi | Thanks Sam! | 17:49 |
| sdake | actually our thi ncontainers run multiple processes | 17:52 |
| sdake | i'd define thin container as somehting that does "one type of job" | 17:52 |
| sdake | in the cae of neutron, the neutron agent container did 4 (?) different jobs | 17:53 |
| sdake | SamYaple made them all do one type of job | 17:53 |
| *** limamauricio has quit IRC | 17:54 | |
| *** athomas has quit IRC | 17:54 | |
| *** limamauricio has joined #kolla | 17:54 | |
| liyi | I though it was a new way to minimise the size of the container, but now I got the idea | 17:55 |
| sdake | its a way to minimize the number of jobs to 1 that a container does | 17:58 |
| sdake | thin container bit of a misnomer i guess liyi :) | 17:58 |
| *** athomas has joined #kolla | 18:00 | |
| liyi | @sdake, thanks for making it clear. be offline now, Have a nice evening! | 18:02 |
| *** liyi has left #kolla | 18:02 | |
| *** Marga__ has quit IRC | 18:02 | |
| *** Marga_ has joined #kolla | 18:03 | |
| *** pbourke has quit IRC | 18:03 | |
| *** achanda has joined #kolla | 18:03 | |
| *** Jeffrey4l__ has quit IRC | 18:06 | |
| *** Marga_ has quit IRC | 18:12 | |
| *** aginwala has joined #kolla | 18:13 | |
| *** mbound has quit IRC | 18:14 | |
| openstackgerrit | Merged openstack/kolla: Add docs to explain why we disabling selinux https://review.openstack.org/280995 | 18:14 |
| *** Marga_ has joined #kolla | 18:18 | |
| sdake | SamYaple the selinux doc is pretty solid | 18:19 |
| *** unicell has joined #kolla | 18:20 | |
| *** prithiv has quit IRC | 18:22 | |
| *** aginwala has quit IRC | 18:23 | |
| SamYaple | sdake: technically its "tracking one process" that defines thin containers | 18:24 |
| *** chandankumar has quit IRC | 18:24 | |
| *** fthiagogv_ has joined #kolla | 18:24 | |
| sdake | i think the definition is open to intereptation | 18:24 |
| sdake | what we know it is not, is doing multiple different types of jobs in the same container ;) | 18:25 |
| sdake | i prefer to bang on fat containers, because those are easy to define :) | 18:25 |
| *** fthiagogv__ has joined #kolla | 18:26 | |
| *** achanda has quit IRC | 18:26 | |
| *** aginwala has joined #kolla | 18:27 | |
| *** fthiagogv has quit IRC | 18:27 | |
| *** fthiagogv_ has quit IRC | 18:29 | |
| sdake | its a shame with heka we can't make all these xtend_starts go in the base file | 18:30 |
| openstackgerrit | Merged openstack/kolla: Set nova hypervisor to qemu in Vagrant https://review.openstack.org/281637 | 18:31 |
| sbezverk | sdake I noticed weird behavior with openvswitch, when I reboot my server and then run ovs-vsctl show, it shows wrong bridge configuration, but if I restart openvswitch service, everything becomes correctly plugged. | 18:33 |
| sdake | sbezverk I think we have some work to do on node restart | 18:33 |
| sdake | sbezverk please file a bug | 18:33 |
| sdake | unfortunately docker doesn't have start ordering | 18:34 |
| sbezverk | sdake got it.. | 18:34 |
| sdake | this is probably something we can fix in the containers themselves | 18:34 |
| sdake | would be nice if it was surgical rather then all over the place though ;) | 18:34 |
| *** prithiv has joined #kolla | 18:35 | |
| *** achanda has joined #kolla | 18:37 | |
| sdake | whats with zuul | 18:41 |
| sbezverk | sdake https://bugs.launchpad.net/kolla/+bug/1547168 | 18:41 |
| openstack | Launchpad bug 1547168 in kolla "wrong connections with openvswitch after server reboot" [Undecided,New] | 18:41 |
| sdake | sbezverk thanks it has been triaged | 18:42 |
| sdake | elemoine_ ping | 18:44 |
| *** Marga_ has quit IRC | 18:45 | |
| *** gfidente has quit IRC | 18:47 | |
| *** Marga_ has joined #kolla | 18:48 | |
| sbezverk | sdake it also requires to restart l3-agent container to refresh namespaces info | 18:51 |
| *** Marga_ has quit IRC | 18:54 | |
| *** Marga_ has joined #kolla | 18:55 | |
| vhosakot | can someone send the output of "px aux | grep docker".. in Ubuntu 14.04, in which file should I add DOCKER_OPTS="-H fd:// --insecure-registry 192.168.122.45:4000" ? | 18:56 |
| vhosakot | /etc/init.d/docker or /etc/init/docker.conf or /etc/default/docker ? | 18:56 |
| *** achanda has quit IRC | 19:03 | |
| *** tzn has quit IRC | 19:03 | |
| vhosakot | is "-H fd://" the right docker daemon socket to use for kolla ? | 19:06 |
| *** tzn has joined #kolla | 19:14 | |
| *** achanda has joined #kolla | 19:17 | |
| *** tzn has quit IRC | 19:18 | |
| *** jasonsb has quit IRC | 19:20 | |
| sdake | d_code around, can you paste your script for centos setup? | 19:24 |
| sbezverk | Anybody could point to the location of kolla's ansible inventory file? It looks like it does not use default /etc/ansible/hosts? | 19:24 |
| *** prithiv has quit IRC | 19:24 | |
| *** aginwala has quit IRC | 19:25 | |
| sdake | sbezverk ansible/inventory/all-in-one | 19:25 |
| sdake | or ansible/invetory/multinode | 19:25 |
| sdake | kolla-ansible deploy defaults to all-in-one | 19:25 |
| vhosakot | I resolved it. "-H fd://" option for docker daemon is just for centos, not for Ubuntu.. after removing "-H fd://", my registry is up :) | 19:26 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla: Add a precheck for MountFlags=shared https://review.openstack.org/281279 | 19:26 |
| vhosakot | I will correct this in the quickstart guide for Ubuntu host | 19:26 |
| rhallisey | SamYaple, can you try that out^^ | 19:26 |
| rhallisey | see if that works for you | 19:27 |
| rhallisey | on ubuntu | 19:27 |
| sbezverk | sdake when I use this command to deploy swift: kolla-ansible deploy --tags swift it skips the ip address I have in all-in-one | 19:27 |
| sbezverk | sdake Here is all-in-one I use, appreciate if you could check what is wrong: http://paste.openstack.org/show/487473/ | 19:36 |
| *** aginwala has joined #kolla | 19:38 | |
| *** vhosakot has quit IRC | 19:42 | |
| *** jtriley has quit IRC | 19:44 | |
| *** vhosakot has joined #kolla | 19:44 | |
| *** jtriley has joined #kolla | 19:48 | |
| *** achanda has quit IRC | 19:51 | |
| SamYaple | sbezverk: did you set 'enable_swift: "yes"' in globals.yml? | 19:53 |
| SamYaple | rhallisey: will check! | 19:53 |
| SamYaple | but it looks good | 19:53 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 19:55 |
| *** achanda has joined #kolla | 19:58 | |
| sbezverk | SamYaple that was it, I did not.. thank you | 20:00 |
| *** gtt116_ has joined #kolla | 20:06 | |
| vhosakot | I changed the port.. I get this error now | 20:07 |
| vhosakot | Error while pulling image: Get http://192.168.122.41:4000/v1/repositories/v2/repositories/kollaglue/ubuntu-source-rsyslog/images: dial tcp 192.168.122.41:4000: getsockopt: connection refused | 20:07 |
| dave-mccowan | does anyone use .header and .footer files to poke through an http proxy for tools/build.py? it's not working for me as documented. | 20:09 |
| *** jmccarthy1 has joined #kolla | 20:11 | |
| *** jmccarthy has quit IRC | 20:12 | |
| *** gtt116 has quit IRC | 20:12 | |
| *** zaneb has quit IRC | 20:12 | |
| *** mandre has quit IRC | 20:12 | |
| *** vincent_vdk has quit IRC | 20:12 | |
| *** zaneb has joined #kolla | 20:12 | |
| *** vincent_vdk has joined #kolla | 20:12 | |
| SamYaple | dave-mccowan: docker 1.10 actually has proxy support without header and footer files | 20:13 |
| *** mandre has joined #kolla | 20:13 | |
| dave-mccowan | SamYaple cool. my environment variable are set and my ansible is 1.10.1. hmmm, why else would i get a network timeout? git, yum, pip, etc. all work. | 20:16 |
| *** aginwala has quit IRC | 20:16 | |
| *** Marga_ has quit IRC | 20:17 | |
| dave-mccowan | i can wget https://index.docker.io/v1/repositories/library/centos/images, but build.py times out getting it. | 20:20 |
| SamYaple | dave-mccowan: hmm i woudl suggest going a small test outside of kolla | 20:25 |
| SamYaple | get it working that way and we can figure out whats wrong | 20:25 |
| dave-mccowan | i tried wget to docker.io, and it works through the proxy. what else to test? | 20:26 |
| SamYaple | a simple dockerfile outside of kolla | 20:27 |
| *** aginwala has joined #kolla | 20:27 | |
| *** vhosakot has quit IRC | 20:27 | |
| *** vhosakot has joined #kolla | 20:28 | |
| sbezverk | SamYaple for swift installation I need to generate account.builder and two other .builder files using swift-ring-builder. I started swift-base container and found this tool there. I generated required files, but they exist within a container, do I have to scp these files back to host or there is another more simple way? | 20:37 |
| *** athomas has quit IRC | 20:39 | |
| *** athomas has joined #kolla | 20:40 | |
| sdake | sbezverk the swift guide details how to do the job | 20:46 |
| sdake | check docs | 20:46 |
| *** Serlex has joined #kolla | 20:49 | |
| *** thumpba has quit IRC | 20:53 | |
| sbezverk | sdake my question about swift is not covered by usual openstack swift operational guide.. it is more container related and how to get files generated inside of a container back to the host.. | 20:55 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Update Instructions for Building Behind HTTP Proxy https://review.openstack.org/282057 | 21:04 |
| SamYaple | sbezverk: to transfer files between containers and host you can use the docker cp command `docker cp <container>:/path/to/file /path/on/host` | 21:05 |
| SamYaple | that works in reverse too | 21:05 |
| *** dwalsh_ has joined #kolla | 21:06 | |
| vhosakot | fixed my port and registry.... still "kolla-ansible deploy" throws the error | 21:06 |
| vhosakot | msg: The requested image does not exist: 192.168.122.41:4000/kollaglue/ubuntu-source-rsyslog:2.0.0 | 21:06 |
| sbezverk | SamYaple that is what I was looking for.. btw swift containers are up and running :-) | 21:07 |
| openstackgerrit | Harm Weites proposed openstack/kolla: Some fixes specific to running with Vagrant https://review.openstack.org/277185 | 21:08 |
| *** tzn has joined #kolla | 21:08 | |
| *** tzn has quit IRC | 21:12 | |
| SamYaple | sbezverk: awesome! | 21:12 |
| vhosakot | I do see this image ubuntu-source-rsyslog | 21:18 |
| *** inc0 has joined #kolla | 21:25 | |
| vhosakot | the registry is looks good.. I can push and pull images... but kolla-ansible fails... | 21:26 |
| vhosakot | can anyone help... I can paste stuff in pastebin | 21:26 |
| inc0 | ayoung, just seen your blog post, thanks:) | 21:31 |
| inc0 | http://adam.younglogic.com/2016/02/holla-kolla/ | 21:31 |
| openstackgerrit | Harm Weites proposed openstack/kolla: Some fixes specific to running with Vagrant https://review.openstack.org/277185 | 21:35 |
| *** asalkeld has joined #kolla | 21:35 | |
| *** mbound has joined #kolla | 21:38 | |
| openstackgerrit | Harm Weites proposed openstack/kolla: Some fixes specific to running with Vagrant https://review.openstack.org/277185 | 21:42 |
| *** tpot has joined #kolla | 21:50 | |
| ayoung | inc0, your welcome. I am working on a second one for running the Keystone functional tests against it | 21:51 |
| inc0 | fyi we added your selinux issue as one of our priorities for N | 21:52 |
| inc0 | so you should have it in repo early N or even earlier | 21:52 |
| Serlex | anyone know where I can set the disk size for vagrant deployment, 20g wasn't enough for operator | 21:53 |
| Serlex | actually nvm | 21:54 |
| *** salv-orlando has joined #kolla | 22:03 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 22:03 |
| *** inc0 has quit IRC | 22:04 | |
| *** salv-orl_ has quit IRC | 22:05 | |
| *** aginwala has quit IRC | 22:14 | |
| *** jtriley has quit IRC | 22:16 | |
| *** aginwala has joined #kolla | 22:16 | |
| *** dwalsh has quit IRC | 22:23 | |
| *** dwalsh_ has quit IRC | 22:23 | |
| *** dave-mccowan has quit IRC | 22:26 | |
| *** JoseMello has quit IRC | 22:28 | |
| *** Marga_ has joined #kolla | 22:29 | |
| SamYaple | sdake: vm functional test passed gate | 22:38 |
| SamYaple | but... only on ovh | 22:39 |
| SamYaple | wierd right? | 22:39 |
| openstackgerrit | Suhail Syed proposed openstack/kolla: Refined documentation regarding registry https://review.openstack.org/282084 | 22:40 |
| sdake | SamYaple rockin dude! | 22:40 |
| sdake | only on ovh? | 22:44 |
| SamYaple | so it seems | 22:45 |
| SamYaple | the others throw no valid host, but im working on it | 22:45 |
| sdake | ovh means what? | 22:49 |
| sdake | sorry super long week already | 22:49 |
| SamYaple | ovh is one of the cloud providers that infra uses | 22:50 |
| sdake | i thought that was what you meant but wasn't sure | 22:50 |
| sdake | so maybe its a oom problem? | 22:50 |
| sdake | or somethign else | 22:50 |
| sdake | maybe virt on virt doesn't work on other cloud provides | 22:50 |
| sdake | are you doing qemu virt? | 22:50 |
| sdake | or kvm? | 22:51 |
| sdake | rax i dont think uses kvm | 22:51 |
| SamYaple | qemu | 22:51 |
| SamYaple | but it doesnt matter if rax uses kvm or not | 22:51 |
| sdake | that is entirely emulated | 22:51 |
| SamYaple | kvm could work on rax on xen | 22:51 |
| sdake | it should work | 22:51 |
| SamYaple | but thats beside the point | 22:51 |
| sdake | didn't know kvm on xen worked ;) | 22:51 |
| * sdake wtbs more underlays!! | 22:52 | |
| SamYaple | its all about cpu flags | 22:52 |
| SamYaple | nothing else | 22:52 |
| sdake | question from daughter "Why does my computer make liquid sounds when it turns on" | 22:54 |
| sdake | "Why its liquid cooled my dear" :) | 22:55 |
| SamYaple | like my servers! | 22:55 |
| *** spisarski has quit IRC | 22:56 | |
| asalkeld | o/ | 23:00 |
| *** spisarski has joined #kolla | 23:00 | |
| sdake | asalkeld yo bro | 23:01 |
| sdake | i slacked yesterday | 23:01 |
| sdake | sorry | 23:01 |
| sdake | email coming | 23:01 |
| asalkeld | np | 23:01 |
| sdake | dmsimard ping re packstack + magnum | 23:01 |
| sdake | dmsimard is the reason magnum isnt in the current-ci repo because of lack of packstack implementation? | 23:02 |
| *** aginwala has quit IRC | 23:02 | |
| *** aginwala has joined #kolla | 23:06 | |
| dmsimard | No | 23:08 |
| dmsimard | We haven't had a promotion to current-passed-ci in a while because trunk keeps breaking | 23:09 |
| sdake | so packstack is implemented for magnum? | 23:12 |
| sdake | dmsimard ^ | 23:12 |
| sdake | i assumed cuurrent-passed-ci is based upon a packstack ci cycle | 23:13 |
| sdake | by trunk do you mean trunk of magnum or trunk of openstack? | 23:13 |
| vhosakot | i need help o/ | 23:18 |
| vhosakot | :) | 23:18 |
| SamYaple | wow! | 23:19 |
| SamYaple | zfsonlinux will be in default repo for 16.04 | 23:19 |
| SamYaple | ubuntu | 23:19 |
| vhosakot | sdake: how is remote debugging done in the team? In the neutron channel, I've used https://tmate.io to do remote-debugging | 23:19 |
| SamYaple | vhosakot: someone opens up a port to root.... | 23:20 |
| *** haplo37 has quit IRC | 23:20 | |
| sdake | vhosakot yup thats how its done, port ro root through their firewall, which is suboptimal since not alot of people can actually do so :) | 23:21 |
| sdake | looking at tmate.io | 23:21 |
| SamYaple | requires trust | 23:21 |
| SamYaple | for the record i like tmate alot | 23:21 |
| vhosakot | yes, open a port to root, on my machine... wow, I'm on o crop network ;) | 23:22 |
| vhosakot | corp* == corporate | 23:22 |
| SamYaple | vhosakot: its called a reverse tunnel | 23:22 |
| SamYaple | even on your orporate netwrk yo ucan do it ;) | 23:23 |
| vhosakot | I see this error | 23:23 |
| vhosakot | TASK: [common | Starting rsyslog container] *********************************** | 23:23 |
| vhosakot | failed: [localhost] => {"failed": true} | 23:23 |
| vhosakot | msg: The requested image does not exist: 192.168.122.41:4000/kollaglue/ubuntu-source-rsyslog:2.0.0 | 23:23 |
| SamYaple | thats a pretty clear cut error | 23:23 |
| sdake | SamYaple i'm ssure corps sec groups would have a kaniption fit if they knew that was happening :) | 23:23 |
| vhosakot | ssh -R ;) | 23:23 |
| SamYaple | i just open a port to my public ip and point it to bash | 23:24 |
| SamYaple | thats just me though | 23:24 |
| *** tzn has joined #kolla | 23:24 | |
| sdake | vhosakot what is on ip .41? | 23:24 |
| sdake | your registry? | 23:24 |
| sdake | and if so, did you build with --push | 23:24 |
| sdake | and --registery 192.168.22.41:4000 | 23:24 |
| vhosakot | the IP address of eth0... the only IP on the Ubuntu box I used to SSH into | 23:24 |
| *** blahRus has quit IRC | 23:24 | |
| vhosakot | yes.. done.. "ps aux | grep docker" returns | 23:25 |
| vhosakot | /usr/bin/docker daemon --insecure-registry 192.168.122.41:4000 | 23:25 |
| sdake | run docker ps -a and paste somewhere not in irc :) | 23:25 |
| vhosakot | sdake: BTW, the "-H fd://" argument mentioned in the quistart guide to setup docker socket is only for CentOS, not for Ubuntu 14.04.. on Ubuntu, the default Unix socket works for docker... | 23:26 |
| * sdake hopes we get a lab session at summit | 23:26 | |
| sdake | vhosakot cool doc fix requested ;) | 23:27 |
| vhosakot | since the error message is not clean, I opened a bug against docker https://github.com/docker/docker/issues/20468#issuecomment-185971073 | 23:27 |
| vhosakot | yes, doc fix coming | 23:27 |
| vhosakot | doc fix, automated kolla install on Ubuntu, --force-cross-build flag... all are coming | 23:27 |
| vhosakot | the help must come too... hahaha :) | 23:27 |
| vhosakot | ok, will send the output in pastebin :) | 23:28 |
| *** tobe has joined #kolla | 23:28 | |
| sdake | docker insepct is about the limit of docker diags ;-) | 23:28 |
| sdake | their logging is terirble imo | 23:28 |
| vhosakot | yup.... i mean /var/log/upstart/docker.log does not have timestamp! what package does that ;) | 23:29 |
| vhosakot | I filed two Vagrant bugs too when installing kolla... Vagrant is just for dev work, not prod | 23:30 |
| sdake | yes that should be clear already in the docs | 23:30 |
| sdake | This guide describes how to use Vagrant to assist in developing for Kolla. | 23:31 |
| sdake | right at top of the vagrant doc | 23:31 |
| sdake | clunky wording, admittidly | 23:31 |
| dmsimard | sdake: magnum doesn't need to be included in packstack for it to be part of RDO or current-passed-ci | 23:32 |
| dmsimard | hell, there's even cloud kitty in our repos | 23:32 |
| dmsimard | what makes a package go from current to current-passed-ci is a pipeline of jenkins jobs that test openstack through various installers (right now: tripleo, packstack, puppet-openstack -- eventually: kolla) | 23:33 |
| sdake | not sure what a cloud kitten is | 23:33 |
| vhosakot | sdake: here you go :) | 23:33 |
| sdake | must be fluffy :) | 23:33 |
| dmsimard | sdake: if all of these installers install openstack and test it successfully (tempest) that set of package is promoted | 23:33 |
| sdake | dmsimard cool thats helpful | 23:34 |
| dmsimard | sdake: maybe if you're more visual: http://versiontracker.dmsimard.com/compare/mitaka-master | 23:34 |
| vhosakot | sdake: yes, the doc clearly says Vagrant is for dev... correct | 23:34 |
| vhosakot | sdake: here you go (with the link this time) http://paste.openstack.org/show/487514/ :) | 23:34 |
| sdake | dmsimard so basically magnum wont be promoted until tripleo, packstack, and puppet-openstack are implemented | 23:34 |
| dmsimard | sdake: nooooooooo | 23:34 |
| dmsimard | it doesn't need to be implemented, it just needs to be part of the set of packages that are tested | 23:35 |
| vhosakot | sdake: let me know if you can see what I've pasted in http://paste.openstack.org/show/487514/ | 23:35 |
| dmsimard | We don't have anything that tests magnum right now | 23:35 |
| dmsimard | As part of that pipeline | 23:35 |
| dmsimard | So that just means that (exaggerating here) if nova works, magnum will go from current to current-passed-ci | 23:35 |
| sdake | so it could be promoted without testing? | 23:35 |
| sdake | got it | 23:35 |
| sdake | thanks | 23:35 |
| *** tobe has quit IRC | 23:35 | |
| *** dims_ has joined #kolla | 23:36 | |
| dmsimard | a lot of things were promoted without being properly tested, we've gotten better at it | 23:36 |
| dmsimard | it's still not perfect but hey, I did a whole talk on that this week :) http://redhat.slides.com/dmoreaus/testing-openstack-is-hard#/ | 23:36 |
| dmsimard | for example, we didn't properly test gnocchi until puppet-openstack had an implementation for it | 23:37 |
| *** dims has quit IRC | 23:38 | |
| dmsimard | so the idea is that we use different installers to try and get as much coverage as we can | 23:38 |
| dmsimard | I even chatted with a guy working on chef cookbooks for openstack today :) | 23:38 |
| *** tzn has quit IRC | 23:39 | |
| *** dave-mccowan has joined #kolla | 23:41 | |
| vhosakot | sdake: did you get a chance to look at http://paste.openstack.org/show/487514/ ? | 23:45 |
| sdake | vhosakot you have some wierd leaked containers | 23:46 |
| sdake | I haec seen this once befoer on centos on my gear | 23:46 |
| sdake | since your using ubuntu it may be a global problem | 23:46 |
| sdake | SamYaple any ideas on above ^^ | 23:46 |
| vhosakot | sdake: leaked ? broken ? I'm able to pull and push form the container | 23:47 |
| vhosakot | from the registry I meant | 23:47 |
| *** openstackgerrit has quit IRC | 23:47 | |
| *** openstackgerrit_ is now known as openstackgerrit | 23:47 | |
| sdake | 7e5562642d95 b1d06b74f1c8 "/bin/sh -c 'chmod 75" 23 hours ago Exited (0) 23 hours ago hopeful_sinoussi | 23:47 |
| sdake | that is a really odd state for a container to be in | 23:47 |
| *** openstackgerrit_ has joined #kolla | 23:47 | |
| sdake | all our containers are NAMED | 23:47 |
| sdake | they dont pick random names | 23:47 |
| sdake | possibly unless your build failed | 23:47 |
| sdake | maybe that is why the containers leak - i know we keep intermediate containers around | 23:48 |
| sdake | or used to at one point | 23:48 |
| vhosakot | ah, yes.. the random names are funny | 23:48 |
| *** openstackgerrit_ is now known as openstackgerrit | 23:48 | |
| SamYaple | sdake: what? | 23:48 |
| sdake | SamYaple see vhosakot's paste of docker ps -a | 23:48 |
| vhosakot | stupefied_golick, loving_kowalevski, cocky_kare haha | 23:48 |
| SamYaple | oh thats most commonly seen on slower backends | 23:48 |
| *** openstackgerrit_ has joined #kolla | 23:48 | |
| vhosakot | SamYaple: http://paste.openstack.org/show/487514/ | 23:48 |
| SamYaple | centos in the gate often had that issue | 23:49 |
| SamYaple | until i switched to btrfs | 23:49 |
| sdake | vhosakot which backend are you using? | 23:49 |
| *** tobe has joined #kolla | 23:49 | |
| SamYaple | it can occur with any, ive seen it most commont with btrfs | 23:49 |
| sdake | I suspect the default - but not sure what that is on ubuntu 14.04 | 23:49 |
| vhosakot | storage backend ? | 23:49 |
| SamYaple | *devicemapper | 23:49 |
| SamYaple | not btrfs | 23:49 |
| sdake | your graph driver | 23:49 |
| sdake | what triggers it? | 23:50 |
| sdake | a build failure? | 23:50 |
| sdake | or is that unknown | 23:50 |
| vhosakot | how can i check it ? | 23:50 |
| vhosakot | /var/lib/docker ? | 23:51 |
| sdake | vhosakot it sat the top of your log | 23:51 |
| sdake | it will say graphdriver and have the type | 23:51 |
| sdake | I run with -s overlay | 23:51 |
| sdake | vhosakot the fact that containers are leaked like that is a bit of a problem | 23:51 |
| sdake | can you file a bug for that please | 23:51 |
| sdake | we can sort out how to fix it later - but we will forget if its not tracked | 23:52 |
| vhosakot | yes.. | 23:52 |
| SamYaple | run `docker info` | 23:52 |
| vhosakot | brb 10 mins | 23:52 |
| SamYaple | vhosakot: ^^ | 23:52 |
| sdake | at pressent I'd use docker rm -f on all the docker containers running at present | 23:52 |
| sdake | you can just use the first 2-3 characters of the id | 23:52 |
| sdake | such as docker rm -f 5c5 | 23:52 |
| sdake | docker rm -f 730 | 23:52 |
| sdake | etc | 23:52 |
| SamYaple | sdake: its not a problem, jsut means vhosakot needs to build with fewer threads | 23:53 |
| SamYaple | or single threaded | 23:53 |
| sdake | would still like a bug so poeple that have that problem can understand the issue | 23:53 |
| sdake | because you say "build with less threads" I hear "buildign with less threads causes it to happen bu twe dont know why" | 23:53 |
| sdake | time to ffeed children bbl | 23:54 |
| sdake | vhosakot but it sounds like SamYaple is suggesting build with less threads ;) | 23:54 |
| *** openstackgerrit_ has quit IRC | 23:55 | |
| *** openstackgerrit_ has joined #kolla | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!