Monday, 2015-08-03

*** dims_ has joined #kolla00:02
*** dimsum__ has quit IRC00:04
*** shadower has quit IRC00:23
*** shadower has joined #kolla00:23
*** dims_ has quit IRC00:50
*** dimsum__ has joined #kolla00:53
*** achanda has joined #kolla01:25
*** dimsum__ has quit IRC01:25
*** dimsum__ has joined #kolla01:28
*** erkules has joined #kolla01:54
*** erkules_ has quit IRC01:56
*** dimsum__ has quit IRC02:02
openstackgerritSam Yaple proposed stackforge/kolla: Adds push to build script  https://review.openstack.org/20826602:04
*** jpeeler has joined #kolla02:49
openstackgerritSam Yaple proposed stackforge/kolla: Add log aggregation for Ansible into 'log_data'  https://review.openstack.org/20827503:08
*** diogogmt has quit IRC03:08
*** ccrouch has left #kolla03:25
*** coolsvap|away is now known as coolsvap03:40
*** unicell1 has joined #kolla04:28
*** unicell has quit IRC04:28
*** daneyon has joined #kolla04:43
*** daneyon has quit IRC04:43
*** daneyon has joined #kolla04:44
openstackgerritSwapnil Kulkarni (coolsvap) proposed stackforge/kolla: Remove service_hosts.sh  https://review.openstack.org/20830705:09
openstackgerritSwapnil Kulkarni (coolsvap) proposed stackforge/kolla: Remove service_hosts.sh  https://review.openstack.org/20830705:11
*** daneyon has quit IRC05:48
*** daneyon has joined #kolla05:49
*** daneyon has quit IRC05:58
*** daneyon has joined #kolla06:02
*** daneyon has quit IRC06:08
*** daneyon has joined #kolla06:09
*** daneyon_ has joined #kolla06:25
*** daneyon has quit IRC06:28
*** daneyon_ has quit IRC06:41
*** achanda has quit IRC07:13
*** achanda has joined #kolla07:17
*** jmccarthy has quit IRC07:29
*** pbourke has quit IRC07:29
*** pbourke has joined #kolla07:33
*** jmccarthy has joined #kolla07:33
*** achanda has quit IRC07:36
*** shadower has quit IRC08:06
*** inc0 has joined #kolla08:21
*** shadower has joined #kolla08:24
inc0hi guys, happy sysadmin day08:29
inc0http://devopsreactions.tumblr.com/post/125543198764/happy-sysadmin-day08:29
SamYapleits always sysadmin day08:33
SamYapleinc0: i am writing a python wrapper for ansible08:33
SamYapleits a PITA08:33
inc0SamYaple, ncurses stuff?08:34
inc0:P08:34
inc0btw...PITA?08:34
SamYaplepain in the ass08:34
SamYapleno inc0 python imports. I do things the right way08:34
*** akwasnie has joined #kolla08:34
inc0you have problems with imports?08:35
inc0what kind of problems?08:35
SamYaplenah its just im basically rewriting the ansible cli stuff08:35
inc0why would you do that?08:35
SamYaplei have to to get our dependacies to be containerized08:36
inc0ahh...shade thing?08:36
SamYaplenot shade in particular08:36
SamYaplejust ansible in general08:36
inc0I mean one we've discussed08:36
SamYaplei need to programatically pull out "changed" and "failed"08:36
inc0shade is an example08:36
SamYaplei _could_ screen scrape for it08:37
SamYaplei guess08:37
SamYaplebut thats horrible08:37
inc0no.08:37
inc0uhh...this all seems hacky:/08:37
inc0and it won't work with other modules, which, unlike shade, has to be ran on host machine right?08:38
inc0have*08:38
inc0and run*08:38
SamYapleitll work for all ansible modules when im done08:38
inc0damn english is hard.08:38
inc0idea is that you'll do a ansible pull from inside container right?08:38
SamYapleit will appear as though you are calling ansible directly, only it will return the changed/failed correctly08:38
*** gfidente has joined #kolla08:39
SamYapleno08:39
*** gfidente has quit IRC08:39
*** gfidente has joined #kolla08:39
SamYaplethats not going to work08:39
inc0then how will that work?08:39
SamYaplethe issue with that is I can't determine whether things changed08:39
SamYapleso instead i hack up the cli and do a quick module that will accept ansible like input in a quick way08:40
SamYaplesince i know exactly wat options will be needed08:40
inc0ok...what problems do we solve now? It's about dependencies on target host?08:40
inc0so shade thing, which requires a lot of stuff we don't want?08:41
inc0or something else?08:41
SamYapleshade is required for ansible modules08:41
SamYaplepython-mysqldb is required for mysql talking08:41
inc0I know, is that a problem you're solving?08:41
SamYapleboth pull in a bunch of crap and gcc08:41
SamYapleyep08:41
inc0ok08:41
inc0and how do you solve it?08:42
inc0what's your idea? since ansible pull won't work as you say (btw, why?)08:42
SamYaple08:39 < SamYaple> the issue with that is I can't determine whether things changed08:42
inc0right, and you want to run ansible stuff inside container and scrap it's output right?08:43
SamYaplewell, i _dont_ want to screen scrape the output.08:43
inc0and btw, how do you run things inside container? you don't ssh to it do you?08:43
SamYapleim using python and importing it so i can know for sure08:43
inc0ok, I know, just want to pinpoint problem itself08:44
SamYaplei just spin up the "ansible" container and call the module i want08:44
SamYapleno ssh08:44
*** Haomeng has quit IRC08:44
*** Haomeng has joined #kolla08:45
inc0as docker-exec call?08:45
SamYapleno there is no "ansible" container running08:46
SamYapleit gets geenrated and destroyed as needed08:46
inc0ah, ok08:46
inc0makes sense08:46
inc0but that means it'll always change right?08:46
SamYapleyea i guess thats right. so well need it running always08:47
inc0one idea would be to add some file "bootstraped"08:47
inc0and check for this one08:47
inc0but that would require to keep contaier08:48
inc0or have it on some volume thing08:48
inc0if that would be host volume, you can use when statement to run it once08:48
inc0but, that would also break on multihost08:48
SamYapleno that wont work inc008:49
SamYaplewe have to run it always08:49
SamYaplei use some modules as healthchecks08:49
SamYaplethe mysql one for example is used to block until mysql is available08:49
inc0ok...why do you run healthcheck there?08:49
SamYapleso it runs always08:49
SamYaplebecause I have to block until mysql is available?08:50
inc0sometimes I have problem following you... you're solving problem of shade having a lot of dependencies (yeah, potentially other packages too, but let's focus on shade)08:50
inc0you run container and block it until mysql is available, then you run endpoint bootstraping and stuff using shade right?08:51
SamYaplewell its not a 'then' its an also08:51
SamYaplei use it to bootstrap users for database, I also use it to do openstack keystone things08:51
inc0ok08:52
inc0let's call it a bootstrap container08:52
inc0so you run bootstrap contaner and wait till mysql is ready08:52
SamYapleok08:52
inc0then you bootstrap mysql users, then you wait for keystone08:52
inc0and bootstrap endpoints08:53
inc0is there anything else this container does?08:53
SamYaplei dont think so, not at the moment at least08:53
inc0so why do you want it to run always? in case of new service being added?08:53
inc0for examle08:53
SamYaplewell i didnt _want_ it to run always, but as you pointed out starting and destroying a container means I awlys have to report a change which is not idempotent08:54
SamYapleso it needs to be running always08:54
inc0it can be08:54
inc0you can use user in database as lock08:54
inc0like "if this user exists, I don't run at all"08:55
SamYaplehow would i know if the user exists?08:55
inc0SELECT user from mysql_users (can't remember exact table name);08:55
SamYapleso i ouwld run that in the always running bootstrap container?08:56
inc0or just try to add it, then it will fail because username is unique08:56
SamYaplewhat problem does that solve?08:56
inc0you'll get idempotency08:56
SamYaplei have idemptonecy already08:56
*** erkules has quit IRC08:56
*** erkules has joined #kolla08:56
SamYapledeleting and recreating the bootstrap container is the problem08:56
SamYapletherefore it must always run08:56
inc0why is that a problem?08:57
SamYaplebecause if i create a container and delete it, i am not being idemptoent08:57
inc0why?08:57
SamYaplebecause those are changed on the system08:57
inc0this particular container has to be idempotent across whole cluster anyway08:57
inc0so you must assume it will run multiple times anyway08:58
SamYaplei dont think you understand08:58
inc0I might not08:58
SamYapleif i have to create and delete this container every time i run ansible then i am not idempotent08:58
inc0brb, unplugging laptop08:58
SamYaplei should run ansible without changing anything if nothing needs changing08:59
*** inc0_ has joined #kolla09:00
*** inc0_ has quit IRC09:02
*** inc0_ has joined #kolla09:03
*** inc0 has quit IRC09:03
*** inc0_ is now known as inc009:03
inc0ok, back09:03
inc0I hate vifi09:03
inc0wifi even09:03
inc0soo...ok, running container isn't idempotent, it will never be because its distributed09:04
SamYaplewhether its distributed or not it wont be idempotent09:04
SamYapleto stop and recreate that is09:04
inc0but things this container does is idempotent09:04
SamYaplecorrect09:05
inc0so you can run it every time09:05
inc0and nothnig will break09:06
SamYapleno one said anything breaks09:06
SamYaplebut starting a container is a change to the system09:06
SamYapleso it needs to stay running09:06
SamYapleotherwise its not idempotent09:06
inc0uhh...its all just for ansible to show not-changed?09:07
SamYapleif all i cared was for ansible ot show "not_changed" i would set changed_when to false09:08
SamYaplei care about starting a container on a system since that is a true change09:08
SamYaplethat has potential to break things09:08
inc0how this will break things?09:08
SamYapleanswer me this, are you saying starting a contaienr should not count as a change?09:09
inc0if it's just bootstrap which tries to do stuff and dies09:10
inc0I'd argue it has to be run each time09:10
SamYaplethat doesnt answer my question09:10
inc0I'd consider it as normal script run09:10
SamYaplethat doesnt answer my question09:10
inc0therefore it's not a change per se09:10
SamYapleit is a change09:10
inc0why did you ask me if you already have answer?:P09:11
SamYapleits not running `ls` its starting a container09:11
SamYapleso we could stop the conversation09:11
SamYapleits a change and therefore must be prevented09:11
inc0from OS point of view there is little difference09:11
SamYaplelies09:11
inc0it's a process09:11
inc0which lives for a moment and dies09:11
inc0even VM is a process09:12
SamYapleits processes with cgroups and changes to the docker daemon09:12
inc0only usually continous09:12
SamYapleand by your logic starting a vm isnt a change if it exists like a script09:12
inc0my point is09:12
inc0we *need* to run bootstraps09:12
inc0each time we change something09:13
inc0because you can add service dynamically09:13
inc0and if containers stays, it won't bootstrap it09:13
SamYaplei dont think you understand the issues at hand, so ill do this and we can talk about the implementation09:13
SamYaplei have idempotency right now09:13
inc0just do a patchset09:14
SamYapleim just moving the dependacies into the container09:14
inc0and we'll discuss there09:14
inc0SamYaple, other thing, do you have Dockerfile templating bp submitted?09:18
SamYaplei dont think so09:19
SamYaplemaybe09:20
SamYapleive been in and out of conciousness since the midcycle09:20
inc0it's that bad?:P09:20
SamYaplecant normally sleep lots of stuff happening09:21
inc0wait till I'll get there, that will break you totally09:21
SamYapleis that confirmed?09:22
*** diga has quit IRC09:34
*** inc0_ has joined #kolla09:48
*** inc0 has quit IRC09:50
openstackgerritSwapnil Kulkarni (coolsvap) proposed stackforge/kolla: Temporary remove gnocchi from kolla-compose & functional tests  https://review.openstack.org/20840309:57
inc0_SamYaple, I'll try different approach - virtualenv in container10:06
inc0_this will fix other problems as well it it'll work10:06
SamYapleinc0_: what problems will a virtualenv in a container solve?10:07
*** Ti-mo has quit IRC10:08
*** walters has quit IRC10:08
*** jpeeler has quit IRC10:08
*** Ti-mo- has joined #kolla10:08
*** jpeeler has joined #kolla10:09
*** walters has joined #kolla10:10
*** daneyon has joined #kolla10:18
*** daneyon has quit IRC10:23
*** dimsum__ has joined #kolla10:34
SamYapleinc0_: aha10:59
SamYapleso i can parse json out10:59
SamYaplei can have ansible wrap the output of ansible and catch the changed/failed11:00
*** coolsvap is now known as coolsvap|away11:06
*** dims_ has joined #kolla11:18
*** dimsum__ has quit IRC11:20
*** rhallisey has joined #kolla11:42
inc0_SamYaple, sorry, I was out11:51
SamYaplenp11:51
akwasniehi11:51
inc0_venv in container will allow you to run shade stuff in normal ansible flow11:51
SamYapledoubtful11:51
inc0_it should be possible11:51
SamYapleakwasnie: hello!11:51
SamYapleinc0_: how will this help at all?11:52
inc0_not sure about mysql stuff, as its C-based, but with enough PYTHONPATH tinkering, it should be doable11:52
akwasnieSamYaple have you done anything about templating Dockerfiles or can I start from scratch?11:52
SamYaplebut i mean, you still have the container11:52
SamYapleakwasnie: i have... give me a second let me see if its around11:52
inc0_yeah, but this one can just run, will not change, and will be much simpler to work with11:53
inc0_it will be volume container essentialy11:53
akwasnieofc sorry11:53
SamYapleinc0_: see i think we are talking the same language here. i have a essneitally "volume" container to do things11:54
SamYapleakwasnie: this is what we agreed on during the midcyle, this is finished teplating for all of our keystone code11:54
SamYaplehttp://paste.openstack.org/show/406297/11:54
*** dims_ has quit IRC11:54
SamYapleto keep things simple, we would like to stick with if blocks11:55
inc0_yeah, but you'll have logic inside container, and I'll have it in normal ansible, that's basically only difference11:55
*** dimsum__ has joined #kolla11:55
SamYapleinc0_: how so? you cant execute normal ansible stuff in a container11:56
akwasnieSamYample is there any bp for this?11:56
SamYapleakwasnie: no11:56
inc0_you'll run container and run sh script inside11:56
inc0_to bootstrap stuff11:56
SamYapleinc0_: wrong11:56
SamYaplehold on11:56
inc0_ansible pull then?11:56
SamYapleill wip this thing11:56
inc0_please do this, will help a lot with discussion11:56
openstackgerritSam Yaple proposed stackforge/kolla: Reduce dependencies on the dest nodes  https://review.openstack.org/20845111:57
SamYaplebtw i determined we _can_ do connection draining on haproxy11:57
SamYapleso we can do service restarts with )% loss11:58
SamYaple0%11:58
*** coolsvap|away is now known as coolsvap11:58
inc0_whats connection draining?11:59
SamYaplei change the wieght on the server in the loadbalancer so no new connections go to it, then i wait until all existing connections close12:00
SamYaplerestart the service then change the weight back12:00
coolsvapSamYaple, akwasnie I also want to work on templating work12:00
akwasniecoolsvap: great12:00
SamYaplecoolsvap: ok. would you like to coordinate the process of templating like harmw did for the config-external/internat stuff?12:01
SamYaplecoolsvap: managing an etherpad and all?12:01
coolsvapSamYaple, sure12:01
SamYaplecoolsvap: ok well you and akwasnie take the lead on this then12:01
coolsvaplet me create a etherpad12:01
SamYapleI am unsure if a blueprint was started12:01
akwasnieI will add bp in a minute12:01
SamYaplewonderful12:01
SamYaplego team!12:02
inc0_I thought as much, we can add this logic to upgrade flow12:02
SamYapleso for the record, we will need to create a second toplevel folder named docker_templates12:02
inc0_but well, this will only turn off APIs12:02
SamYapleso we will have docker and docker_templates12:02
SamYaplewe have t ostage the templates beside the existing code until its ready12:02
inc0_RPC calls will still balance over nodes12:02
coolsvapSamYaple, yes, i have a bit of context from midcycle for this12:03
SamYaplecool12:03
SamYapleinc0_: i send rabbit through the LB so i can do this too12:03
inc0_but not consume through LV12:03
inc0_LB12:03
SamYapleyea it does12:03
SamYaplei mean i ensure no client is connecting directly through the node going down12:04
SamYapleso it will get proxeied through the other nodes if the node thats going down is master12:04
SamYaple_but_ the other nodes know how to safely retry12:04
inc0_yeah, I mean indirect connections12:05
SamYaplelike i said the other nodes know how to properly retry and the client never has to12:05
SamYapleinderect is ok12:05
SamYaplecutting off the one you are talking to is bad12:05
inc0_if they know how to retry:(..12:05
SamYaplewell for openstack12:05
SamYaplethey do actually a good job of it12:06
inc0_cinder doesn't for example;)12:06
SamYaplebut now it doesnt have too!12:06
inc0_but yeah, connection draining is cool12:06
*** daneyon has joined #kolla12:07
*** dimsum__ has quit IRC12:08
coolsvapakwasnie, https://etherpad.openstack.org/p/kolla-dockerfile-template12:12
*** daneyon has quit IRC12:12
coolsvapakwasnie, not sure if you got the earlier msg, here's the etherpad: https://etherpad.openstack.org/p/kolla-dockerfile-template12:15
*** dimsum__ has joined #kolla12:22
akwasniecoolsvap thanks12:24
akwasniehere is the bp https://blueprints.launchpad.net/kolla/+spec/dockerfile-template12:24
SamYaplesynergy, we have achieved it12:24
SamYapleinc0_: did you look at the review?12:25
*** dims_ has joined #kolla12:27
*** dims__ has joined #kolla12:29
*** dimsum__ has quit IRC12:29
*** dimsum__ has joined #kolla12:32
*** dims_ has quit IRC12:32
*** dims__ has quit IRC12:34
*** dwalsh has joined #kolla12:38
inc0_mstachow, could you link 3o poc for rhallisey12:39
inc0_?12:39
inc0_config external stuff12:39
mstachowyou mean link to github?12:39
inc0_SamYaple, yeah, I did, seems simple enough now12:40
inc0_yup12:40
rhalliseymstachow, yea where are you working12:40
mstachowrhallisey https://github.com/intelsdi-x/puppet_kolla.git12:41
mstachowbut this is only PoC12:41
rhalliseycool thx12:41
mstachowit's dirty as hell12:41
mstachow>:)12:41
rhalliseyno problem12:41
*** dims_ has joined #kolla12:45
*** dimsum__ has quit IRC12:46
SamYaplemorning rhallisey12:52
rhalliseySamYaple, morning12:52
*** dims_ has quit IRC12:53
SamYaplegot the kolla dependacies situation solved for ansible12:53
SamYapleI would bet yo ucan do the same thing or puppet12:53
rhalliseyoh nice! What's the solution12:53
rhalliseyI was trying some stuff out on Friday with no success12:53
SamYapleits a wip but it does work. I containerize all of the deps and then have ansible istalled in that container12:54
SamYapleI then call ansible in that container and parse the output of _that_ ansible for changed/failed12:54
*** dimsum__ has joined #kolla12:54
SamYapleso i maintain idempotency12:54
SamYapleno scripts on my part12:54
SamYaplethe output is json too, so no screen scraping12:55
SamYapleno passwords hitting the disk12:55
SamYaplehttps://review.openstack.org/#/c/208451/12:55
*** dimsum__ is now known as dims12:55
rhalliseyI'll have to try it out13:00
SamYapleyea im not sure how puppet works, but im positive we could adopt the same model13:00
SamYapleif needed13:00
SamYaplethis reduces the deps down to only docker and docker-py o nthe dest hosts13:00
rhalliseyessentially what you doing is exaclty what we do with heat13:01
SamYaplei made tripleo! w00t13:02
inc0_we're playing with kok now13:02
inc0_kolla on kolla13:02
rhalliseyya I haven't don KoK yet13:02
SamYaplekolla in dind on tripleo13:02
inc0_kolla-setup heat to deploy tripleo templates+puppets that deploy kolla-based overcloud13:03
rhalliseyinc0_, right13:03
SamYapleI, for one, welcome our new Kolla overlord13:03
inc0_we need ironic container tho...13:03
inc0_someone have done that I think13:03
inc0_I mean was doing it13:04
inc0_jpeeler ...13:04
rhalliseyya I was going to take a stab at it13:09
SamYaplewont ironic require nova and then keystone?13:09
rhalliseyyea, what are you asking though?13:12
SamYaplei dont really understand triple-o i guess13:13
SamYaplei mean i get it on the highlevel :)13:13
SamYaplebut the initial openstack deploy i dont understand13:13
*** jtriley has joined #kolla13:34
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org has been restarted in an attempt to improve performance.13:39
*** sdake has joined #kolla13:41
sdakemorning13:41
inc0_morning sdake13:44
*** ccrouch has joined #kolla13:51
*** daneyon has joined #kolla13:56
coolsvapmorning sdake :)13:56
*** daneyon has quit IRC14:01
*** ccrouch has quit IRC14:04
sdakesamyaple around14:05
*** ccrouch has joined #kolla14:05
*** ccrouch has left #kolla14:05
*** ccrouch has joined #kolla14:11
*** ccrouch has left #kolla14:11
*** ccrouch1 has joined #kolla14:11
openstackgerritSteven Dake proposed stackforge/kolla: Add Ansible support for Heat  https://review.openstack.org/20552814:15
openstackgerritSteven Dake proposed stackforge/kolla: Bootstrap the heat images properly  https://review.openstack.org/20824814:15
openstackgerritSteven Dake proposed stackforge/kolla: Include oslo_reports in installation of heat  https://review.openstack.org/20822614:15
*** sdake has quit IRC14:16
*** sdake has joined #kolla14:24
sdakehttps://review.openstack.org/#/c/208226/14:24
sdakecan i get quick reviews on this ^^ harmw samyaple rhallisey jpeeler14:24
sdakepbourke?14:25
SamYaplesdake: ping14:26
sdakewound me samyaple14:26
*** unicell has joined #kolla14:26
rhalliseysdake, shouldn't that be in the base image ?14:26
rhalliseycinder also needs oslo reports14:26
sdakeit is in the base image of heat14:26
rhalliseyno base centos14:26
sdakei wasn't aware cinder needed that14:27
*** unicell1 has quit IRC14:27
rhalliseylet me find the old bug..14:27
sdakei will edit in a moemnt after conversation with sam is done14:27
rhalliseykk14:27
sdakewhen i connect to my intrenal network i dont have irc access for some annoying reqason14:28
sdakeshoot samyaple whats up14:29
SamYaplesdake: i dknt know14:30
SamYapleyou said my name14:30
sdakeoh wanted to let you know heat does work14:30
sdakebut whaty ousee is unfinished job14:30
SamYaplei left reviews14:30
sdakeok14:30
sdakei left responses14:30
sdakei'll brb14:31
SamYapleits not actually bootstraping, you know that right14:31
sdakeabsolutely know that14:31
SamYapleok cool14:31
sdakei just wantd to push my work14:31
*** absubram has joined #kolla14:31
sdakeinc case, i got hit by a bus or something14:31
rhalliseysdake, I take that back, cinder needs oslo-policy not oslo-reports14:31
SamYaplerhallisey: i think the issue with heat is it doesnt have athe package dep on it, even though it needs it14:32
SamYapleso its a broken package, this doesnt prevent it from being used in cinder14:32
*** echoingumesh has joined #kolla14:33
*** ccrouch1 has quit IRC14:36
sdakeif folks could just approve that change, no other packae needs oslo reports14:36
sdakei have tested a build last night at ike midnight14:36
*** ccrouch1 has joined #kolla14:38
SamYaplestarted on the logging container14:42
SamYapleit appears we will need to use rsyslog after all :/14:42
SamYaplenot everything can log to a file (keepalived and haproxy)14:42
*** jtriley has quit IRC14:43
*** dwalsh has quit IRC14:44
sdakecomon gate14:53
sdakejesh if we hae to wait ike 4 hours toay for that patch to merge i'm going to strangle someone14:53
*** achanda has joined #kolla14:53
SamYaplei want my stackalytics stuff to merge14:53
sdakedims may be able to help14:53
sdakei think they do weekly maintenaceo n the wekends tho so it wont go live until the weekend14:54
SamYapleyea i got it in late on friday :(14:54
SamYaple6 patches14:54
SamYaple_six_14:55
SamYaplei shouldnt be an openstack dev14:55
sdakeheat going on 15 patches ;-)14:55
sdakerhallisey are you going to have time tthis cycle to do ansible for cinder14:55
SamYaplebut i was just adding my name to the stackatylics thing14:55
sdakeafter heat is working i guess i'll do magnum14:56
sdakewhich wil default to n14:56
sdakewe need some cat to take on horizon as well14:57
sdakei guess i'll make blueprints for all the remaining containers14:57
sdakerather services14:57
* sdake needs to have a 1:1 with launchpad14:58
*** barra204 has joined #kolla14:58
sdakejasonsb did you make a blueprint for logging14:58
SamYaplesdake: he did14:58
SamYapleits in discussion14:58
sdakegot a link?14:58
sdakeso I can put it in l314:58
SamYapleblueprints kolla14:59
sdakegroan14:59
rhalliseysdake, ya I'll give it a go14:59
sdakerhallisey nice :)15:00
sdakeso I know I am not big on exceptions15:00
sdakebut openstack has a standard policy of allowing features in past the feature freeze deadline15:00
sdakethese are called FFEs15:00
sdakeI'm not usre if the TC or PTL decides or if its a combination15:01
sdake(feature freeze exceptions)15:01
SamYaplealright im going home, maybe for the night ill be off15:03
*** achanda has quit IRC15:04
sdakeok guys FFEs are decided by ptl15:06
sdakeunless we have the release_managed tag which we dont have15:06
sdakejpeeler you around15:06
sdakejpeeler i talked to most of the core team about the mission change, wanted toget your ack/nack15:07
* jpeeler is still catching up on email15:07
jpeelerwas it posted to the list? i'll get to it soon if so15:07
sdakeplease the deadline is today to veto15:09
*** barra204 has quit IRC15:09
sdakehttps://review.openstack.org/#/c/206789/15:10
sdakejpeeler how his married life ;)15:10
sdakecna you look at the mission that review15:10
sdakethats what the tc trimmed us down to15:10
sdakeit actually expands out scope not reduces it15:10
sdakewhich is ok15:10
sdakesince we have ultimate decision factor on our scope by what we do15:10
jpeelermarried life is great, still adjusting though heh15:11
* jpeeler looks now15:11
sdakeour midcycle rocked15:11
sdaketoo bd you missed it15:11
*** jtriley has joined #kolla15:12
*** dwalsh has joined #kolla15:13
*** thumpba has joined #kolla15:14
*** shakamunyi has joined #kolla15:14
sdakemandre you around15:15
*** shakamunyi is now known as barra20415:16
jpeelersdake: this review seems fine. is there a +1 vote i need to add anywhere?15:24
sdakeno just tell me your happy with the mission statement change15:25
*** barra204 has quit IRC15:26
jpeelerhas it changed since i last looked at it? you're talking about this specifically - https://etherpad.openstack.org/p/kolla-manifesto15:26
sdakethat has changed yes15:30
sdakeits inthe review15:30
sdakein the yaml file15:30
sdake  mission: >104515:31
sdake    To provide production-ready containers and deployment tools for operating104615:31
sdake    OpenStack clouds.104715:31
openstackgerritMerged stackforge/kolla: Include oslo_reports in installation of heat  https://review.openstack.org/20822615:35
SamYaplesdake: techinically the deadline passed15:35
SamYapleyou said 00:00 Aug 3rd UTC15:35
SamYapleits 15:3615:35
jpeelerah ok, yeah it's fine anyway15:37
* jpeeler has forgotten how to use gerrit and forgot to read the actual file15:37
*** SourabhP has joined #kolla15:38
sdakeyou wer eoverwhelmed by the rocking commit message we created I know ;-)15:39
SamYaplejpeeler: does anyone really _know_ how to use gerrit?15:39
*** tfukushima has joined #kolla15:39
jpeelertrue on both points15:40
*** banix has joined #kolla15:40
*** barra204 has joined #kolla15:41
*** daneyon has joined #kolla15:45
*** blahRus has joined #kolla15:47
*** daneyon has quit IRC15:50
*** inc0_ has quit IRC15:50
*** daneyon has joined #kolla15:51
*** daneyon has quit IRC15:51
*** daneyon has joined #kolla15:51
*** openstackgerrit_ has joined #kolla15:53
*** daneyon_ has joined #kolla15:54
*** daneyon has quit IRC15:56
*** ccrouch1 has quit IRC16:03
*** ccrouch has joined #kolla16:04
*** ccrouch has left #kolla16:04
*** ccrouch has joined #kolla16:05
*** unicell has quit IRC16:13
*** sdake has quit IRC16:16
*** rhallisey has quit IRC16:18
*** sdake has joined #kolla16:21
sdakedamn forgot to put a bug id in oslo-reports16:21
sdakenext time remind me plz :)16:21
*** diga has joined #kolla16:23
*** diogogmt has joined #kolla16:32
SamYaplesdake: bug id16:36
*** unicell has joined #kolla16:37
sdaketoo late16:41
sdakethe release has been done16:41
sdakeenjoy ;)16:41
*** sdake has quit IRC16:43
bmacethat is some real dropping the mic stuff there.. release done, sdake out! :)16:43
*** sdake has joined #kolla16:47
sdakeI am loki of Asgard16:47
sdakeand I am burdened with glorious purpose ;)16:47
*** vbel has quit IRC16:50
*** vbel has joined #kolla16:50
*** rhallisey has joined #kolla16:51
*** jasonsb has quit IRC16:54
*** banix has quit IRC16:55
*** coolsvap is now known as coolsvap|away16:58
bmacei see sdake, global control and it all started from being the ptl on an open source project.  nobody will ever see it coming, like the spanish inquisition!17:00
sdakeyikes17:00
*** dwalsh has quit IRC17:00
bmacethat was a monty python gag btw.. if you haven't watched it you should.17:01
*** akwasnie has quit IRC17:03
*** dwalsh has joined #kolla17:03
*** jtriley has quit IRC17:11
*** diogogmt has quit IRC17:15
*** unicell1 has joined #kolla17:16
*** unicell has quit IRC17:16
*** banix has joined #kolla17:18
*** echoingumesh has quit IRC17:20
*** echoingumesh has joined #kolla17:20
*** echoingumesh has quit IRC17:21
*** echoingumesh has joined #kolla17:22
*** achanda_ has joined #kolla17:26
*** SourabhP has quit IRC17:27
openstackgerritVikas Jain proposed stackforge/kolla: Fix incorrect config-external in horizon  https://review.openstack.org/20857017:29
*** echoingumesh has quit IRC17:33
*** diogogmt has joined #kolla17:34
*** jtriley has joined #kolla17:36
sdakesamyaple ping re templating installs17:43
*** echoingumesh has joined #kolla17:45
sdakehttps://blueprints.launchpad.net/kolla/+spec/dockerfile-template17:46
*** ccrouch has quit IRC17:52
*** ccrouch has joined #kolla17:56
sdakemy tube amp and feickart record player are so fantastic18:00
sdakejust need a cabinet to put all this stuff on18:00
sdakeand i'll be "done" with my gear setup :)18:01
*** jasonsb has joined #kolla18:01
sdakemy record player has ateast 100 hours of breakin left too so its just going to get better :)18:02
sdakeand some of my tubes have 50+ hours left of breakin18:02
*** openstackgerrit_ has quit IRC18:07
*** achanda has joined #kolla18:14
*** achanda_ has quit IRC18:16
*** SourabhP has joined #kolla18:17
*** Haomeng has quit IRC18:22
daneyon_sdake we need to add the kollaglue/centos-rdo-haproxy to dockerhub18:27
daneyon_^ repo18:27
sdakeI will push liberty2 images today18:28
daneyon_sdake the kollaglue/centos-rdo-haproxy repo need to be created on docker hub before you can push the L2 image18:28
daneyon_I do not have permissions to create the repo18:29
sdakewhen we push the liberty2 images it will create the repo18:29
daneyon_ok, I'll mod my config to use local images until L2 is pushed18:36
*** tfukushima has quit IRC18:38
*** bmace has quit IRC18:42
*** gfidente is now known as gfidente|afk18:46
*** bmace has joined #kolla18:55
daneyon_sdake fyi: https://bugs.launchpad.net/kolla/+bug/148105419:00
openstackLaunchpad bug 1481054 in kolla "Can't Deploy Kolla Ansible All-in-One" [Undecided,New]19:00
sdakei know what that is19:01
sdakeinternal_network must be a free ip address on your network19:01
rhalliseyI just skip haproxy :P19:01
sdakeor internal_inetwork_address19:01
sdakeor whatever it is19:01
sdakewhat happens is haproxy forwards allt he requests to the appropriate services19:01
sdakefrom the global ip address19:02
sdakethat haproxy manages19:02
sdakeI know, intuitiive huh? :)19:02
sdakedaneyon_ feel free to close that  out ;)19:04
sdakeor keep it open and we can use it for a doc fix against the /etc directory documentation19:05
daneyon_sdake let me try that. rhallisey I set the enable_haproxy to "no" and ansible is still processing the haproxy role19:10
daneyon_sdake maybe another option is to bind the services being proxied by haproxy to the loopback interface19:10
rhalliseydaneyon_, ya that didn't work for me either so I edited inventory/all-in-one and commented out haproxy19:11
rhalliseythat prevented it for being used19:11
sdakeyou need haproxy19:11
sdakejust set it to a empty ip address19:11
rhalliseyk19:11
sdakekolla will have undefined results without the haproxy container operational19:12
rhalliseysdake, where do I set that19:14
sdakerhallisey int he etc directory dude19:14
sdakedid you rtfm :)19:14
rhalliseywhat's rtfm19:14
sdakered the foxtrot manual19:15
sdakeread19:15
rhalliseyok I am setting it.. I didn't know what it was for19:16
rhalliseysorry still my 2nd time attempting to deploy..19:17
sdakethere is a deployment manual in the docs directoroy19:21
sdakegranted its imperfect19:21
sdakealso set registry to "missing"19:21
sdakeor pull or whatever it is19:22
sdakekolla_external_address: "openstack.example.com"19:22
sdakekolla_internal_address: "10.10.10.254"19:22
sdakekolla_external_address: "openstack.example.com"19:22
sdakekolla_internal_address: "10.10.10.254"19:22
sdakeIsorry for double paste19:22
sdakemistyle19:22
sdakeI set these to a free ip address in my network19:23
sdakedocker_pull_policy: "always"19:23
sdakeset this to missing19:23
sdakenetwork_interface: "eth0"19:23
sdakeneutron_external_interface: "eth1"19:23
sdakeset those to your eth0 and eth1 on your box19:23
*** sdake has quit IRC19:28
*** bmace has quit IRC19:33
rhalliseyI set internal and external to my host ip19:39
rhalliseydon't know if that's bad19:39
daneyon_sdake here is my latest problem trying to deploy: https://gist.github.com/danehans/95409886ced6b34f200b19:50
*** echoingumesh has quit IRC19:50
*** echoingumesh has joined #kolla19:51
*** banix has quit IRC19:54
*** mfalatic has joined #kolla20:01
*** sdake has joined #kolla20:01
*** banix has joined #kolla20:02
sdakedaneyon_ did that work for you20:05
*** sdake_ has joined #kolla20:10
*** sdake has quit IRC20:13
daneyon_sdake changing int and ext addresses to 127.0.0.1 worked, but I then ran into another issue: https://gist.github.com/danehans/95409886ced6b34f200b20:32
sdake_ccrouch around20:33
sdake_no, you want to change to an unused interfce20:33
sdake_127.0.0.1 is used20:33
sdake_just pick an ip address out of your network that has nothing assigned to it20:33
sdake_for example, I have 192.168.1.14820:33
sdake_nothign else will use .14820:33
sdake_looking at gist moment20:33
sdake_daneyon_ I have never seen that trace before20:34
sdake_daneyon_ and i've been beating on this stuff daily for 3 weeks20:34
sdake_my first inclination would be to fixt the 127.0.01 thing, then to a cleanup-containers -fv20:35
sdake_then reboot20:35
sdake_which distro arey ou using?20:35
sdake_did you install the shade depepndencies?20:35
sdake_not installing the sahde dep may result in something like that, I'm not sure i've never tried running without it prior20:36
sdake_I use fedora 21, and AIO works like a champ20:36
sdake_I use ubuntu 14..04, AIO works like a champ20:37
sdake_but I had to install the shade deps first20:37
sdake_in some cases including the entire toolchain to build shade20:37
sdake_after you get through with my Q's if you coudl pot your globals.yml for me to have a look :)20:38
sdake_post20:38
sdake_FRUEDEN SLIP!20:38
sdake_note you should probbly reboot if you chnage the internal address20:39
sdake_it mya damage things, I am not sure20:39
sdake_after cleanup-containers -fv20:39
daneyon_sdake_ i am using F21 on the host and centos rdo containers20:40
sdake_cool that should work like a champ20:40
daneyon_sdake_ I installed all the pkgs/dep identified in the ansible deploy doc20:40
sdake_so pip show shade shows shade installed?20:41
daneyon_one min20:41
sdake_it is possible you have a newer version of shade then me or a differnet version of ansible which are broke20:41
sdake_oh, one key thing20:41
sdake_not in the documentation20:42
sdake_I run ansible as sudo20:42
sdake_running as sudo is mandatory unfortunately atm20:42
rhalliseywhat version and ansible and shade do we need20:43
sdake_one moment20:43
sdake_i'm running shade 0.11.020:45
sdake_docker-py 1.2.3 (this is mandatory!)20:46
sdake_ansible-1.9.2-1.fc21.noarch20:46
sdake_docker-io-1.7.0-1.git21cb6eb.fc21.x86_6420:47
sdake_newer versions of docker-py will not work20:47
sdake_like the 1.3 series will not operate - problems with the api versioning20:47
daneyon_sdake: https://etherpad.mozilla.org/y3JRiEFC0e20:49
daneyon_sdake i have been running ansible as root20:50
sdake_running ansible from pip20:50
sdake_?20:50
sdake_I am using ansible from packaging20:50
daneyon_sdake i tried ansible last week from yum pkgs and had problems. SamYaple suggested i try the pip version20:50
sdake_i think that was bad advice tbh :)20:51
sdake_try pip remove ansible20:51
sdake_or maybe its uninstall20:51
daneyon_sdake are u running docker 1.7?20:51
sdake_ya docker 1.720:52
daneyon_ok20:52
sdake_I have a different build the nyou20:52
sdake_I found that build to be flakey20:52
sdake_I cant tell you where I got my fc21 version, it apepars missing on koji20:54
sdake_someone probably deleted it20:54
sdake_I think docker 1.6.2 should do the job20:55
sdake_(stock yum upgrade version)20:55
daneyon_but docker seems to be working fine outside of ansible deploy20:55
sdake_ya its probably not docker20:55
sdake_my guess is it s your pip installed docker or your 127.0.0.1 address20:56
sdake_I dont' even know what that 127.0.0.1 for kolla_internal_address would do, but nothing good for certain20:56
*** thumpba has quit IRC20:56
sdake_the kolla_internal_address is a HAProxy/keepalived VIF20:56
daneyon_aand binding to 127.0.0.1 seems to work and should. haproxy binds the load-balnced services to the real ip/port and then hits the services on the backend using 127.0.0.120:56
daneyon_right20:57
daneyon_bith int and ext are virtual IP's used by haproxy20:57
sdake_I would highly recommend changing that from 127.0.0.1 before continuing20:57
sdake_what can it hurt?20:57
sdake_do you not have a free ip to use?20:57
sdake_the external_interface_address i htnk is just a DNS mapping to the internal address20:58
sdake_but samyaple was not clear on that point20:58
daneyon_sdake_ added by ansible configs to the etherpaad20:59
*** SourabhP has quit IRC20:59
*** SourabhP has joined #kolla21:00
daneyon_sdake_ i only have the 1 ip. i can try using an ip on the subnet that doesn't respond, but it may cause problems down the line.. Christine Hicks said all other IPs have been assigned.21:00
sdake_do you hae any other interfaces inthe mchine?21:01
sdake_if so give one a fake addess like 10.100.1.121:01
sdake_and then set your internal_interface to 10.100.1.221:01
sdake_your config looks good21:02
daneyon_sdake_ i can create a fake interface and give it an ip... i'll give that a try21:03
sdake_what I would try just to get down the road a bit is to sue an ip that doens't repsond to ping21:03
sdake_just temporary like21:03
sdake_then we can try workaround21:03
sdake_we can try the other workarounds today after we use this temporary non-responsive ip address21:03
sdake_what could possibly go wrong ;-)21:03
sdake_there is one other thing I did before installing the dependencies that seemed to help alot21:05
sdake_I ran pip install -U wheel pip21:06
sdake_then reinstalled all my dependencies21:06
*** rhallisey has quit IRC21:07
sdake_so step 1 is to use an ip address that is non responsive on your main interface21:09
sdake_clenaup-containers -fv and reboot21:09
sdake_step 2 is to run pip uninstall ansible21:09
sdake_step 3 is to yum install ansible21:09
sdake_it would be great if we could get to the root cause of the problem you ahve21:10
*** bmace has joined #kolla21:13
*** daneyon_ has quit IRC21:14
*** daneyon has joined #kolla21:15
sdake_rhallisey are you likewise jammed up?21:18
*** jtriley has quit IRC21:19
sdake_jpeeler when you get unstuck from email21:21
sdake_jpeeler I'd like you tog ive the ansible deployment a spin21:21
sdake_you can use a vm if you like21:21
daneyonsdake_ take a look at https://etherpad.mozilla.org/y3JRiEFC0e21:26
daneyoni can't seem to get mysql to bind to my real ip and have haproxy use a fake ip21:26
sdake_i set external and internal address to the same thing21:27
sdake_you have them set to 85 and 8621:28
sdake_I have them both set to .14821:28
sdake_kolla_internal_address should be .85 if that is your fake address21:28
sdake_is your machine's ip .86?21:28
sdake_daneyon ^21:29
daneyonI just set my kolla_external_address and kolla_internal_address to the fake ip and set database_address: "172.29.74.86" (real ip)21:30
sdake_leave database address at whatever it defaults to21:30
sdake_and make your internal and external address match21:31
sdake_as a non-used ip in your network21:31
sdake_or one you ahe borrwed temporarily21:31
sdake_if you do what you are doing, none of the services will work21:32
sdake_because there are other addresses besides "datbasae_address" used tobind the services to the correct ip addresses21:32
daneyonsdake_ no luck: https://etherpad.mozilla.org/y3JRiEFC0e21:37
sdake_so 85 is an unused interface correct?21:37
*** banix has quit IRC21:37
daneyonyes21:37
sdake_her eis my database address:21:39
sdake_database_address: "{{ kolla_internal_address }}"21:39
sdake_your etherpad is a different database address21:39
sdake_you ahve database_address: "{{ network_interface }}"21:40
sdake_i htink you ened to clenaup-containers and reboot between failed attempts too ;-)21:40
sdake_kolla atm in ansible does not restart failed containers properly imo21:41
daneyonsdake let me change the db addr to "{{ kolla_internal_address }}"21:41
sdake_if a container fails to start, and then you change the config, it doesn't really restart it even though th econfig has changed21:42
sdake_that has been my experience atleast21:42
sdake_or rather starts then fails21:43
sdake_and then kolla gets jammed up21:43
daneyonsdake_ changed db addr to match your but still same problem.21:44
sdake_did you cleanup-containers -fv21:45
sdake_and reboot21:45
daneyoni did clean -f21:45
sdake_just tobe clear, 85 is the VIF interface, and .86 i your local box?21:45
daneyonlet me try -fv21:45
sdake_-fv just deletes volumes21:45
daneyonyes .85 is a fake ip .86 is how i manage my box21:46
sdake_network_interface: "em1"21:46
sdake_neutron_external_interface: "enp1s0"21:46
sdake_you hae those set?21:47
sdake_one thing I noticed is you have the interfaces backwards from boot order21:47
sdake_this may be a problem for the implementation21:47
*** dwalsh has quit IRC21:47
sdake_network_interface: "enp10s0"21:48
sdake_neutron_external_interface: "enp9s0"21:48
sdake_can you cd to /opt/kolla21:48
sdake_and find a configuration file relating to an openstack service21:48
sdake_and see what it sets bind to?21:48
sdake_bind_host = 192.168.1.10021:48
*** jasonsb has quit IRC21:51
*** bmace has quit IRC21:55
*** bmace has joined #kolla21:56
*** thumpba has joined #kolla22:19
*** thumpba has quit IRC22:29
*** thumpba has joined #kolla22:41
*** thumpba has quit IRC22:41
*** jasonsb has joined #kolla22:45
*** dims_ has joined #kolla22:51
*** dims has quit IRC22:52
*** dims has joined #kolla22:53
*** dims_ has quit IRC22:56
*** absubram has quit IRC23:05
*** dims has quit IRC23:29
*** echoingumesh has quit IRC23:40
*** echoingumesh has joined #kolla23:41
*** echoingumesh has quit IRC23:45
*** blahRus has quit IRC23:45
*** banix has joined #kolla23:47
*** banix has quit IRC23:50
*** vinkman has joined #kolla23:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!