Wednesday, 2015-07-15

*** barra204 has quit IRC00:03
*** thetrav has joined #kolla00:10
thetravAnyone on the Kolla team in Melbourne?00:13
sdakethetrav none that I am aware of are in AU00:13
thetravbummer00:13
sdakewhy may I ask? :)00:14
thetravI'm in Melbourne, I want to meet them, try and set up some in person collaboration00:15
sdakeroger00:15
sdakewell we are having a midcycle, you can attend remotely if you like 28th and 29th00:15
sdakeit will be a good chance to meet the team00:15
thetravI get two wisdom teeth removed on the 26th, I won't be much good for anything that week00:16
sdakebummer that sucks00:16
sdakei remember that experience vividly00:16
sdakenot pleasant00:16
thetravyeah, I've been putting it off for a while.00:16
sdakeatleast you will get pain killers ;-)00:16
sdakei had mine done at 1600:16
thetravsupposedly easier to bounce back the younger you are00:17
sdakedefinately true00:17
sdakeat 6 your jaw isn't hardened either00:17
sdake16 that is00:17
thetravtackling it mid 30's was probably not the best plan00:17
sdakeso its easier to extract them00:17
sdakewell it probably has to be done now whethre you like or not I suspect :(00:17
thetravcorrect00:17
thetravI'll be fine00:18
sdakemy dad was 68 when he had his out00:18
thetravit'll be another item ticked off the bucket list00:18
sdakehe dodged it all those years but finally the roots got infected in almsot all of them at same time00:18
thetravalong with paragliding and canyon swing :P00:18
sdakemy friend busted his leg up pretty bad paragliding00:19
sdakenever heard of canyon swinging00:19
sdakeya just read about it00:20
sdakelooks friegenting00:20
sdakebut safer then paragliding :)00:20
*** shadower has quit IRC00:23
*** shadower has joined #kolla00:23
sdake5 star rating, 534 reviews00:23
sdakemust be memorable ;)00:23
sdakewith my luck the rope would break00:26
*** diogogmt has joined #kolla00:27
*** sdake_ has joined #kolla00:28
*** vinkman has joined #kolla00:28
*** achanda has quit IRC00:29
*** sdake has quit IRC00:31
*** vinkman has left #kolla00:41
*** vinkman has joined #kolla00:41
thetravheh, yeah was terrifying to make the jump00:43
thetravI'm not great with heights either, get a lot of vertigo00:43
*** bitblt has quit IRC00:44
thetravfantastic rush though00:45
thetravand once you've left the platform it's like flying, especially the pendulum00:46
*** jtriley has joined #kolla00:49
openstackgerritMartin André proposed stackforge/kolla: Fix symlinks to config-{internal,external}.sh  https://review.openstack.org/20144500:49
*** jtriley has quit IRC00:54
*** dims__ has quit IRC00:54
*** jasonsb has quit IRC00:58
*** jasonsb has joined #kolla00:59
*** thetrav has quit IRC01:00
*** jruano has joined #kolla01:02
*** jasonsb has quit IRC01:04
sdake_the build from source infrastructure is pretty sweet01:11
sdake_i've almost finished heat in only a few hours01:11
*** erkules_ has joined #kolla01:16
*** erkules has quit IRC01:18
*** bradjones has quit IRC01:26
*** dims_ has joined #kolla01:28
*** bradjones has joined #kolla01:30
*** bradjones has quit IRC01:30
*** bradjones has joined #kolla01:30
sdake_from source is much smaller as well01:33
*** jruano has quit IRC01:35
*** dolpher has joined #kolla01:37
*** unicell has quit IRC01:39
vinkmanis it documented somewhere on how to do source builds?01:43
openstackgerritSteven Dake proposed stackforge/kolla: Add build from source for Heat  https://review.openstack.org/20188601:45
openstackgerritSteven Dake proposed stackforge/kolla: Add build from source for Heat  https://review.openstack.org/20188601:46
*** thetrav has joined #kolla01:47
*** jtriley has joined #kolla01:50
*** jtriley has quit IRC01:55
*** thetrav has quit IRC02:05
*** jasonsb has joined #kolla02:06
*** coolsvap|away has quit IRC02:10
*** unicell has joined #kolla02:10
*** coolsvap|away has joined #kolla02:12
*** dims_ has quit IRC02:17
*** boh_ricky has joined #kolla02:23
boh_rickyhi everyone02:24
*** unicell1 has joined #kolla02:29
*** unicell has quit IRC02:29
sdake_hey boh_ricky how you doing02:33
sdake_most people are not here at this time02:34
*** sdake_ is now known as sdake02:34
sdakewe have alot of east coast not much west coast and alot of apac/emea02:34
boh_rickyyeah02:35
*** unicell1 has quit IRC02:35
boh_rickyi  am in CHina02:36
*** unicell has joined #kolla02:36
boh_rickyUTC+802:36
sdakecool ya its definately good time for apac now02:36
sdake8pm on the west coat atm02:37
sdakeso you said you wantedt ow ork on memcached iirc02:37
boh_rickysdake ,i am interesting in the bp memcached container.02:38
sdakecool02:38
sdakei'll assign to you what is your launchpad id02:38
boh_rickymay be i can try,if possible02:38
boh_rickyboh.ricky02:38
sdakecool your all set02:39
sdakefeel free to ask questions :)02:39
sdakerecommend reading ansible-multi.rst02:39
sdakein the specs directory02:39
sdakeit explains how our three config strategies work02:39
boh_rickyok isee it02:40
sdakewe have config internal, which you will want to do first I suspect02:40
sdakethen we have a config external with two subtypes02:40
sdakethe config external just takes an external configuration file that ansible sets up02:40
sdakeand feeds it in02:40
sdakei'd focus on just getting config intrnal to work to start02:41
sdakewe do development in the centos directory02:41
sdakedocker/centos/rdo02:41
sdakewe lead with that02:42
sdakeand then we follow up with everything else02:42
*** sdake_ has joined #kolla02:43
*** coolsvap has quit IRC02:45
*** sdake has quit IRC02:46
*** sdake__ has joined #kolla02:46
boh_rickyok02:49
*** sdake_ has quit IRC02:50
boh_rickythanks for what u said02:51
SamYaplehello people02:55
SamYaplepbourke: congrats02:55
SamYaplesdake__: around?02:55
sdake__shoot samyaple02:56
SamYaplere: docker push same layer many times02:56
SamYaplei played with layer squashing all night. i have a mostly done python script for it. i believe I can work around the issue02:57
openstackgerritSteven Dake proposed stackforge/kolla: Add build from source for Magnum  https://review.openstack.org/20189502:57
SamYaplealso, it looks like we can take the currently 770MB centos-base image and smash it into 400MB, but that isn't confirmed yet02:58
sdake__these from source builds are about 100-200mb smaller02:58
SamYapleyes02:58
*** sdake__ is now known as sdake02:58
sdakeI don't get why02:59
SamYapleno extra dependacies are installed02:59
SamYapleand im fairly certain they are missing dependacies that wont show up until they are run later02:59
sdakeopenstack is pretty good about requirements.txt02:59
SamYaplethats not the only requirements :)03:00
sdakewhat is layer squashing03:00
SamYaplethe centos-base consists of about 15 layers. I can squash that to one large layer03:01
SamYaplewe talked about this a month or so back03:01
sdakeby tar untar ?03:04
*** achanda has joined #kolla03:10
SamYaplekinda?03:10
SamYapleits all in python, nothing hits the disk03:10
SamYaplewell except the new layer03:10
*** rbergero1 has quit IRC03:19
*** dims_ has joined #kolla03:19
sdakesamyaple cool so after that goes in, nova and neutron ansible stuff will be hitting repo ?03:21
SamYaplenova has been in the repo03:21
sdakeoh nice03:21
sdakeso we are just missing neutron?03:21
SamYapleneutron is mostly done, but ive been having issues with it03:21
SamYapleim also having to do hackish things wihle waiting on the haproxy container03:22
sdakewaiting for it to be completed you mean?03:22
SamYapleinc0 is working on it03:22
sdakeok03:22
SamYaplebut he is tryin gto bundle the container and ansible playbooks which wont fly03:22
SamYaplewe need the container now, then well test implementations03:23
sdakei dont see nova in the repo under the roles directory03:23
*** dims_ has quit IRC03:23
SamYapleah nvm its just in review from a few days back03:24
SamYaplethought it merged03:24
SamYaplehttps://review.openstack.org/#/c/200881/03:24
SamYapletake care of that will ya03:24
openstackgerritMerged stackforge/kolla: Fixes login failure to Horizon dashboard  https://review.openstack.org/20108103:24
sdakereviewing now03:24
sdakewhy remoe the glance registry and api ports?03:25
sdakeremove?03:25
sdakeis that an error?03:25
SamYapleits moved to a different file03:26
SamYapleit was an error to be in that file03:26
openstackgerritMerged stackforge/kolla: Fix symlinks to config-{internal,external}.sh  https://review.openstack.org/20144503:26
sdakewhy do you create empty config files03:32
sdakewhy create them at all I guess is my question03:32
SamYaplethey have to exist03:32
SamYaplethe config needs a rework and those files need comments03:32
sdakelooks pretty solid03:33
sdakedont see any errors03:33
sdakei approved it03:33
SamYaplecool03:34
sdakenow for neutron, then i'll take a stab at heat03:34
sdakeand we can break the remaining ones up among the team03:34
SamYaplewe will need a discussion of the configs once everyone sees how i have planned for them to work. likely people will reject it at first but it is the most flexible option. we need to make it less rough around the edges though03:34
sdakeya well lets follow the model you ahve laid out03:35
SamYaplebut luckily it is all "advanced" usage so it doesnt affect anything else03:35
sdakeonce we experience it and get it all rolling03:35
sdakewe will be  in a better place to offer change suggestions03:35
SamYaplethat was my thinking as well03:35
openstackgerritMerged stackforge/kolla: Ansible Nova w/ libvirt support  https://review.openstack.org/20088103:36
sdakespeaking of which, how do I launch this single node03:36
SamYapleplus it will be easy to make large changes quickly since the playbooks have so few lines of code03:36
SamYaplesdake: thats super easy03:36
sdakemake me feel dumb why dont ya :)03:37
SamYaplehttps://review.openstack.org/#/c/199077/6/docs/ansible-deployment.md03:37
SamYaplei dont me intuitive, i mean easy :)03:37
SamYapleinstall those deps03:37
SamYaplethen read the paragraph afterward and run that ansible-playbook command03:37
SamYapleyoull likely need to read the rest of the doc as well, but dont worry about configuring the inventory file at all03:38
SamYapleall-in-one inventory file never needs to be adjusted03:38
sdaketonight i'd like to get multinode going too03:40
sdakeif I can find ethernet cables to do the job03:40
SamYapleplease document all of your "first time" issues, even if it is user error03:41
SamYapleim kinda in a development bubble of "i know how things work and my environment is setup"03:41
SamYaplehard to write docs for that03:41
sdakeyes03:42
sdakeetc directory03:42
sdakewhat do I do with that03:42
SamYapleput the files in etc? same procedure as every other openstack project03:43
SamYapleetc/kolla -> /etc/kolla03:43
sdakevery few people actually do that with openstack ;)03:44
SamYapleall the packages do03:44
SamYaplewell, at least teh ubuntu packages do03:45
sdakeper mdenided on /usr/share/kolla03:46
sdakewhat is supposed to go in /usr/share/kolla?03:46
SamYaplenothing. i have a todo in the defaults.yml telling me to find and kill references to that03:46
SamYapleright now i think templates get saved there03:46
SamYapleso nothing is "supposed to", but currently things are03:47
sdakeis /usr/share/kolla/templates supposed to be a directory?03:47
SamYapleyea03:47
SamYapleit should be getting created for oyu03:47
sdakeshould I run as sudo?03:48
SamYaplewhen running locally, yea youll need to until all the permissions are worked out03:48
SamYaple(i think that directory is the last permission issue)03:48
sdakefatal: [localhost] => {'msg': 'One or more items failed.', 'failed': True, 'changed': False, 'results': [{'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute u'ansible_eth0'", 'failed': True}]}03:49
sdakemy eth0 is actually em103:49
SamYapleyou did not update the /etc/kolla/global.yml did you03:49
sdakeno I didn't catch that part03:49
SamYapleok can you make note of that so we can update teh docs03:50
sdakeyes of course03:51
sdakestarting containers takes a long time03:52
sdakeis the color yellow ok?03:52
SamYapleyellow means changed03:53
SamYaplegreen means unchanged03:53
SamYaplered is bad03:53
sdakeshould the booting process tae a long time03:53
sdakelike on order of 2-3 minutes or more?03:53
SamYaplethe containers starting probably dont take a while, its probably recreating the container after a bootstrap (so it is waiting for the previous one to stop)03:53
SamYapleno. mine completes in under 2 minutes all the way through nova03:54
sdakeits on keystone atm03:54
sdakecreaitng database03:54
sdakei bet it is pulling the containers03:55
sdakecould explain the lag03:55
SamYapledid you read the docs?03:55
sdakeno03:55
sdakewhy read the docs? ;-)03:55
SamYaplefor locally built images you need to change 'docker_pull_policy: "missing"' in globals.yml03:55
*** achanda has quit IRC03:56
*** diogogmt has quit IRC03:56
*** achanda has joined #kolla03:57
sdakeseems to get jammed up on keystone db creation03:58
SamYapledid you change the line and then remove the containers. and then rebuild the containers?03:59
sdakei did cleanup-containers -f03:59
SamYaplebecause you have likely pulled the containers and overridden your local ones03:59
SamYaplethats not gonna do anything i dont thikn03:59
sdakethat permanently removes any container contents in the system03:59
sdakebut not images03:59
SamYapleoh that just removes all containers04:00
SamYaplewithout the -v option04:00
SamYapleyikes04:00
sdakefailed: [localhost] => {"failed": true}04:00
SamYaplelots of orphaned volumes04:00
sdakemsg: unable to connect, check login credentials (login_user, and login_password, which can be defined in ~/.my.cnf), check that mysql socket exists and mysql server is running04:00
sdakeFATAL: all hosts have already failed -- aborting04:00
sdakeya its a dev tool04:00
sdakei get that my disk will fill up ;)04:00
SamYapledid you read setup the vip in the globals.yml?04:01
sdakeno04:01
sdakewhere is that, in the docs?04:01
sdakedamn I guess I better read the docs04:01
SamYaplehttps://github.com/stackforge/kolla/blob/master/docs/ansible-deployment.md04:02
SamYapleenjoy04:02
sdakei don't recall reviewing these docs04:04
sdakenice job!04:04
SamYaplepbourke has been cleaning them up. he ran into all of the issues you just talked about and then documented them04:04
sdakeGATHERING FACTS ***************************************************************04:07
sdakeok: [localhost]04:07
sdakeTASK: [glance | Creating the Glance service and endpoint] *********************04:07
sdakefatal: [localhost] => One or more undefined variables: 'glance_api_port' is undefined04:07
sdakeFATAL: all hosts have already failed -- aborting04:07
sdakePLAY RECAP ********************************************************************04:07
sdake           to retry, use: --limit @/root/site.retry04:07
sdakelocalhost                  : ok=39   changed=16   unreachable=1    failed=004:07
sdakegetting further, and yes i've rtfme now :)04:07
sdakethis is master04:07
SamYapleyou running with the just merged nova master?04:07
sdakeyes04:07
SamYapledamn let me check04:08
sdakei just git pulled04:08
sdakeyou movd that to the etc directory04:09
sdakeI think that is the cause04:09
sdakewe have to be careful with those sorts of moves in the future04:09
sdakelike not in the next month but long term ;-)04:10
SamYaple/etc/kolla/defaults.yml should have the glance_api_port04:10
SamYapleyea well need an update script04:10
sdakemine does not because I copied from pre-nova merge04:10
SamYapleitll have to compare/merge the yml files to update them i think04:10
SamYapleor something along those lines04:10
sdakeglance is jammed up on waiting for bootstrap container to exit04:15
sdakedoes it have a timer?04:15
sdakei just fired off sudo again without cleaning up the containers04:15
SamYapleyea dont do that04:15
sdakeideally we need to tolerate failures of our own making :)04:15
SamYapleeh well see what we tolerate. alot of these failures will only happen in dev environments04:16
SamYaplewaht we *need to tolerate04:16
sdakedocker blocking is a perfectly reasonable thing for a poorly behaving system to do :)04:16
SamYaplei doubt thats what happening though04:17
sdakethat error that just happeend after I git updated, that oculd happen again - that is not a dev environment that coudl happen in production04:17
SamYaplei dont think you know whats actually happening04:17
sdakeanywy cleaned up and pulled trigger on it again04:18
sdakeyour right I have no idea04:18
sdakeall I know is it bloced and didn't exit or finish :)04:18
*** achanda has quit IRC04:18
SamYaplethere are a few bits we need to make better, but im not even close to worrying about them since most of the crashes are caused by tihng only devs will have an issue with04:19
sdakeTASK: [glance | Waiting for bootstrap container to exit] **********************04:19
sdakeafter a clenaup04:19
sdakestuck there04:19
SamYaplethats normal, whats breaking in the container?04:20
SamYapleyou did a full container rebuild, yes?04:20
sdakeyesterday or so yes04:20
*** coolsvap|away is now known as coolsvap04:20
sdake[sdake@bigiron tools]$ sudo docker logs a104:20
sdake[sdake@bigiron tools]$04:20
SamYapleyou need to rebuild al lyour containers. you just did a git pull and i know nova touched at least nova containers04:21
sdakenot sure why that would make glance jam up at bootstrapping?04:21
SamYaplei dont know what glance container you previously built or what branch you are on04:22
sdakei am on master04:22
SamYaplethe containers you built arent from master04:22
sdakemaster - t1 days04:22
sdakemy containers are 6 days old04:22
sdakekollaglue/centos-rdo-glance-api                        latest              ba7ff05c0f31        4 days ago          743.7 MB04:23
sdakerather 4 days04:23
SamYapleok? rebuild the containers04:23
sdakedid you change something in glance in the last 4 days that would make the container work?04:24
sdakerelated to bootstrapping?04:24
SamYaplei have no idea what was merged in the bast 4 days. and i dont have time to look04:24
SamYaplejust rebuild the containers from master and then we can see if the issue still exists04:24
sdakeok t - 1 hour to new containers bbiaf04:24
SamYapleuse the new build script04:24
SamYaplei mean if you want it done quickly that is04:24
sdakeya new containers did it04:41
sdakethat is not ideal :(04:41
SamYaplewhat is not ideal?04:43
SamYapleto me that sounds like working as designed04:44
sdakelocks up on an upgrade?04:44
sdakehow is it suppose to be designed to do that :)04:44
SamYapleyou werent doing an upgrade04:45
SamYaplewe have no upgrade anything yet04:45
SamYapleupgrades are a whole different best04:45
SamYaplebeast*04:45
SamYapleright now the only "upgrade" is it will use a new container if it is available after you have already setup your environment04:46
SamYaplein your case you hadnt built the new container04:46
SamYapleor setup the environment04:46
*** boh_ricky has quit IRC04:46
SamYaplei dont understand what you would be expected04:46
sdakelocalhost                  : ok=95   changed=61   unreachable=0    failed=004:47
sdakedoes that mean success?04:47
SamYapleno failures == success04:47
SamYapleno run the command again ;)04:48
sdakewhich openrc creds is it going to use?04:48
SamYaplewhat do you mean?04:48
*** boh_ricky has joined #kolla04:48
sdakesource .openrc04:48
sdaketo interact with openstack?04:48
SamYaplewe arent generating that yet04:48
sdakewhich creds does it use04:48
sdakehow do you do testing?04:49
sdakeso I am running a second time, its redeploying the existing system?04:49
SamYaplei have an openrc file04:49
sdakemind sharing04:49
SamYaplesure, sec04:49
sdakei want to see stuff actually work vs run playbook command s:)04:50
SamYapleyou ask quesitons about the ansible output and i explain them, then you complain about not "seeing how things actually owrk"?04:50
sdakei wasn't complaining?04:51
sdakewhat I mean is I want to see openstack work04:51
sdakeI want to interact with openstack04:51
SamYapleit kinda feels that way. but i know its you just hammering at it04:51
sdakeglance would be good enough ;)04:51
sdakewell if glance works, I'll think what has been done so far is fantastic ;)04:51
*** jtriley has joined #kolla04:51
sdakeso not complaining04:52
sdakelooks pretty fast04:52
sdakelooks to work pretty well04:52
SamYaplehttp://paste.openstack.org/show/376415/04:52
SamYaplethats my openrc04:52
sdakelooks fairly bug free minus update process to /etc files04:52
sdakethat looks like a can of worms04:52
SamYapleglance image-create --name "cirros-0.3.4-x86_64" --file ~/cirros-0.3.4-x86_64-disk.img --disk-format qcow2 --container-format bare --progress04:52
*** boh_ricky has quit IRC04:53
SamYapleafter sourcing your creds ^^ that will upload to glance04:53
SamYapleassuming oyu have that image in that place ;)04:53
SamYapleand while you cant boot an instance, most of the nova things still work (services check in, you can do flavors and stuff)04:53
sdake[sdake@bigiron kolla]$ glance image-list04:54
sdakeUnable to determine the Keystone version to authenticate with using the given auth_url. Identity service may not support API version discovery. Please provide a versioned auth_url instead. error=Unable to establish connection to http://192.168.1.1.100:3535704:54
sdakeERROR: Unable to load glance-api-keystone from configuration file /usr/share/glance/glance-api-dist-paste.ini.04:55
sdakeGot: ImportError('No module named cachetools',)04:55
sdakefrom glance-api04:55
* sdake grumbles at rdo04:56
SamYaplewell thats new ;)04:56
SamYapleim so happy04:56
* SamYaple is holding on until all source installs04:56
sdakeHypervisorUnavailable: Connection to the hypervisor is broken on host: bigiron.broked.org04:56
*** jtriley has quit IRC04:56
sdakefrom nova-compute04:56
sdakenote I had a a 6-10 second sleep in there to prevent hat prolem04:57
SamYapleis that just a log message, or is the contaienr broken?04:57
sdakewhich one, nova or glance?04:57
SamYaplenova04:57
sdakethey are both in a restart state04:57
SamYapleyea seems like upstream issues. nova-compute shouldnt be crashing just because it cant talk to libvirt04:58
SamYaplebut ill check it out tonight04:58
sdakeoh it does04:58
sdakethere is a sleep 6 or something in the code to prevent it currently04:58
SamYaplenot on my system. it retries04:58
sdakeya its definatley retrying then does this:04:59
sdakeINFO - This container has already been configured; Refusing to copy new configs04:59
sdakethen loops04:59
SamYapleno retired in the service itself, not the docker container04:59
sdakei looked at he code 3-4 months ago it had no retry in it05:00
sdakenot sure about master05:00
sdakewhich is why I added the hack rather then fixing properly05:00
sdakebecause compose would try to restart libvirt as well05:00
sdakebut that shouldn't be a problem here05:01
SamYapleyea my point was the compute service shouldnt be bombing out05:02
SamYaplei can run it without libvirt without it bombing out (but it will be "broken"05:02
sdakeya I agree compute *should* just retry to connect to libvirt, but as of 3-4 months ago it exited instead05:03
SamYaplewill have to test05:03
sdakedan berrange doesn't think tcp is a support model that makes sense for openstac + libvirt05:04
SamYaplehe must not like live-migration05:04
sdakei dont know05:06
sdakeso i fixed the cachetools problem05:06
sdakein glance-api05:06
sdake(by pip installing it in the container)05:06
sdakenow I get:05:06
sdake[sdake@bigiron ansible]$ glance image-list05:06
sdakeUnable to determine the Keystone version to authenticate with using the given auth_url. Identity service may not support API version discovery. Please provide a versioned auth_url instead. error=Unable to establish connection to http://192.168.1.1.100:3535705:06
*** rbergeron has joined #kolla05:06
SamYaplethe reference was to the fact that live-migration leverages libvirt, which requires tcp to communicate05:06
SamYapleyea its all bungled sdake05:07
SamYaplethat wont work05:07
SamYapleit probably never bootstrapped properly05:07
sdaketypo05:07
*** achanda has joined #kolla05:07
sdake[sdake@bigiron kolla]$ glance image-list05:07
sdake+----+------+05:07
sdake| ID | Name |05:07
sdake+----+------+05:07
sdake+----+------+05:07
sdakevictory :)05:07
sdakesweet05:07
sdakewell heat should be easy enough - I'll knock that out tomorrow05:07
SamYaplewait i thought it was broken in all containers05:07
sdakei had a typo in my openrc05:08
sdakethe bootstrapping thing was fixed by rebuildilng like oh 1-2 hours ago05:08
SamYapleah i see what you mean05:08
sdakecool glance image create works05:10
sdakewell our config model is fantastic in comparison to every other deployment tool05:11
sdakeworks well out of the box05:11
sdakecan be customized in any fashion05:11
sdake 2-3 minute deployment05:12
SamYaplerealistically it is more like 15m05:12
SamYapleonce everything is up05:13
SamYapleand thats assuming you have have all the contaienrs downloaded05:13
sdakefor how many nodes?05:14
sdake3 nodes = 15 minutes with local registry?05:14
SamYaplelocal registry I would say ~100 nodes in 15 minutes05:14
SamYaplethen youll probably be breaking the bandwidth on the local side05:15
SamYaplethe number of nodes does not affect the total deploy time significantly05:15
sdakethe network can go faster then docker can from whatI can tell05:15
SamYaplethe way the containers are downloaded (local vs internet) will be the biggest factor05:15
SamYaplenot when you have 100 nodes pulling from one registry05:15
sdakerather registry05:16
sdakealthoug hthat was registryv1 code05:16
sdakewith registry v1, registry was cpu bound not io bound05:17
sdakeand I only have gige ethernet05:17
SamYaplei run both v1 and v2 registry for testing05:17
SamYaplei can saturate gig with both of them05:17
SamYaple_if_ i have multiple lcients pulling05:17
sdakeso pretty cool you said you are able to go from 1 to 2 to 3 nodes05:20
sdakeis that difficult?05:20
sdakei need to get my hardware plugged in to try it, but i'm curious what you think the steps may be05:20
SamYapleits simply a matter of specifying a different inventory file. i change nothing else05:20
sdakeso you uncomment the first line?05:21
sdakenm05:21
sdakeyou use the multinode invenotry file?05:21
SamYapleyea, but you will aactually have to change that file to fit your environment05:22
SamYapleunlike the all-in-one inventory file05:22
sdakeya from support to my ips?05:22
SamYapleI would suggest hostnames, but ips should work (have not tested that)05:22
sdakeand ssh_user05:22
SamYapleyou can just remove that05:23
SamYapleit has to be root05:23
SamYaplewhich is default05:23
sdakeso I need more then just database and message broker right?05:23
sdakethat is on the tbd list?05:23
SamYaplehttps://github.com/stackforge/kolla/blob/master/ansible/inventory/multinode05:23
sdakei see, i was looking at old code05:24
sdaketoo many trees!05:24
SamYaplesdake: yea gotta fix the directory access issue. we should be able to run al lthe playbooks without using root 9assuming the guest user is a docker user)05:24
sdakeseems like managing this inventory file willbe painful05:28
SamYapledisagree. but ansible supports many many ways to handle the inventory05:28
SamYapleoutside of the kolla scope05:28
SamYaplethe fact is if i have 50 computes named 'company-compute01' ... 'company-compute50' I can specify them with one line in ansible05:29
sdakeperhaps for now, not sure long trem05:29
sdakethe reason being it appears you ahve to hae implementation understanding of kolla to deploy an inventory fie05:30
SamYaplewhy? you only add your host in one section05:30
sdakeso under cmpute you add compute nodes?05:30
SamYapleyup05:30
sdakeand under support you add controller nodes?05:30
SamYapleyup05:30
SamYaplesupport can be changed btw. its just a "thats what was there" name05:31
SamYaplei have no prefernce. it doesnt affect anything outside the inventory05:31
sdakethat is all you ahve to do to configure the inventory file?05:32
SamYapleyup05:32
sdakelist your controller and compute nodes?05:32
SamYapleyup05:32
SamYaple(and later storage and network nodes)05:32
sdakeit has other implemetnation details, can those be abstracted in some other file?05:32
sdake[keystone:children]05:32
SamYaplelike i said, ansible supports many many ways to manage inventory05:32
sdakeeg^^05:33
SamYapleincluding tying into a database05:33
SamYapleeh maybe but then it actually does get complicated05:33
SamYaplewe have to do "our own thing"05:33
sdakefile based management is good05:33
SamYapleim trying not to do that in the kolla base05:33
sdakewher ei struggle is [keystone:children]05:33
sdakethat will blow up someone when they go "what does that mean"05:34
SamYaplei put comments in explicitly telling you to ignore that05:34
SamYapleonly if you know what you are doing modify that05:34
sdakeand I explicitly didn't read it ;-)05:34
SamYaplethat comes from docs05:34
SamYaplei dont take any responsibility then05:34
sdakeusers do all kinds of wired things ;)05:34
SamYapleand thats not the packages fault05:34
SamYapleif you go modify and uncomment a bunch of random lines and expect it to work, thats on the user05:35
sdakeya i hear that, but anything in /etc is fair game to most admins05:37
sdakewhen shits on fire, "hey look I bet if I change this i'm golden"05:37
SamYaplei agree. but they cant expect fair game to also make it work05:37
sdakeand then they are not golden05:37
sdakethey are torpeedoed05:37
SamYaplenot my problem05:37
SamYapleyou fuck up your configs, thats on you05:37
SamYapleyou cant make this idiot proof05:37
SamYaplethose are configurable. you config them if you know what they do05:38
sdakere neutorn, are you implementing ovs or linux bridge05:40
SamYapleovs05:40
SamYaplebut a when conditional should make this quickly supporting both05:40
sdakecool05:40
sdakei need both05:40
SamYaplei cant promise both by L205:41
sdakeunderstood05:41
sdakeboth by l3 is good for me ;)05:41
*** inc0 has joined #kolla05:41
SamYapleL3 should be doable i think05:41
sdakemaybe able to help there as well05:41
* SamYaple is not a big linuxbridge guy so unsure of potentiall issues05:41
sdakemay be able to get daneyon to do some work there - he is a linuxbridge pro05:42
inc0morning05:42
SamYaplehonestly it already works in a container, so that should be enough05:42
sdakehey inc005:42
SamYaplemorning inc005:42
inc0whats up?05:42
sdakejust playing with ansible deployment of kolla05:42
sdakeseems to work minus nova ;)05:42
inc0I'll take a look05:42
SamYaplenova should be working minus networking sdake05:42
sdakehavey ou tried it yet?05:43
SamYapleif its not thats an issue05:43
sdakesamyaple nova is restarting in my environment05:43
SamYaplek looking now05:43
sdakethe api05:43
inc0I haven't tried nova yet, if that's what you asking. I've tried ansible a lot05:43
sdakeoh you have tried sam's work in progress then?05:43
inc0I've even made haproxy ansiblish Sam;)05:44
sdakeits looking more like work done rather then in progress05:44
inc0yeah, I'm about to submit haproxy with ansible05:44
SamYaplewhich i will -205:44
SamYaplesperate them05:44
inc0two separate patches Sam05:44
SamYapleexcelent05:44
SamYapleMAHAHAAH05:44
inc0don't worry, I know git enough to make patch series05:44
inc0I've solvd this role problem05:45
SamYapleoh yea05:45
SamYaple?05:45
inc0by adding group vars to groups we want to have vip of, for example keystone05:45
inc0we add keystone:vars and there specify vip port and so on05:45
sdakehttp://ur1.ca/n4mvb -> http://paste.fedoraproject.org/244476/9391491405:46
sdakesamyaple logs ^^05:46
inc0and few rather longer statements in jinja allowed me to get these vars from haproxy role05:46
inc0even if host groups were mutually exclusive05:47
sdakethe thing is samyaple that should work after it cycles05:47
SamYaplesdake: yea something else is going on05:47
sdakewhich implies it is missing a bindmount or some other error05:47
sdakelet me double check i rebuilt the containers properly05:47
SamYapleprobably missing sock shared or something05:47
sdakeare you able to replicate it?05:47
sdakeor only me05:47
SamYapleinc0: pretty sure i know what you are talking about, and why i was trying to tell you you should upload the work sooner. that isn't where we specify ports05:48
SamYaplebut upload the patch and ill review it05:48
SamYaplesdake: i am checking now05:48
SamYapleits complaining about cachetools in all the containers05:48
inc0SamYaple, we can specify this port in default config05:49
SamYapleinc0: we already do05:49
inc0and then use variable in inventory, it's working05:49
SamYaplethat was my point05:49
SamYaplehttps://github.com/stackforge/kolla/blob/master/etc/kolla/defaults.yml#L2905:49
inc0I don't use inventory as config if that's what you're thinkig05:49
SamYaplewe have no need for group_vars though05:50
inc0its like vip_port={{ glance_api_port }}05:50
SamYaplenot at the moment05:50
inc0well, this seems to be best place for this type of logic tho05:50
sdakekollaglue/centos-rdo-nova-base                         latest              22e11545e9a8        4 days ago          757.5 MB05:50
SamYaplethe big question here, have you tested that the HUP for haproxy reload new configs as well?05:50
sdakeI think buidl.py is busted with -T 100005:50
SamYaplesdake: youre using cache05:51
inc0I'm on it today, wanted to finish what I've started with ansible05:51
inc0and get this to working point05:51
SamYaplewasted effort though. this has to be discussed. the container is top priority05:51
inc0I've learned ansible, so no, not wasted effort05:51
SamYaplebut youre holding up the project by learning ansible05:52
SamYaplei need to get this all stable by L205:52
SamYapleif you had pushed the container, I could have haproxy done05:52
inc0yeah, but we need more than 1 person who knows ansible right?05:52
SamYapleagreed, but i am doing the base bits for ansible to get it stable now05:52
SamYaplei will _not_ be doing all the other container05:53
SamYaplei will only be guiding05:53
SamYaplethats where the ocmmunity is learning ansible, that was the plan that was agreed to05:53
inc0I'm a rebel I guess;)05:53
SamYaplei know, I am the same05:54
sdakeso back on this using cache thing, how do I fix that05:54
SamYaplei can't fault you for it05:54
SamYaplesdake: my guess is pip install cachetools in base05:54
SamYapleoh you mean the docker build issue05:54
sdakehuh?05:54
sdakeyes05:54
SamYaplemerge this https://review.openstack.org/#/c/201433/05:54
sdakei already fixed cachetools05:54
SamYaplefixes that05:54
SamYapleargparser was going its job as designed, but i misunderstood how it was 'type'ing the values05:56
SamYaplebool will not work as such05:56
SamYaplesdake: did you file a bug for cachetools? if not, can you? it should probably go in base05:57
SamYapleit is affecting at least glance and nova05:58
sdakeya05:58
sdakei'm working on it05:58
openstackgerritSteven Dake proposed stackforge/kolla: Install cachetools via pip in base image  https://review.openstack.org/20192306:02
SamYaplesdake: instead of several pip installs, why not pip install all three at the same time?06:05
sdakei am hopeful we can remove them one at a time06:05
sdakebut it doesn't really matter06:05
sdakewhat does our haautoproxy container do and do we need that in the code bsae anymore?06:06
inc0I was kinda wondering the same thing06:07
SamYaplei have no idea06:07
SamYaplewe should drop it unless you build off of it inc006:07
inc0nope, I'm doing my container from scratch06:07
inc0basing a bit on yaodu06:07
sdakei'll filea bug06:07
sdakesomeone else can fix pls06:08
sdakei need to hit the sack06:08
SamYapleYAODU: the gold standard06:08
inc0config of haproxy is not that nice tho....hardcoded ports and such...06:08
inc0:P06:08
SamYaple(kolla playbooks actually run a fair bit faster than yaodu, and are much more flexible)06:08
inc0evolution its called06:08
SamYaplestill, its nice to honor our seniors06:08
inc0yup, we learn from them then bury them06:09
inc0anyway, I'm going to work06:09
inc0be back in about 40mins I guess06:09
inc0cyas06:09
sdakehttps://bugs.launchpad.net/kolla/+bug/147467906:10
openstackLaunchpad bug 1474679 in kolla "Remove dead haautorpoxy code" [Medium,Triaged]06:10
sdakeenjoy06:10
inc0I'll tackle it later todat06:10
sdakewait for this build to finish06:10
inc0today06:10
sdakethanks inc006:10
sdakeand see if nova is woring06:10
*** inc0 has quit IRC06:10
SamYaplecool sdake06:11
SamYaplei need to try the patch and rebuild06:11
sdakeI hope to have from source done this week06:19
sdakesince it takes about 1-2 hours to do a container06:19
sdakeI did two this afternoon06:19
sdakewith paul's work we should have all the containers needed for from source + ansible06:20
SamYaplegood cuase i need that for my and pauls talk06:20
sdakeassuming any of our talks get voted in :)06:21
SamYaplelast year the talks were..... lets just say im hopefully bordering on confident06:22
sdakekolla was #1 voted talk prior to summit06:22
sdakehowever there was only one of them06:23
sdakeif there are 4 or 5 ,who knows what will happen ;-)06:23
sdakewhat is subject of your and puals' talk06:23
SamYapleive got a "operators - everything you need to know about kolla" lab06:24
SamYapleand customer openstack from source with kolla06:25
SamYaplecustom*06:25
*** tobe has joined #kolla06:28
sdakeno module futurist06:34
* sdake groans06:34
sdakebedtime06:34
sdakenight06:34
SamYaplewhat?06:34
sdakenova schedule is looking for something called futurist06:34
sdakeload_one_plugin06:35
sdake    plugin = ep.load(require=verify_requirements)06:35
sdake  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2260, in load06:35
sdake    entry = __import__(self.module_name, globals(),globals(), ['__name__'])06:35
sdake  File "/usr/lib/python2.7/site-packages/oslo_messaging/_executors/impl_eventlet.py", line 19, in <module>06:35
sdake    import futurist06:35
sdakeImportError: No module named futurist06:35
SamYapleugh06:35
SamYaplethats builtin python i thought06:35
sdakesame with novncproxy06:35
sdakerest start up06:35
SamYaplebut you can pip install futurist if you want06:35
sdakei want to go to bed actually so i'm going to do that - have 7am meeting06:36
SamYaplego man go06:36
sdakesee ya later - feel free to fix if yo ulike ;)06:36
SamYaplenever06:36
SamYaplegot training on containres today06:36
SamYaple(i wish i was kidding, its containers 101)06:36
*** ashu has joined #kolla06:45
*** ashu has left #kolla06:45
*** ashu has joined #kolla06:50
*** ashu has quit IRC06:51
*** juggler has quit IRC06:52
*** ashuk has joined #kolla06:56
*** ashuk has quit IRC06:58
*** ashuk has joined #kolla06:58
*** ashuk has left #kolla06:59
*** inc0 has joined #kolla07:00
*** sdake has quit IRC07:07
*** shardy has joined #kolla07:10
*** akwasnie has joined #kolla07:17
*** shardy_ has joined #kolla07:18
*** shardy has quit IRC07:19
*** dims_ has joined #kolla07:20
*** shardy_ has quit IRC07:23
*** shardy has joined #kolla07:24
*** dims_ has quit IRC07:26
*** erkules_ is now known as erkules07:55
*** erkules has joined #kolla07:55
*** achanda has quit IRC08:02
pbourkeSamYaple: can you not skip it?08:37
SamYapleskip skip what what?08:37
pbourkethe container training08:40
SamYapleoh yea. no. I have to "beta" it for the team08:41
jmccarthyo/ Morning ! I've somehow introduced a problem for myself in swift containers I'm building, relating to kolla-common.sh - any suggestions about the complaint:08:41
jmccarthy/start.sh: line 10: set_configs: command not found08:41
SamYaplejmccarthy: you need to rebuild all the containers, specifically the bse08:41
pbourkejmccarthy: make sure you're building with --no-use-released-parent08:42
jmccarthyI'm using --release tho ?08:42
jmccarthyOk - will rebuild base to see if that helps as well08:43
SamYaplejmccarthy: you will have t orebuild the base, without cache too08:43
*** dolpher has quit IRC08:44
jmccarthySamYample: is there a specific argument for that ?08:45
SamYaplefor the regular build script, i believe -N08:45
jmccarthyOh right yea --no-cache08:45
*** jtriley has joined #kolla08:53
*** athomas has quit IRC08:55
*** jtriley has quit IRC08:58
jmccarthypbourke: I should use --no-use-released-parent for building base is it ? (I'm using --release across the board at the minute)08:59
pbourkejmccarthy: no first build the base, then build the rest with --no-use-released-parent08:59
pbourkeotherwise the others will base themselves off of the dockerhub base09:00
*** mickt has joined #kolla09:03
SamYaplepbourke: when you are the second core to +2 a review, also +1 Workflow09:03
pbourkeSamYaple: ok09:04
SamYaple+109:04
jmccarthypbourke: Ok I'll try that way, I thought I wanted my locally built ones to all be 'latest'09:05
micktI was wondering if/when the dev-quickstart kollaglue/centos-rdo-glance-data:latest issue might be resolved. There is also similar issue with neutron-linuxbridge-agent.09:07
micktPulling glancedata (kollaglue/centos-rdo-glance-data:latest)...  not found09:07
SamYaplemickt: we have not pushed containers to Docker for the L1 branch yet. I would recommend building your own containers for now09:09
micktI tried using build all but horizon fails with django issue09:09
SamYaplei believe the fix for that was merged into master mickt, but nevertheless Horizon doesn't stop the othe rcontainers from working09:10
SamYaplewe are waiting on unstream Horizon fixes to truly fix the container09:11
mickttrue, I'm interested in horizon. :(09:11
micktI'll "watch this space" :)09:12
SamYapleyea sorry. iirc the issue was with a new version of django breaking upstream horizon09:12
SamYaplei dont follow the issue to closely09:12
pbourkemickt: pick up marks patch09:14
pbourkemickt: it's currently up for review09:14
SamYaplepbourke: pretty sure that merged already09:15
pbourkeoh09:15
pbourkemickt: then git pull09:15
pbourke:)09:15
*** athomas has joined #kolla09:28
coolsvapSamYaple, I am trying to build the ceilometer container, but I am getting permission denied for start.sh, have we seen similar issues with previous containers?09:51
*** tobe has quit IRC09:54
coolsvappbourke, ^^09:55
pbourkecoolsvap: not that im aware of09:56
SamYaplecoolsvap: sounds like start.sh doesnt have +x set09:56
coolsvapSamYaple, it has in the repo, but not inside the container09:56
coolsvapi changed ADD instruction in Dockerfile to COPY but no change09:57
pbourkecoolsvap: the start.sh scripts for ceilometer do indeed appear to be missing +x09:57
SamYapleyep. config-internal.sh needs +x too09:58
SamYaplecalled it09:58
pbourkefind docker/common/ceilometer/ -name start.sh | xargs chmod +x09:58
SamYaplemake sure there isn't a bug/patch for it09:59
coolsvapSamYaple, but its a bit weird, the links in docker/centos/binary showing +x09:59
coolsvap:(09:59
SamYaplecoolsvap: those are symlinks09:59
pbourkesounds like we could use a gate for checking these scripts09:59
SamYaplepbourke: no we just need to be setting hte perms inside the dockerfile like civilized people09:59
coolsvapSamYaple, yes10:00
pbourkeeven better10:00
SamYaplecoolsvap: symlinks dont reflect the permissins of the file they reference10:00
coolsvapSamYaple, hmm10:01
*** dims_ has joined #kolla10:03
coolsvapSamYaple, thx! seems the container is starting now10:04
SamYaplew00t10:04
SamYapleif no bug: file bug10:04
SamYapleif no patch: submit patch10:04
*** dims_ has quit IRC10:05
openstackgerritSwapnil Kulkarni proposed stackforge/kolla: Update executable permissions on scripts  https://review.openstack.org/20203210:15
*** tobe has joined #kolla10:17
*** weiyu-001 has joined #kolla10:30
harmwguys, I won't be attending tonight's meeting10:38
harmwlocal bedtime10:38
*** mstachow has joined #kolla10:39
*** coolsvap is now known as coolsvap|away10:41
*** tobe has quit IRC10:48
*** rhallisey has joined #kolla11:14
openstackgerritMerged stackforge/kolla: Update executable permissions on scripts  https://review.openstack.org/20203211:17
openstackgerritMerged stackforge/kolla: Improve build.py  https://review.openstack.org/20143311:20
*** weiyu-001 has left #kolla11:21
*** openstack has joined #kolla11:38
rhalliseyman what a relief11:39
SamYapleive got that ceph patch pending me actually testing it11:40
SamYapleyou dont have to use a loopback "dev" for ceph, raw folder works fine11:40
rhalliseySamYaple, oh you already have ceoh11:40
rhalliseyI was going to try and implement it11:40
rhalliseyadding disk support shouldn't be hard11:40
SamYaplehttps://github.com/SamYaple/yaodu/tree/master/ansible/roles/docker_build/templates/ubuntu/ceph11:41
SamYaplei just ported it from yaodu11:41
rhalliseyI just need get a setup for it going11:41
SamYapleeasy peasy11:41
SamYaplerhallisey: how do i get this to retest the zuul gate? https://review.openstack.org/#/c/201923/11:44
rhalliseyehh I thought it was check recheck11:45
rhalliseylooks like it din't work tho11:45
SamYaplerecheck does the jenkins check jobs11:45
SamYaplei think reverify is th gate jobs11:45
SamYapleah maybe because i said recheck first its running the check jobs11:46
SamYaplethatll take 45 minutes11:46
*** dwalsh has joined #kolla11:47
*** dims_ has joined #kolla11:47
rhalliseyya probably11:48
rhalliseySamYaple, are you going to mid cycle11:48
SamYapleyea, but RAX aint paying :/11:48
rhalliseyboo11:48
SamYapleypu11:49
rhalliseyis it far?11:49
SamYapleitll cost me ~800$ roundtrip with hotel and food11:49
rhalliseynot bad11:49
SamYapleand vacation days :/11:49
rhalliseyI think it will be worth it11:50
SamYaplei do too, thats why im going but still11:50
rhalliseyya that's annoying11:50
rhalliseysince it is work related11:50
SamYaplethey dont think so11:50
rhallisey:(11:51
rhalliseysome day11:51
SamYapleits what im banking on11:51
SamYapleI think the summit will be the game changer. have some shakeup power then11:51
rhalliseyhow many talks did you submit?11:51
SamYaple1 with sdake, 1 with pbourke, 1 kolla lab with pbourke and a Racker i work with11:52
rhalliseyoh cool11:53
rhalliseyall kolla?11:53
SamYapleyup11:53
rhalliseywow there's going to be like 15 kolla submissions11:54
SamYapleall different though, i believe11:55
rhalliseyya I think so11:55
SamYapleive been careful not to step on any toes11:55
rhalliseysame11:55
SamYaplethats exciting11:55
inc03 submissions from our end:)11:55
*** dolpher has joined #kolla11:55
inc0incl our joint rhallisey11:55
inc0let's own this summit11:56
SamYaplewe gotta vote up ALL the kolla talks11:56
inc0yeah, about time I learn what's this is about11:56
SamYaplei dont know how many votes are typical for a tlak to be accepted11:57
inc0in vancouver I was on 2 talks total;)12:05
inc0Steven's about kolla and my own12:05
*** dolpher has quit IRC12:12
*** tobe has joined #kolla12:15
SamYaplehey one of my labs is an operators guide to kolla12:16
SamYaplei will be discussing some bits about upgrades12:16
SamYaplewe should probably coordinate a bit on that subject12:16
rhalliseySamYaple, inc0 Slower and I are going to talk about doing an upgrade through tripleo + kolla12:18
*** tobe has quit IRC12:18
rhalliseyso heat would do the orchestration12:19
SamYapleyea but the actual process itself, i dont think we have any docs on that12:19
*** tobe has joined #kolla12:19
rhalliseyya we have none12:19
SamYaplewe should probably figure that out :P12:20
akscrambuffer 212:20
rhalliseyI've don't one without vm migration12:21
rhalliseywas pretty straightforward12:21
rhalliseys/don't/done12:21
*** akwasnie has quit IRC12:21
openstackgerritMerged stackforge/kolla: Install cachetools via pip in base image  https://review.openstack.org/20192312:24
*** tobe has quit IRC12:24
inc0with lm its not that bad12:29
inc0but due to neutron major fuckups, its hard without lm12:29
inc0also we're speaking about control plane downtime12:29
inc0live upgrades are whole different story:)12:29
*** dims_ has quit IRC12:41
*** jmccarthy has quit IRC13:00
*** jmccarthy has joined #kolla13:01
*** dims_ has joined #kolla13:06
*** rbergeron has quit IRC13:11
*** rbergeron has joined #kolla13:11
vincent_vdkhi, if i build all kilo containers, should that work "out of the box"/13:12
rhalliseyvincent_vdk, yes, just follow the docs13:14
rhalliseyany issues you have just bring them up here13:15
vincent_vdkrhallisey: just got hit by the cachetools i think13:15
vincent_vdkjust pulled the from the repo and rebuilding13:15
rhalliseyok13:16
vincent_vdklets see what happens13:17
*** dolpher has joined #kolla13:19
SamYaplevincent_vdk: youll also need "futurist"13:23
SamYaplefor nova13:23
SamYaplegot a patch13:23
*** dolpher has quit IRC13:24
*** shardy_ has joined #kolla13:29
*** shardy has quit IRC13:31
vincent_vdkSamYaple: was that merged already?13:31
openstackgerritSam Yaple proposed stackforge/kolla: Add needed module 'futurist' to Nova  https://review.openstack.org/20210813:32
SamYaplenope13:32
SamYaplebut if you can convince two cores its important you can get it merged quickly ;)13:32
SamYaplerhallisey harmw pbourke ^13:33
openstackgerritRyan Hallisey proposed stackforge/kolla: Start tgtd in cinder-volume container  https://review.openstack.org/20013913:33
pbourkewe really need dockerfile style guidelines13:33
SamYaplepbourke: just hold out13:33
SamYaplewe can talk about them before templating work begins13:34
pbourkefair enough13:34
SamYapleI plan on tackling all templating during L313:34
pbourkeno rpm for futurist I take it?13:34
SamYapleThis includes docker style fixes13:34
SamYaplepbourke: it didnt like the rpm for me13:35
SamYapleor the build failed rather13:35
*** shardy_ has quit IRC13:35
SamYaplei didnt investigate13:35
*** shardy has joined #kolla13:35
*** diogogmt has joined #kolla13:36
*** mstachow has quit IRC13:37
*** jtriley has joined #kolla13:38
SamYaplecan someone build new libvirt containers and confirm libvirt is _not_ busted?13:38
SamYapleit is failing for me 'Failed to connect to socket /var/run/dbus/system_bus_socket'13:39
rhalliseyit's been working for me, but I haven't rebuilt in a few days13:40
SamYapleare you in a position to do a full rebuild, no cache?13:40
SamYaplei think upstream broke this13:40
rhalliseyya let me put one of my machines to work13:43
*** jtriley has quit IRC13:45
SamYapleheading home13:46
SamYapleprobably wont be on the meeting today due ot time13:46
*** prad has joined #kolla13:47
*** jtriley has joined #kolla13:55
openstackgerritRyan Hallisey proposed stackforge/kolla: Start tgtd in cinder-volume container  https://review.openstack.org/20013914:03
*** diogogmt has quit IRC14:04
*** jtriley has quit IRC14:06
*** jtriley has joined #kolla14:08
*** shardy_ has joined #kolla14:16
*** shardy has quit IRC14:17
*** shardy_ has quit IRC14:21
*** shardy has joined #kolla14:22
*** pbourke has quit IRC14:22
openstackgerritRyan Hallisey proposed stackforge/kolla: Start tgtd in cinder-volume container  https://review.openstack.org/20013914:22
*** jtriley has quit IRC14:22
*** pbourke has joined #kolla14:22
rhallisey^ that worked for me14:22
rhalliseysomeone else can try it out if they like14:23
*** sdake has joined #kolla14:23
rhalliseysdake,  37a97bb2-9cc9-4cfb-be65-850327d2ac36 |     in-use     | None |  1   |     None    |  false   | 2f3c1167-0fba-4131-a146-e5786edd08d714:24
rhalliseyhttps://review.openstack.org/#/c/200139/1314:24
rhalliseytry it out, the volume is attaching now14:24
sdakecool what did you fix :)14:25
sdakemorning folks14:25
sdakedid anyone ffix the futurist bug14:25
rhalliseysdake, there's a review for it14:25
rhalliseyhttps://review.openstack.org/#/c/202108/14:25
sdakeagree with pbourke14:26
sdakesamyaple around?14:26
pbourkehe said he was heading home about 30 mins ago14:26
sdakepbourke can you submit a review fo rthat14:26
sdakeand rhallisey+sdake can approvve14:27
pbourkesdake: hmm SamYaple doesn't like people touching his patches14:27
pbourkeI can submit an alternative?14:27
sdakeyes a new patch14:27
sdakesame bug id14:27
pbourkek one sec14:27
sdakedifferent change id14:27
*** absubram has joined #kolla14:29
openstackgerritPaul Bourke proposed stackforge/kolla: Add new Nova requirement 'futurist' to base  https://review.openstack.org/20213814:30
*** absubram has quit IRC14:31
sdakerhallisey/jpeeler can one of you approve that patcha bove pls14:31
rhalliseyk14:31
*** jtriley has joined #kolla14:31
*** absubram has joined #kolla14:31
*** inc0 has quit IRC14:34
openstackgerritRyan Hallisey proposed stackforge/kolla: Start tgtd in cinder-volume container  https://review.openstack.org/20013914:35
openstackgerritRyan Hallisey proposed stackforge/kolla: Start tgtd in cinder-volume container  https://review.openstack.org/20013914:35
sdakepbourke did samyaple finish up neutron yesterday?14:38
pbourkesdake: no idea14:38
pbourkehe was ansible-ising it?14:39
sdakeyes14:40
pbourkei haven't seen any reviews for that yet14:40
sdakebummer14:40
sdakethe ansible implementation looks pretty sweet14:41
sdakei wonder if we shouldn't have a container just for db upgrades14:41
pbourkewe kind of do with the bootstrap task14:43
*** blahRus has joined #kolla14:43
pbourkeits the same container it just runs once with a bootstrap flag14:43
sdakebootstrap is a different operation then upgrade no?14:44
pbourkewell it generally runs db_sync but I guess it's not clearly defined14:45
SamYaplesdake: yea14:45
sdakerhallisey i try out your patch around 9am once i haverebuilt all my images14:45
pbourkewaiting for SamYaple to return and tell me im wrong ;)14:45
pbourkeaha14:45
SamYapleyoure wrong14:45
rhalliseysdake, kk14:45
SamYapleits onloy needed by nova14:45
SamYapleis that what you were tlaking about?14:45
sdakeno we are talkingabout db migration upgradess14:45
SamYapleoh14:45
sdakeand how to handle that14:45
SamYaplehmm14:46
SamYaplefor config-external that will vary depending on official upgrade proceedures14:46
SamYaplepbourke: why did you submit a new patch...14:47
SamYaplei already had a patch for that and you commented on it14:47
pbourkeSamYaple: sdake needed it through14:47
SamYaplei had a patch for it...14:48
sdakesamyaple i asked for it14:48
SamYaplei get that. i HAD a patch already14:48
*** absubram has quit IRC14:48
sdakeI just want to get on my day with cinder14:48
sdakei saw yoru patch, i thought futurist would be better in the base14:49
SamYapleisnt that what reviews are for.14:49
sdakeyou mean why I didn't leave a review?14:50
sdake7am - brain no worky14:50
sdake6 hrs sleep14:50
SamYaplewhy did a new patch get submitted at all14:50
sdakeyou know I dont operate well in the morning14:50
pbourkeyou werent around to update it, and I didnt want to touch yours14:50
SamYaplei was around. we dont need updates inside a 30 minute mark14:51
SamYaplejeez people i was just heading home14:51
sdakeya my bad samyaple i thought you were off for the night14:51
SamYaplei really dont like setting that as a precedent14:51
sdakei thought youw ere gone for the day bro14:51
sdakeand didn't want to block for 12 hrs on it14:51
SamYaplefair lets just agree not to make that common practice?14:51
sdakeagree shit happens14:51
pbourkeno harm meant14:51
SamYapleI dont want multipatches14:52
SamYaplekewl14:52
SamYapleso on the subject of database upgrades. that only happens on new openstack versions14:52
sdakerhallisey i assigned that lock path bug to me since I fixed it ok?14:53
rhalliseysdake, ya take14:53
rhalliseyit14:53
SamYaplein the case of all of our services that can be solved by an explict call to the KOLLA_BOOTSTRAP function after stopping all the contaienrs accessing teh DB14:53
sdakerhallisey my bosses are keen to see me do real technical work not just chat in a channel all day ;)14:53
SamYaplebut with the advent of live db upgrades that will need thought14:53
rhalliseysdake, I thought I fixed that back in the day, but you def fixed the most recent bug14:53
sdakepbourke thanks for review on heat - so looks good then?14:55
* sdake yays for doing real techncial work ;)14:55
sdakewhat was wrong with magnum - the review comments don't come out for some reason14:55
sdakewere the links busted?14:55
*** shardy_ has joined #kolla15:03
pbourkesdake: yeah Im not sure how you created them but they seem to be normal files with a a textual pointer to the target rather than symlinks15:04
*** shardy has quit IRC15:04
pbourkecheck `ls -l` locally and you dont see the a --> b notation15:05
sdakecp -aR15:05
pbourkegerrit also didnt tag them with the "symbolic link" thingy15:06
pbourkeSamYaple: what happens if something is done in bootstrap that shouldn't be done on upgrade15:06
*** diogogmt has joined #kolla15:07
*** shardy_ has quit IRC15:08
*** shardy has joined #kolla15:09
*** mickt has quit IRC15:10
sdakedid inc0 finish up haproxy?15:11
pbourkewhy are there so many missing dependencies popping up lately :( what are rdo up to?15:14
pbourkeSamYaple: my futurist review seems to be still in the queue, I have no problem if you want to pull it and re-instate yours15:16
sdakethey take master15:16
sdakepbourke lets not disrupt the queueu - i want master woring soon :)15:17
pbourkesdake: they're kilo rpms though?15:17
sdakeno they are master15:17
pbourkeoh15:17
pbourkeI did *not* know that15:17
sdakein rpm format15:17
sdakethat is what delorean is15:18
sdakeit packages master into rpms15:18
sdakebut it doesn't set the depependencies properly15:18
sdakemake sense?15:21
sdakesince it doesn't set deps properly and the dep creation trails the rpm creation we have to pip install stuff15:21
pbourkeyup15:22
sdakesamyaple q15:28
sdakesamyaple if i reboot my system with ansible olla containers on them, will they restart automtically?15:28
pbourkethey will15:28
pbourkebecause of the docker restart policy15:29
*** sdake has quit IRC15:32
*** sdake has joined #kolla15:32
openstackgerritMerged stackforge/kolla: Add new Nova requirement 'futurist' to base  https://review.openstack.org/20213815:39
sdakesamyaple neutron coming along? :)15:43
*** inc0 has joined #kolla15:49
*** dwalsh has quit IRC15:53
*** diogogmt has quit IRC16:01
*** diogogmt_ has joined #kolla16:01
inc03 presentations about kolla submitted16:02
inc0well, 2 and a half16:02
sdakeyou submitted 3?16:02
rhalliseyinc0, nice16:02
inc0I submitted 2, and rhallisey 116:02
bmacethis is going to be an all kolla summit :)16:02
rhalliseyyerp16:03
pbourkespeaking = free ticket?16:03
sdakeya or a commit to an openstack namesapced project16:03
pbourkewill be a nice feeling if/when we get into big tent16:03
*** unicell has quit IRC16:05
inc0sdake, do we have checklist for getting to OS namespace?16:05
inc0what's left to be done?16:05
*** sdake has quit IRC16:08
openstackgerritJames McCarthy proposed stackforge/kolla: Swift WIP  https://review.openstack.org/20217116:11
*** sdake has joined #kolla16:11
jmccarthyGo easy on me ;) Wanted to try and get something in before todays meeting16:12
*** unicell has joined #kolla16:13
rhalliseyjmccarthy, cool!16:13
*** unicell has quit IRC16:15
*** inc0 has quit IRC16:15
pbourkehmm there's no dupe on launchpad?16:17
pbourke(dupe status for bugs)16:17
*** bradjones has quit IRC16:20
*** akscram has quit IRC16:20
openstackgerritPaul Bourke proposed stackforge/kolla: Swift WIP  https://review.openstack.org/20217116:20
*** bradjones has joined #kolla16:20
*** bradjones has quit IRC16:20
*** bradjones has joined #kolla16:20
*** akscram has joined #kolla16:20
*** jasonsb_ has quit IRC16:22
pbourkeah there is16:31
sdakeso ansible looks pretty solid16:35
sdakeI think we can start working on the other services now16:35
bmaceoh?  that is awesome.  it works well multi-node or mostly still single node?16:36
sdakemaster now builds and starts properly as well :)16:36
sdakeI haven't tried multi-node16:36
sdakebut samyaple did asy multinode works well16:36
sdakeI plan to try today16:36
sdakewe are missing neutron16:36
bmacefantastic.16:36
sdakewhich is a bit unforttuantely :)16:36
bmaceah, well, who needs networks? :)16:36
sdakenetworking - overrated16:37
bmacethat is what usb sticks are for.. and interns16:37
harmwsdake: wasn't Sam working on neutron?16:37
sdakehttp://ur1.ca/n4rou -> http://paste.fedoraproject.org/244705/4369782816:38
sdakesamyaple ^^ nova-compute failing to start in master16:38
bmacethey need more cores on infra :)16:38
sdakefreshly built images no caching16:38
sdakeharmw yup16:38
*** unicell has joined #kolla16:49
sdakesamyaple there is no pid=host for libvirt's start16:50
sdakethat is why we get the cannot connect error I think16:51
sdakeit will connect on ubuntu but not on fedora16:51
sdakeharmw or pbourke do you know how to pass pid=host to ansible's start module?16:51
harmwnope, I know it's a compose feature that magically gets replaced with some docker var16:52
harmwprobably16:52
sdakeI added it to compose16:54
sdakelooks like its in the docker module already17:04
*** loth has quit IRC17:26
*** diga has joined #kolla17:27
*** loth has joined #kolla17:28
*** achanda has joined #kolla17:29
*** achanda has quit IRC17:39
*** jasonsb has joined #kolla17:46
*** achanda has joined #kolla17:48
openstackgerritJeff Peeler proposed stackforge/kolla: Improve logging and prepare for gate  https://review.openstack.org/20221617:54
jpeeleri didn't actually mean to submit that yet, but i think it's fine17:54
sdakeif anyone has ideas on this one https://bugs.launchpad.net/kolla/+bug/147496918:00
openstackLaunchpad bug 1474969 in kolla "Ansible implementation of Nova doesn't work on systemd based systems" [Undecided,New]18:00
*** dwalsh has joined #kolla18:01
*** slagle_ is now known as slagle18:08
*** athomas has quit IRC18:19
*** athomas has joined #kolla18:32
sdakeany ansible wizards have any idea why this doesn'twor? http://ur1.ca/n4skv -> http://paste.fedoraproject.org/244749/3698530918:35
*** juggler has joined #kolla18:37
*** achanda has quit IRC18:43
harmwsdake: any specific errormsg?18:49
harmwdoes default(None) work like that?18:50
sdakeharmw no19:00
sdakeharmw i am in #ansible gtting help now ;)19:00
sdakecome watch its fun :)19:00
openstackgerritJeff Peeler proposed stackforge/kolla: Improve logging and prepare for gate  https://review.openstack.org/20221619:07
sdakeansible cats say we need to conditionalize our start.yml19:10
sdakei dont now how to do that19:10
*** achanda has joined #kolla19:11
harmwah, I think that involves things like when: and such19:14
harmwsam has already done some pieces with that19:14
bmacethere looks to be an example of using when and 'None' in bootstrap.yml19:15
sdakewhich one bmace19:18
sdakethey said we can set container_pid to !!null in each container19:18
* sdake groans19:18
*** prad has quit IRC19:19
*** jasonsb has quit IRC19:20
*** jasonsb has joined #kolla19:20
*** jasonsb has quit IRC19:21
*** jasonsb has joined #kolla19:21
*** prad has joined #kolla19:22
*** dolpher has joined #kolla19:23
bmaceroles/database/tasks/bootstrap.yml19:26
*** prad has quit IRC19:28
*** prad has joined #kolla19:40
*** sdake_ has joined #kolla19:41
*** sdake has quit IRC19:45
*** dolpher has quit IRC19:47
*** sdake_ has quit IRC19:52
*** sdake has joined #kolla20:00
sdakewell I tried http://ur1.ca/n4t9q -> http://paste.fedoraproject.org/244779/9904321420:01
sdake but no beuno20:01
*** jmccarthy1 has joined #kolla20:06
jmccarthy1o/20:07
sdakeo/20:07
jmccarthy122:00 UTC is todays meeting isn't it ?20:08
sdake22:00 utc is in 2 hours20:09
sdakebut yup20:09
jmccarthy1Yep ok cool, on crappy home pc blah20:09
jmccarthy1sdake: I managed to get a WIP in for swift (with pdb help !) it still needs some sorting out, but any and all feedback appreciated !20:11
sdakecool i'll check it out20:11
jmccarthy1I'm really hoping it will all come together for L220:11
sdakethat woudl be sweet, I"ve been trying to get folks to implement swift for oh like 8 months or so ;-)20:11
jmccarthy1What sort of timelines would - or ok like patch for that blueprint would need to sorted out by when doyou think roughly to get the over the line ?20:12
jmccarthy1It might be tight but I'm trying to be optimistic =)20:13
sdake[sdake@bigiron kolla]$ nova hypervisor-list20:14
sdake+----+---------------------+-------+---------+20:14
sdake| ID | Hypervisor hostname | State | Status  |20:14
sdake+----+---------------------+-------+---------+20:14
sdake| 1  | bigiron.broked.org  | up    | enabled |20:14
sdake+----+---------------------+-------+---------+20:14
sdakeyay ;-)20:14
sdakesorry for spam ;)20:14
sdakenotraelly but ya :)20:14
jmccarthy1I don't recall who, but someone told me the first commit is the toughest - totally agree (even tho it's a WIP)20:15
jmccarthy1In hindsight I think now maybe I could have done that earlier, and then discuss, iterate, repeat - but anyway moving things along at least20:16
jmccarthy1pdb: you around by chance ?20:17
openstackgerritSteven Dake proposed stackforge/kolla: Add pid_mode to libvirt  https://review.openstack.org/20226420:21
sdakedunno first committed to open source 15+ years ago20:22
sdakeso I've lost track of what i t was like to have that "new" feeling to it :)20:23
openstackgerritSteven Dake proposed stackforge/kolla: Add pid_mode to libvirt  https://review.openstack.org/20226420:24
*** bmace has quit IRC20:30
*** bmace has joined #kolla20:44
sdakeharmw I got your linked in request but my client doen't do language conversion, can you mail to steven.dake@gmail.com, this is my real linked in address20:44
sdakei dont speak whatever language was there ;)20:44
harmwlol, weird20:45
sdakegmail does translation automatically20:45
harmwit should've been just plain English20:45
sdakeit definatey was not20:46
harmwthere20:46
openstackgerritJeff Peeler proposed stackforge/kolla: WIP: add new tox target for parallel gate building  https://review.openstack.org/20229420:59
jpeelersdake: i'm not going to be able to come to the meeting today, but i have the project-config changes staged too for once that WIP is finished ^21:00
sdakefor which21:00
jpeelerparallel image building using build.py21:01
sdakeoh in the gate you mean21:01
jpeelernon-voting job, figured i shouldn't submit until the tox target is in place21:01
sdakecoool21:01
*** jpeeler has quit IRC21:02
harmwsdake: /me won't attend either21:04
sdakemaybe I wont attend ;-)21:05
harmwdont think so21:05
*** unicell has quit IRC21:06
*** dims_ has quit IRC21:07
*** jmccarthy1 has quit IRC21:14
*** rhallisey has quit IRC21:26
*** jtriley has quit IRC21:30
*** sdake_ has joined #kolla21:30
*** sdake_ has quit IRC21:30
*** rhallisey has joined #kolla21:30
*** sdake_ has joined #kolla21:30
*** dwalsh has quit IRC21:32
*** sdake has quit IRC21:33
*** sdake has joined #kolla21:43
*** sdake_ has quit IRC21:46
*** shardy has quit IRC21:47
vinkmanHmmm, any ideas on this one: fatal: [localhost] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute 'ipv4'", 'failed': True}21:52
sdakevinkman did you read the docs and set your interface name n the globals.yml (I htin its this file)21:53
vinkmanThis one : network_interface: "eth2"21:54
vinkmanYup set21:54
sdakeodd not sure although samyaple will likely know the cause21:54
vinkmanAhhh… Got IT! my eth2 has no IP since it is the public one, has to change the api_interface to eth0 (which does have an IP)21:56
*** jmccarthy1 has joined #kolla21:57
*** pbourke_ has joined #kolla21:57
*** mstachow has joined #kolla21:59
*** gfidente has quit IRC21:59
sdakemeeting now in #openstack-meeting-421:59
*** dwalsh has joined #kolla22:00
*** achanda has quit IRC22:06
*** achanda has joined #kolla22:06
*** britthouser has quit IRC22:07
*** jruano has joined #kolla22:10
*** pbourke_ has left #kolla22:11
rhalliseydiga, around?22:16
*** dwalsh has quit IRC22:23
mstachowthat was quick22:30
jmccarthy1Any swifties out there with feedback for swift-container wip, https://review.openstack.org/#/c/202171/ don't be shy ! Hoping to get this in for L222:38
jmccarthy1Zzz night !22:38
mstachownight ! (:22:39
*** mstachow has quit IRC22:39
*** jmccarthy1 has quit IRC22:39
mandrebye22:41
*** prad has quit IRC22:45
*** dims_ has joined #kolla22:51
*** rhallisey has quit IRC22:53
*** akwasnie has joined #kolla22:55
*** achanda_ has joined #kolla23:02
*** achanda has quit IRC23:02
*** diogogmt_ has quit IRC23:05
*** blahRus has quit IRC23:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!