2014-06-24T00:00:12 right, you would say "this_is_a_network_node: true" not "sysctl.net.ipv4.ip_forward: 1" 2014-06-24T00:00:19 yerp 2014-06-24T00:03:14 so, I think you would still need to support a runtime sysctl-set-value, right? 2014-06-24T00:03:46 * greghaynes is not wondering if https://review.openstack.org/#/c/101784/1 needs to be smart-ish and detect if its build or runtime 2014-06-24T00:03:51 s/not/now 2014-06-24T00:11:06 *** ddieterly has joined #tripleo 2014-06-24T00:12:18 *** ddieterly has quit IRC 2014-06-24T00:12:52 *** ddieterly has joined #tripleo 2014-06-24T00:13:07 *** yamahata has quit IRC 2014-06-24T00:14:29 *** Penick has quit IRC 2014-06-24T00:15:35 *** jerryz has joined #tripleo 2014-06-24T00:17:07 *** ddieterly has quit IRC 2014-06-24T00:17:14 lifeless: is there an updated tutorial to play tripleo, deploying seed, under, overcloud? 2014-06-24T00:17:27 lifeless: is there a way to turn debug logs back on for neutron so i can see what’s going on in this patch? https://review.openstack.org/#/c/101447/ 2014-06-24T00:20:39 kevinbenton: check out the ExtraConfig parameter in overcloud.yaml 2014-06-24T00:20:48 neutron.conf supports adding options via that 2014-06-24T00:21:08 *** rlandy has quit IRC 2014-06-24T00:21:41 *** matsuhashi has joined #tripleo 2014-06-24T00:23:09 *** nati_ueno has joined #tripleo 2014-06-24T00:24:26 so what’s the process of doing that for a specific neutron patch? 2014-06-24T00:25:30 greghaynes: ^^ 2014-06-24T00:26:05 *** eghobo has quit IRC 2014-06-24T00:31:17 jerryz: are you asking about a video? I'm not aware of such a thing 2014-06-24T00:31:50 but http://docs.openstack.org/developer/tripleo-incubator/devtest.html (generated from devtest.sh) essentially does that in text 2014-06-24T00:31:52 tchaypo: not necessarily a video. doc.o.o or something 2014-06-24T00:34:05 tchaypo: thanks. so the devtest scripts is the literally the procedures. 2014-06-24T00:34:24 it's *a* procedure 2014-06-24T00:34:45 I'd expect that if you're doing your own deployment you'll probably find parts of the procedure that don't quite do what you need 2014-06-24T00:35:26 but our scripts aim to be "small pieces, loosely joined" so it should be easy for you to tweak pieces to get a procedure that suits you 2014-06-24T00:35:37 tchaypo: got you 2014-06-24T00:35:53 but running through the devtest scripts first should, in theory, give you an introduction to the bits and how they fit together, and give you a starting point for your own deployment 2014-06-24T00:36:57 tchaypo: what automation or CM tool do you use to deploy your CD overcloud? 2014-06-24T00:37:35 tchaypo: devtest looks devstack kind of thing to me. 2014-06-24T00:38:42 good question, I should know the answer to that 2014-06-24T00:39:08 for CI we use jenkins and zuul, and they drive a slightly modified process - our CI infra has a few things that don' 2014-06-24T00:39:14 jerryz: we do use devtest, because it deploys production ready - we keep moving stuff out of it and into stable repos as time goes by 2014-06-24T00:39:27 t change much preconfigured 2014-06-24T00:39:57 SpamapS: greghaynes FWIW e20084f21aa80e47ce219b5cc050ec2a57ab3fea 2014-06-24T00:40:30 The slight-modified devtest.sh we use for CI is at http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_devtest.sh - but that's CI, not CD 2014-06-24T00:40:59 greghaynes: kevinbenton is looking into the thing that broke overcloud gre networking 2014-06-24T00:41:20 * jerryz reading 2014-06-24T00:41:30 jerryz: tchaypo: meeet each other - jerryz is bringing up the huawei region 2014-06-24T00:41:54 lifeless: dprince mentioned in the patch that the heat auth errors were normal 2014-06-24T00:41:58 jerryz: oh hey, you're my new favorite person! 2014-06-24T00:42:11 SpamapS: 101305 is ready for +A 2014-06-24T00:42:23 lifeless: so i need to figure out how it’s still breaking but for some reason all of the logging is gone now 2014-06-24T00:42:49 kevinbenton: righto so thats because we run defaults :) 2014-06-24T00:42:58 uhm 2014-06-24T00:43:12 tchaypo: nice to talk to you. going to have questions bugging you. 2014-06-24T00:43:40 kevinbenton: the review that we were using for this had a patch set to make everything default to debug, we don't have a generic knob for CI to say 'this run should be debug', but it sounds like a super useful thing to have 2014-06-24T00:45:37 *** e0ne has joined #tripleo 2014-06-24T00:45:45 A change was merged to openstack/tripleo-image-elements: Enable ip nonlocal binds https://review.openstack.org/101305 2014-06-24T00:46:37 lifeless: sounds a potential zuul parameter to me. 2014-06-24T00:47:19 we run all tests as debug 2014-06-24T00:47:23 do you guys not do that? 2014-06-24T00:47:33 clarkb: we dont 2014-06-24T00:48:34 *** e0ne has quit IRC 2014-06-24T00:48:35 clarkb: part of running defaults 2014-06-24T00:48:47 Dear cores - after yesterday's adventure, https://review.openstack.org/#/c/92749/ is back to having one +2 and a couple of +1s; the only change from the last time it had 3 +2s is that I've merged it with some changes gfidente made. Your attention would be appreciated 2014-06-24T00:48:48 clarkb: a knob would be great or just debug open for rechecks. that would also reduce log volumes 2014-06-24T00:48:54 clarkb: it means we get to show devs what their logging /actually is like/ 2014-06-24T00:49:08 lifeless: we do that by only indexing > debug 2014-06-24T00:49:17 so if you want to take advantage of tooling you need sufficient logging 2014-06-24T00:49:42 tchaypo: one +2. 2014-06-24T00:50:35 clarkb: we'd need to run non-default to do that. 2014-06-24T00:50:49 clarkb: which violates a fairly important principle. 2014-06-24T00:50:56 clarkb: we'd welcome a change to the defaults :) 2014-06-24T00:51:05 well defaults shouldn't be debug 2014-06-24T00:51:10 but imo you should always test with debug 2014-06-24T00:51:11 clarkb, lifeless: missed e-r part that rely on debug mode, never mind 2014-06-24T00:51:14 because well testing 2014-06-24T00:51:18 clarkb: depends what you're testing 2014-06-24T00:51:44 clarkb: if you can't analyse the fault remotely with the logs we get, the logs are insufficient, because thats production too 2014-06-24T00:51:51 I agree 2014-06-24T00:52:04 but you have to have a realistic approach to the problem 2014-06-24T00:52:08 clarkb: we do :) 2014-06-24T00:52:12 ok 2014-06-24T00:54:42 lifeless: yes, that's all it has now. It had three but then I had to rebase.. 2014-06-24T00:54:56 kevinbenton: https://review.openstack.org/#/c/100374/ is the "debug all the things" patch 2014-06-24T00:55:07 that is one way to do it 2014-06-24T00:55:31 IMO there's just no good guidelines for what should be DEBUG and would should be INFO 2014-06-24T00:55:37 so people throw everything in DEBUG 2014-06-24T00:55:50 tchaypo: ah yeup. anyhow -1 2014-06-24T00:55:57 SpamapS: there is a doc now on the wiki. 2014-06-24T00:56:02 oh good 2014-06-24T00:56:04 SpamapS: the big issue IMO is that most devs are not ops. 2014-06-24T00:56:28 anyhow, I have to write this up. excuse me. 2014-06-24T00:56:31 Reasons its good for us to do !debug 2014-06-24T00:57:27 kevinbenton: the passthrough config is another way, if youre wanting something more permanent I would look into doing passthrough config, but for debugging editing the config like in the debug all the things patch might be easiest 2014-06-24T00:57:50 *** dkehn has joined #tripleo 2014-06-24T00:58:01 greghaynes: wht kevinbenton needs is his Neutron patch run in debug mode. 2014-06-24T00:58:55 *** yamahata has joined #tripleo 2014-06-24T00:59:02 ah 2014-06-24T01:01:32 kevinbenton: diskimage-builder/elements/source-repositories/README.md has some info on how to do a build with a specific ref, basically DIB_REPOREF_neutron=refs/..., and then you can edit the neutron.conf in triple-image-elements similar to review 100374 to enable debugging 2014-06-24T01:01:48 if I understand what youre trying to do this time :p 2014-06-24T01:02:11 greghaynes: in CI:) 2014-06-24T01:02:16 ugh 2014-06-24T01:02:35 * greghaynes finds example 2014-06-24T01:02:43 greghaynes: since I very much doubt kevinbenton has the activation energy to get setup on TripleO dev right now. 2014-06-24T01:02:50 haha, that fair 2014-06-24T01:02:55 *** chuckC has quit IRC 2014-06-24T01:03:04 greghaynes: I suspect a toci patch to fiddle things is probably the most viable route 2014-06-24T01:03:13 yep 2014-06-24T01:03:30 greghaynes: e.g. to bring in the neutron patch 2014-06-24T01:05:23 *** eghobo has joined #tripleo 2014-06-24T01:05:30 *** eghobo has quit IRC 2014-06-24T01:05:46 *** eghobo has joined #tripleo 2014-06-24T01:07:17 *** morganfainberg has quit IRC 2014-06-24T01:07:18 *** morganfainberg_Z is now known as morganfainberg 2014-06-24T01:08:12 *** ddieterly has joined #tripleo 2014-06-24T01:11:33 *** bthurber has quit IRC 2014-06-24T01:13:24 Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 2014-06-24T01:20:34 Could we achieve that witht he usual reverts mechanism 2014-06-24T01:20:38 ? 2014-06-24T01:20:49 see that patch ^ 2014-06-24T01:20:54 I *think* that might work ;) 2014-06-24T01:22:38 I was thinking of the temprevert section above. But you're right, we're not trying to revert anything here, we're trying to pin a particular change. 2014-06-24T01:23:01 Giulio Fidente proposed a change to openstack/tripleo-incubator: Move user's group membership setup in separate script https://review.openstack.org/93954 2014-06-24T01:26:53 Hrm. 2014-06-24T01:27:19 I was going to say that I'd like to see a comment on the line above, as we do in the temprevert section, explaining why this line is here 2014-06-24T01:27:33 but then the demon on my other shoulder told me "nonsense, that's what git blame is for" 2014-06-24T01:27:36 haha 2014-06-24T01:27:41 so now I'm not sure which to listen to 2014-06-24T01:27:45 that shouldnt ever be merged 2014-06-24T01:28:14 that'd be why it's W:-1 2014-06-24T01:29:13 Urr 2014-06-24T01:29:26 https://wiki.openstack.org/w/index.php?title=TripleO&diff=0&oldid=56304 has made me aware that in our page we link to github repos rather than git.openstack 2014-06-24T01:29:32 That seems... contrary to my expectations.. 2014-06-24T01:29:49 fixitfixitfixit 2014-06-24T01:31:46 lifeless: https://etherpad.openstack.org/p/tripleo-heat-chunking-plan 2014-06-24T01:31:52 * tchaypo kicks off a run to test https://review.openstack.org/#/c/101438/3 2014-06-24T01:32:45 oh great, CI fail due to cloud sha being invalid 2014-06-24T01:32:56 greghaynes: Haha. We should totally find the youtube clip of Fry doing that 2014-06-24T01:33:10 :) 2014-06-24T01:34:29 *** cwolferh has quit IRC 2014-06-24T01:34:43 *** cwolferh has joined #tripleo 2014-06-24T01:34:52 There we go. 2014-06-24T01:35:02 tchaypo: www.youtube.com/watch?v=xhy7dXWjpAA 2014-06-24T01:35:39 Argh. Mirror sync: universe/p/percona-xtradb-cluster-5.5 last seen 2014-06-24T01:36:20 StevenK, at 3.36 localtime, that was hard 2014-06-24T01:37:04 hit me so badly I wanted to go to sleep crying for not having fixed much 2014-06-24T01:37:10 Haha 2014-06-24T01:37:17 :) 2014-06-24T01:38:01 * gfidente salute everyone 2014-06-24T01:38:16 *** gfidente has quit IRC 2014-06-24T01:43:50 *** chuckC has joined #tripleo 2014-06-24T01:44:14 *** nosnos has joined #tripleo 2014-06-24T01:45:35 *** e0ne has joined #tripleo 2014-06-24T01:45:54 SpamapS: some quick thoughts there. 2014-06-24T01:46:30 lifeless: yes they were some quick thoughts 2014-06-24T01:46:30 :) 2014-06-24T01:46:37 SpamapS: mine, I meant :) 2014-06-24T01:46:58 *** eguz has joined #tripleo 2014-06-24T01:46:59 *** e0ne has quit IRC 2014-06-24T01:47:21 lifeless: the idea of not changing merge.py has crossed my mind. 2014-06-24T01:47:23 *** e0ne has joined #tripleo 2014-06-24T01:48:37 lifeless: and agree the the all hosts I/O may be generally useful. 2014-06-24T01:50:39 *** eghobo has quit IRC 2014-06-24T01:51:07 *** eguz has quit IRC 2014-06-24T01:52:01 *** e0ne has quit IRC 2014-06-24T01:52:32 lifeless: ty.. I will work backwards anyway and so I might find that I can just manually write the binding document and call make overcloud.yaml as you suggest. 2014-06-24T01:53:21 *** nati_ueno has quit IRC 2014-06-24T01:56:52 boom 2014-06-24T01:57:23 greghaynes: next HA review plox 2014-06-24T01:58:19 lifeless: I'm waiting to see if you open the sprint with "Welcome, thanks for coming. HA ALL THE THINGS!" 2014-06-24T01:58:57 s/.*/\1 NOW/ 2014-06-24T01:59:28 lifeless: I was pondering ALL THE THINGS RIGHT NOW, but didn't want to make you seem needy :-P 2014-06-24T01:59:37 lifeless proposed a change to openstack/tripleo-specs: TripleO and backwards compatibility https://review.openstack.org/102081 2014-06-24T01:59:51 *** jp_at_hp has quit IRC 2014-06-24T02:00:06 now, I need someone to express strong support for ^ right now, to set the tone up right 2014-06-24T02:00:15 also I need food 2014-06-24T02:00:28 so, -> that. 2014-06-24T02:00:31 SpamapS: ^ 2014-06-24T02:00:42 looks like the two I wanted failed CI :/ 2014-06-24T02:02:26 the nova thing? 2014-06-24T02:02:45 bbiaw 2014-06-24T02:02:52 That too, but https://review.openstack.org/#/c/83883/ and the patch it depends on 2014-06-24T02:04:34 on, no, not the nova thing 2014-06-24T02:05:01 one is dns fail and the other might be legit 2014-06-24T02:09:53 Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Add outputs and inputs for all hosts https://review.openstack.org/102082 2014-06-24T02:13:26 lifeless: hrm. ControlVirtualIP complicates things. :-P 2014-06-24T02:14:09 actually hm.. maybe I don't need to do an overcloud.yaml CONTROLSCALE=0 .. I should be able to just use nova-compute-instance.yaml 2014-06-24T02:15:35 *** ddieterly has quit IRC 2014-06-24T02:16:59 lifeless: so just to check that I'm on the right track - I'm currently trying to verify that the change in https://review.openstack.org/#/c/101438/3 does what I expect (create a new VIP on the overcloud control nodes), then I'm going to do another change to make haproxy listen on that vip as well 2014-06-24T02:17:10 lifeless: is that roughly what you were thinking? 2014-06-24T02:18:34 Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Add ControlVirtualIP as an overcloud output https://review.openstack.org/102085 2014-06-24T02:23:14 *** morazi has quit IRC 2014-06-24T02:31:02 *** mestery has quit IRC 2014-06-24T02:31:15 *** mestery has joined #tripleo 2014-06-24T02:33:19 SpamapS: this seems relevant to my interests 2014-06-24T02:34:07 *** pcrews has quit IRC 2014-06-24T02:34:20 SpamapS: in 101438 I'm adding a new VIP to be used for things that want to be accessible from outside the cluster. I'm guessing I should add it as an output too? 2014-06-24T02:34:34 If I understand heat correctly, an "output" can be used as "input" elsewhere in the template, right? 2014-06-24T02:36:32 So I'd be adding it as an output so that eg the bit of the heat template that sets up haproxy could consume it, is what I'm thinking 2014-06-24T02:42:01 *** killer_prince is now known as lazy_prince 2014-06-24T02:42:36 *** lazy_prince is now known as killer_prince 2014-06-24T02:42:49 tchaypo: sounds like it 2014-06-24T02:43:12 tchaypo: I don't think clints work is relevant to your work 2014-06-24T02:43:29 * tchaypo does some reading about heat 2014-06-24T02:43:33 tchaypo: outputs are for inspecting output of the template-as-progra 2014-06-24T02:43:36 program 2014-06-24T02:43:54 how do you do that inspection? 2014-06-24T02:43:55 *** killer_prince is now known as lazy_prince 2014-06-24T02:45:07 looking at https://review.openstack.org/#/c/100848 2014-06-24T02:45:31 *** lazy_prince is now known as killer_prince 2014-06-24T02:45:35 *** e0ne has joined #tripleo 2014-06-24T02:45:57 if I'm understanding correctly - the fn:getatt says to look in "NovaCompute0" for the attribute "networks"; then the fn::select picks out just the attribute of networks called ctlplane; then the fn::select picks out just the 0th element of ctlplane 2014-06-24T02:46:00 is that roughtly right? 2014-06-24T02:49:16 *** e0ne has quit IRC 2014-06-24T02:49:47 *** ddieterly has joined #tripleo 2014-06-24T02:52:57 *** mestery_ has joined #tripleo 2014-06-24T02:54:10 *** nosnos has quit IRC 2014-06-24T02:54:12 *** ddieterly has quit IRC 2014-06-24T02:54:45 *** nosnos has joined #tripleo 2014-06-24T02:55:01 *** mestery has quit IRC 2014-06-24T02:59:06 *** nosnos has quit IRC 2014-06-24T03:00:16 *** untriaged-bot has joined #tripleo 2014-06-24T03:00:16 Untriaged bugs so far: 2014-06-24T03:00:16 https://bugs.launchpad.net/diskimage-builder/+bug/1332458 2014-06-24T03:00:17 Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] 2014-06-24T03:00:18 https://bugs.launchpad.net/diskimage-builder/+bug/1332521 2014-06-24T03:00:19 Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] 2014-06-24T03:00:20 *** untriaged-bot has quit IRC 2014-06-24T03:00:28 tchaypo: that review doesn't have outputs 2014-06-24T03:01:12 100848? I know, I'm haring off on a tangent looking at heat template changes to learn more about it 2014-06-24T03:01:41 gets 2014-06-24T03:01:44 yes 2014-06-24T03:02:10 Fn::Join seems pretty simple. OS::Heat::RandomString looks handy. 2014-06-24T03:02:37 *** vinsh has joined #tripleo 2014-06-24T03:02:39 *** e0ne has joined #tripleo 2014-06-24T03:03:38 When you use ref: to do a lookup... I'm going to have to figure out the scoping rules later. 2014-06-24T03:05:54 *** nosnos has joined #tripleo 2014-06-24T03:07:03 okay steamophiles, the next "community choice" group has only one correct answer - it's the group with kerbal space program at -40% and simcity4 for $5 2014-06-24T03:07:37 *** e0ne has quit IRC 2014-06-24T03:11:25 *** akuznetsov has joined #tripleo 2014-06-24T03:11:36 *** ddieterly has joined #tripleo 2014-06-24T03:13:28 tchaypo: in the REST API to get the outputs you fetch the stack 2014-06-24T03:15:23 lifeless: ping 2014-06-24T03:15:56 tchaypo: ugh.. that is a choice that leads to me never leaving the house for 3 weeks. 2014-06-24T03:16:03 *** ddieterly has quit IRC 2014-06-24T03:16:32 SpamapS: sounds good, right? 2014-06-24T03:17:13 When I'm using the command-line client and I "heat stack-show undercloud" and there's one part of the output that has a list of key:value pairs - those are the outputs? 2014-06-24T03:17:46 tchaypo: I can't even begin to tell you how poisonous that would be to my life. ;) 2014-06-24T03:17:49 I can't build anything today 2014-06-24T03:17:52 + heat stack-create -e /home/polleyj/.cache/tripleo/undercloud-env.json -t 360 -f /home/polleyj/.cache/tripleo/tripleo-heat-templates/undercloud-vm-ironic.yaml undercloud 2014-06-24T03:17:55 ERROR: The Parameter (PowerManager) was not defined in template. 2014-06-24T03:17:55 * SpamapS is a recovering gamaholic 2014-06-24T03:18:05 please tell me I'm not the only one seeing that.. 2014-06-24T03:18:20 you're not the only one seeing that 2014-06-24T03:18:24 * SpamapS lies 2014-06-24T03:18:33 *** matsuhashi has quit IRC 2014-06-24T03:19:05 *** matsuhashi has joined #tripleo 2014-06-24T03:21:44 SpamapS: you were addicted to gama ? 2014-06-24T03:24:13 *** matsuhashi has quit IRC 2014-06-24T03:27:29 *** ramishra has joined #tripleo 2014-06-24T03:29:44 fabbione: pong 2014-06-24T03:30:35 tchaypo: you've got a non-ironic parameter in your heat env 2014-06-24T03:30:40 tchaypo: and heat no like that 2014-06-24T03:31:14 and importantly 2014-06-24T03:31:23 ~/.devtestrc has USE_IRONIC=1 2014-06-24T03:31:28 * greghaynes is sad https://review.openstack.org/#/c/93104/ got abandoned 2014-06-24T03:31:49 given that its needed for HA and the one it was abandoned for looks not close to mergable 2014-06-24T03:32:19 * tchaypo turns that off and tries again 2014-06-24T03:34:44 *** nosnos has quit IRC 2014-06-24T03:35:18 *** nosnos has joined #tripleo 2014-06-24T03:37:18 *** ramishra_ has joined #tripleo 2014-06-24T03:38:27 *** eghobo has joined #tripleo 2014-06-24T03:38:32 *** ramishra has quit IRC 2014-06-24T03:39:37 *** nosnos has quit IRC 2014-06-24T03:45:38 *** e0ne has joined #tripleo 2014-06-24T03:47:05 tchaypo: likely caused by changing that parameter without rm'ing your -env.json 2014-06-24T03:47:17 I have been meaning to write up a bug report for that 2014-06-24T03:47:49 *** jml has quit IRC 2014-06-24T03:48:02 *** CaptTofu_ has quit IRC 2014-06-24T03:48:07 ok, offlineish 2014-06-24T03:48:11 *** e0ne has quit IRC 2014-06-24T03:52:24 *** michchap has quit IRC 2014-06-24T03:52:37 *** michchap has joined #tripleo 2014-06-24T03:55:30 *** jml has joined #tripleo 2014-06-24T04:04:41 *** GheRivero has quit IRC 2014-06-24T04:04:54 *** davidlenwell_ has joined #tripleo 2014-06-24T04:04:58 *** GheRivero has joined #tripleo 2014-06-24T04:05:13 *** davidlenwell has quit IRC 2014-06-24T04:05:53 *** SlickNik has quit IRC 2014-06-24T04:06:08 *** SlickNik has joined #tripleo 2014-06-24T04:10:04 *** matsuhashi has joined #tripleo 2014-06-24T04:11:38 *** ddieterly has joined #tripleo 2014-06-24T04:16:47 *** ddieterly has quit IRC 2014-06-24T04:19:33 *** nosnos has joined #tripleo 2014-06-24T04:24:32 *** killer_prince is now known as lazy_prince 2014-06-24T04:25:58 *** e0ne has joined #tripleo 2014-06-24T04:29:12 lifeless, greghaynes: sorry, I had to leave in a rush earlier for a meeting 2014-06-24T04:29:40 lifeless, greghaynes: what was the consensus on the best way to have the CI test my neutron patch on tripleo with debug enabled? 2014-06-24T04:30:35 *** e0ne has quit IRC 2014-06-24T04:37:07 kevinbenton: once our CI is unwedged from a new trusty image I think https://review.openstack.org/#/c/102073/ will work 2014-06-24T04:38:12 greghaynes: cool. what does unwedged mean in this context? 2014-06-24T04:40:47 kevinbenton: a new trusty cloudimg got released, I think were caching one (but not both) of the image or sha256sums... so we keep failing tests with invalid sha 2014-06-24T04:41:13 *** lazy_prince has quit IRC 2014-06-24T04:41:22 *** morganfainberg_L has joined #tripleo 2014-06-24T04:41:35 greghaynes: oh okay. is this something that will clear up soon or should I check back tomorrow? 2014-06-24T04:42:11 I think I can clear it up, will take a minute 2014-06-24T04:42:31 lifeless: cool. thanks 2014-06-24T04:42:31 there is a patchset which does cache-busting requests 2014-06-24T04:46:14 if I can find it... doesn't look like its merged 2014-06-24T04:46:32 88084 2014-06-24T04:46:48 so thats rechecking now 2014-06-24T04:47:06 in a few minutes re-run greghaynes's one. It will be behind the other and should DTRT :) 2014-06-24T04:49:26 DTRT? 2014-06-24T04:49:33 kevinbenton: Do The Right Thing 2014-06-24T04:49:57 *** pcrews has joined #tripleo 2014-06-24T04:50:06 oooh, a new acronym :-) 2014-06-24T04:51:49 Ng: ^ might want to finish that patch off :> in your copious spare time 2014-06-24T04:57:52 *** rakesh_hs4 has joined #tripleo 2014-06-24T04:58:01 *** nati_ueno has joined #tripleo 2014-06-24T05:00:48 *** eghobo has quit IRC 2014-06-24T05:01:04 *** eghobo has joined #tripleo 2014-06-24T05:01:51 *** CaptTofu_ has joined #tripleo 2014-06-24T05:02:01 *** bmahalakshmi has joined #tripleo 2014-06-24T05:06:17 *** cwolferh has quit IRC 2014-06-24T05:06:27 *** CaptTofu_ has quit IRC 2014-06-24T05:07:27 *** ajc_ has joined #tripleo 2014-06-24T05:07:54 lifeless: do you have a link to some logs from a recent job that passed? I want to compare some outputs 2014-06-24T05:09:04 *** lazy_prince has joined #tripleo 2014-06-24T05:11:39 *** ddieterly has joined #tripleo 2014-06-24T05:16:02 *** ddieterly has quit IRC 2014-06-24T05:16:17 Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Separate Overcloud into plan and stack https://review.openstack.org/102106 2014-06-24T05:22:24 *** lazy_prince is now known as killer_prince 2014-06-24T05:29:52 *** amitpp has joined #tripleo 2014-06-24T05:41:37 *** rwsu has quit IRC 2014-06-24T05:49:05 *** gcha has quit IRC 2014-06-24T05:51:09 *** morganfainberg_L has quit IRC 2014-06-24T06:02:37 *** rdopieralski has joined #tripleo 2014-06-24T06:04:49 *** robsparker has quit IRC 2014-06-24T06:11:41 *** ddieterly has joined #tripleo 2014-06-24T06:12:32 That moment when people insist you remove the "as of $DATE we support" from your patch, so you do, then people say "this should say "as of $DATE"" 2014-06-24T06:12:41 That other moment when you realise you have nested ""s 2014-06-24T06:15:51 *** dshulyak_ has joined #tripleo 2014-06-24T06:15:53 *** killer_prince is now known as lazy_prince 2014-06-24T06:16:04 *** ddieterly has quit IRC 2014-06-24T06:16:14 *** ddieterly has joined #tripleo 2014-06-24T06:21:25 *** ddieterly has quit IRC 2014-06-24T06:22:11 oh wait 2014-06-24T06:22:20 easycount is produced by the AEC, not by a third party? 2014-06-24T06:22:31 fuck this "commercial in confidence" shit then. that's a fucking crock. 2014-06-24T06:25:14 James Polley proposed a change to openstack/tripleo-incubator: Add details of which OS releases are tested in CI https://review.openstack.org/92749 2014-06-24T06:25:38 tchaypo: AEC is a commision, so has no commercial interests, right? 2014-06-24T06:25:56 huh. 2014-06-24T06:26:02 * tchaypo forgot to switch windows 2014-06-24T06:26:07 Haha 2014-06-24T06:26:11 StevenK: incorrect, they run elections for profit 2014-06-24T06:26:33 according to latest post on linux-aus, total of like 1.5 million in profit each year, compared with 160 million to run a federal election 2014-06-24T06:26:34 All elections, or the state/federal ones are not for profit? 2014-06-24T06:28:40 *** mrunge has joined #tripleo 2014-06-24T06:30:07 *** jang has joined #tripleo 2014-06-24T06:31:03 *** jml has quit IRC 2014-06-24T06:33:13 *** jml has joined #tripleo 2014-06-24T06:38:17 *** gcha has joined #tripleo 2014-06-24T06:44:09 *** gfidente has joined #tripleo 2014-06-24T06:44:40 *** ramishra_ has quit IRC 2014-06-24T06:46:20 *** ramishra has joined #tripleo 2014-06-24T06:46:22 Woo, progress. 2014-06-24T06:46:28 the state and federal ones are not for profit 2014-06-24T06:46:37 My setup-neutron in os-cloud-config can create nets. 2014-06-24T06:46:41 they're required by law to run some of the union elections; not sure if they're for profit or not 2014-06-24T06:49:53 *** CaptTofu_ has joined #tripleo 2014-06-24T06:50:48 guys, still there? 2014-06-24T06:50:54 I left you a while back 2014-06-24T06:51:02 gfidente: welcome back 2014-06-24T06:51:07 gfidente: i hope you got some sleep 2014-06-24T06:51:11 I did! 2014-06-24T06:51:16 gfidente: but your comment made me cry. 2014-06-24T06:51:17 have to buy some cofee though 2014-06-24T06:51:29 you don't have coffee supplies at home? 2014-06-24T06:51:32 ... are you at home? 2014-06-24T06:51:45 those are finished 2014-06-24T06:51:48 speaking of supplies at home, i need to contact my dealer 2014-06-24T06:52:17 tchaypo, you mean the comment on which distro version we support? 2014-06-24T06:52:35 yeah, the comment suggestiong we should have "as of $DATE ..." 2014-06-24T06:52:51 not your fault, it's how i wanted it, but consensus seems to be that it's better without 2014-06-24T06:53:08 wait let me check the reviews :) 2014-06-24T06:53:23 I wanted to ask though if you know anything about SHA failing for the ubuntu images? 2014-06-24T06:53:25 patchset 8 is where it changed :) 2014-06-24T06:53:34 oh, i think greghaynes said something about that 2014-06-24T06:53:58 *** ifarkas has joined #tripleo 2014-06-24T06:54:25 *** CaptTofu_ has quit IRC 2014-06-24T06:56:11 tchaypo, oh I saw the comments on patch 8 2014-06-24T06:57:04 I think I can be happy if we use Currently too :) 2014-06-24T06:58:03 phew 2014-06-24T06:59:05 *** eguz has joined #tripleo 2014-06-24T07:01:38 gfidente: if you have a momeny, I'd appreciate a review on https://review.openstack.org/#/c/97657/5 2014-06-24T07:02:45 bnemec and i both feel it's too early to be breaking backwards compatibility, 2014-06-24T07:03:14 *** eghobo has quit IRC 2014-06-24T07:04:30 but we were involved in putting the code in its current state, maybe we're being too conservative 2014-06-24T07:05:59 *** eguz has quit IRC 2014-06-24T07:09:57 tchaypo, breaking backwards means you're not sourcing the file 2014-06-24T07:10:05 but script is not overwriting the passwords anyway 2014-06-24T07:10:39 *** openstackgerrit has quit IRC 2014-06-24T07:10:48 not sourcing the file in $PWD; sure; but that only breaks people who were using devtest before my patch in march, and had password files created somewhere else, and are used to using them 2014-06-24T07:11:25 his script doesn't overwrite the passwords; t:q 2014-06-24T07:11:46 so the thing is that I think we should make the passwords customizable 2014-06-24T07:12:25 bnemec tweaked the scripts so that for any password already in the file it will be left untouched, but any new passwords added to the script will be appended to the file 2014-06-24T07:12:43 yep which I think is great 2014-06-24T07:12:48 so calling it on an exisitng file will touch nothing, but when we have changes such as the recent one that added a ceilometer password, they get added to existing files 2014-06-24T07:13:12 *** pblaho has joined #tripleo 2014-06-24T07:13:45 so I am of the idea that this is an actual feature, not only something good for compatibility 2014-06-24T07:13:50 but I'd make it more explicit 2014-06-24T07:14:23 * tchaypo waits for your comment on review 2014-06-24T07:14:43 I'd write the file in a location which the user can customize, but keep the change 2014-06-24T07:14:54 ok sorry 2014-06-24T07:14:57 commenting there 2014-06-24T07:14:58 and have $TRIPLEO_ROOT be the default location? 2014-06-24T07:15:10 yes 2014-06-24T07:15:33 I can get behind that. It means that existing devs with existing files will have to, just once, move their files into $TRIPLEO_ROOT, or export something in their ~/.devtestrc 2014-06-24T07:15:33 *** nati_ueno has quit IRC 2014-06-24T07:16:13 yes I understand that, but in its current state it is sort of 'hidden' while I think it is a good feature to have 2014-06-24T07:18:44 _1 2014-06-24T07:18:46 urr 2014-06-24T07:18:48 +1 also 2014-06-24T07:18:51 if I can type 2014-06-24T07:18:58 in other news it's time for me to head pubwards 2014-06-24T07:21:34 *** jcoufal has joined #tripleo 2014-06-24T07:32:09 *** beekneemech has joined #tripleo 2014-06-24T07:32:17 *** bnemec has quit IRC 2014-06-24T07:34:54 *** pblaho is now known as pblaho|afk 2014-06-24T07:36:59 *** dshulyak_ has quit IRC 2014-06-24T07:37:24 *** jprovazn has joined #tripleo 2014-06-24T07:39:09 *** andreaf has joined #tripleo 2014-06-24T07:45:57 *** jtomasek has joined #tripleo 2014-06-24T07:52:46 *** pblaho|afk is now known as pblaho 2014-06-24T08:05:01 *** jp_at_hp has joined #tripleo 2014-06-24T08:12:23 *** amitpp has quit IRC 2014-06-24T08:14:43 *** fandi has joined #tripleo 2014-06-24T08:19:46 *** akrivoka has joined #tripleo 2014-06-24T08:24:03 *** dtantsur|afk is now known as dtantsur 2014-06-24T08:29:37 *** bogdando has quit IRC 2014-06-24T08:30:37 *** ramishra has quit IRC 2014-06-24T08:32:20 *** ramishra has joined #tripleo 2014-06-24T08:33:32 *** jcoufal has quit IRC 2014-06-24T08:34:16 *** jcoufal has joined #tripleo 2014-06-24T08:34:49 *** athomas has joined #tripleo 2014-06-24T08:38:09 *** CaptTofu_ has joined #tripleo 2014-06-24T08:38:51 *** ramishra has quit IRC 2014-06-24T08:42:37 *** CaptTofu_ has quit IRC 2014-06-24T08:42:45 *** martyntaylor has joined #tripleo 2014-06-24T08:46:21 *** martyntaylor has quit IRC 2014-06-24T08:50:57 *** martyntaylor has joined #tripleo 2014-06-24T08:59:18 *** pblaho is now known as pblaho|afk 2014-06-24T09:00:18 *** untriaged-bot has joined #tripleo 2014-06-24T09:00:18 Untriaged bugs so far: 2014-06-24T09:00:19 https://bugs.launchpad.net/diskimage-builder/+bug/1332458 2014-06-24T09:00:20 Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] 2014-06-24T09:00:21 https://bugs.launchpad.net/diskimage-builder/+bug/1332521 2014-06-24T09:00:22 Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] 2014-06-24T09:00:23 *** untriaged-bot has quit IRC 2014-06-24T09:08:23 *** pblaho|afk is now known as pblaho 2014-06-24T09:10:54 *** e0ne has joined #tripleo 2014-06-24T09:11:55 *** akrivoka has quit IRC 2014-06-24T09:16:07 *** pelix has joined #tripleo 2014-06-24T09:23:04 *** ramishra has joined #tripleo 2014-06-24T09:23:08 morning 2014-06-24T09:23:50 * Ng hopes his neighbour's kids continue to practice their basic violin at this volume all day, it's extremely soothing to hear someone violate a shrill musical instrument when you've managed about 3 hours of sleep 2014-06-24T09:24:01 hang on, nope, that's not right at all ;) 2014-06-24T09:24:07 *** rdopieralski has quit IRC 2014-06-24T09:28:01 *** ramishra has quit IRC 2014-06-24T09:31:02 Ng: Sarah is learning the violin, and it sounds rather lovely when she practises :-) 2014-06-24T09:31:22 StevenK: she's not 6 and physically incapable of playing a violin nicely :) 2014-06-24T09:32:01 I now feel bad for all the people I annoyed when I was learning violin as a little kid :) 2014-06-24T09:33:13 Ng: When are you buying a quarter size violin for your eldest as payback? :-) 2014-06-24T09:33:39 StevenK: never, I will encourage my kids to learn things that can be trivially electronicsised and thus played with headphones ;) 2014-06-24T09:33:51 piano, drums, that sort of thing 2014-06-24T09:33:56 Electronic violins also exist 2014-06-24T09:34:30 they do indeed, but I'm not sure there are cheap ones 2014-06-24T09:34:58 but it's a lot of years since I was into violin 2014-06-24T09:35:03 Musical instrument and cheap and diametrically opposed concepts. 2014-06-24T09:35:11 s/cheap and/cheap are/ 2014-06-24T09:35:31 *** akrivoka has joined #tripleo 2014-06-24T09:35:59 I'm not sure I agree, cheap keyboards aren't unplayable, and the cheap electronic drumkits are really surprisingly good these days 2014-06-24T09:36:16 when I was taking drum lessons, they were thousands of pounds, but you can get an entirely acceptable kit for under £200 now 2014-06-24T09:36:25 Huh 2014-06-24T09:37:06 One of the friends of the family had a cheap electronic drumkit when I was growing up, and it was utterly terrible. 2014-06-24T09:38:14 I'm actually hoping I can get the kids into drumming, because I really miss playing and this house is the first time in a couple of decades that I've legitimately had space to own a kit :) 2014-06-24T09:38:39 It's London, lies. :-P 2014-06-24T09:39:01 hehe 2014-06-24T09:39:19 StevenK: well once the kids are a bit older, I can start taking over their play room with more suitable toys :D 2014-06-24T09:39:25 games consoles, drumkits 2014-06-24T09:39:46 programmable lego! 2014-06-24T09:40:02 Hahaha 2014-06-24T09:40:33 chemistry sets! 2014-06-24T09:41:03 *** bmahalakshmi has quit IRC 2014-06-24T09:41:23 Oooh, what could possibly go wrong 2014-06-24T09:41:30 nothing* 2014-06-24T09:41:43 *** amitpp has joined #tripleo 2014-06-24T09:41:56 Rike will hate it, but I will claim that all of the above are good for the kids ;) 2014-06-24T09:42:18 How do you get Copper Sulfate stains out of the carpet? :-P 2014-06-24T09:43:17 *** bmahalakshmi has joined #tripleo 2014-06-24T09:44:11 StevenK: easy, we don't have carpet on that floor of the house ;) 2014-06-24T09:44:37 Ng: Tiles, floorboards, *shiver* lino? 2014-06-24T09:44:54 StevenK: the funky interlocking boards 2014-06-24T09:45:10 one continuous, smooth surface across four rooms and a hallway \o/ 2014-06-24T09:45:21 Wait 2014-06-24T09:45:46 so I have my son (who is 3) playing a lot with a kalimba and the kaossilator 2014-06-24T09:45:55 Give it 5 to 7 years, and it probably won't be so smooth 2014-06-24T09:46:05 I'm not sure the kaossilator was a good move 2014-06-24T09:46:07 StevenK: probably true :) 2014-06-24T09:46:22 gfidente: ♥ kalimbas 2014-06-24T09:46:37 dict can't spell anything with those two words 2014-06-24T09:47:20 Ng, indeed it is a very nice sounding thing and he enjoys the sound it makes 2014-06-24T09:47:40 the kaossilator instead, drives us all crazy 2014-06-24T09:47:41 gfidente: I think I shall get one for my 3 year old :) 2014-06-24T09:47:47 *** openstackgerrit has joined #tripleo 2014-06-24T09:47:55 yeah they make a really nice sound 2014-06-24T09:48:20 and are kind of "easy" to use 2014-06-24T09:50:48 *** dtantsur is now known as dtantsur|lunch 2014-06-24T09:53:14 Chris Jones proposed a change to openstack/diskimage-builder: Last ditch effort to correct a wrong shasum. https://review.openstack.org/88084 2014-06-24T09:54:25 *** fandi has quit IRC 2014-06-24T10:00:54 *** rcarrillocruz has joined #tripleo 2014-06-24T10:03:02 *** rcarrill` has quit IRC 2014-06-24T10:05:14 *** hewbrocca has joined #tripleo 2014-06-24T10:14:21 *** athomas has quit IRC 2014-06-24T10:19:02 *** nosnos has quit IRC 2014-06-24T10:19:33 guys, regarding the overcloud networking, I noticed instances get by default a float on the same ctlplane we use for undercloud/overcloud interconnection 2014-06-24T10:19:37 *** nosnos has joined #tripleo 2014-06-24T10:20:12 I'm not sure if this is customizable though, is it possible to avoid that? anyone has a pointer to the relevant settings? 2014-06-24T10:23:14 *** athomas has joined #tripleo 2014-06-24T10:23:31 *** matsuhashi has quit IRC 2014-06-24T10:24:07 *** matsuhashi has joined #tripleo 2014-06-24T10:24:09 *** nosnos has quit IRC 2014-06-24T10:24:47 *** amitpp has quit IRC 2014-06-24T10:26:24 *** CaptTofu_ has joined #tripleo 2014-06-24T10:28:08 *** matsuhashi has quit IRC 2014-06-24T10:29:53 Jan Provaznik proposed a change to openstack/tripleo-image-elements: Check if stunnel.connect_ip is set https://review.openstack.org/102177 2014-06-24T10:31:03 *** CaptTofu_ has quit IRC 2014-06-24T10:33:26 *** e0ne has quit IRC 2014-06-24T10:33:36 *** markmc has joined #tripleo 2014-06-24T10:41:36 *** pblaho is now known as pblaho|afk 2014-06-24T10:47:54 Ladislav Smola proposed a change to openstack/tripleo-image-elements: Allowing to optionally run sensors-detect https://review.openstack.org/102182 2014-06-24T10:56:39 Jan Provaznik proposed a change to openstack/tripleo-heat-templates: Add parameters for setting up keystone keys/certs in undercloud https://review.openstack.org/102183 2014-06-24T10:57:36 Jan Provaznik proposed a change to openstack/tripleo-incubator: Generate overcloud keystone keys/certs https://review.openstack.org/95101 2014-06-24T11:00:06 A change was merged to openstack/tripleo-incubator: Require and set mode 0600 on heat env caches https://review.openstack.org/100994 2014-06-24T11:01:02 *** e0ne has joined #tripleo 2014-06-24T11:07:23 *** jprovazn has quit IRC 2014-06-24T11:11:55 *** rdopieralski has joined #tripleo 2014-06-24T11:15:59 good morning 2014-06-24T11:16:33 *** xuhaiwei has quit IRC 2014-06-24T11:18:36 *** yamahata has quit IRC 2014-06-24T11:21:57 please review https://review.openstack.org/#/c/79289/ (Adds local storage Boot support on PXE failures). It has under gone multiple review and is now ready for merge.. 2014-06-24T11:26:44 *** bmahalakshmi has quit IRC 2014-06-24T11:28:18 Tomas Sedovic proposed a change to openstack/tripleo-specs: Remove merge.py from TripleO Heat Templates https://review.openstack.org/97939 2014-06-24T11:29:16 *** bmahalakshmi has joined #tripleo 2014-06-24T11:31:06 *** jprovazn has joined #tripleo 2014-06-24T11:36:21 *** pblaho|afk is now known as pblaho 2014-06-24T11:43:35 *** bmahalakshmi has quit IRC 2014-06-24T11:44:59 *** bmahalakshmi has joined #tripleo 2014-06-24T11:50:49 *** ddieterly has joined #tripleo 2014-06-24T11:53:11 *** e0ne_ has joined #tripleo 2014-06-24T11:54:34 Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 2014-06-24T11:55:31 lifeless: greghaynes and who else it concerned ^ 2014-06-24T11:56:35 *** e0ne has quit IRC 2014-06-24T11:57:09 *** weshay has joined #tripleo 2014-06-24T11:59:21 *** ramishra has joined #tripleo 2014-06-24T11:59:42 *** CaptTofu_ has joined #tripleo 2014-06-24T12:00:24 *** morazi has joined #tripleo 2014-06-24T12:01:16 *** rakesh_hs4 has quit IRC 2014-06-24T12:01:46 *** jdob has joined #tripleo 2014-06-24T12:07:33 *** rlandy has joined #tripleo 2014-06-24T12:16:40 *** ddieterly has quit IRC 2014-06-24T12:19:00 *** dprince has joined #tripleo 2014-06-24T12:20:28 *** dtantsur|lunch is now known as dtantsur 2014-06-24T12:21:09 *** bmahalakshmi has quit IRC 2014-06-24T12:31:19 Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 2014-06-24T12:36:06 *** edmund1 has joined #tripleo 2014-06-24T12:39:11 *** dshulyak_ has joined #tripleo 2014-06-24T12:46:42 d0ugal: ping 2014-06-24T12:46:52 pblaho: hey 2014-06-24T12:47:22 d0ugal: hi 2014-06-24T12:47:44 will you be ok if I do +A on https://review.openstack.org/#/c/94523/ if gates pass? 2014-06-24T12:48:25 pblaho: sure 2014-06-24T12:48:48 pblaho: my suggestion was only minor, which is why I went with a +1 still :) 2014-06-24T12:48:52 d0ugal: ok... I was just asking ... 2014-06-24T12:48:56 thanks! 2014-06-24T12:48:57 d0ugal: yeah... 2014-06-24T12:49:04 np 2014-06-24T12:49:12 * pblaho hopes gates will be ok 2014-06-24T12:50:33 *** ajc_ has quit IRC 2014-06-24T12:53:53 did we fix the ubuntu SHA mismatch? am I safe to recheck? 2014-06-24T12:56:27 *** dshulyak_ has quit IRC 2014-06-24T12:59:30 *** julim has joined #tripleo 2014-06-24T12:59:33 *** ddieterly has joined #tripleo 2014-06-24T12:59:47 *** sseago has quit IRC 2014-06-24T12:59:51 *** sseago_ has joined #tripleo 2014-06-24T13:00:36 *** Matt1 has joined #tripleo 2014-06-24T13:13:33 *** lazy_prince has quit IRC 2014-06-24T13:13:52 *** andreaf has quit IRC 2014-06-24T13:20:11 *** sseago__ has joined #tripleo 2014-06-24T13:20:15 *** sseago_ has quit IRC 2014-06-24T13:30:09 *** pblaho_ has joined #tripleo 2014-06-24T13:30:27 *** pblaho has quit IRC 2014-06-24T13:31:39 *** casanch1 has joined #tripleo 2014-06-24T13:38:40 *** ramishra has quit IRC 2014-06-24T13:39:14 *** ramishra has joined #tripleo 2014-06-24T13:39:33 *** ramishra has quit IRC 2014-06-24T13:39:45 *** ramishra has joined #tripleo 2014-06-24T13:41:38 *** pblaho__ has joined #tripleo 2014-06-24T13:41:47 *** pblaho__ is now known as pblaho 2014-06-24T13:42:43 *** mestery_ is now known as mestery 2014-06-24T13:45:37 *** pblaho_ has quit IRC 2014-06-24T13:52:46 *** pblaho is now known as pblaho|afk 2014-06-24T13:56:01 *** martyntaylor has left #tripleo 2014-06-24T13:57:58 *** pblaho|afk is now known as pblaho 2014-06-24T13:58:22 *** pblaho has quit IRC 2014-06-24T14:04:46 morning 2014-06-24T14:04:52 *** beekneemech is now known as bnemec 2014-06-24T14:08:40 lifeless: are you traveling right now? isn't it like the middle of the night where you are? 2014-06-24T14:08:57 also, morning :) 2014-06-24T14:12:54 *** edmund1 has quit IRC 2014-06-24T14:16:08 Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 2014-06-24T14:16:22 jdob: I'm in sunnyvale right now 2014-06-24T14:16:41 ahh, that makes more sense :) 2014-06-24T14:17:39 Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 2014-06-24T14:30:20 *** cwolferh has joined #tripleo 2014-06-24T14:33:53 *** killer_prince has joined #tripleo 2014-06-24T14:37:08 *** andreaf has joined #tripleo 2014-06-24T14:40:09 Phil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration when UI not specified https://review.openstack.org/102018 2014-06-24T14:40:33 *** killer_prince has quit IRC 2014-06-24T14:44:27 Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 2014-06-24T14:44:39 *** killer_prince has joined #tripleo 2014-06-24T14:46:07 *** jprovazn is now known as jprovazn_afk 2014-06-24T14:52:48 Phil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration when UI not specified https://review.openstack.org/102018 2014-06-24T14:53:44 Phil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration https://review.openstack.org/102018 2014-06-24T14:54:56 *** bthurber has joined #tripleo 2014-06-24T14:58:11 *** morganfainberg_L has joined #tripleo 2014-06-24T14:59:08 *** andrearosa has quit IRC 2014-06-24T15:00:06 *** nati_ueno has joined #tripleo 2014-06-24T15:00:16 *** untriaged-bot has joined #tripleo 2014-06-24T15:00:16 Untriaged bugs so far: 2014-06-24T15:00:16 https://bugs.launchpad.net/diskimage-builder/+bug/1332458 2014-06-24T15:00:17 Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] 2014-06-24T15:00:18 https://bugs.launchpad.net/tripleo/+bug/1333649 2014-06-24T15:00:19 Launchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress] 2014-06-24T15:00:22 https://bugs.launchpad.net/diskimage-builder/+bug/1332521 2014-06-24T15:00:22 *** untriaged-bot has quit IRC 2014-06-24T15:00:23 Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] 2014-06-24T15:01:30 *** chuckC has quit IRC 2014-06-24T15:01:59 *** marun has joined #tripleo 2014-06-24T15:11:13 *** casanch1 has quit IRC 2014-06-24T15:11:16 *** jcoufal has quit IRC 2014-06-24T15:19:50 *** ramishra has quit IRC 2014-06-24T15:30:03 *** ramishra has joined #tripleo 2014-06-24T15:32:00 all, are there plans to switch from mysql-binary to the distro packaged versions of mysql and/or mariadb? 2014-06-24T15:33:49 *** marun has quit IRC 2014-06-24T15:37:36 *** marun has joined #tripleo 2014-06-24T15:40:54 lifeless ^^ ? 2014-06-24T15:41:55 gfidente: I think you can do a build like that if you want to, there's certainly been patch sets up to permit that 2014-06-24T15:42:16 lifeless, yeah I've seen elements for packaged mariadb as well 2014-06-24T15:43:02 but is there a plan to switch per default? 2014-06-24T15:45:01 I don't think so 2014-06-24T15:45:22 percona pacakges aren't available everywhere yet 2014-06-24T15:46:42 got it, thanks 2014-06-24T15:49:37 *** ramishra has quit IRC 2014-06-24T15:50:09 *** ramishra has joined #tripleo 2014-06-24T15:52:37 *** chuckC has joined #tripleo 2014-06-24T15:53:20 Kevin Benton proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 2014-06-24T15:53:39 *** eghobo has joined #tripleo 2014-06-24T15:53:46 *** eghobo has quit IRC 2014-06-24T15:54:38 *** ramishra has quit IRC 2014-06-24T15:55:00 *** eghobo has joined #tripleo 2014-06-24T15:55:21 *** jtomasek has quit IRC 2014-06-24T15:56:24 lifeless: shouldn’t this line be gone now? https://github.com/openstack-infra/tripleo-ci/blob/master/toci_devtest.sh#L35 2014-06-24T15:59:01 *** yamahata has joined #tripleo 2014-06-24T15:59:17 kevinbenton: ah - the code for that checks the commit # and merge logic, but yes, lets remove it 2014-06-24T16:00:24 lifeless: does that code affect jobs run by ‘check experimental'? 2014-06-24T16:00:52 kevinbenton: everything but is has some smarts. 2014-06-24T16:01:01 lifeless proposed a change to openstack-infra/tripleo-ci: Unrevert Neutron's ovs-agent patch https://review.openstack.org/102281 2014-06-24T16:01:14 re-un-revert. ;) 2014-06-24T16:01:20 kevinbenton: you can however rebase your test patch on top of ^ to ensure you're not affected by it 2014-06-24T16:01:21 *** eghobo has quit IRC 2014-06-24T16:01:24 Ihave a call now, sorry 2014-06-24T16:01:31 *** ramishra has joined #tripleo 2014-06-24T16:01:39 *** eghobo has joined #tripleo 2014-06-24T16:01:40 lifeless: no prob. i just commented it out on mine 2014-06-24T16:07:17 *** martyntaylor has joined #tripleo 2014-06-24T16:11:34 *** martyntaylor has quit IRC 2014-06-24T16:16:38 *** dtantsur is now known as dtantsur|afk 2014-06-24T16:17:42 *** nati_ueno has quit IRC 2014-06-24T16:18:35 *** e0ne_ has quit IRC 2014-06-24T16:18:47 *** andreaf has quit IRC 2014-06-24T16:19:12 *** andreaf has joined #tripleo 2014-06-24T16:31:37 *** rdopieralski has quit IRC 2014-06-24T16:31:59 *** martyntaylor has joined #tripleo 2014-06-24T16:32:04 *** martyntaylor has left #tripleo 2014-06-24T16:33:16 *** yjiang5 is now known as yjiang5_away 2014-06-24T16:39:48 A change was merged to openstack/tripleo-incubator: Allow use of dib-utils https://review.openstack.org/100003 2014-06-24T16:41:30 *** jcoufal has joined #tripleo 2014-06-24T16:42:38 *** noslzzp has quit IRC 2014-06-24T16:43:40 *** noslzzp has joined #tripleo 2014-06-24T16:46:22 A change was merged to openstack/diskimage-builder: avoid failure if /lib/firmware doesn't exist https://review.openstack.org/100026 2014-06-24T16:49:11 *** ramishra has quit IRC 2014-06-24T16:51:24 *** rcarrill` has joined #tripleo 2014-06-24T16:51:31 *** rcarrillocruz has quit IRC 2014-06-24T16:51:53 *** yjiang5_away is now known as yjiang5 2014-06-24T16:54:36 *** pelix has quit IRC 2014-06-24T16:56:09 *** athomas has quit IRC 2014-06-24T16:56:24 *** ramishra has joined #tripleo 2014-06-24T16:58:36 *** dshulyak_ has joined #tripleo 2014-06-24T17:02:13 *** mrunge has quit IRC 2014-06-24T17:06:49 *** dtantsur|afk is now known as dtantsur 2014-06-24T17:07:06 Endre Karlson proposed a change to openstack/tripleo-image-elements: Allow ha_mode to be set. https://review.openstack.org/102292 2014-06-24T17:12:52 A change was merged to openstack/tripleo-image-elements: Switch from upstart to os-refresh-config for MySQL server-id's https://review.openstack.org/93039 2014-06-24T17:12:53 Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 2014-06-24T17:15:12 *** akrivoka has quit IRC 2014-06-24T17:18:34 A change was merged to openstack/tripleo-image-elements: Do not use cat in sysctl-set-value https://review.openstack.org/101116 2014-06-24T17:21:38 *** martyntaylor1 has joined #tripleo 2014-06-24T17:25:10 jprovazn_afk: you seee my note on https://review.openstack.org/#/c/93104/ ? 2014-06-24T17:25:45 *** rwsu has joined #tripleo 2014-06-24T17:26:27 *** nealph has quit IRC 2014-06-24T17:27:21 greghaynes: u got time to signoff to changes ? ;9 2014-06-24T17:27:47 referring to 102198? 2014-06-24T17:28:24 yah, for one 2014-06-24T17:28:32 I have it open :) 2014-06-24T17:29:21 :) 2014-06-24T17:29:35 https://review.openstack.org/#/c/102292/ this also 2014-06-24T17:29:47 should fix issues with healing upon partitioning 2014-06-24T17:30:35 ooo nice 2014-06-24T17:32:05 aye, testing my awk fu 2014-06-24T17:35:25 jp_at_hp: have you had any joy with https://bugs.launchpad.net/tripleo/+bug/1318750 ? 2014-06-24T17:35:28 Launchpad bug 1318750 in tripleo "overcloud create failing; "Command output matched '(CREATE|UPDATE)_FAILED'" (dup-of: 1317923)" [Critical,In progress] 2014-06-24T17:35:29 Launchpad bug 1317923 in tripleo "tripleo failing to come up - baremetal configuration broked" [Critical,Fix released] 2014-06-24T17:35:47 *** openstackgerrit has quit IRC 2014-06-24T17:36:09 i'm having an issue with an authentication failure from nova baremetal to one of the nodes and I came across that bug 2014-06-24T17:36:25 which has the same error in the attached log snippet, not sure its related 2014-06-24T17:36:55 *** openstackgerrit has joined #tripleo 2014-06-24T17:37:40 *** akuznetsov has quit IRC 2014-06-24T17:38:44 *** e0ne has joined #tripleo 2014-06-24T17:40:09 *** jprovazn_afk has quit IRC 2014-06-24T17:40:16 *** e0ne has quit IRC 2014-06-24T17:40:30 *** andreaf has quit IRC 2014-06-24T17:40:47 *** ramishra has quit IRC 2014-06-24T17:42:26 *** e0ne has joined #tripleo 2014-06-24T17:43:30 greghaynes: morning :) 2014-06-24T17:43:33 greghaynes: next HA patch ? 2014-06-24T17:43:34 hihi 2014-06-24T17:44:32 ooh. potentially good news. Think we've got a fix for eventlet. 2014-06-24T17:44:45 mind you, I've said that about six times now. 2014-06-24T17:44:46 jang: yay 2014-06-24T17:44:50 jang: noo 2014-06-24T17:44:57 but I think we caught the bug that's actually been killing us 2014-06-24T17:45:47 lifeless: hrm, youre running me dry as for ones that have passed CI 2014-06-24T17:47:46 dumdumdummmmmmm duuuum! 2014-06-24T17:47:51 *** jp_at_hp has quit IRC 2014-06-24T17:49:47 http://logs.openstack.org/77/102177/1/check-tripleo/check-tripleo-undercloud-precise/7494d73/logs/seed_logs/nova-compute.txt.gz 2014-06-24T17:49:53 PXE deploy failed? 2014-06-24T17:49:57 *** mrunge has joined #tripleo 2014-06-24T17:49:58 Havent seen that one before 2014-06-24T17:50:53 huh, no such file or directory 2014-06-24T17:53:04 lifeless: Theres two that *should* pass CI soon and are needed, ill ping when done 2014-06-24T17:53:58 greghaynes: same here - logs.openstack.org/35/96235/6/check-tripleo/check-tripleo-undercloud-precise/323ebb2/logs/seed_logs/nova-compute.txt.gz 2014-06-24T17:54:05 aye 2014-06-24T17:54:16 apparently the seeds are 2G VMs, which will be hitting the adm64 footprint limit 2014-06-24T17:54:44 *** Penick has joined #tripleo 2014-06-24T17:55:07 Ng: ping 2014-06-24T17:56:01 Ng: is there an hp2 bastion now? network folk want to put the network into final configuration 2014-06-24T17:56:59 *** jcoufal has quit IRC 2014-06-24T17:57:04 *** yjiang5 is now known as yjiang5_away 2014-06-24T17:57:05 greghaynes: check the output of free in the host state 2014-06-24T17:57:57 total used free shared buffers cached 2014-06-24T17:57:57 Mem: 2.0G 1.6G 387M 448K 13M 78M 2014-06-24T17:57:57 -/+ buffers/cache: 1.5G 478M 2014-06-24T17:57:57 Swap: 0B 0B 0B 2014-06-24T17:58:10 tight but not impossible 2014-06-24T17:58:19 I'd be a lot happier if that was a 4G VM 2014-06-24T17:58:58 dprince: hey - so - suggestions on how to deploy/fix that? 2014-06-24T17:59:08 dprince: I'm still not entirely familiar with the layout within rh1 2014-06-24T17:59:20 any chance to get some views on https://review.openstack.org/#/c/102292/ ? 2014-06-24T17:59:41 greghaynes: filing a bug ? 2014-06-24T17:59:58 yep 2014-06-24T18:00:27 Right. That github repo may have the latest and greatest in. It's chock full of debug crap but there's the gem of a working eventlet in there, we think. Proving it out with ironic and tidying it back up for shoving upstream, for tomorrow. 2014-06-24T18:00:39 Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 2014-06-24T18:01:30 jang: did you get a suggestion in reply to your mail? or you just kept plugging at the issue ? 2014-06-24T18:03:17 oh, bnemec might have already 2014-06-24T18:03:19 https://bugs.launchpad.net/tripleo/+bug/1316350 2014-06-24T18:03:20 Launchpad bug 1316350 in tripleo "Deployment failure due to missing iscsi target" [Medium,Triaged] 2014-06-24T18:06:29 Haneef Ali proposed a change to openstack/tripleo-image-elements: Configure keystone with apache https://review.openstack.org/102304 2014-06-24T18:10:26 lifeless: jsut kept plugging at it. My first patch had "moved" the problem, we kept on with it until we'd plugged every leak. 2014-06-24T18:10:48 now we jsut need to figure out what the minimal invasive patch is. 2014-06-24T18:11:21 A change was merged to openstack/os-collect-config: Updated from global requirements https://review.openstack.org/96234 2014-06-24T18:11:34 I swear, though, coroutines are actually _harder_ to grok than straight up continuations. It'd be a hell of a lot simpler to recast eventlet atop that as an abstraction. 2014-06-24T18:12:32 is it my patch that makes it bork or someone elses http://logs.openstack.org/92/102292/1/check-tripleo/check-tripleo-ironic-seed-precise/1368845/console.html ? 2014-06-24T18:13:25 jang: funnily enough, thats been said before :) 2014-06-24T18:14:19 ekarlso: seed didn't come up proplerly; what was your patch changing ? 2014-06-24T18:14:46 rabbitmq.config 2014-06-24T18:15:15 could be it :> 2014-06-24T18:15:35 ekarlso: you can check if the config came up properly 2014-06-24T18:15:43 lifeless: where ? 2014-06-24T18:17:15 ekarlso: in the logs tgz 2014-06-24T18:17:33 lifeless: how bad of form would it be to un-abandon https://review.openstack.org/#/c/93104/ 2014-06-24T18:17:46 seems like it was already 2x+2 whish is why I want to 2014-06-24T18:17:52 oh, wrong config format : p 2014-06-24T18:18:20 lifeless: are you requesting that we increase the size of the seed VMs? 2014-06-24T18:18:32 *** echohead has joined #tripleo 2014-06-24T18:19:18 lifeless: I'd have to do the match... but like we spoke yesterday the intent was to be able to save space and use a 2 Gig seed for CI (I think) 2014-06-24T18:19:18 dprince: or if we needed to go to i386, that we do that. I would rather 4G seeds though - because we IO thrash if the BM deploy nodes are too small 2014-06-24T18:19:54 lifeless: how urgently is this needed? 2014-06-24T18:20:15 dprince: but right now we have 2G seed and amd64 nodes, that fails intermittently in my tests - we're too close to the RAM ceiling 2014-06-24T18:20:30 lifeless: The cost is we'll have less capicity on the RH rack I think 2014-06-24T18:21:01 lifeless: of course we can do it, but we'll be running even less jobs. 2014-06-24T18:22:26 lifeless: I'd like to see us bring on more nodes first, I suppose and hold off on sending anything which breaks the seeds in this regard until we do 2014-06-24T18:22:53 lifeless: which patch in particular are you concerned about? 2014-06-24T18:22:56 dprince: what about 3G ndoes w/i386 ? 2014-06-24T18:23:24 dprince: 2014-06-24T18:23:27 bah 2014-06-24T18:23:36 https://bugs.launchpad.net/tripleo/+bug/1316350 <- thinking that might be a low memory symptom 2014-06-24T18:23:37 Launchpad bug 1316350 in tripleo "Deployment failure due to missing iscsi target" [Medium,Triaged] 2014-06-24T18:23:45 lifeless: seems like a reasonable compromise but again I'll have to check the numbers 2014-06-24T18:23:51 *** martyntaylor1 has left #tripleo 2014-06-24T18:24:00 StevenK: where is your patch to fixup testenv creation metadata? 2014-06-24T18:24:09 StevenK: we should get that in to any rebuild dprince does 2014-06-24T18:24:29 dprince: if we do rebuild, I'd like to follow along in a screen session or something, to see how its down in that rack 2014-06-24T18:25:13 Endre Karlson proposed a change to openstack/tripleo-image-elements: Allow ha_mode to be set. https://review.openstack.org/102292 2014-06-24T18:25:15 lifeless: sure. I'm fine with that. Honestly this would be something nice to do in mid-July perhaps. 2014-06-24T18:26:01 dprince: do you mean at the sprint? 2014-06-24T18:26:12 lifeless: we have good stability right now... I hate to rock the boat just because we think a small seed causes this iscsi failure though. 2014-06-24T18:26:12 dprince: some training for tripleo-cd-admins there would rock 2014-06-24T18:26:22 lifeless: yes, the sprint 2014-06-24T18:26:24 +1 2014-06-24T18:27:51 greghaynes: not sure what you are +1'ing. We'll take it as a general amen, tripleO rocks sort of thing 2014-06-24T18:28:16 *** e0ne has quit IRC 2014-06-24T18:28:35 yep, just trying to cheer the project along 2014-06-24T18:28:42 *** e0ne has joined #tripleo 2014-06-24T18:28:44 +1 to cd-admins training though :) 2014-06-24T18:28:50 *** vinsh has quit IRC 2014-06-24T18:29:08 greghaynes: when u mean to compare the file, do a md5sum or so of the 2 hosts files ? 2014-06-24T18:29:11 https://review.openstack.org/#/c/102198/3/elements/hosts/os-refresh-config/configure.d/51-hosts 2014-06-24T18:29:38 ekarlso: I mean figure out if were actually going to modify it, and if so log what were going to change 2014-06-24T18:29:54 greghaynes/lifeless: I need this done before ssh access to the RH cloud will get fixed https://review.openstack.org/#/c/101576/ 2014-06-24T18:30:01 I could see that being useful for knowing what our hosts content was during a specific o-c-c run 2014-06-24T18:30:32 greghaynes: that sounds like doing more advanced stuff then the awk that's currently there :| 2014-06-24T18:33:05 The gist is our guys don't like using a single account for access (for obvious security reasons) 2014-06-24T18:33:08 hehe, WCPGW with a single account and a ton of ssh keys 2014-06-24T18:33:09 And there isn't a reasonable way to map the keys to usernames/IRC names otherwise 2014-06-24T18:33:58 cp file, do awk, compare 2014-06-24T18:34:08 ? 2014-06-24T18:34:18 *** jp_at_hp has joined #tripleo 2014-06-24T18:34:23 greghaynes: can you run the meeting today?\ 2014-06-24T18:34:30 If its a bunch of work then im fine not doing it, seemed simple though 2014-06-24T18:34:37 lifeless: sure 2014-06-24T18:34:49 do youall have macros for all those #links things you do? 2014-06-24T18:34:56 I notice youall spam lots quicly 2014-06-24T18:35:00 greghaynes: see the wiki page 2014-06-24T18:35:41 dprince: so its basically fine 2014-06-24T18:36:08 dprince: I'm just struggling to remember how we get *those* keys into the admin 'default' keypair in the CI region underclouds, so that we have access to the overcloud 2014-06-24T18:39:00 lifeless: So, in the case where the undercloud is the gateway/access node (i.e. what the HP rack did) I don't think there is an issue. 2014-06-24T18:39:32 lifeless: in the case where there is a bastion there would need to be a second sync script to allow you to access the undercloud as well. 2014-06-24T18:39:48 lifeless: I use port forwarding 2014-06-24T18:40:02 lifeless: not port forwarding, ssh forwarding :) 2014-06-24T18:40:52 lifeless: we can have more scripts for whatever. All I really care about initially is how we store these keys... and keeping them in files per user is a step in the right direction 2014-06-24T18:41:20 dprince: yep, like I say I'm fine with that. Its their use in the admin keypair that I'm trying to remember. 2014-06-24T18:41:49 dprince: how is that setup in the rh rack? or do you use the id_rsa of the user account the deploy is intiated from ? 2014-06-24T18:42:05 lifeless: I don't think they are used there. 2014-06-24T18:42:09 ah 2014-06-24T18:42:12 lifeless: at least they aren't on our rack 2014-06-24T18:42:18 greghaynes: would doing a diff be sufficient ? ; ) 2014-06-24T18:42:26 so in the hp1 region I had it up so we can all ssh directly in as heat-admin@ci-overcloud.... 2014-06-24T18:42:31 using our own keys 2014-06-24T18:42:49 ekarlso: totally 2014-06-24T18:43:22 lifeless: that is fine. But in our region everything is configured off of a seed host. As TripleO doesn't bootstrap that seed host I'd say lets just also have a script to sync things there too 2014-06-24T18:44:11 lifeless: if someone wants to go directly from bastion to undercloud... that is fine too but I would rather consolidate things on the seed host as much as possible. As sort of an administration node 2014-06-24T18:44:36 dprince: FWIW I marked it +A about 3m back 2014-06-24T18:45:00 lifeless: we're just talking here man, not trying to pressure you :) 2014-06-24T18:45:06 dprince: I know 2014-06-24T18:45:15 dprince: just letting you know since you need it 2014-06-24T18:45:27 dprince: I have to go do this PTL webinar now 2014-06-24T18:45:45 lifeless: thanks, TTL 2014-06-24T18:45:57 A change was merged to openstack-infra/tripleo-ci: Revert "Try reverting Neutron ovs-agent change" https://review.openstack.org/101513 2014-06-24T18:47:50 Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 2014-06-24T18:47:57 there you go greghaynes 2014-06-24T18:50:24 we on alternate meeting time today? 2014-06-24T18:50:39 oh, right 2014-06-24T18:50:41 lifeless: ^ 2014-06-24T18:51:05 oh 2014-06-24T18:51:19 I thought it was original. Easy :) 2014-06-24T18:51:26 tchaypo: tag, you get to run the meeting :> 2014-06-24T18:55:52 Oh wow I didn't know there was an ical for the meetings 2014-06-24T18:55:53 lovely 2014-06-24T18:56:55 ekarlso: reviewed your other RMQ patch 2014-06-24T18:57:45 I didnt realise rmq had that config value, that makes me a lot happier 2014-06-24T18:57:47 *** jdob has quit IRC 2014-06-24T18:58:05 *** jdob has joined #tripleo 2014-06-24T18:59:06 *** jtomasek has joined #tripleo 2014-06-24T18:59:29 *** jcoufal has joined #tripleo 2014-06-24T19:01:02 greghaynes: i discovered it when testing it here by partitioning the cluster it doesn't autoheal 2014-06-24T19:01:04 ;) 2014-06-24T19:01:26 awesome 2014-06-24T19:01:26 *** morganfainberg_L has quit IRC 2014-06-24T19:01:41 *** morganfainberg_L has joined #tripleo 2014-06-24T19:02:04 *** morganfainberg_L has quit IRC 2014-06-24T19:02:16 *** morganfainberg_L has joined #tripleo 2014-06-24T19:02:17 *** yamahata__ has joined #tripleo 2014-06-24T19:02:46 ccrouch - that fix should be in the tripleo codebase already, as per https://bugs.launchpad.net/tripleo/+bug/1317923 2014-06-24T19:02:48 Launchpad bug 1317923 in tripleo "tripleo failing to come up - baremetal configuration broked" [Critical,Fix released] 2014-06-24T19:03:04 *** yamahata_ has quit IRC 2014-06-24T19:03:12 A change was merged to openstack/tripleo-incubator: Split ssh keys into files per user. https://review.openstack.org/101576 2014-06-24T19:03:26 *** jp_at_hp has quit IRC 2014-06-24T19:08:33 *** vinsh has joined #tripleo 2014-06-24T19:09:41 *** jdob has quit IRC 2014-06-24T19:09:54 *** jdob has joined #tripleo 2014-06-24T19:12:04 *** jang1 has joined #tripleo 2014-06-24T19:12:25 *** mrunge has quit IRC 2014-06-24T19:21:11 bnemec: did you submit a patch for the tox.ini yet, or did I misread your email and you're still working on it? 2014-06-24T19:21:28 greghaynes: what u want to call the ha_mode then ? 2014-06-24T19:21:42 jdob: https://review.openstack.org/#/c/102291/ 2014-06-24T19:22:22 ah, crud, dib-utils isnt in my dashboard yet, that's why 2014-06-24T19:22:40 Ng: are you watching the webcast, or is it your evil clone? 2014-06-24T19:24:33 *** killer_prince has quit IRC 2014-06-24T19:24:36 thanks bnemec 2014-06-24T19:25:52 jdob: np 2014-06-24T19:40:30 *** dshulyak_ has quit IRC 2014-06-24T19:41:56 *** noslzzp has quit IRC 2014-06-24T19:42:04 dprince: ping 2014-06-24T19:42:27 kevinbenton: hi 2014-06-24T19:43:55 dprince: would this have affected the tests for my neutron patch? https://review.openstack.org/102281 2014-06-24T19:43:56 dprince: whoops https://review.openstack.org/#/c/101513/ 2014-06-24T19:44:31 it would have removed one of the commits mine depended on if i understand it correctly 2014-06-24T19:44:57 kevinbenton: oh, did you reference the same (already closed I thought) ticket? 2014-06-24T19:45:17 if you reference the ticket it doesn't apply the revery 2014-06-24T19:45:26 kevinbenton: ah, yes. I think it might have. 2014-06-24T19:45:48 kevinbenton: no, though. THe commit was different 2014-06-24T19:45:55 kevinbenton: plus we'd see it in the logs 2014-06-24T19:45:56 dprince: yeah, i see the id changed 2014-06-24T19:46:52 kevinbenton: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_devtest.sh#n20 2014-06-24T19:47:05 Jay Dobies proposed a change to openstack/tripleo-specs: Spec for Tuskar Juno Plan REST API changes https://review.openstack.org/94720 2014-06-24T19:47:24 dprince: well that looks for a bug number, right? 2014-06-24T19:48:02 dprince: XXX isn’t exactly a valid bug :-) 2014-06-24T19:48:58 kevinbenton: Correct, So this wouldn't have caused the problems. 2014-06-24T19:49:24 *** jcoufal has quit IRC 2014-06-24T19:49:48 greghaynes: when u mean a diff name it would rock my world with a suggestion :) 2014-06-24T19:49:55 kevinbenton: FWIW, I tried your patch 4-5 times locally today (on real bare metal) and it works for me. So I think it may be a race that occurs only on smaller/virtual test environments. 2014-06-24T19:49:57 kevinbenton: did you see my note about the GRE ports missing on the br-tun bridge? 2014-06-24T19:50:07 regards to https://review.openstack.org/#/c/102292/2 2014-06-24T19:50:49 dprince: yes, but that can be related to lots of things 2014-06-24T19:51:11 dprince: without the debug logs for the vswitch agent there is nothing to work on 2014-06-24T19:54:33 dprince: I was hoping it would be as easy as this, but it doesn’t seem so https://review.openstack.org/#/c/102073/2/heat-templates/testenv-workers.yaml 2014-06-24T19:55:39 *** jtomasek has quit IRC 2014-06-24T19:58:58 *** Penick has quit IRC 2014-06-24T20:00:38 dprince: if you have a moment to look at https://review.openstack.org/#/c/99460/ -- looks like it is passing everything now 2014-06-24T20:01:36 devananda: ack, thanks. Glad to see that. let me re-review it in a minute 2014-06-24T20:02:12 kevinbenton: left you some comments on 102073. You are on the right track... I think we just need two sets of DIB_REPOREF variables though 2014-06-24T20:03:22 kevinbenton: one for your neutron patch (you've got that already). And another for a new tripleo-heat-templates branch which enables neutron debug logging. Alternately you could just hack the tripleo-image-elements neutron.conf file directly and reference that project/patch instead. 2014-06-24T20:03:42 *** Penick has joined #tripleo 2014-06-24T20:03:52 kevinbenton: make sense? Once we have that you should essentially get the results you are looking for, ie. your neutron patch with debug logging enabled 2014-06-24T20:04:14 *** spzala has joined #tripleo 2014-06-24T20:05:48 *** e0ne has quit IRC 2014-06-24T20:06:21 *** e0ne has joined #tripleo 2014-06-24T20:08:42 *** e0ne has quit IRC 2014-06-24T20:09:39 *** jml has quit IRC 2014-06-24T20:10:08 devananda: 99460 is +A 2014-06-24T20:10:11 Kevin Benton proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 2014-06-24T20:10:14 dprince: like that? ^^ 2014-06-24T20:10:38 dprince: forgot to remove the debug: true 2014-06-24T20:10:39 dprince: awesome, thanks 2014-06-24T20:10:56 *** dtantsur is now known as dtantsur|afk 2014-06-24T20:12:05 kevinbenton: Hmmm, actually I led you on the wrong path. TripleO image elements isn't installed by DIB!! 2014-06-24T20:12:18 lifeless: ^ scheduling fixes should be landing soon 2014-06-24T20:12:23 ekarlso: *shrug* why not just use the name of the config option in rabbit 2014-06-24T20:12:29 rabbit.cluster_partition_handling 2014-06-24T20:12:32 *** e0ne has joined #tripleo 2014-06-24T20:12:49 kevinbenton: I'm not sure we have a good way of enabling neutron debugging at this time. What we really need is Zuul dependencies :( 2014-06-24T20:13:12 *** jml has joined #tripleo 2014-06-24T20:14:54 kevinbenton: I think we might should an an option to the tripleo-incubator (the devtest scripts themselves) to enable DEBUG via an environment variable. 2014-06-24T20:15:57 dprince: why not just debug by default in these projects? to keep the logs small? 2014-06-24T20:17:15 Endre Karlson proposed a change to openstack/tripleo-image-elements: Allow ha_mode to be set. https://review.openstack.org/102292 2014-06-24T20:17:24 kevinbenton: yes, keeps the logs small. Also, we want production defaults 2014-06-24T20:18:03 kevinbenton: Would you like me to take this? 2014-06-24T20:18:32 dprince: cant we use ExtraParams 2014-06-24T20:18:36 to enable debugging 2014-06-24T20:18:45 dprince: yes, i know nothing about tripleo so i’m in over my head 2014-06-24T20:18:48 so export OVERCLOUD_EXTRA_CONFIG 2014-06-24T20:19:21 greghaynes: thanks, good idea 2014-06-24T20:19:27 * dprince double checks... 2014-06-24T20:19:35 neutron.conf does support passthrough 2014-06-24T20:20:20 *** e0ne has quit IRC 2014-06-24T20:20:55 greghaynes: yes, it does. Are you up for wiring https://review.openstack.org/#/c/102073/3 up with that approach? 2014-06-24T20:20:56 *** e0ne has joined #tripleo 2014-06-24T20:21:09 yep 2014-06-24T20:21:27 greghaynes: if not I'll take your suggestions and see what works best 2014-06-24T20:21:40 greghaynes: thanks man :) 2014-06-24T20:21:44 np 2014-06-24T20:22:31 kevinbenton: cool, hopefully we'll get you debug info soon then 2014-06-24T20:23:59 *** dprince has quit IRC 2014-06-24T20:24:23 *** e0ne has quit IRC 2014-06-24T20:26:56 *** eguz has joined #tripleo 2014-06-24T20:28:02 *** davidlenwell_ is now known as davidlenwell 2014-06-24T20:28:12 *** eguz has quit IRC 2014-06-24T20:28:17 *** ChanServ sets mode: +v davidlenwell 2014-06-24T20:28:30 *** eguz has joined #tripleo 2014-06-24T20:29:46 Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 2014-06-24T20:29:57 *** e0ne has joined #tripleo 2014-06-24T20:30:48 *** howleyt_ has joined #tripleo 2014-06-24T20:30:53 *** eghobo has quit IRC 2014-06-24T20:32:20 Howdy, anyone around that could approve https://review.openstack.org/#/c/81920/ It has two +2's already. 2014-06-24T20:32:20 devananda: huh? 2014-06-24T20:32:37 howleyt_: not until we have CI ok 2014-06-24T20:32:49 ci looks like it passed 2014-06-24T20:33:31 oh gertty hadn't picked that up. cool 2014-06-24T20:33:44 +a'd 2014-06-24T20:33:48 A change was merged to openstack/tripleo-image-elements: Add keepalive option to rabbitmq config https://review.openstack.org/81920 2014-06-24T20:35:25 hmm, no reviews on 102081 yet. 2014-06-24T20:36:40 lifeless: scheduler bug in nova that wasn't picking up host stats from ironic -- the fix was just approved. 2014-06-24T20:36:56 lifeless: once it lands, you may want to poke hp internal folks who were waiting for that 2014-06-24T20:43:09 lifeless: how important do you think it will be that the chunking shim be able to take any and all heat CLI options and pass them along to the respective underlings? 2014-06-24T20:48:16 *** nati_ueno has joined #tripleo 2014-06-24T20:49:33 The meeting was the early one today? 2014-06-24T20:50:03 unsure what early is for us 2014-06-24T20:50:07 its 00:00 PST 2014-06-24T20:50:29 which is technically *very* early 2014-06-24T20:50:42 Ie 12 hours from now 2014-06-24T20:51:04 Oh 2014-06-24T20:51:32 That'd be why lifeless is declining to run sit today 2014-06-24T20:52:56 greghaynes, lifeless: Thought I'd pounce when I saw the CI passing. Thanks. 2014-06-24T20:53:23 I woke up, saw that he'd tagged me at 04:51, and thought I'd misjudged and slept through 2014-06-24T20:56:15 Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 2014-06-24T20:56:22 bnemec: thanks for the repolocation protip 2014-06-24T20:58:31 *** jdob has quit IRC 2014-06-24T21:00:17 *** untriaged-bot has joined #tripleo 2014-06-24T21:00:17 Untriaged bugs so far: 2014-06-24T21:00:17 https://bugs.launchpad.net/diskimage-builder/+bug/1332458 2014-06-24T21:00:19 Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] 2014-06-24T21:00:19 https://bugs.launchpad.net/tripleo/+bug/1333807 2014-06-24T21:00:20 Launchpad bug 1333807 in tripleo "RabbitMQ cluster doesn't autoheal after a partition" [Undecided,New] 2014-06-24T21:00:21 https://bugs.launchpad.net/tripleo/+bug/1333649 2014-06-24T21:00:22 Launchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress] 2014-06-24T21:00:23 https://bugs.launchpad.net/diskimage-builder/+bug/1332521 2014-06-24T21:00:25 *** untriaged-bot has quit IRC 2014-06-24T21:00:25 Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] 2014-06-24T21:02:35 *** nati_ueno has quit IRC 2014-06-24T21:02:51 *** nati_ueno has joined #tripleo 2014-06-24T21:03:56 Looking for core on https://review.openstack.org/#/c/102177/ 2014-06-24T21:04:04 devananda: I don't know who's waiting for it - I just heard of it now 2014-06-24T21:04:07 lifeless: ^ HA patch that passed CI 2014-06-24T21:04:21 SpamapS: I think its very unimportant. Its not in a heat repo. 2014-06-24T21:06:12 A change was merged to openstack/tripleo-image-elements: Check if stunnel.connect_ip is set https://review.openstack.org/102177 2014-06-24T21:08:34 *** julim has quit IRC 2014-06-24T21:12:17 *** cwolferh has quit IRC 2014-06-24T21:15:02 lifeless: I got it working anyway. :-P 2014-06-24T21:15:21 lifeless: should help this have minimal impact on the way everything works. 2014-06-24T21:15:30 Just working on pointing it at a real Heat now 2014-06-24T21:21:09 lifeless: curious if youve read what happened on https://review.openstack.org/#/c/93104/ 2014-06-24T21:21:17 its needed for HA 2014-06-24T21:21:20 greghaynes: i have not 2014-06-24T21:21:37 got abandoned when it was 2x +2 for a patch that seems pretty far from being merged 2014-06-24T21:21:37 greghaynes: meetig, back soon 2014-06-24T21:21:40 ok 2014-06-24T21:21:57 greghaynes: if you want it alive, I think you can do that or ping clarkb 2014-06-24T21:22:14 you all can do it now 2014-06-24T21:22:23 Cores have (un)abandon rights 2014-06-24T21:22:25 *** eguz has quit IRC 2014-06-24T21:22:26 Yep, hoping for at least a second opinion that im not missing something 2014-06-24T21:22:36 I do see a restore change button :) 2014-06-24T21:22:50 this is one of the new shiny gerrit 2.8 features 2014-06-24T21:28:14 *** nealph has joined #tripleo 2014-06-24T21:29:23 lifeless: added you as a core reviewer on https://review.openstack.org/#/c/96527/. I think you have background on what's happening there...thanks again for working through the context with me 2014-06-24T21:31:35 *** jang has quit IRC 2014-06-24T21:32:47 *** eghobo has joined #tripleo 2014-06-24T21:36:11 *** eghobo has quit IRC 2014-06-24T21:36:26 *** e0ne has quit IRC 2014-06-24T21:36:30 *** eghobo has joined #tripleo 2014-06-24T21:43:34 nealph: whats wrong with it ? 2014-06-24T21:43:47 nealph: it looks like its passed CI and review 2014-06-24T21:44:24 greghaynes: I don't think they are mutually exclusive, no? fixing up reset-db is fine, on the path to deleting it :) 2014-06-24T21:45:40 Yes, 93104 is just a subset of the other 2014-06-24T21:49:57 greghaynes: so... you have the force, greg 2014-06-24T21:50:09 restored 2014-06-24T21:51:50 *** yjiang5_away is now known as yjiang5 2014-06-24T21:51:58 greghaynes: r11 is really quite different to r10 2014-06-24T21:52:21 oh my, it is 2014-06-24T21:53:21 I'm not sure r11 will pass 2014-06-24T21:53:27 *** nati_ueno has quit IRC 2014-06-24T21:53:29 have you looked at why it failed ? 2014-06-24T21:53:30 Its been working for me 2014-06-24T21:53:48 Yap greghaynes that pause_minority thing for fixes teh partitioning issue when a node comes up again, no manual intervention needed : p 2014-06-24T21:54:00 *** nati_ueno has joined #tripleo 2014-06-24T21:54:17 lifeless: pypi fail 2014-06-24T21:54:21 ah kk 2014-06-24T21:54:22 ekarlso: w00t 2014-06-24T21:54:27 well its rechecking now 2014-06-24T21:55:12 lifeless: when we switched to non-package install for mysql I think the data massaging part of that script became irrelevant 2014-06-24T21:55:52 greghaynes: also I tested the /etc/hosts fix, works like a charm : I 2014-06-24T21:56:17 Now I can reboot a node without the fear of it never coming back to the living : P 2014-06-24T22:00:17 *** jtomasek has joined #tripleo 2014-06-24T22:01:23 lifeless: just lookin' for the +2/approval. 2014-06-24T22:04:54 *** jpeeler has quit IRC 2014-06-24T22:07:07 *** e0ne has joined #tripleo 2014-06-24T22:07:55 *** jtomasek has quit IRC 2014-06-24T22:09:08 *** e0ne_ has joined #tripleo 2014-06-24T22:12:25 *** e0ne has quit IRC 2014-06-24T22:12:48 *** e0ne_ has quit IRC 2014-06-24T22:13:38 *** nijaba has quit IRC 2014-06-24T22:15:40 *** nijaba has joined #tripleo 2014-06-24T22:15:40 *** nijaba has quit IRC 2014-06-24T22:15:40 *** nijaba has joined #tripleo 2014-06-24T22:16:53 Longgeek proposed a change to openstack/tripleo-image-elements: loading of package `pbr` will break setuptools in tripleo https://review.openstack.org/102367 2014-06-24T22:26:04 lifeless: how does heat decide the number of nodes for compute and controller, and the role mapping to the nodes? 2014-06-24T22:28:38 *** markmc has quit IRC 2014-06-24T22:31:22 *** noslzzp has joined #tripleo 2014-06-24T22:35:37 *** yamahata has quit IRC 2014-06-24T22:36:43 *** nijaba_ has joined #tripleo 2014-06-24T22:36:51 *** nijaba has quit IRC 2014-06-24T22:45:45 *** nijaba_ has quit IRC 2014-06-24T22:46:39 *** boris-42 has quit IRC 2014-06-24T22:47:51 *** yamahata has joined #tripleo 2014-06-24T22:50:44 *** eghobo has quit IRC 2014-06-24T22:51:12 *** Penick has quit IRC 2014-06-24T22:51:13 *** nijaba has joined #tripleo 2014-06-24T22:54:38 *** howleyt_ has quit IRC 2014-06-24T22:58:13 *** Penick has joined #tripleo 2014-06-24T22:58:37 *** nijaba has quit IRC 2014-06-24T23:00:00 jerryz: Heat does not decide 2014-06-24T23:00:18 jerryz: we generate a template that has the number of nodes 2014-06-24T23:00:42 jerryz: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/Makefile 2014-06-24T23:00:59 jerryz: COMPUTESCALE and CONTROLSCALE 2014-06-24T23:01:29 SpamapS: yeah, i see that. but how can i specify which is which 2014-06-24T23:02:15 *** boris-42 has joined #tripleo 2014-06-24T23:02:32 SpamapS: say, one BM node has larger storage, which is for cinder 2014-06-24T23:03:09 jerryz: you'd crete a flavor that matches that node size, and specify the right flavor for cinder nodes in the template 2014-06-24T23:03:32 jerryz: if this is for the tripleo region, I wouldn't bother - just run all the nodes the same size, and run all services on all of them 2014-06-24T23:03:41 jerryz: no need to super-optimise things 2014-06-24T23:04:24 *** chuckC has quit IRC 2014-06-24T23:06:44 lifeless: ok. i see. that is at the time registering the BM node. 2014-06-24T23:07:47 greghaynes: hey, was the latest result set for this patch supposed to have debug info? https://review.openstack.org/#/c/102073/ 2014-06-24T23:07:56 lifeless: but tripleO script itself just hardcodes to flavor baremetal 2014-06-24T23:08:16 kevinbenton: I just saw that, am testing now. Those logs look like it didnt have debug info set, right? 2014-06-24T23:08:28 jerryz: yes, we're very simple today 2014-06-24T23:08:29 greghaynes: yeah, doesn’t seem to 2014-06-24T23:08:42 *** yamahata has quit IRC 2014-06-24T23:09:02 *** e0ne has joined #tripleo 2014-06-24T23:09:13 lifeless: another question, how do you get the mac address of the real BM machines not VM simulators 2014-06-24T23:09:44 *** ddieterly has quit IRC 2014-06-24T23:09:47 greghaynes: should be able to see rpc messages in http://logs.openstack.org/73/102073/5/check-tripleo/check-tripleo-overcloud-f20/d704ad5/logs/overcloud-NovaCompute0-brah7pocqqon_logs/neutron-openvswitch-agent.txt.gz 2014-06-24T23:10:54 lol, I think it might be because I didnt capitalize default 2014-06-24T23:10:57 in the config section 2014-06-24T23:11:08 *** Penick has quit IRC 2014-06-24T23:12:49 *** e0ne has quit IRC 2014-06-24T23:13:12 *** ifarkas has quit IRC 2014-06-24T23:16:22 jerryz: we pass it in with the IPMI details - in nodes.json 2014-06-24T23:18:10 oh wait, it didnt set that at all 2014-06-24T23:18:28 *** Penick has joined #tripleo 2014-06-24T23:18:29 er, it didnt use the exta params to fill out the config file 2014-06-24T23:20:05 lifeless: i assume this nodes.json file needs lab admin's input, or some hardware discovery tool. 2014-06-24T23:20:12 Chris Krelle proposed a change to openstack/diskimage-builder: update zypper element https://review.openstack.org/102382 2014-06-24T23:21:15 Chris Krelle proposed a change to openstack/diskimage-builder: update zypper element https://review.openstack.org/102382 2014-06-24T23:23:06 hrm, looks like the extra config thing should have worked: http://logs.openstack.org/73/102073/5/check-tripleo/check-tripleo-overcloud-f20/d704ad5/console.html#_2014-06-24_21_53_43_551 2014-06-24T23:23:12 jerryz: lab admin yeah 2014-06-24T23:23:20 jerryz: you need IPMI credentials, can't discover that :) 2014-06-24T23:24:23 ok, yea, I think its a capitalization issue, retrying... 2014-06-24T23:24:48 *** eghobo has joined #tripleo 2014-06-24T23:24:55 Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 2014-06-24T23:25:22 kevinbenton: http://logs.openstack.org/73/102073/5/check-tripleo/check-tripleo-overcloud-f20/d704ad5/logs/overcloud-controller0-ogh64dgoubfk_logs.tar.xz has /etc in it 2014-06-24T23:25:31 kevinbenton: so you can see at the bottom the value is being set 2014-06-24T23:25:32 *** eghobo has quit IRC 2014-06-24T23:25:53 *** eghobo has joined #tripleo 2014-06-24T23:26:28 *** morazi has quit IRC 2014-06-24T23:28:33 *** gfidente has quit IRC 2014-06-24T23:35:13 *** chuckC has joined #tripleo 2014-06-24T23:36:28 Chris Krelle proposed a change to openstack/diskimage-builder: Ensure cache directory exists before use https://review.openstack.org/102386 2014-06-24T23:37:28 huh http://logs.openstack.org/54/93554/10/check-tripleo/check-tripleo-overcloud-f20/a44beb4/console.html#_2014-06-24_22_21_43_145 2014-06-24T23:37:32 so close! 2014-06-24T23:37:53 lifeless: ok, I've got something that _tries_ to work.. let me squash and push for review 2014-06-24T23:38:09 SpamapS: woo 2014-06-24T23:38:09 have to run get kids now.. but I think I could have a chunked stack deploying tonight 2014-06-24T23:38:16 SpamapS: fantastico 2014-06-24T23:41:25 Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-incubator: Tool to tie disconnected stacks together https://review.openstack.org/102388 2014-06-24T23:41:51 about half of those lines are, sadly, copy/paste from python-heatclient 2014-06-24T23:42:08 no time to refactor it to expose its parsers and keystone-client-endpoint-determining goodness. 2014-06-24T23:43:43 greghaynes: I wonder if theres another issue now... 2014-06-24T23:44:05 nah, im sure we fixed *all* the bugs :p 2014-06-24T23:44:21 so when nodes have been powered off and then powered back on w rmq I get: mkdir: cannot create directory ‘/var/run/rabbitmq’: Permission denied in logs :) 2014-06-24T23:44:39 is /var/run wiped on boot or ? 2014-06-24T23:45:06 * SpamapS is out 2014-06-24T23:45:28 *** Penick has quit IRC 2014-06-24T23:45:32 something to note is that even if rabbitmq comes back, there is not much hope of galera coming back after a full cluster reboot 2014-06-24T23:45:43 if you shut it all down at once 2014-06-24T23:45:53 so not sure if that puts a damper on your plans 2014-06-24T23:45:59 lifeless: i see in devtest grizzly is used for overcloud. is there plan for icehouse? 2014-06-24T23:46:27 greghaynes: we're more checking up on how stuff behaves and what intervention is needed 2014-06-24T23:46:40 ok, just making sure you were aware 2014-06-24T23:47:34 ekarlso: unsure about the /var/run thing, I dont think we wipe it though 2014-06-24T23:48:07 jerryz: huh? We use HEAD 2014-06-24T23:49:52 greghaynes: I'll check tmrw what happens then 2014-06-24T23:52:15 *** Penick has joined #tripleo 2014-06-24T23:57:41 *** yamahata has joined #tripleo 2014-06-24T23:57:41 *** xuhaiwei has joined #tripleo