Friday, 2014-10-10

StevenKgreghaynes: Yes, and? :-)00:00
tchaypothis sounds like a case for *dramatic music* devpi-server!00:00
tchaypowhich would allow us to cut dev releases and push them to our devpi server, then promote them upstream if/when the associated changes land00:00
tchaypoallows us to "pip install" and make sure that the package we release is the package that earlier passed CI00:01
greghaynesYea, that seems reasonable from the perspective of us not having a bunch of git managing code to install from git00:03
greghaynesI would still argue that t-i-e and t-i should be testing HEAD on os-cloud-config though00:04
tchaypoall it needs is for the wonderful folk in the infra team to donate some of their copious free time00:04
*** beekneemech has joined #tripleo00:09
*** bnemec has quit IRC00:09
tchaypoI hate the fact that "ERROR" is a status in nova-list00:14
tchaypoI can't tell the difference between a node that was in status ERROR before I told nova to delete it00:14
tchaypoand one that used to be in some other status but had an error int he middle of being deleted00:14
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Add ironic to enable-packages-install
*** yamahata has joined #tripleo00:16
openstackgerritA change was merged to openstack/diskimage-builder: Check for epel before installing it
lifelessgreghaynes: hey00:29
lifelessgreghaynes: so pip cache - avoids having multiple how-to-install codepaths00:29
lifelessgreghaynes: when we can just add the pip index00:30
*** TheJulia has quit IRC00:30
*** daneyon_ has quit IRC00:31
*** daneyon has joined #tripleo00:31
*** daneyon has quit IRC00:32
*** jml has quit IRC00:32
*** daneyon has joined #tripleo00:33
lifelessgreghaynes: w.r.t. testing releases vs non-releaees00:33
lifelessgreghaynes: I think we need to do both: one is what we tell users to do.00:33
*** jml has joined #tripleo00:35
*** cwolferh has quit IRC00:37
*** daneyon has quit IRC00:37
*** ci-overcloud has joined #tripleo00:44
ci-overcloud************** ci-overcloud complete status=1 ************00:44
*** ci-overcloud has quit IRC00:44
*** ramishra has joined #tripleo00:44
*** rwsu has quit IRC00:47
*** ramishra has quit IRC00:49
*** panda has quit IRC00:53
*** panda has joined #tripleo00:54
*** spzala has quit IRC00:57
*** Andy_Yan has joined #tripleo01:07
*** jerryz_ has joined #tripleo01:07
tchayposo frustrating01:09
tchaypoI want to log into an overcloud node, but it's not accessible on the network01:10
tchayposo I'm going in over serial, but it times out before it can process the password I'm giving it01:10
*** jerryz_ has quit IRC01:14
* StevenK teaches tchaypo about the evilness of getty -a01:15
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add svc-map to cinder-api
openstackgerritA change was merged to openstack/diskimage-builder: Move install bin from rpm-distro to yum
StevenKrbrady: Could you use the Change-ID, rather than linking to github?01:21
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add svc-map to cinder-api
*** ramishra has joined #tripleo01:32
*** weshay has quit IRC01:34
*** nosnos has joined #tripleo01:39
*** rainya has joined #tripleo01:47
*** ramishra has quit IRC01:47
*** rainya has quit IRC02:01
*** ramishra has joined #tripleo02:05
*** dsneddon_ has quit IRC02:13
*** dsneddon_ has joined #tripleo02:13
StevenKERROR: The Parameter (keystoneSslCertificate) was not defined in template.02:18
*** lazy_prince has quit IRC02:23
openstackgerritA change was merged to openstack/tripleo-image-elements: Correct slo middleware position swift proxy-server pipeline
*** dsneddon_ has quit IRC02:33
*** killer_prince has joined #tripleo02:33
*** killer_prince is now known as lazy_prince02:33
*** dsneddon_ has joined #tripleo02:34
*** ramishra has quit IRC02:37
*** ci-overcloud has joined #tripleo02:47
ci-overcloud************** ci-overcloud complete status=1 ************02:47
*** ci-overcloud has quit IRC02:47
*** openstackgerrit has quit IRC02:58
*** ramishra has joined #tripleo03:01
*** untriaged-bot has joined #tripleo03:01
untriaged-botNo untriaged bugs so far! \o/03:01
*** untriaged-bot has quit IRC03:01
StevenKgreghaynes: Ah ha, you missed a change in setup-clienttools to symlink generate-ssl-cert03:03
*** yamahata has quit IRC03:05
*** yamahata has joined #tripleo03:05
StevenKgreghaynes: You may be able to no longer symlink generate-keystone-pki in the same branch, I'm not sure if it is used03:05
*** ramishra has quit IRC03:11
StevenKgreghaynes: Ah ha haha.03:13
StevenKgreghaynes: generate-ssl-cert only accepts short form arguments, and devtest_seed uses --heatenv03:13
StevenKAnd devtest_undercloud was using -heatenv03:16
StevenKtchaypo: commit 66b970d964f9f91d2b0404487400ac78d0a20f71 in -infra/config03:21
*** nosnos has quit IRC03:26
*** nosnos has joined #tripleo03:26
*** nosnos has quit IRC03:30
*** ci-overcloud has joined #tripleo03:41
ci-overcloud************** ci-overcloud complete status=1 ************03:41
*** ci-overcloud has quit IRC03:41
*** ramishra has joined #tripleo03:42
*** ramishra has quit IRC03:47
*** akuznetsov has quit IRC03:50
*** pcrews has quit IRC03:55
*** pradeep has joined #tripleo04:09
* tchaypo pokes at pbr04:21
tchaypowhy are you not paying attention to my requirements.txt?04:21
*** tzumainn has quit IRC04:31
*** nosnos has joined #tripleo04:32
*** akuznetsov has joined #tripleo04:56
*** rha has quit IRC05:06
*** lazy_prince has quit IRC05:12
*** pensu has joined #tripleo05:13
*** pensu has left #tripleo05:15
*** pensu has joined #tripleo05:16
*** loki184 has joined #tripleo05:18
*** dshulyak_ has joined #tripleo05:19
greghaynesStevenK: generate-ssl-cert does accept a heat env05:23
StevenKgreghaynes: --heat_env , and the help only mentions short form05:26
*** rushiagr_away is now known as rushiagr05:26
StevenKgreghaynes: And does this also require a t-h-t change, or am I missing something?05:27
greghaynesStevenK: hrm, help says its positional05:27
greghaynesthis could have been a derp on my part05:28
greghaynesactually, no05:28
greghaynespositional might be ok05:28
StevenKmetavar is also pointless05:28
StevenKgreghaynes: Yeah, I think you want to patch generate-ssl-cert to look more like generate-keystone-pki to save porting issues05:29
greghaynesugh, I think its more complicated05:30
*** lazy_prince has joined #tripleo05:30
greghaynesbasically the naming scheme we currently use for keystone certs is not so great to adopt for generating $service certs05:31
greghaynesI think what ill do is add a param to generate-keystone-pki that makes it forward compat, change t-h-t to the new format, then switch t-i over to use generate-ssl-pki05:32
StevenKgreghaynes: I keep seeing {Ca,Keystone}SslCertificate or somesuch was not defined in template with the new os-cloud-config release, too05:32
greghaynesthus the :(05:33
greghaynesbut the param is KeystoneSigningCertificate05:33
*** ramishra has joined #tripleo05:33
StevenKDo we want a revert and a 0.1.13 brown paper bag release to get ducks in a row?05:33
greghaynesso if we follow that format well have stuff like HorizonSigningCertificate05:33
greghayneswell, I would prefer to fix this in generate-keystone-pki if possible and then have generate-ssl-cert support the new naming scheme05:34
greghayneswhich I think I can do by adding some detection logic in generate-keystone-pki05:34
StevenKgreghaynes: Sure, but aren't we broken at the moment?05:34
greghaynesthats possible05:35
greghaynesnow, how did this pass tests05:35
StevenKHuh, maybe we aren't.05:35
StevenKOh, I think we are05:35
*** pradeep1 has joined #tripleo05:36
greghaynesI am actually surprised we arent05:36
*** pradeep has quit IRC05:36
StevenKgreghaynes: That's ... comforting05:36
*** takadayuiko_away has left #tripleo05:36
greghaynessomehow it passed tests...05:37
StevenKgreghaynes: I'm running a local build now, let's see05:38
*** ramishra has quit IRC05:38
greghaynesStevenK: So, you know how you said were doing pki init in two places05:38
greghaynesStevenK: I wonder if thats why it passed tests, and that commit makes us fall back to the second way we ini keystone certs05:39
StevenKI'm wondering if it's due to not testing tip of os-cloud-config during os-cloud-config changes. Which has broken us before05:40
greghaynesI thought we fixed that?05:40
greghaynesif we didnt then yes, that would also cause this05:40
StevenKgreghaynes: No, hence my banging the drum that lifeless keeps banging of pip index05:41
greghaynesso yea, that would be it. And were probably saved in CI because infra's pypi mirrors havent sync'd the new release05:41
StevenKI'm not certain about that05:42
greghaynesthe best part - were going to have to re-wait for a new release, sync, etc before we can get unbroken05:42
StevenKgreghaynes: Yes, which why I'm saying do we want to revert and then get a ptl group member to release 0.1.1305:43
StevenKWe don't have to wait a week for a release :-)05:43
greghayneswell, in the meanwhile we should do the hacky install from git fix so we start testing os-cloud-config from HEAD on os-cloud-config patches05:43
greghaynesI mean, we may as well turn off the tripleo-* jobs for os-cloud-config at this point05:45
*** loki184 has quit IRC05:48
StevenKgreghaynes: Yeah. :-(05:53
*** akuznetsov has quit IRC05:53
StevenKI had a branch up for -incubator that checked ZUUL_REF, but it was -2'd05:53
StevenKOr something05:53
greghayneswhat value does ZUUL_REF give you?05:54
greghaynestrying to figure out how in toci to detect the repo being tested, I imagine thats how?05:54
*** dsneddon_ has quit IRC05:54
*** dsneddon_ has joined #tripleo05:55
StevenKgreghaynes: You might be able to find the patch on gerrit, I don't think I have it locally any more05:55
*** andreaf has joined #tripleo05:55
greghaynesStevenK: Ah, its because it shouldnt live in tripleo-incubator05:56
* greghaynes makes patch in toci05:56
*** jml has quit IRC05:57
StevenKERROR: The Parameter (CaSslCertificateKey) was not defined in template.05:59
*** jml has joined #tripleo05:59
*** loki184 has joined #tripleo06:01
*** pradeep1 has quit IRC06:02
StevenKgreghaynes: Aren't you missing an openstack-tools ?06:05
*** pradeep has joined #tripleo06:06
*** mrunge has joined #tripleo06:11
StevenKgreghaynes: And sorry for the massive revert :-(06:13
greghayneseh, the world is broken06:13
greghayneswe do need to find someone who can cut releases, though06:14
StevenKmarios: Halp!06:14
StevenKgreghaynes: One of,members06:15
*** akuznetsov has joined #tripleo06:17
mariosStevenK: sup06:23
StevenKmarios: We need landed, and a new release of os-cloud-config cut to stop the world burning06:24
mariosStevenK: k, np. lemme get some more caffeinne and i'll get on it06:25
*** d0m0reg00dthing has joined #tripleo06:29
*** rha has joined #tripleo06:29
tchaypoNg: if you're around tonight and have time, I'd like to poke at hp2 with you.06:32
*** k4n0 has joined #tripleo06:34
*** ramishra has joined #tripleo06:34
*** regebro has quit IRC06:37
*** regebro has joined #tripleo06:38
*** jcoufal has joined #tripleo06:39
*** ramishra has quit IRC06:39
mariosStevenK: has anyone investigated enough to know what in particular this breaks (e.g which params are changed)06:39
*** jprovazn has joined #tripleo06:41
StevenKmarios: heat stack-create -e /home/steven/.cache/tripleo/undercloud-env.json -t 360 -f /home/steven/.cache/tripleo/tripleo-heat-templates/undercloud-vm-ironic.yaml undercloud06:41
StevenKERROR: The Parameter (CaSslCertificateKey) was not defined in template.06:41
greghaynesyea, itll also break keystonesigningcert param06:42
mariosStevenK: ok. is that a typo - is this something we want in templates? etc06:43
greghaynesnot a typo, its just a patch that wasnt properly CI'd and shouldnt have been merged yet06:44
greghaynes(I wrote it)06:44
mariosgreghaynes: yeah just saw that on the original. OK - has os-cloud-config already been cut this week?06:44
StevenKI think so06:44
greghaynesyep, it was just cut with this patch which is why the world is now breaking06:44
greghaynesthe patch actually merged a while ago06:45
mariosOK. going to +A and let it merge. release should take 10 mins once it's there.06:45
mariosStevenK: greghaynes: quid pro quo... would be great if in the next couple of weeks you guys can do the release ;)06:46
StevenKHeh he06:46
StevenKmarios: Neither of us are members of tripleo-release06:47
greghaynesAgreed, I should really learn how06:47
*** d0m0reg00dthing has quit IRC06:47
StevenKgreghaynes: There is a wiki page with following the bouncing ball06:47
*** d0m0reg00dthing has joined #tripleo06:47
mariosStevenK: that can be fixed in no time (any existing member can assign you). if that is the problem, go ahead and volunteer and I will definitely add you if no-one else does06:47
*** d0m0reg00dthing has quit IRC06:47
*** d0m0reg00dthing has joined #tripleo06:48
StevenKmarios: Sounds like you should go ahead and add greghaynes and I, then :-)06:48
mariosStevenK: rather do it after someone has committed in a given meeting - otherwise we may as well go ahead and put everyone there. imo. I don't think there is any documented practice for that.06:49
*** kevinbenton has quit IRC06:50
*** d0m0reg00dthing has quit IRC06:50
*** dshulyak_ has quit IRC06:50
*** d0m0reg00dthing has joined #tripleo06:50
*** d0m0reg00dthing has quit IRC06:53
*** d0m0reg00dthing has joined #tripleo06:53
StevenKmarios: Patch has merged, can haz release?06:56
*** rdopieralski has joined #tripleo06:59
mariosStevenK: ^^^ ignore, is in zzul07:03
marioszuul even07:03
*** andreaf has quit IRC07:06
*** pensu has quit IRC07:16
*** dtantsur|afk is now known as dtantsur07:22
*** ifarkas has joined #tripleo07:24
*** k4n0 has quit IRC07:24
*** k4n0 has joined #tripleo07:28
*** killer_prince has joined #tripleo07:30
*** jml has quit IRC07:31
*** ramishra has joined #tripleo07:35
*** jml has joined #tripleo07:35
*** ramishra has quit IRC07:39
*** akuznetsov has quit IRC07:46
*** pensu has joined #tripleo07:52
*** dtantsur is now known as dtantsur|bbl07:54
*** jistr has joined #tripleo07:56
*** athomas has joined #tripleo08:00
*** derekh has joined #tripleo08:15
*** lucasagomes has joined #tripleo08:27
*** akrivoka has joined #tripleo08:33
shadowerlsmola: the defaults in the templates peter and I wrote are based on the ones in the current ones08:33
*** ramishra has joined #tripleo08:33
shadowerwe had to make some concessions due to heat bugs08:34
shadowerlsmola: there'd have to be a compelling reason to have the  default parameter values in the non-mergepy templates different from the ones08:35
lsmolashadower: well the reason is that current default value doesn't work for me08:36
shadowerlsmola: specificaly?08:37
lsmolashadower: so they overwrite the values in bash08:37
shadowerlsmola: by bash you mean devtest?08:38
lsmolashadower: or Instack08:38
lsmolashadower: most of it is messed up in storage templates08:38
lsmolashadower: or is related to it08:39
lsmolashadower: and e.g. NeutronPublicInterface is defaulted correctly only in one template, rest is blank08:40
shadowerso: I'm open to this but 1) we should keep both the mergepy and the non-mergepy versions in sync and 2) maybe there's a good reason for the temlate defaults that just doesn't work for our non-production environment08:40
shadowerhm the last bit doesn't feel right, I'm fine with fixing that08:40
*** pblaho has joined #tripleo08:41
shadoweralthough I'm not keen on duplicating the defaults across all the templates (some values are used in all four roles + the main overcloud template)08:41
shadowerbut we can sort that out later, I'll look into the NeutronPublicInterface thing. Or feel free to send a patch08:42
lsmolashadower: I don't get the duplicating? Each template should have the correct default in it08:42
*** Andy_Yan has quit IRC08:43
shadowerlsmola: but the defaults for at least some of the params have to be kept in sync. If you change one, you'll have to change it everywhere08:43
lifelessgreghaynes: StevenK: FWIW derekh *wrote* the pip index thing.08:43
lifelessgreghaynes: StevenK: its just not merged.08:43
lsmolashadower: well yeah, if they are suppose to be the same08:44
shadowerlsmola: plus, they're not actually ever used in reality because you pass the values to Heat specificaly08:44
lsmolashadower: well they are used by Tsukar as defaults08:44
*** gfidente has joined #tripleo08:45
lsmolashadower: also the NeutronPublicInterface and OVERCLOUD_LIBVIRT_TYPE, will be different for real baremetal deployment08:46
lsmolashadower: for this case, it should be either blank in template, or probably defaulted to baremetal deployment08:47
lsmolashadower: but for UI, I would say that all parameters being set in Devtest, should be blank in template08:49
lsmolashadower: cause their defaults are for some reason unusable08:50
shadowerlsmola: well then the user will change them in the UI. Just like you can change them when using the environment08:50
lsmolashadower: otherwise it will be hard to default it in UI or to check that user has to pick the right one08:50
derekhlifeless: greghaynes StevenK I missed some context there but I guess your talking about
derekhI can renew it if ye want08:51
shadowerlsmola: so: I'm fine with picking better defualts. But it ought to happen in both template types and should have agreement from people who krow what each default is for08:51
shadower(I'm not necessarily that person)08:51
lsmolashadower: well in that case, I am voting for setting them blank08:52
shadowerlsmola: feel free to propose that on the ml08:52
shadoweror send a patch to tht08:52
lsmolashadower: cause the decision for the right default will happen in bash/UI08:52
lsmolashadower: you want to first change it in devtest templates, then in yours?08:54
*** ramishra has quit IRC08:55
shadowerlsmola: yes. The non-mergepy alternatives should track the mergepy ones as closely as possible. That will make them easier to test, debug, merge and keep in sync08:56
shadowerlsmola: what you're proposing is an unrelated change08:56
shadowerso it should be in a separate commit08:56
lsmolashadower: ok, that means I need some workaround for now09:01
pblahohi all... can someone clarify for me what NOT_REGISTERED as results for jenkins job means? see
*** akuznetsov has joined #tripleo09:05
*** jp_at_hp has joined #tripleo09:10
*** rlandy has joined #tripleo09:15
*** akrivoka has quit IRC09:15
*** ramishra has joined #tripleo09:19
tchaypoI think you'll get a better answer in -infra09:19
tchaypofrom memory it's something like the testenv that was allocated to run that test ailed to respond09:19
tchaypounless it means that the name of the test changed unexpectedly09:19
*** dsneddon_ has quit IRC09:22
*** dsneddon_ has joined #tripleo09:23
*** ramishra has quit IRC09:29
*** killer_prince has quit IRC09:31
pblahotchaypo: thanks...09:32
*** jang has quit IRC09:38
*** nosnos has quit IRC09:39
*** jml has quit IRC09:40
*** nosnos has joined #tripleo09:40
*** killer_prince has joined #tripleo09:41
*** jml has joined #tripleo09:42
*** nosnos has quit IRC09:44
StevenKderekh: Why did you abandon it? It looks like a good start09:45
*** killer_prince has quit IRC09:47
*** lazy_prince has quit IRC09:47
*** lazy_prince has joined #tripleo09:48
*** pelix has joined #tripleo09:57
*** d0m0reg00dthing has quit IRC09:58
*** jang has joined #tripleo09:59
*** ramishra has joined #tripleo10:00
*** jang1 has joined #tripleo10:00
*** athomas has quit IRC10:00
*** kevinbenton has joined #tripleo10:03
*** ramishra has quit IRC10:04
*** athomas has joined #tripleo10:07
*** akrivoka has joined #tripleo10:15
*** jprovazn has quit IRC10:36
*** loki184 has quit IRC10:40
derekhStevenK: as far as I remember it worked but we wanted to test against released library versions instead (I've forgotten the details), anyways I think it makes sense to renew it so we don't get bitten by problems sneaking passed the CI10:43
*** pblaho has quit IRC10:49
*** ramishra has joined #tripleo11:00
*** ramishra has quit IRC11:05
*** jprovazn has joined #tripleo11:06
*** dtantsur|bbl is now known as dtantsur11:08
*** yamahata has quit IRC11:09
*** dshulyak_ has joined #tripleo11:15
*** dsneddon_ has quit IRC11:33
*** lucasagomes is now known as lucas-hungry11:33
*** dsneddon_ has joined #tripleo11:34
*** lazy_prince is now known as killer_prince11:37
*** k4n0 has quit IRC11:42
StevenKderekh: So I think the only change I would make is to install releases, except if we're testing against something like os-*-config etc11:49
derekhStevenK: Yup, I'll dust it off and submit a new version so we can give it a whirl11:50
StevenKderekh: \o/11:53
*** jistr has quit IRC12:00
*** ramishra has joined #tripleo12:01
*** dshulyak_ has quit IRC12:03
*** ramishra has quit IRC12:06
*** dprince has joined #tripleo12:08
*** weshay has joined #tripleo12:09
*** ramishra has joined #tripleo12:09
shadowerlsmola: +A?
shadowertests passed, has two +2s12:12
lsmolashadower: done12:12
*** jdob has joined #tripleo12:18
*** jistr has joined #tripleo12:19
*** lucas-hungry is now known as lucasagomes12:25
*** ramishra has quit IRC12:28
shadowerlxsli: that comment about "watching the world burn", was that about the use of all caps or something else?12:28
*** rhallisey has joined #tripleo12:48
lxslishadower: I was just joking, Tomas added a comment12:48
lxsliwhich is a good thing yet somehow completely unprecedented12:48
shadowerlxsli: right, I wasn't sure I got the joke12:55
shadowerthought it was about the all caps (which was never supposed to see the light of day)12:55
*** rushiagr is now known as rushiagr_away12:55
*** tzumainn has joined #tripleo12:55
shadowerlxsli: thanks for looking at the patch btw, much appreciated12:55
*** ramishra has joined #tripleo12:59
*** ramishra has quit IRC13:00
*** ramishra has joined #tripleo13:01
*** morazi has joined #tripleo13:03
*** ramishra has quit IRC13:05
*** jcoufal has quit IRC13:07
*** _crobertsrh is now known as crobertsrh13:11
*** julim has joined #tripleo13:12
*** jistr is now known as jistr|bbl13:16
*** jcoufal has joined #tripleo13:25
*** regebro has quit IRC13:32
*** rushiagr_away is now known as rushiagr13:44
*** jprovazn has quit IRC13:47
*** jehb has joined #tripleo13:52
*** akrivoka has quit IRC14:00
*** sdake has quit IRC14:15
*** openstackgerrit has joined #tripleo14:16
*** sdake has joined #tripleo14:22
*** rlandy has quit IRC14:27
*** pensu has left #tripleo14:27
*** ifarkas has quit IRC14:29
SpamapSjdob: \o/ thanks for doing releases. :)14:30
jdobnp  :)14:31
*** ramishra has joined #tripleo14:31
*** pradeep has quit IRC14:33
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: New node registration and upload dialogs
greghayneslifeless: Re: - Agreed that derekh's patch is the way to go but his patch needs a little work and fixing this hackily now seems worth it given that we basically are lying about CI passing on these repos (which just broke the world)14:36
*** dprince has quit IRC14:38
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: New node registration and upload dialogs
*** pcrews has joined #tripleo14:42
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: New node registration and upload dialogs
*** jml has quit IRC14:46
*** jml has joined #tripleo14:48
greghaynesderekh: Maybe you can answer - on seems like the idea is to just create a local pip mirror, how would you handle the case of a patch to os-apply-config or similar where we need to test HEAD?14:49
greghaynesoh, youre using the repo provided by zuul, building an sdist, and sticking it in the mirror?14:50
*** rdopieralski has quit IRC14:50
*** akrivoka has joined #tripleo14:55
derekhgreghaynes: Yup, I'll have a new version ready monday (gotta go in a bit), if you want to do something with it, in the meantime then fire ahead but it shouldn't take me long monday15:00
greghaynesderekh: I was going to poke around with it a bit today15:01
derekhgreghaynes: ok15:01
greghaynesderekh: Something else that occured to me - o-a-c and os-cloud-config we use on the 'jump host'15:02
*** dsneddon_ has quit IRC15:02
greghaynesso we need to also export the pip mirror config there15:03
derekhgreghaynes: yup15:04
*** dsneddon_ has joined #tripleo15:05
openstackgerritA change was merged to openstack-infra/tripleo-ci: Remove cherry pick's
*** rwsu has joined #tripleo15:05
openstackgerritErik Colnick proposed a change to openstack/tripleo-image-elements: Create 'neutron-router' image element
openstackgerritTomas Sedovic proposed a change to openstack/tripleo-specs: Remove from TripleO Heat Templates
openstackgerritA change was merged to openstack/tuskar-ui: Setup swift endpoint correctly
*** pensu has joined #tripleo15:14
*** pensu has quit IRC15:19
openstackgerritA change was merged to openstack/tripleo-incubator: Rotate an SSH key for tchaypo
*** ramishra has quit IRC15:27
*** andreaf has joined #tripleo15:27
*** pensu has joined #tripleo15:35
*** pradeep has joined #tripleo15:36
*** mrunge has quit IRC15:36
*** jp_at_hp has quit IRC15:38
*** pensu has quit IRC15:39
greghaynesderekh: oh, another thing youll love - the HA job is failing often due to out of memory15:46
SpamapSOpenStack, why you so big?15:48
SpamapSeat vegeble15:48
*** morazi has quit IRC15:48
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Enable dracut deploy ramdisks
*** dtantsur is now known as dtantsur|afk15:49
greghaynesMaybe we should focus on its mass during kilo15:49
greghaynesSpamapS: One thought I had - I know we bumped some of the mysql buffer size setings so they were usable for prod, but maybe it would be reasonable to CI with these at values that make more sense on 3gb vms15:50
*** jistr|bbl is now known as jistr15:50
SpamapSI thought we bumped them _down_15:54
derekhgreghaynes: again ? Increasing the VM's to 3G seemed to help a few weeks ago15:54
derekhSpamapS: greghaynes: I have a feeling its at 100Meg in CI  /me could be wrong15:55
SpamapSYeah I believe it is.15:55
SpamapSIt may be the same issue they saw in devstack testing.15:55
greghaynesderekh: but yes, definitely getting OOM's15:55
SpamapSToo many worker processes.15:55
greghaynesI was fiddling with that setting in keystone and nova-api without much success15:56
derekhgreghaynes: your correct I love it ;-)15:56
SpamapSProd wants many and the RAM is worth it. CI wants wide scalability and is not so rich in RAM. ;)15:56
greghaynesIll try and figure out the decreasing worker count thing though since thats the best idea so far...15:57
derekhgreghaynes: any idea how many of the failures are because of RAM ? we're not getting anywhere as many failures as we used to15:57
derekhgreghaynes: swapfile ...15:57
greghaynesderekh: All the fails I could find in the past few days (that werent legitimate fails and such) were from that15:57
greghaynesgranted, not a ton15:57
greghaynesoff-hand seemed like ~10-20%15:58
greghayneswhich is waaaay better than before :)15:58
derekhgreghaynes: ok, would be nice to get that a bit lower but I think if its consintently below 10% we could consider making it voting15:59
SpamapSgreghaynes: randomly selected OOM's killed, or always mysqld?16:00
SpamapSsometimes the kernel decides mysql is the pig and roasts it16:00
*** pradeep1 has joined #tripleo16:01
greghaynesMysqld wasnt actually being killed, I just assumed that would be a low hanging fruit16:02
greghaynesbut sounds like we already picked that one16:02
*** pradeep has quit IRC16:04
*** andreaf has quit IRC16:05
*** yamahata has joined #tripleo16:11
*** derekh has quit IRC16:11
*** pensu has joined #tripleo16:13
*** cody-somerville has quit IRC16:16
*** cwolferh has joined #tripleo16:17
openstackgerritgreghaynes proposed a change to openstack/tripleo-image-elements: Use keystone identity_uri instead of auth_*
*** jdob has quit IRC16:22
greghayneskeystone: Could not find user: neutron16:23
greghaynesthats interesting16:23
greghaynesor is that normal?16:23
*** yamahata has quit IRC16:24
*** jdob has joined #tripleo16:26
*** dshulyak_ has joined #tripleo16:31
*** jdob has quit IRC16:32
*** ChuckC__ has joined #tripleo16:32
*** akrivoka has quit IRC16:32
*** akrivoka has joined #tripleo16:33
*** noslzzp has joined #tripleo16:35
*** jcoufal has quit IRC16:41
*** ChuckC__ has quit IRC16:42
*** slagle has quit IRC16:44
*** gfidente has quit IRC16:45
*** akrivoka has quit IRC16:46
*** daneyon has joined #tripleo16:58
*** pradeep has joined #tripleo16:59
*** pradeep1 has quit IRC17:00
*** lucasagomes is now known as lucas-dinner17:01
greghaynesDIB reviewers - infra is hoping to get merged and a DIB release cut to work around some issue they are having which I dont entirely understand all the details of17:02
greghaynesbut I imagine they basically just want tar's for some reason which isnt that crazy17:02
* clarkb explains17:04
* greghaynes waves hands vigurously17:04
clarkbwe are using DIB to build our jenkins slave images now (some of them anyways). By default DIB outputs a qcow2 image which works great with hpcloud, but rackspace needs VHD/VPC images because Xen. I think it would be excellent if my image building tool handled the variety of formats I need for me hence that patch17:05
clarkbso we do a single DIB builds and get two images, one that goes to hpcloud and one that goes to rax17:05
clarkbthe secondary goal of this feature is to make it easy for us to publish a variety of formats of the slave images for all the people17:06
clarkbwe would like to have a location where anyone can grab the slave image in the format that suits them and just boot it17:07
*** athomas has quit IRC17:16
*** pweiss has joined #tripleo17:17
openstackgerritErik Colnick proposed a change to openstack/tripleo-image-elements: Create 'neutron-router' image element
beekneemechgreghaynes: clarkb: I left a comment on the review.  I think there's one more thing that should be moved, but otherwise looks fine.17:24
clarkbbeekneemech: just remove the signal reset and let it fall through to the end of the process execution?17:28
beekneemechclarkb: Or move the signal reset after the cleanup_dirs call in disk-image-create like it would have been before.17:30
beekneemechAlthough I'm wondering if we could just leave the handler set and let it do the cleanup.17:30
*** killer_prince has quit IRC17:30
*** rameshg87 has joined #tripleo17:31
openstackgerritClark Boylan proposed a change to openstack/diskimage-builder: Allow for multiple image outputs from raw source
*** daneyon has quit IRC17:37
*** rameshg871 has joined #tripleo17:39
*** rameshg871 has quit IRC17:39
*** rameshg871 has joined #tripleo17:40
*** rameshg87 has quit IRC17:42
*** chuckC_ has quit IRC17:43
rameshg871hello tripleo-cores, request you to have a look at as it is required for Ironic iLO drivers in openstack for juno release. thanks.17:45
*** ChuckC has quit IRC17:50
*** jml has quit IRC17:57
SpamapSrameshg871: reviewed17:59
*** daneyon has joined #tripleo17:59
rameshg871SpamapS: that was very quick, thanks.18:00
*** jml has joined #tripleo18:01
*** jlabocki has joined #tripleo18:01
rameshg871SpamapS: regarding efiboot.img18:01
jlabockihola sdake18:01
SpamapSrameshg871: yes?18:02
rameshg871SpamapS: it is available in some OS built trees, but not in any of the OS packages18:02
rameshg871SpamapS: but it's not exactly what we want thought18:02
jlabockilarks sdake - looking at and I think it's out of date . There is no build script in any more18:03
SpamapSrameshg871: ah.. it seems like a special situation so maybe shipping the binary in git is worth the trouble we might get into.18:03
*** noslzzp has quit IRC18:03
SpamapSmy concern would be that we now have to take responsibility for monitoring efiboot and making sure we ship new releases of diskimage-builder whenever there's an issue.18:03
rameshg871SpamapS: the concern is it doesn't work with the later version of grub.18:03
jlabockisdake larks looks like it was replaced by this?
rameshg871SpamapS: just thought of avoiding user's headache in "you should have xx version of grub for this to work"18:04
larsksjlabocki: we have changed the build mechanism, yes.18:04
larsksjlabocki: documented in that document.18:04
larsksLet me know if you have any questions, because I'm responsible for that change...18:04
rameshg871SpamapS: is it okay to ship the binary in that case ?18:04
jlabockilarks: Can someone update or at least redact the instructions from ?18:05
SpamapSrameshg871: what OS's ship the wrong version of grub?18:05
jlabockilarsks: so if I wanted to just build locally I would run ./build —release ?18:06
rameshg871SpamapS: it doesn't work on ubuntu 14.04, but works on ubuntu 13.1018:06
jlabockibut what if I wanted to build all the images?18:06
larsksjlabocki: no, never use --release, unless you really mean to overwrite the official images.18:06
jlabockiofficial images?18:07
rameshg871SpamapS: i have raised a bug in grub, but no response so far
jlabockiwhere are official images stored?18:07
uvirtbotLaunchpad bug 1378658 in grub2 ""no such device" error from grub2 search command" [Undecided,New]18:07
jlabockiwhat registry?18:07
larsksjlabocki: E.g., things in the kollaglue/ namespace.18:07
jlabockiah, ok18:07
jlabockibase images18:07
larsksThat is, the idea is that if you're *working* on someething, don't tromp on those images unless you really mean it.18:07
larsksBecause other people may be using them.18:07
jlabockigot it18:07
jlabockiso, I set my own namespace18:07
larsksYOu would generally create a .buildconf file in the root of your working copy with NAMESPACE=yourname and TAG=latest18:08
rameshg871SpamapS: i have left a provision for user to build their own image if they don't want to use the prebuilt efiboot.img (setting DIB_UEFI_ISO_BUILD_EFIBOOT to 1)18:08
jlabockiso how do I build based on what I have in my CWD?18:08
jlabockiI did that18:08
jlabockiok, I added the TAG=latest18:08
larsksThen for a specific image just run ./build in the image directory18:08
jlabockito my .buildconf18:08
larsks./build -p18:08
SpamapSrameshg871: thanks, I'll poke cjwatson to see if he can take a look at that bug.18:08
larsksThis will build and push to your docker hub namespace.18:08
rameshg871SpamapS: when the problem is fixed in grub, we can put this as default behaviour18:08
jlabockiok, what about if I want to do all images?18:08
rameshg871SpamapS: thanks18:08
jlabockido I need to loop that mysefl?18:09
rameshg871SpamapS: i will address other comments and raise the review again.18:09
larsksjlabocki: Right now, yes.18:09
jlabockilarks: so like this?
jlabockiso for namespace I could append my.registry.address:port/repositoryname to .buildconf?18:11
jlabockiassuming it hasn't been tested18:11
jlabockilarks ^^18:11
SpamapSrameshg871: thanks. I think I'm comfortable going forward w/ the binary include, though I think we might want to mark that bug as affecting diskimage-builder so we know to fix it later.18:11
larsksjlabocki: we're all just using docker hub, so e.g., I just set NAMESPACE=larsks.18:11
larsksI havne't tried with private registries but I guess it should work.18:12
jlabockilarsks: I need to use an internal registry18:12
jlabockilet me try it18:12
*** daneyon has quit IRC18:12
*** jistr has quit IRC18:13
*** daneyon has joined #tripleo18:13
*** rameshg871 has quit IRC18:13
*** rameshg87 has joined #tripleo18:13
rameshg87SpamapS: just a quick question18:18
rameshg87SpamapS: did you mean we can document the new variable IMAGE_ROOT_FS_UUID here: ??18:19
*** pelix has quit IRC18:20
SpamapSrameshg87: yes. And it probably needs a DIB_ prefix so we don't interfere with other programs inside elements that might want to use the same variable name.18:22
openstackgerritErik Colnick proposed a change to openstack/tripleo-image-elements: Remove redundancies between neutron-router and neutron-network-node
openstackgerritRamakrishnan G proposed a change to openstack/diskimage-builder: 'iso' element to build bootable ISO images.
rameshg87SpamapS: okay, will take care of it .. :)18:28
*** akuznetsov has quit IRC18:29
openstackgerritRamakrishnan G proposed a change to openstack/diskimage-builder: 'iso' element to build bootable ISO images.
rameshg87SpamapS: ^^^ :)18:30
*** rameshg87 has quit IRC18:31
*** ChuckC has joined #tripleo18:31
*** akuznetsov has joined #tripleo18:33
openstackgerritRyan Brady proposed a change to openstack/tuskar-ui: Enable ExtraConfig Parameter in Tuskar UI
*** chuckC_ has joined #tripleo18:35
*** ChuckC has quit IRC18:36
*** ChuckC has joined #tripleo18:36
openstackgerritRyan Brady proposed a change to openstack/tuskar-ui: Enable ExtraConfig Parameter in Tuskar UI
lifelessgreghaynes: AIUI the only thing missing in derekh's patch was shoving the repo behind http to work around old-pip issues.18:39
greghaynesok, im going to play with the patch and try and get it good to go18:46
greghayneswe also need to setup the pip mirror stuff on the jump host since we call os-cloud-config and o-a-c directly from devtest18:47
greghaynesbut thats easy18:47
*** openstackgerrit has quit IRC18:48
*** akuznetsov has quit IRC18:52
*** openstackgerrit has joined #tripleo18:54
*** akuznetsov has joined #tripleo19:08
*** pradeep has quit IRC19:10
*** jml has quit IRC19:11
*** jml has joined #tripleo19:13
*** rushiagr is now known as rushiagr_away19:18
openstackgerritRyan Brady proposed a change to openstack/tuskar-ui: Enable ExtraConfig Parameter in Tuskar UI
*** rainya has joined #tripleo19:29
*** dkehn has quit IRC19:41
*** dkehn has joined #tripleo19:43
*** ChuckC__ has joined #tripleo19:43
*** ChuckC__ has quit IRC19:47
*** jehb has quit IRC19:53
jlabockilarks: <— failed when trying to build mariadb container "One of the configured repositories failed (Fedora 20 - x86_64),"19:56
jlabockilarsks ^^19:56
*** pensu has quit IRC19:58
larsksjlabocki: sounds like a transient repository access error. They happen occasionally with fedora. Either that our you have a local docker config problem.20:00
larsksTry again and see what happens v20:00
jlabockiI've tried multiple times20:01
jlabockiover 30 minutes20:01
jlabockikeeps happening20:01
jlabockitrying to rabbitmq build now20:01
*** dshulyak_ has quit IRC20:03
larsksIs name resolution working inside docker on your system?20:07
jlabockilarsks: as soon as this docker build is done I'll tell you (I deleted base image, waiting for fedora-rdo-base20:08
jlabockiyou know, you could --rm=true on docker build20:08
jlabockito remove intermediate containers after a successful build20:08
jlabockimight speed things up20:08
jlabockilarks: interesting, rabbitmq worked20:13
jlabockilet me try mariadb again20:13
jlabockilarsks: I lied20:14
jlabockiit failed20:14
jlabockilet me launch a container interactively and check name resolution20:14
*** andreaf has joined #tripleo20:15
*** crobertsrh is now known as _crobertsrh20:16
openstackgerritJeff Peeler proposed a change to stackforge/kolla: Update start/stop scripts with updated paths/names
*** jtomasek has quit IRC20:34
jpeelerlarsks: what do i need to look at to figure out why minions are not showing up? on my latest (completed) stack deploy, kubecfg list minions doesn't return any ips20:35
larsksjpeeler:  are you running it on the master or on a minion?20:35
jpeelerrunning from master, although i thought it work either place20:36
jlabockilarsks: problem was with firewalld (arg, switched back to iptables) I can push to private repo20:39
larskspossibly.  let me just make sure locally...20:39
jlabockiThe push refers to a repository [] (len: 1)20:39
larsksjpeeler: yeah, should work on the master. Look in /etc/kubernetes/apiserver20:39
larsksIs "MINION_ADDRESSES" set?20:39
jpeelerlarsks: it is20:41
*** akuznetsov has quit IRC20:41
jpeelerlarsks: it is20:41
jpeeleroops, sorry20:42
openstackgerritOpenStack Proposal Bot proposed a change to openstack/os-collect-config: Updated from global requirements
openstackgerritOpenStack Proposal Bot proposed a change to openstack/tuskar: Updated from global requirements
larsksjpeeler: Hmmm. Does "curl http://localhost:4001/v2/keys/registry/minions | jq ." show anything?  Output should look like:
larsksAnd if you "ps -fe | grep apiserver", is there a --machines argument listing the minions?20:46
jpeelerthere is a machines argument, looks as expect20:47
jpeelerbut the former reports "key not found"20:47
larsksany chance I can get access to this environment?20:48
jpeelerlarsks: give me a few min20:51
*** akuznetsov has joined #tripleo20:52
tchaypogood morning everybody!20:56
tchaypo has been open for a month and is now on its 5th patchset. Could we get this landed please?20:57
*** untriaged-bot has joined #tripleo21:00
untriaged-botNo untriaged bugs so far! \o/21:00
*** untriaged-bot has quit IRC21:00
*** julim has quit IRC21:00
*** noslzzp has joined #tripleo21:08
clarkbany ideas on why failed? I have run it twice with patch as is and patch rebased locally against one of my images with no issues21:26
clarkbwhich leads me to think it is something in that particular ramdisk image I am hitting21:26
clarkbif I want to reproduce that ramdisk build do I need to grab elements from some repo other than dib?21:26
openstackgerritDaneyon Hansen proposed a change to stackforge/kolla: Closes-Bug: 1379442
*** lucas-dinner has quit IRC21:33
*** noslzzp has quit IRC21:41
*** akuznetsov has quit IRC21:44
*** killer_prince has joined #tripleo21:44
*** killer_prince is now known as lazy_prince21:44
*** jang2 has joined #tripleo21:46
beekneemechclarkb: I think it's this:
clarkboh its calling its own save image from an element?21:48
beekneemechThat calls finish_image, and since we're not unsetting the trap there anymore it's probably trying to clean up twice.21:48
clarkbthis isn't verboten?21:48
beekneemechramdisks are kind of weird.21:48
beekneemechThat's probably why there's special casing for ramdisk builds too.21:49
*** akuznetsov has joined #tripleo21:51
*** ccrouch has quit IRC22:03
*** akuznetsov has quit IRC22:06
*** ccrouch has joined #tripleo22:20
*** jang2 has quit IRC22:22
*** rainya has quit IRC22:40
*** openstackgerrit has quit IRC22:54
*** rwsu has quit IRC22:59
*** andreaf has quit IRC23:03
*** andreaf has joined #tripleo23:03
*** andreaf has quit IRC23:08
*** chuckC_ has quit IRC23:13
*** ChuckC has quit IRC23:22
*** ccrouch has quit IRC23:52

Generated by 2.14.0 by Marius Gedminas - find it at!