Wednesday, 2015-03-11

*** r-daneel has quit IRC00:01
fungiso it turns out the cert is _not_ signed by the same key, even though i ordered it the same way through the same account00:02
openstackgerritSteve Baker proposed openstack-infra/project-config: Use ssh publisher for heat-templates-buildimages
fungifor some reason namecheap used "COMODO RSA Domain Validation Secure Server CA" for that one as opposed to "PositiveSSL CA 2" on our others00:02
fungii wonder if all the recent certs are signed with the same as and this is the first it's come to our attention00:04
openstackgerritDonald Stufft proposed openstack-infra/puppet-openstackid: Use a set of secure ciphers optimizing for security and performance
fungisince that ca is sha2 instead of sha100:05
dstufftclarkb: there ^ dropped 3DES and expanded the commit message to give more details00:05
fungichecking other recent additions for confirmation00:05
*** e0ne has quit IRC00:06
*** dteselkin has quit IRC00:08
*** katyafervent has quit IRC00:08
*** tsufiev_ has quit IRC00:08
*** rakhmerov has quit IRC00:08
*** marun has quit IRC00:09
openstackgerritClark Boylan proposed openstack-infra/nodepool: Remove duplicate code
dstufftfungi: oh, that makes sense, ssllabs complained about extra certs too00:10
*** katyafervent has joined #openstack-infra00:10
*** hichihara has joined #openstack-infra00:10
fungidstufft: yep, just piecing together the correct chain so i can update this and the other four affected systems ordered since that changed00:10
*** tsufiev_ has joined #openstack-infra00:14
openstackgerritClark Boylan proposed openstack-infra/project-config: Run nodepool coverage as a check job
fungidstufft: cinerama: now works with openssl s_client -connect -CAfile /etc/ssl/certs/AddTrust_External_Root.pem00:15
openstackgerritClark Boylan proposed openstack-infra/project-config: Run nodepool coverage as a check job
fungii've got puppet momentarily suspended with what i think is the correct chain installed on the server00:15
fungiassuming that looks good i'll update it in hiera00:15
*** rakhmerov has joined #openstack-infra00:15
cineramafungi: cool00:16
fungi-CApath /etc/ssl/certs works too for my systems00:16
fungicinerama: dstufft: thanks for spotting this. the ask and groups sites are currently also affected by the same problem, as are certs for two servers we haven't switched to https yet00:17
*** dteselkin has joined #openstack-infra00:17
*** tsufiev_ has quit IRC00:17
*** rakhmerov has quit IRC00:17
*** dteselkin has quit IRC00:17
*** katyafervent has quit IRC00:17
anteayadhellmann: +A 161447 sorry that took so long00:18
greghaynesclarkb: haha, my test blows up pretty hard00:18
anteayadhellmann: and sorry again to the incorrect review00:18
fungii need to remember to start manually validating certificate chains when we bring servers up00:18
*** camunoz has joined #openstack-infra00:18
*** katyafervent has joined #openstack-infra00:18
dstufftfungi: deploying certs is way harder than it should be00:18
clarkbgreghaynes: perfect :) thats how my change ended up becoming 4 changes. it tickled the bugs and they needed fixing00:18
*** achanda has quit IRC00:18
*** achanda has joined #openstack-infra00:18
*** sdake_ has joined #openstack-infra00:19
*** dims_ has joined #openstack-infra00:20
greghaynesclarkb: goes into a polling loop and then eventually results in a sql deadlock00:20
clarkbwow thats amazing00:20
*** mtanino has quit IRC00:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: WIP: test nodepool upload failures
greghaynesneed to AFK for a few but ^ if youre interested in seeing00:22
*** dims has quit IRC00:22
*** dims_ has quit IRC00:22
*** dims has joined #openstack-infra00:23
*** achanda has quit IRC00:23
*** dteselkin has joined #openstack-infra00:24
*** gokrokve has quit IRC00:24
clarkbgreghaynes: oh ya I think you need to rebase onto my stack to get some of the fixes involved00:24
clarkbgreghaynes: I would try that at least and see if it blows up less badly00:24
*** katyafervent has quit IRC00:24
*** dteselkin has quit IRC00:24
*** ivar-lazzaro has quit IRC00:25
*** tsufiev_ has joined #openstack-infra00:25
*** rakhmerov has joined #openstack-infra00:26
*** katyafervent has joined #openstack-infra00:27
openstackgerritMerged openstack-infra/project-config: import aeromancer to stackforge
clarkbfyi new project creation is still broken until and get in00:29
*** dteselkin has joined #openstack-infra00:30
clarkbnibalizer: your gist makes me wish you used \t00:30
*** claudiub has quit IRC00:30
nibalizerwhat gist?00:31
clarkboh ww but that package script thing00:31
clarkbI was very confused initially00:31
nibalizeroh that was just grosssss hax for the lulz00:31
clarkbjhesketh: re 163188 feel free to push new patchset and add a co authored by00:32
clarkbjhesketh: I am about to call it a day so happy if someone else continues to work on it00:33
jheskethclarkb: will do00:33
jheskethno worries :-)00:33
clarkbjhesketh: you will probably end up rewriting the whole thing and then just take authorship in git :)00:33
*** david-lyle_afk has quit IRC00:34
clarkbjhesketh: the reason I didn't initially put it in the payload was that data and headers are two different parameters00:34
clarkbjhesketh: so I thought they were distinct, but you are right it would be nice to keep that data together to make the uploader not very opinionated00:34
jheskethclarkb: I figure the headers will be part of the payload, so you can define individual headers for each file00:35
jheskethmaybe payload is the wrong term00:36
clarkbjhesketh: ya I get what you mean though00:36
clarkbjhesketh: its a file attribute not an uploader attribute00:36
jheskethoh, but we have to do it for the X number of files we're uploading00:37
*** garyh has quit IRC00:37
jheskeththat's a little lame/trickier00:37
openstackgerritClark Boylan proposed openstack-infra/nodepool: Remove duplicate code
fungiafter closer inspection of the hiera file, was the only one with an incorrect chain cert. the other four i've ordered since the transition had the correct (new) chains00:41
clarkbfungi: thank you for debugging00:41
fungii've reenabled puppet on it, and seems happy now too00:43
*** spzala has joined #openstack-infra00:43
*** sarob has joined #openstack-infra00:45
*** ghostpl_ has joined #openstack-infra00:47
*** yamamoto has quit IRC00:47
*** dboik has joined #openstack-infra00:47
openstackgerritgreghaynes proposed openstack-infra/nodepool: WIP: test nodepool upload failures
greghaynesclarkb: ^ still goes boom00:49
greghaynesI can debug00:49
openstackgerritTristan Cacqueray proposed openstack-infra/project-config: Test new zuul split layout
*** tiswanso has joined #openstack-infra00:50
*** dboik_ has joined #openstack-infra00:52
*** tiswanso has quit IRC00:52
*** tiswanso has joined #openstack-infra00:53
*** zul has joined #openstack-infra00:54
*** wenlock has joined #openstack-infra00:54
*** dboik has quit IRC00:55
openstackgerritJoe Gordon proposed openstack-infra/elastic-recheck: Add fingerprint for bug 1430592
openstackbug 1430592 in OpenStack-Gate "testtools-1.7.0 triggering pkg_resources.VersionConflict: (unittest2 0.5.1 (/usr/lib/python2.7/dist-packages), Requirement.parse('unittest2>=1.0.0'))" [Undecided,New]
*** ghostpl_ has quit IRC00:55
anteayatristanC jeblair fungi clarkb pleia2 jhesketh jogo, anyone else interested in the election tooling discussion: ttx (though I know you are out)00:56
*** wenlock has quit IRC00:59
*** bhuvan has quit IRC00:59
*** pblaho has quit IRC01:05
*** ivar-lazzaro has joined #openstack-infra01:09
openstackgerritThanh Ha proposed openstack-infra/jenkins-job-builder: Add FindBugs plugin support
*** ChuckC has quit IRC01:10
*** liusheng has quit IRC01:12
*** liusheng has joined #openstack-infra01:12
*** ddieterly has joined #openstack-infra01:14
*** ddieterl_ has joined #openstack-infra01:15
*** yamamoto has joined #openstack-infra01:15
*** zz_dimtruck is now known as dimtruck01:18
*** ddieterly has quit IRC01:19
pleia2anteaya: thanks!01:19
anteayapleia2: :)01:20
*** crc32 has joined #openstack-infra01:20
asselinclarkb, mmedvede fyi, saw some patches to fix that issue in devstack. Only affecting icehouse & juno. master's fine. I had a different error for that branch.01:20
*** crc32 has quit IRC01:20
lifelesstesttools 1.7.1 up which should have the fixed wheel :)01:21
*** david-lyle_afk has joined #openstack-infra01:21
*** bhuvan has joined #openstack-infra01:21
*** hughhalf is now known as hugh_afk01:22
*** crc32 has joined #openstack-infra01:22
*** ivar-lazzaro has quit IRC01:25
*** dims has quit IRC01:28
*** unicell has quit IRC01:28
*** dims has joined #openstack-infra01:29
*** salv-orlando has quit IRC01:30
*** devlaps has joined #openstack-infra01:31
*** dims has quit IRC01:34
*** banix has quit IRC01:36
*** SumitNaiksatam has quit IRC01:39
*** dims has joined #openstack-infra01:40
openstackgerritRamy Asselin proposed openstack-infra/project-config: Add puppet-openstackci project
*** ChuckC has joined #openstack-infra01:42
*** macjack has joined #openstack-infra01:46
*** ChuckC has quit IRC01:46
*** ChuckC has joined #openstack-infra01:47
*** devlaps has quit IRC01:51
*** patrickeast has quit IRC01:51
*** rwsu is now known as rwsu-afk01:52
mtreinishjeblair: ok sure, I was fine either way with ahving a cookiecutter for it. You could use cookiecutter to make a dir in the repo too, which is what I thought you were going to do01:53
*** otter768 has joined #openstack-infra01:53
*** dims_ has joined #openstack-infra01:55
*** changbl has joined #openstack-infra01:56
*** Ryan_Lane has quit IRC01:58
*** dims has quit IRC01:58
*** otter768 has quit IRC01:58
*** woodster_ has quit IRC02:00
*** sdake_ has quit IRC02:04
*** sdake_ has joined #openstack-infra02:05
*** harlowja_ is now known as harlowja_away02:09
*** yamahata has quit IRC02:13
jeblairmtreinish: oh, that might be a reason to keep doing it then.  let's talk more tomorrow02:13
openstackgerritJames E. Blair proposed stackforge/gertty: Add indexes to revision table
jeblairtchaypo: ^ that makes that part of the change sync much much much much much much much faster for me02:14
mtreinishjeblair: sure02:14
* jeblair eods02:15
*** sdake_ has quit IRC02:20
*** gokrokve has joined #openstack-infra02:26
*** coolsvap has joined #openstack-infra02:29
tchaypolet me give it a try02:30
*** salv-orlando has joined #openstack-infra02:31
openstackgerritSpencer Krum proposed openstack-infra/system-config: Refactor o_p::base into o_p::template
*** nelsnelson has joined #openstack-infra02:33
*** david-lyle_afk has quit IRC02:36
openstackgerritSpencer Krum proposed openstack-infra/system-config: Default to a bogus rsa key on puppetmaster
*** salv-orlando has quit IRC02:38
*** ghostpl_ has joined #openstack-infra02:39
*** mfink_ has joined #openstack-infra02:40
*** sdake_ has joined #openstack-infra02:41
*** ghostpl_ has quit IRC02:44
*** sdake_ has quit IRC02:45
*** spzala has quit IRC02:54
*** carl_baldwin has joined #openstack-infra02:56
*** hdd_ has quit IRC02:57
*** salv-orlando has joined #openstack-infra02:57
*** achanda has joined #openstack-infra02:58
*** banix has joined #openstack-infra03:01
*** dims_ has quit IRC03:02
*** armax has quit IRC03:09
*** salv-orlando has quit IRC03:10
*** achanda has quit IRC03:10
*** coolsvap is now known as coolsvap|afk03:15
*** mrda has joined #openstack-infra03:16
mrdaHey infra - just to confirm, we don't log meeting rooms? (i.e. and this is a deliberate choice?03:17
mrdaSorry - just found what I was looking for at
*** armax has joined #openstack-infra03:17
*** david-lyle_afk has joined #openstack-infra03:17
*** resker has joined #openstack-infra03:18
mtreinishmrda: the meetings are treated separately:
mrdathanks mtreinish03:18
*** ddieterl_ has quit IRC03:18
*** gokrokve has quit IRC03:18
*** ashleighfarnham has joined #openstack-infra03:20
*** armax has quit IRC03:20
*** esker has quit IRC03:21
*** coolsvap|afk is now known as coolsvap03:22
*** markvan has quit IRC03:28
*** mpopow_ has joined #openstack-infra03:32
*** mpopow_ has quit IRC03:32
*** markvan has joined #openstack-infra03:33
*** mpopow_ has joined #openstack-infra03:33
*** stevemar has quit IRC03:34
*** mpopow has quit IRC03:34
*** armax has joined #openstack-infra03:35
*** armax has quit IRC03:40
*** jgrimm is now known as zz_jgrimm03:40
*** mpopow_ has quit IRC03:46
*** mpopow_ has joined #openstack-infra03:46
*** zz_jgrimm is now known as jgrimm03:47
*** garyh has joined #openstack-infra03:49
*** tiswanso has quit IRC03:49
*** patrickeast has joined #openstack-infra03:53
*** mpopow_ has quit IRC03:53
*** otter768 has joined #openstack-infra03:54
*** dhritishikhar has joined #openstack-infra03:56
*** otter768 has quit IRC03:59
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Add option to set swift delete after header
*** asselin_ has joined #openstack-infra04:01
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: zuul-swift-upload use logging for debug msgs
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Add option to set swift delete after header
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: zuul-swift-upload use logging for debug msgs
*** dims has joined #openstack-infra04:03
*** markvan has quit IRC04:05
clarkbjhesketh one comment on the delete after change04:06
jheskethclarkb: I can't see the comment sorry?04:07
*** achanda has joined #openstack-infra04:07
clarkbugh phone commenting04:07
*** markvan has joined #openstack-infra04:07
clarkbthe delete_afters in swift_form_post() should be removed04:08
jheskethheh, go to bed?04:08
jheskethah, missed that, thakns04:08
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Add option to set swift delete after header
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: zuul-swift-upload use logging for debug msgs
*** garyh has quit IRC04:10
*** dims has quit IRC04:10
*** dimtruck is now known as zz_dimtruck04:10
*** BharatK has joined #openstack-infra04:14
*** miqui has quit IRC04:14
*** ujuc has joined #openstack-infra04:23
*** bhuvan has quit IRC04:24
*** ghostpl_ has joined #openstack-infra04:28
*** dhritishikhar has quit IRC04:33
*** ghostpl_ has quit IRC04:33
*** harlowja_at_home has joined #openstack-infra04:35
*** amotoki has joined #openstack-infra04:36
*** hdd_ has joined #openstack-infra04:38
*** crc32 has quit IRC04:39
*** unicell has joined #openstack-infra04:42
*** dboik_ has quit IRC04:42
*** dboik has joined #openstack-infra04:43
*** banix has quit IRC04:48
*** patrickeast has quit IRC04:48
*** ddieterly has joined #openstack-infra04:49
*** nelsnelson has quit IRC04:52
*** ddieterly has quit IRC04:53
*** salv-orlando has joined #openstack-infra04:57
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/shade: Add unit test for create_server
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/shade: Add basic unit test for shade.openstack_cloud
*** carl_baldwin has quit IRC05:01
*** xyang1 has quit IRC05:02
*** bhuvan has joined #openstack-infra05:05
*** salv-orlando has quit IRC05:10
*** salv-orlando has joined #openstack-infra05:12
*** yfried|afk is now known as yfried_05:14
*** salv-orlando has quit IRC05:16
*** Alexandra_ has joined #openstack-infra05:17
*** koolhead17 has joined #openstack-infra05:18
*** koolhead17 has quit IRC05:18
*** koolhead17 has joined #openstack-infra05:18
*** deepakcs has joined #openstack-infra05:18
*** dboik has quit IRC05:20
*** dhritishikhar has joined #openstack-infra05:20
*** dboik has joined #openstack-infra05:20
*** BharatK has left #openstack-infra05:25
openstackgerritQiming Teng proposed openstack-infra/project-config: new-project: stackforge/senlin
*** doug-fish has quit IRC05:28
*** yfried_ is now known as yfried|afk05:28
*** salv-orlando has joined #openstack-infra05:29
*** doug-fish has joined #openstack-infra05:29
*** achanda has quit IRC05:31
*** yfried|afk is now known as yfried_05:31
*** salv-orlando has quit IRC05:34
*** achanda has joined #openstack-infra05:34
*** salv-orlando has joined #openstack-infra05:35
*** salv-orlando has quit IRC05:35
*** garyk has quit IRC05:35
*** unicell has quit IRC05:37
*** unicell has joined #openstack-infra05:37
*** koolhead_ has joined #openstack-infra05:38
*** koolhead17 has quit IRC05:40
*** salv-orlando has joined #openstack-infra05:41
*** mrmartin has joined #openstack-infra05:41
*** dhritishikhar has quit IRC05:42
*** salv-orlando has quit IRC05:45
*** achanda has quit IRC05:48
*** unicell has quit IRC05:48
*** unicell has joined #openstack-infra05:49
*** bnemec has quit IRC05:49
*** ddieterly has joined #openstack-infra05:50
*** bnemec has joined #openstack-infra05:52
*** achanda has joined #openstack-infra05:52
*** ddieterly has quit IRC05:54
*** otter768 has joined #openstack-infra05:55
*** achanda has quit IRC05:58
*** otter768 has quit IRC06:00
*** achanda has joined #openstack-infra06:01
*** dboik has quit IRC06:01
*** yfried_ is now known as yfried|afk06:01
*** jyuso1 has quit IRC06:01
*** camunoz has quit IRC06:03
*** salv-orlando has joined #openstack-infra06:04
*** camunoz has joined #openstack-infra06:05
*** BharatK has joined #openstack-infra06:06
*** yfried|afk is now known as yfried_06:06
*** ashleighfarnham has quit IRC06:06
*** salv-orlando has quit IRC06:08
*** mrmartin has quit IRC06:08
*** harlowja_at_home has quit IRC06:11
*** Sukhdev has joined #openstack-infra06:12
*** punith has joined #openstack-infra06:13
*** achanda has quit IRC06:14
*** dmellado has quit IRC06:15
*** salv-orlando has joined #openstack-infra06:15
*** ghostpl_ has joined #openstack-infra06:17
*** dmellado has joined #openstack-infra06:19
openstackgerritSwati Sharma proposed openstack/requirements: Adding weasyprint to download Horizon reports in PDF format
*** salv-orlando has quit IRC06:20
*** salv-orlando has joined #openstack-infra06:21
*** ghostpl_ has quit IRC06:21
*** yfried_ is now known as yfried|afk06:23
*** yfried|afk is now known as yfried_06:26
*** salv-orlando has quit IRC06:26
*** salv-orlando has joined #openstack-infra06:27
*** andreykurilin_ has joined #openstack-infra06:28
*** kiseok7_ has quit IRC06:31
*** kiseok7_ has joined #openstack-infra06:32
*** salv-orlando has quit IRC06:32
*** Sukhdev has quit IRC06:33
*** Alexandra_ has quit IRC06:38
*** salv-orlando has joined #openstack-infra06:38
BharatKping AJaeger06:41
*** salv-orlando has quit IRC06:43
*** resker has quit IRC06:49
*** esker has joined #openstack-infra06:50
*** ddieterly has joined #openstack-infra06:51
openstackgerritSudheesh proposed openstack-infra/project-config: project config changes for openstack-docker cookbook
*** ildikov has quit IRC06:51
*** ddieterly has quit IRC06:55
*** andreykurilin_ has quit IRC06:57
*** hdd_ has quit IRC06:57
*** kevinbenton has quit IRC06:59
*** HeOS has quit IRC07:00
AJaegermorning, BharatK07:02
*** garyk has joined #openstack-infra07:03
*** rlandy has joined #openstack-infra07:03
*** fdegir has joined #openstack-infra07:05
BharatKAJaeger: Good Morning07:09
BharatKAJaeger: Regarding the patch
BharatKAJaeger: As per this patch it create jobs with name "gate-tempest-dsvm-full-sheepdog" and gate-tempest-dsvm-full-glusterfs07:10
BharatKAJaeger: But I want to add this job to check pipeline of Cinder's project07:10
AJaegerSo, what's your problem?07:11
BharatKAJaeger: Having check makes more sense I guess, then having "gate-tempest-dsvm-full-glusterfs"07:11
AJaegerBharatK, that'S just naming, we're not consistent here. No need to change it07:12
BharatKAJaeger: Please have a look at chat with Clark07:12
deepakcsAJaeger, the larger point being, some folks haev raised Q on why its caleld gate, when its non voting and only runs during check pipeline07:12
deepakcsAJaeger,  so we were thinking to make it {pipeline} and jobs can specify gate/check as they like it. We want to call ours as check not gate07:13
deepakcsAJaeger, also sheepdog's job was with prefix check (before that patch) and the patch hardcodes it to gate !07:14
deepakcsAJaeger, I would propose to make it a variable, which can then be controlled by individual jobs' wish, what say ?07:14
AJaegerDid you see my email regarding the -nv?07:14
BharatKAJaeger: Yeah I have seen that.07:15
deepakcsAJaeger, yes, but thats for voting/nv, how does that relate to this ?07:15
AJaegerit's something you should keep in mind when doing the changes ;)07:15
sabarittx, dhellmann: we need for glance k3. Can you help us with the review ?07:16
yolandajhesketh, saw your comments about that zuul param. We had this setting downstream for ages, i've always seen that there, and it looks it does nothing? do you remember having that zuul_override parameter at some time?07:16
deepakcsAJaeger, oh yes, i agree, but do u agree to make the variable {pipeline} instead of hardcodign it to gate for plugin template ?07:16
*** che-arne has joined #openstack-infra07:16
AJaegerdeepakcs, check or gate is for me just naming, I don't really care.07:17
deepakcsAJaeger, technically check is before +2, gate is for the final check, so there is a diff and the name causes confusion if used incorrectly07:17
*** ghostpl_ has joined #openstack-infra07:17
AJaegerWe'Re not consistent here at all ;(, see how all the infra jobs are named:
deepakcsAJaeger, I know and ack the inconsistency, but i feel the job owners should be given the freedom to choose check/gate07:19
AJaegerdeepakcs, yes, you're right on the naming but as long as there's no clear policy, I will not reject patches that add further gates.07:19
AJaegerdeepakcs, then do it. I'm not blocking your patch - or do I?07:19
deepakcsAJaeger, no you are not, just wanted to discuss with you before BharatK sends the patch07:20
AJaegerAh, now I see.07:20
deepakcsAJaeger, we just need the freedon to choose check Vs gate in the job name for each job07:20
deepakcsAJaeger, right now for devstack plugin jobs its hardcoded to gate-<name>07:20
deepakcsAJaeger, we want to change it to {pipeline}-<name>07:20
deepakcsAJaeger, and {pipeline} is defiend by the actual job template07:21
*** punith has quit IRC07:21
*** tnovacik has joined #openstack-infra07:21
*** sarob has quit IRC07:22
*** ghostpl_ has quit IRC07:23
deepakcsAJaeger, maybe its clear once BharatK sends a patch07:23
AJaegerdeepakcs, go ahead and change it the way most projects setup with {pipeline} and {branch-designator} to allow usage of -nv as well, see for example how grenade-dsvm-ironic-sideways is setup in jenkins07:23
*** scheuran has joined #openstack-infra07:25
*** yfried_ is now known as yfried|afk07:26
*** yfried|afk is now known as yfried_07:26
*** michchap_ has joined #openstack-infra07:28
*** michchap has quit IRC07:28
deepakcsAJaeger, cool, we will send the patch soon07:31
*** hashar has joined #openstack-infra07:37
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Removed grunt:server
*** che-arne has quit IRC07:39
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Added sane result set indicators.
*** mrunge has joined #openstack-infra07:45
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Enabled eslint:no-use-before-define
*** shardy has joined #openstack-infra07:48
*** che-arne has joined #openstack-infra07:49
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: Add neutron-aiopcpu-dvr experimental job
*** ddieterly has joined #openstack-infra07:51
GheRiveromorning all07:52
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Enabled eslint:space-infix-ops
*** Somay has joined #openstack-infra07:54
*** coolsvap is now known as coolsvap|brb07:54
*** ddieterly has quit IRC07:56
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Enabled eslint:no-shadow
*** otter768 has joined #openstack-infra07:56
*** mpaolino has joined #openstack-infra07:57
*** otter768 has quit IRC08:01
*** chlong has quit IRC08:01
*** gampel has quit IRC08:03
*** jcoufal has joined #openstack-infra08:04
*** markus_z has joined #openstack-infra08:04
*** salv-orlando has joined #openstack-infra08:05
*** Qiming has joined #openstack-infra08:06
openstackgerritQiming Teng proposed openstack-infra/project-config: new-project: stackforge/senlin
*** _shaps_ has joined #openstack-infra08:06
QimingAJaeger, thanks for the review, new patch submitted, ... hope addressed issues you pointed out08:07
AJaegerQiming, thanks for the quick update. I'll review later.08:08
AJaegerMorning, GheRivero !08:08
openstackgerritFatih Degirmenci proposed openstack-infra/jenkins-job-builder: Pass through git revision to triggered jobs
*** e0ne has joined #openstack-infra08:11
*** koolhead_ has quit IRC08:12
*** _shaps_ has quit IRC08:13
*** salv-orlando has quit IRC08:15
*** mrodden has quit IRC08:18
*** ghostpl_ has joined #openstack-infra08:18
*** Somay has quit IRC08:18
openstackgerritSerg Melikyan proposed openstack-infra/project-config: Fix for gate-murano-congress-devstack-dsvm
*** sarob has joined #openstack-infra08:22
*** HeOS has joined #openstack-infra08:23
openstackgerritSumit Naiksatam proposed openstack-infra/project-config: Create experimental job for group-based-policy
*** mrodden has joined #openstack-infra08:24
*** ghostpl_ has quit IRC08:25
*** mpaolino has quit IRC08:25
*** salv-orlando has joined #openstack-infra08:25
*** salv-orlando has quit IRC08:25
*** mpavone has joined #openstack-infra08:26
*** coolsvap|brb is now known as coolsvap08:26
*** sarob has quit IRC08:27
*** mrmartin has joined #openstack-infra08:28
*** mrmartin has quit IRC08:29
*** ildikov has joined #openstack-infra08:33
*** devvesa has joined #openstack-infra08:36
*** mpaolino has joined #openstack-infra08:38
*** arxcruz has joined #openstack-infra08:44
*** dizquierdo has joined #openstack-infra08:45
openstackgerritTeemu Patja proposed stackforge/python-jenkins: Add a get_nodes method.
openstackgerritTeemu Patja proposed stackforge/python-jenkins: Add a get_nodes method
*** ddieterly has joined #openstack-infra08:52
openstackgerritBharat Kumar Kobagana proposed openstack-infra/project-config: Modifying devstack plugin job template
openstackgerritBharat Kumar Kobagana proposed openstack-infra/project-config: Adding 'check-tempest-dsvm-full-glusterfs' voting job
*** ddieterly has quit IRC08:56
*** yfried_ is now known as yfried|pto09:01
*** jedimike has joined #openstack-infra09:03
*** ominakov_ has joined #openstack-infra09:04
*** mrmartin has joined #openstack-infra09:05
*** ssam2 has joined #openstack-infra09:05
*** jlanoux has joined #openstack-infra09:08
*** ujuc has quit IRC09:12
openstackgerrityolanda.robla proposed openstack-infra/puppet-zuul: Add fetch_url parameters to zuul.conf
*** Somay has joined #openstack-infra09:13
openstackgerritMartin Mágr proposed openstack-infra/project-config: Updated Packstack project
*** afazekas_ has joined #openstack-infra09:16
*** mpaolino has quit IRC09:16
openstackgerritMartin Mágr proposed openstack-infra/project-config: puppet-openstack update
*** ghostpl_ has joined #openstack-infra09:19
openstackgerritMartin Mágr proposed openstack-infra/project-config: puppet-openstack update
*** mpaolino has joined #openstack-infra09:21
*** yfried|pto is now known as yfried|afk09:23
*** yfried|afk is now known as yfried|pto09:25
*** jistr has joined #openstack-infra09:25
*** hichihara has quit IRC09:25
*** e0ne has quit IRC09:27
*** kevinbenton has joined #openstack-infra09:27
*** ihrachyshka has joined #openstack-infra09:28
*** teran has quit IRC09:30
*** zz_johnthetubagu is now known as johnthetubaguy09:35
*** ghostpl_ has quit IRC09:35
*** ghostpl_ has joined #openstack-infra09:37
*** ihrachyshka has quit IRC09:37
*** ghostpl_ has quit IRC09:38
*** mpaolino has quit IRC09:38
*** ihrachyshka has joined #openstack-infra09:38
*** ghostpl_ has joined #openstack-infra09:41
*** salv-orlando has joined #openstack-infra09:44
*** _shaps_ has joined #openstack-infra09:45
*** ociuhandu has joined #openstack-infra09:45
*** claudiub has joined #openstack-infra09:45
*** teran has joined #openstack-infra09:46
*** salv-orlando has quit IRC09:46
*** mpaolino has joined #openstack-infra09:47
*** jistr has quit IRC09:50
*** _shaps_ has quit IRC09:50
*** ociuhandu has quit IRC09:50
*** teran has quit IRC09:51
*** jistr has joined #openstack-infra09:52
*** ociuhandu has joined #openstack-infra09:53
*** Qiming has quit IRC09:53
*** ddieterly has joined #openstack-infra09:53
*** yamamoto has quit IRC09:54
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Enable pxe_ipa-nv on ironic-python-agent
*** ghostpl_ has quit IRC09:57
*** otter768 has joined #openstack-infra09:57
*** ddieterly has quit IRC09:58
*** bhuvan has quit IRC09:59
*** dizquierdo has quit IRC09:59
*** mfink_ has quit IRC10:01
SomayHello, can anyone review my patch here -
*** otter768 has quit IRC10:02
*** ghostpl_ has joined #openstack-infra10:03
*** deepakcs has quit IRC10:04
*** ghostpl_ has quit IRC10:06
*** jp_at_hp has joined #openstack-infra10:07
*** salv-orlando has joined #openstack-infra10:10
*** _nadya_ has joined #openstack-infra10:11
*** ghostpl_ has joined #openstack-infra10:11
*** ghostpl_ has quit IRC10:14
*** salv-orlando has quit IRC10:14
openstackgerritTeemu Patja proposed stackforge/python-jenkins: Add a get_nodes method
*** salv-orl_ has joined #openstack-infra10:15
*** salv-orl_ has quit IRC10:15
*** e0ne has joined #openstack-infra10:16
*** dims has joined #openstack-infra10:18
*** yamamoto has joined #openstack-infra10:19
*** praveenkumar is now known as bloodbath10:22
mkodererhi folks, I have some issues with registering some of my team members10:23
mkoderergit review -s fails with "fatal: ICLA contributor agreement requires current contact information."10:23
*** chlong has joined #openstack-infra10:23
mkodererbut it's not possible to store the contact information .. we get a "Cannot store contact information" in gerrit10:24
*** sarob has joined #openstack-infra10:24
*** tnovacik has quit IRC10:25
*** BobBall_AWOL is now known as BobBall10:26
*** sarob has quit IRC10:30
*** pc_m has joined #openstack-infra10:30
*** bloodbath is now known as praveenkumar10:31
*** che-arne has quit IRC10:31
*** _nadya_ has quit IRC10:31
*** che-arne has joined #openstack-infra10:31
*** teran has joined #openstack-infra10:37
*** jistr has quit IRC10:38
*** jistr has joined #openstack-infra10:39
*** vigneshvar has joined #openstack-infra10:40
*** kmartin has quit IRC10:41
tchaypoI thought I remembered a feedback box on the website10:42
*** koolhead17 has joined #openstack-infra10:42
tchaypoif I could find it again I’d use it to report “a second pass just fot you” on
*** teran has quit IRC10:42
*** koolhead_ has joined #openstack-infra10:42
*** koolhead17 has quit IRC10:42
*** kevinbenton has quit IRC10:45
*** teran has joined #openstack-infra10:46
*** mfink_ has joined #openstack-infra10:48
*** salv-orlando has joined #openstack-infra10:48
*** asselin has quit IRC10:52
*** hyakuhei has joined #openstack-infra10:54
*** ddieterly has joined #openstack-infra10:54
*** ociuhandu has quit IRC10:54
openstackgerrityolanda.robla proposed openstack-infra/system-config: Parameterize url to clone puppet modules
hyakuheiThe security group has been loaned a pair of HSMs from HP to be used by various OpenStack projects (Anchor, Barbican, Castellan) we could probably rack them in our public cloud datacentre space but it would make more sense to get them setup with the rest of the OpenStack infrastructure - I’m not really sure who to talk to about the options/next steps?10:55
*** ociuhandu has joined #openstack-infra10:56
*** bhuvan has joined #openstack-infra10:56
*** mrmartin has quit IRC10:57
*** ddieterly has quit IRC10:58
*** bhuvan has quit IRC11:01
*** claudiub has quit IRC11:01
mkodererok fixes the issue :)11:02
*** dizquierdo has joined #openstack-infra11:05
*** ghostpl_ has joined #openstack-infra11:08
*** yamamoto has quit IRC11:08
*** che-arne has quit IRC11:12
*** claudiub has joined #openstack-infra11:12
*** che-arne has joined #openstack-infra11:13
*** ghostpl_ has quit IRC11:15
*** ghostpl_ has joined #openstack-infra11:15
*** tnovacik has joined #openstack-infra11:16
*** che-arne has quit IRC11:17
*** che-arne has joined #openstack-infra11:17
*** claudiub has quit IRC11:21
*** e0ne is now known as e0ne_11:21
*** che-arne has quit IRC11:21
*** che-arne has joined #openstack-infra11:22
*** sarob has joined #openstack-infra11:25
*** e0ne_ is now known as e0ne11:26
*** che-arne has quit IRC11:27
*** tosky has joined #openstack-infra11:27
*** che-arne has joined #openstack-infra11:27
*** yamamoto has joined #openstack-infra11:29
*** dhritishikhar has joined #openstack-infra11:29
*** ldnunes has joined #openstack-infra11:30
*** sarob has quit IRC11:31
*** ghostpl_ has quit IRC11:32
*** ghostpl_ has joined #openstack-infra11:33
*** che-arne has quit IRC11:36
*** che-arne has joined #openstack-infra11:37
*** pelix has joined #openstack-infra11:39
*** dizquierdo has quit IRC11:39
*** dhritishikhar has quit IRC11:43
*** che-arne has quit IRC11:44
*** che-arne has joined #openstack-infra11:44
*** yamamoto has quit IRC11:46
jedimikewhat's the git repo for the bandersnatch puppet module?11:47
*** Ala has quit IRC11:49
*** mpaolino has quit IRC11:50
*** e0ne is now known as e0ne_11:51
*** Ala has joined #openstack-infra11:51
*** ddieterly has joined #openstack-infra11:54
*** esker has quit IRC11:55
*** dizquierdo has joined #openstack-infra11:56
*** e0ne_ has quit IRC11:56
*** otter768 has joined #openstack-infra11:58
*** ddieterly has quit IRC11:59
*** otter768 has quit IRC12:02
*** mpaolino has joined #openstack-infra12:03
*** erlon has joined #openstack-infra12:04
*** mpaolino has quit IRC12:05
*** mpavone has quit IRC12:05
*** macjack has quit IRC12:07
*** vigneshvar has quit IRC12:07
*** johnthetubaguy is now known as zz_johnthetubagu12:11
*** markvoelker has joined #openstack-infra12:14
*** ghostpl_ has quit IRC12:17
mordredjedimike: modules/openstack_project/manifests/pypi_mirror.pp12:18
*** e0ne has joined #openstack-infra12:18
mordredjedimike: in system-config12:18
mordredjedimike: it seems we did not split out a module for that12:18
jedimikemordred, so I just got daily snapshots with configurable retention policy working with bandersnatch, should I start a new module for it?12:19
*** dprince has joined #openstack-infra12:20
*** aysyd has joined #openstack-infra12:21
*** mpavone has joined #openstack-infra12:21
*** ghostpl_ has joined #openstack-infra12:22
mordredjedimike: yes. I think a new module for that would be the right choice12:23
jedimikemordred, great, we really need it for gozer. Snapshots were easy in the end, configure bandersnatch not to purge removed packages, and copy the simple index into a snapshot-${date} directory12:25
jedimikethe retention policy just sets to keep the last n days of snapshots12:25
*** sarob has joined #openstack-infra12:27
*** BharatK has quit IRC12:27
*** claudiub has joined #openstack-infra12:28
*** _shaps_ has joined #openstack-infra12:31
*** sarob has quit IRC12:31
*** dims has quit IRC12:33
*** pblaho has joined #openstack-infra12:34
*** jlanoux has quit IRC12:34
*** dims has joined #openstack-infra12:34
*** funzo has quit IRC12:35
*** adalbas has joined #openstack-infra12:36
openstackgerritMike Heald proposed openstack-infra/project-config: Project for puppet-bandersnatch
*** claudiub has quit IRC12:38
*** Somay has quit IRC12:39
*** zz_johnthetubagu is now known as johnthetubaguy12:39
*** dboik has joined #openstack-infra12:41
*** esker has joined #openstack-infra12:44
*** Somay has joined #openstack-infra12:46
*** esker has quit IRC12:47
*** baoli has joined #openstack-infra12:47
*** dboik has quit IRC12:49
*** esker has joined #openstack-infra12:50
*** esker has quit IRC12:51
*** yamamoto has joined #openstack-infra12:51
*** spredzy is now known as spredzy|Afk12:51
*** bswartz has quit IRC12:51
*** tnovacik has quit IRC12:52
*** baoli has quit IRC12:52
*** dkliban_afk is now known as dkliban12:52
*** baoli has joined #openstack-infra12:53
*** gordc has joined #openstack-infra12:53
*** ildikov has quit IRC12:57
*** tnovacik has joined #openstack-infra12:59
*** ddieterly has joined #openstack-infra13:01
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added story types
*** hyakuhei has quit IRC13:03
*** mattfarina has joined #openstack-infra13:04
*** banix has joined #openstack-infra13:04
*** banix has quit IRC13:05
mrungehello, can please someone give me a helping hand?13:08
mrungeIt's about
mrunge(raise cap for Django)13:08
*** amotoki has quit IRC13:08
mrungethat's the only change, jenkins complains afterwards about missing django_nose, which is listed in horizons test-requirements13:09
mrungeI'm just unsure, how to proceed from here and how to fix that.13:09
*** esker has joined #openstack-infra13:10
*** ildikov has joined #openstack-infra13:11
*** banix has joined #openstack-infra13:13
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added story types
*** amotoki has joined #openstack-infra13:15
*** jlanoux has joined #openstack-infra13:15
banixjeblair: Any chance you can have a second look at
*** sdake_ has joined #openstack-infra13:15
*** tiswanso has joined #openstack-infra13:16
*** ildikov_ has joined #openstack-infra13:17
*** dboik has joined #openstack-infra13:17
*** tiswanso has quit IRC13:18
*** ildikov has quit IRC13:19
*** Longgeek has joined #openstack-infra13:19
*** tiswanso has joined #openstack-infra13:19
banixany infra cores who could review:
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Tasks branches, milestones and projects ids
*** cdent has joined #openstack-infra13:22
*** dustins has joined #openstack-infra13:22
*** banix has quit IRC13:22
*** bswartz has joined #openstack-infra13:25
*** peristeri has joined #openstack-infra13:25
fungimkoderer: are your associates still having trouble? see for troubleshooting suggestions13:25
*** mriedem has joined #openstack-infra13:27
fungimkoderer: nevermind i see further down you sorted it out13:27
*** sarob has joined #openstack-infra13:27
fungii need to link up those two questions in ask.o.o13:28
fungiit's too bad hyakuhei wandered off. i had already discussed his topic at a high level with someone else (mostly pointing out the challenges associated with that proposition)13:29
*** banix has joined #openstack-infra13:29
*** garyh has joined #openstack-infra13:29
*** _nadya_ has joined #openstack-infra13:29
*** spredzy|Afk is now known as spredzy13:30
*** jcoufal has quit IRC13:31
*** jcoufal has joined #openstack-infra13:32
*** banix has quit IRC13:32
*** dboik has quit IRC13:32
*** sarob has quit IRC13:33
*** dboik has joined #openstack-infra13:33
*** dboik has quit IRC13:33
*** dboik has joined #openstack-infra13:34
*** gordc_ has joined #openstack-infra13:36
*** gordc has quit IRC13:42
mordredfungi: I mean, yeah13:42
mordredfungi: even in a world where we have infra-cloud existing, it's still kinda problematic13:42
*** tiswanso has quit IRC13:42
mordred_until_ then it's impossible13:42
*** Longgeek has quit IRC13:43
fungiright, rob clark (hp, ossg ptl) e-mailed me yesterday asking who to get in touch with about it, and i laid out the challenges when i replied13:44
fungimostly that we'd need the same hardware available in additional service providers, as a start, before we could really support something like that for a gating job13:45
fungiand i didn't even get into asking if it already had mainline kernel support13:46
fungionce someone from hp+barbican engaged us as a group, i figured we'd aim them at third-party testing13:47
fungibut wanted to see if any of the rest of the team had other suggestions13:47
*** annegentle has joined #openstack-infra13:49
fungigiven that hsm/tpm hardware usually just has a defined api in the driver for interacting with it, i have to wonder why it's necessary to have the actual hardware present. seems like it wouldn't be tough to just mock if the goal is to test barbican itself13:49
*** mrunge is now known as mrunge_bbl13:49
*** devvesa has quit IRC13:50
fungibut i really don't have any details as to what they're trying to do with it13:50
*** dkranz has joined #openstack-infra13:51
*** eharney has joined #openstack-infra13:55
*** Longgeek has joined #openstack-infra13:55
*** r-daneel has joined #openstack-infra13:55
*** e0ne has quit IRC13:57
*** e0ne has joined #openstack-infra13:57
*** changbl has quit IRC13:58
*** otter768 has joined #openstack-infra13:59
*** jgrimm is now known as zz_jgrimm14:00
*** gordc_ is now known as gordc14:00
*** fdegir has quit IRC14:00
openstackgerritMerged openstack-infra/project-config: Fix for gate-murano-congress-devstack-dsvm
*** fdegir has joined #openstack-infra14:01
*** otter768 has quit IRC14:03
*** ddieterly has quit IRC14:05
*** tsg_ has joined #openstack-infra14:05
*** tgohad has joined #openstack-infra14:05
*** tgohad has quit IRC14:05
*** ddieterl_ has joined #openstack-infra14:05
*** yamamoto has quit IRC14:05
*** yamamoto has joined #openstack-infra14:05
*** cody-somerville has joined #openstack-infra14:06
*** craigbr has joined #openstack-infra14:06
*** garyh has quit IRC14:06
*** nelsnelson has joined #openstack-infra14:06
openstackgerritChris Dent proposed openstack/requirements: Update gabbi to 0.12.0
dhellmannsabari: +214:07
*** changbl has joined #openstack-infra14:09
*** che-arne has quit IRC14:10
*** che-arne has joined #openstack-infra14:10
openstackgerritBharat Kumar Kobagana proposed openstack-infra/project-config: Modifying devstack plugin job template
*** gokrokve has joined #openstack-infra14:13
*** ayoung is now known as ayoung_afk14:13
openstackgerritThanh Ha proposed openstack-infra/jenkins-job-builder: Add FindBugs plugin support
*** _shaps_ has quit IRC14:15
*** kgiusti has joined #openstack-infra14:20
*** hyakuhei has joined #openstack-infra14:20
*** salv-orlando has quit IRC14:23
*** timcline has joined #openstack-infra14:23
*** bknudson has joined #openstack-infra14:24
*** ildikov_ has quit IRC14:26
*** esker has quit IRC14:26
*** sigmavirus24_awa is now known as sigmavirus2414:27
*** wenlock has joined #openstack-infra14:28
*** sarob has joined #openstack-infra14:29
*** timcline_ has joined #openstack-infra14:31
*** salv-orlando has joined #openstack-infra14:31
openstackgerritJames E. Blair proposed stackforge/gertty: Only sync parent commit once
jeblairtchaypo: ^ that should make a difference too14:32
*** e0ne is now known as e0ne_14:32
jeblairfungi: i agree with everythng you said re hsm14:32
fungijeblair: maybe someone from that group will stick around in irc long enough to actually hear it next time ;)14:33
*** gokrokve_ has joined #openstack-infra14:33
*** banix has joined #openstack-infra14:33
*** sarob has quit IRC14:33
*** e0ne_ is now known as e0ne14:33
jeblairit's okay, channel's logged14:34
*** timcline has quit IRC14:34
*** eharney has quit IRC14:34
fungitoo true14:35
*** Somay has quit IRC14:35
*** radez_g0n3 is now known as radez14:36
*** gokrokve has quit IRC14:36
*** mtanino has joined #openstack-infra14:36
jeblair  File "/home/corvus/git/venv/local/lib/python2.7/site-packages/requests/packages/urllib3/util/", line 87, in create_connection14:37
jeblair    raise err14:37
jeblairProtocolError: ('Connection aborted.', error(101, 'Network is unreachable'))14:37
jeblairrequests.ProtocolError does not exist....14:37
jeblairdoes that mean the ProtocolError returned to my app is something like requests.packages.urllib3.ProtocolError ?14:38
*** esker has joined #openstack-infra14:38
fungiyeah, something like that14:39
fungiyeah, courtesy of "the vendoring"14:40
fungibecause requests is calling its vendored urllib3 rather than whatever may be installed14:41
clarkbI think thats a requests bug as they are supposed to wrap all exceptions for you so the requests exception base class is base of everything14:41
fungisigmavirus24 would know, if he's around14:41
jeblairclarkb: yeah, i was wondering if that was supposed to be invisible to me, or if the urllib3 api is part of the requests api...14:41
sigmavirus24jeblair: what version?14:41
*** BharatK has joined #openstack-infra14:42
sigmavirus24Yeah that's fixed in newer ones14:42
sigmavirus24urllib3 added a new exception in place of an old one without considering backwards compat so we missed it14:42
fungigotta love that14:42
jeblairsigmavirus24: oh cool.  do you know if that becomes a requests.ConnectionError ?14:42
sigmavirus24It should14:42
*** marcusvrn has joined #openstack-infra14:43
sigmavirus24jeblair: If I remember the bug correctly =)14:43
*** mrmartin has joined #openstack-infra14:43
*** dboik_ has joined #openstack-infra14:43
*** ChuckC_ has joined #openstack-infra14:43
sigmavirus24fungi: why do you think Cory and I have become far more involved in urllib3?14:43
jeblairsigmavirus24: ok, so i should just need a version bump, thanks14:43
*** _shaps_ has joined #openstack-infra14:43
sigmavirus24It's my perpetual problem that I need to become further involved in the dependencies of the projects I maintain14:43
fungispeaking of...14:43
fungihashar: who's managing the etherpad instance for wmf?14:44
*** ChuckC has quit IRC14:44
jeblairsigmavirus24: there is an end in sight!  somewhere around the c compiler and assembler...14:44
*** wenlock has quit IRC14:44
*** wenlock has joined #openstack-infra14:44
sigmavirus24jeblair: nah, will need to start working on the kernel with it's code of conflict now =P14:44
fungihashar: i've volunteered to help etherpad upstream get cves for their security vulnerabilities and announce them in better ways than twitter and a blog, and was goign to see if anyone from wmf and/or mofo wanted to pitch in too14:44
*** craigbr has quit IRC14:44
*** marcusvrn1 has quit IRC14:45
*** claudiub has joined #openstack-infra14:45
*** dboik has quit IRC14:46
*** tonytan4ever has joined #openstack-infra14:46
jeblairfungi: that's awesome14:46
jeblairfungi: did you connect with them on irc or mail or...?14:47
fungiand then via e-mail to confirm contact info14:47
hasharfungi: let me find :)14:47
fungithe primary contributor for etherpad is fairly active in #etherpad14:47
fungion freenode14:47
jeblairthere's also an #etherpad-lite-dev, know anything about it?14:48
clarkbany additional testing on etherpad-dev people want to do before I propose an update?14:48
*** ChuckC_ has quit IRC14:49
jeblairclarkb: it looked good to me14:49
*** nelsnelson has quit IRC14:49
hashariirc there is a version that depends on  libreoffice and whatever language, and another version which is a rewrite using nodejs14:49
fungiyeah, seems to be working fine14:49
hasharthough dont quote me on that :)14:49
*** nelsnelson has joined #openstack-infra14:50
clarkbfungi ya I think #etherpad is not the one we use unless etherpad-lite moved in there14:50
fungihashar: the etherpad-lite nodejs rewrite has basically supplanted the old google app thing14:50
jeblairi think they may have14:50
jeblair14:47 -!- Topic for #etherpad: The Etherpad Foundation - Devs in #etherpad-lite-dev - http://etherpad.org14:50
hasharsounds good14:50
fungiclarkb: it's all one now apparently14:50
clarkbI see14:50
jeblairthere is no etherpad but etherpad-lite14:51
fungiafaict the old etherpad is dead14:51
*** _shaps_ has quit IRC14:52
*** carl_baldwin has joined #openstack-infra14:52
*** yfried|pto is now known as yfried|afk14:52
*** cody-somerville has quit IRC14:53
hasharfungi: you can poke  which spams the appropriate persons. Add in "Alexandros Kosiaris <>"  who did our Debian package14:53
fungihashar: thanks14:53
*** funzo has joined #openstack-infra14:53
hasharfungi: our package is obviously on our Gerrit,n,z14:53
*** zz_jgrimm is now known as jgrimm14:55
hasharmeanwhile, I had an ugly debian package for Zuul :]  Based on Paul Belanger work from last year14:55
*** marcusvrn1 has joined #openstack-infra14:56
*** marcusvrn has quit IRC14:58
*** yfried|afk is now known as yfried|pto14:59
*** yamamoto has quit IRC15:00
openstackgerritClark Boylan proposed openstack-infra/system-config: Deploy latest version of etherpad-lite
*** dangers_away is now known as dangers15:00
clarkbsomething like ^ I am not entirely sure if that will update the git repo properly :/ I really don't want to ensure => latest on a specific commit if I can avoid it15:00
clarkbI can construct a test case with puppet to test though15:01
clarkbhashar: do you know what node.js you run for that?15:02
*** stevemar has joined #openstack-infra15:02
*** dannywilson has joined #openstack-infra15:02
*** dannywilson has quit IRC15:02
jeblairclarkb: are you sure you don't want to ensure latest develop branch?15:03
*** marcusvrn1 has quit IRC15:03
*** dannywilson has joined #openstack-infra15:03
jeblairperhaps you don't want to risk an upgrade during a summit?15:03
clarkbjeblair: ya, I think we want latest develop on the -dev server then we can pick from that if things work15:03
fungiclarkb: any reason not to stick with 1.5.1?15:03
hasharclarkb: probably 0.8 from Precise.15:03
*** eharney has joined #openstack-infra15:03
clarkbjeblair: otherwise we may end up with broken during important times15:04
clarkbhashar: precise has 0.6 I thought?15:04
clarkbfungi: tested latest commit, if we want to test specific releases we can do that too15:04
fungithey do seem to be tagging releases with some regularity now
*** jcoufal has quit IRC15:05
*** jcoufal has joined #openstack-infra15:05
clarkboddly git describe says we are on 1.5.0-stuff15:05
jeblairclarkb: an alternative would be to run develop always and freeze before summits and unfreeze after.  but your process works for me, esp if they will start issuing cves.15:05
fungibut yeah, tracking develop tip seems fine too15:06
fungiand we have some control if we manually update the explicit commit15:06
hasharclarkb: yeah 0.6.12, but we have backported 0.8.x to Precise15:06
clarkbbut 1.5.1 is merged into develop so likely just not tagged properly15:06
clarkbhashar: I see15:07
*** banix has quit IRC15:07
*** armax has joined #openstack-infra15:07
*** fdegir1 has joined #openstack-infra15:08
*** marcusvrn has joined #openstack-infra15:09
ttxsabari: done15:09
*** craigbr has joined #openstack-infra15:09
*** fdegir has quit IRC15:09
*** vponomaryov has left #openstack-infra15:11
*** vponomaryov has joined #openstack-infra15:11
*** banix has joined #openstack-infra15:12
*** kmartin has joined #openstack-infra15:12
*** fdegir1 has quit IRC15:13
*** dizquierdo has quit IRC15:13
*** marcusvrn has quit IRC15:14
openstackgerritJames E. Blair proposed stackforge/gertty: Upgrade to requests 2.5.3
openstackgerritJames E. Blair proposed stackforge/gertty: Add a 30 second timeout for requests
jeblairi'm hoping that last change will help with gertty being hung on network traffic when you close your laptop and leave the conference wifi in the middle of a sync15:15
*** ChuckC_ has joined #openstack-infra15:16
*** banix has quit IRC15:16
*** banix has joined #openstack-infra15:17
*** carl_baldwin has quit IRC15:18
*** bhuvan has joined #openstack-infra15:19
*** teran has quit IRC15:20
*** teran has joined #openstack-infra15:20
*** craigbr has quit IRC15:21
fungithat almost sounds like a bug you've experienced personally ;)15:21
*** tiswanso has joined #openstack-infra15:21
*** craigbr has joined #openstack-infra15:23
*** zz_dimtruck is now known as dimtruck15:24
*** _shaps_ has joined #openstack-infra15:25
*** dizquierdo has joined #openstack-infra15:25
*** ujuc has joined #openstack-infra15:25
*** gokrokve_ has quit IRC15:27
*** hogepodge has quit IRC15:28
*** david-lyle_afk is now known as david-lyle15:28
openstackgerritMerged openstack-infra/reviewstats: Overdue update for the ironic project group
*** sarob has joined #openstack-infra15:30
*** mtanino is now known as mtanino_away15:30
openstackgerritMerged openstack-infra/reviewstats: Deactivate jpeeler from Heat core
*** garyh has joined #openstack-infra15:31
*** unicell1 has joined #openstack-infra15:35
*** ildikov_ has joined #openstack-infra15:35
*** cody-somerville has joined #openstack-infra15:35
*** cody-somerville has quit IRC15:35
*** cody-somerville has joined #openstack-infra15:35
*** sarob has quit IRC15:36
*** unicell has quit IRC15:36
*** maune has quit IRC15:37
*** Sukhdev has joined #openstack-infra15:37
*** che-arne has quit IRC15:38
openstackgerritClark Boylan proposed openstack-infra/project-config: Run nodepool-coverage as a check job
*** mpaolino has joined #openstack-infra15:38
*** mpaolino has quit IRC15:39
*** dannywilson has quit IRC15:40
openstackgerritTimothy R. Chavez proposed openstack-infra/zuul: Add a dequeue command
*** rfolco_ has joined #openstack-infra15:41
*** mpaolino has joined #openstack-infra15:41
*** deepakcs has joined #openstack-infra15:41
fungidstufft: mrmartin: see results of testing at
*** afazekas_ has quit IRC15:42
*** ashleighfarnham has joined #openstack-infra15:42
fungidstufft: mrmartin: so ie<11 and windows phone up to 8.1 are broken. no idea if we care?15:43
*** rfolco has quit IRC15:44
*** Longgeek has quit IRC15:45
*** marcusvrn has joined #openstack-infra15:45
*** dboik_ has quit IRC15:47
*** marcusvrn1 has joined #openstack-infra15:47
*** dboik has joined #openstack-infra15:47
*** tosky has quit IRC15:48
*** mpaolino has quit IRC15:49
*** marcusvrn has quit IRC15:49
openstackgerritJames E. Blair proposed stackforge/gertty: Display times in local tz
*** tosky has joined #openstack-infra15:52
*** unicell1 has quit IRC15:52
*** marcusvrn has joined #openstack-infra15:52
jeblairfungi: ^ i don't think that change will affect you since your local tz is set to utc.  but i added an option for people like me (possible set size of 1) who have their clocks set to government issued timezones but still want gerrit times displayed in utc (the current behavior)15:53
fungijeblair: i'll take it for a spin anyway15:53
*** marcusvrn1 has quit IRC15:54
*** coolsvap is now known as coolsvap|afk15:54
mrmartinfungi: what is broken? the output of ?15:55
*** rwsu-afk is now known as rwsu15:55
openstackgerritTimothy R. Chavez proposed openstack-infra/zuul: Add a dequeue command
*** hdd_ has joined #openstack-infra15:55
*** armax has quit IRC15:55
*** armax has joined #openstack-infra15:55
fungimrmartin: if we permanently implement dstufft's hardening proposal in 163228 then internet explorer before 11 won't work with (for lack of ssl agreement in https), and internet explorer on windows phone before 8.1 similarly15:57
fungimrmartin: so basically asking for input weighing the cost vs benefit15:57
mrmartinfungi: and what you suggest? We need to apply this ssl patch for groups portal too15:57
anteayaihrachyshka: you submitting to the third-party-announce mailing list yesterday and I replied to you and asked for two changes in the submission, are you going to resubmit15:58
fungimrmartin: i have a feeling very few people using our services use internet explorer, so not supporting fairly old internet explorer releases seems like an acceptable compromise to me, but i haven't actually analyzed browser strings in our apache access logs to find out. i should probably do that now15:59
openstackgerritMerged openstack/requirements: Bump glance_store to latest version
mrmartinfungi, give minute, I have access some google analytics15:59
ihrachyshkaanteaya, hm? I haven't seen anything...15:59
*** otter768 has joined #openstack-infra16:00
*** katyafervent is now known as katyafervent_awa16:00
*** yamahata has joined #openstack-infra16:00
ihrachyshkaanteaya, was it smth in public?16:00
*** ashleighfarnham has quit IRC16:00
mrmartinwe have 41% for IE16:00
openstackgerritMerged openstack-infra/project-config: Set experimental pipeline precedence to 'normal'
*** patrickeast has joined #openstack-infra16:00
fungimrmartin: any feeling for what ie versions? like 10 and older?16:01
*** purp_away has quit IRC16:01
mrmartinand just 25% of Internet Explorer traffic is < 1116:01
mrmartinso not16:01
mrmartinso just the 25% is Internet Explorer 1116:01
*** purp_2 has joined #openstack-infra16:01
mrmartinit means, around 30% is using IE < v1116:01
anteayaihrachyshka: I rejected you post and left you a reply16:02
hasharfungi: mediawiki has a concept of browser grades.    Full support is IE 8+16:02
anteayamy expectation is that you recieve a reply16:02
*** hyakuhei has quit IRC16:02
*** purp_2 is now known as purp16:02
dstufftfungi: hm16:02
dstufftfuthat's not right16:02
hasharfungi: we deprecated support for IE 6 / IE 7 in the last release.  (that is from a top 10 website)16:03
*** sarob has joined #openstack-infra16:03
mrmartinyeah, but if the ssl patch broke under IE11, i guess it's a problem16:03
dstufftprecise should have all of those ciphers (at least openssl ciphers did)16:03
ihrachyshkaanteaya, have you?
dstufftone moment16:03
anteayawell I did16:04
*** katyafervent_awa is now known as katyafervent16:04
anteayaI did't approve it16:04
openstackgerritAssaf Muller proposed openstack/requirements: Pin pylint to 1.4.1
*** otter768 has quit IRC16:04
anteayasince in my reply I pointed out that the top paragraph didn't accurately convey the purpose of the mailing list16:04
*** thedodd has joined #openstack-infra16:04
anteayawhich third party people get very very confused about16:04
anteayaso your characterization didn't help16:04
dstufftfungi: oh, it's apache2's fault >:[16:05
anteayaso I had asked you to change or delete it16:05
dstufftapache2 on precise didn't have ECDHE16:05
fungiyeah, that's fairly new for apache (2.4.x iirc)16:05
*** hyakuhei has joined #openstack-infra16:05
mrmartinso, upgrade everything to trusty?16:05
*** jlanoux has quit IRC16:06
fungieventually sure16:06
fungifor now, we can probably scale back that change to remove fewer options16:06
dstufftlemme see what non ECDHE ciphers those support16:06
anteayaihrachyshka: it isn't a list where third party people hang out16:07
hasharwikimedia has worked on the ssl ciphersuite already. I can probably find you some ciphers chains that works for us16:07
anteayaihrachyshka: it is a list for announcement pertainent to third party operators from infra16:07
dstufftok, it supports AES+12816:07
dstufftso just need to add those in16:07
dstufftpatch incoming16:08
ihrachyshkaanteaya, ok, is there any problem beyond wording? the email was probably approved, since it's in archives, so I guess we don't revert it for wording reasons.16:08
fungianteaya: ihrachyshka: right, we encourage third-party ci operators to subscribe to that ml so that they can be reached en masse, since we doubt they pay attention to higher-volume lists like openstack-dev16:08
clarkbsdague: where do I put that xml file with the custom cpu model? I am digging in libvirt docs and not finding much16:09
anteayathey don't pay attention to anything16:09
dstufftfungi: oh did you apply the patch w/o merging?16:09
fungidstufft: temporarily by hand yes16:09
*** cody-somerville has quit IRC16:09
*** marun has joined #openstack-infra16:09
ihrachyshkafungi, ok, so is there any problem that requires an *action* from me (not just regret due to apparently bad wording)?16:10
fungianteaya: ihrachyshka: but i wouldn't say it's only for infra to contact them, just that the infra team has had the most frequent need to do so16:10
hasharah cipher suite we use:
*** enikanorov__ has joined #openstack-infra16:10
anteayaihrachyshka: the nature of the post is fine, the contextulization of why it is there is misleading16:10
*** enikanorov has quit IRC16:10
anteayaand I have found any pothole in the road collects quite a few who then expect to be dug out16:11
*** ominakov_ has quit IRC16:11
*** ashleighfarnham has joined #openstack-infra16:11
*** ominakov_ has joined #openstack-infra16:11
anteayaand they expect infra to do the digging16:11
ihrachyshkaanteaya, then that's the problem, their attitude.16:11
dstufftfungi: can you try that?16:11
*** mescanef_ has joined #openstack-infra16:12
clarkbsdague: it looks like virsh can list them but not define them?16:12
anteayaihrachyshka: if you can change that I'll buy you beer16:12
anteayaihrachyshka: I've been working on that for over a year16:12
anteayaihrachyshka: you have a magic wand that does the job and the job is all yours16:12
*** rossella_s has quit IRC16:12
fungidstufft: thanks, i'm retesting now with that in place16:12
anteayafungi: and yes that is fair, but it also isn't for the world16:12
hashardstufft: i commented on your change with some links,unified16:13
fungianteaya: agreed, hence the moderation16:13
*** jvrbanac has quit IRC16:13
*** erikmwilson has quit IRC16:13
*** markmcclain has quit IRC16:13
*** jlk has quit IRC16:13
*** d34dh0r53 has quit IRC16:13
*** Ng has quit IRC16:13
*** mescanef has quit IRC16:13
*** briancurtin has quit IRC16:13
*** sdake__ has joined #openstack-infra16:13
*** esheffield__ has joined #openstack-infra16:13
anteayaihrachyshka: my current strategy is to be very careful in word usage around them, so that when they fall down and flail they can ask for help but we are not obliged to dig them out16:14
*** matt__ has joined #openstack-infra16:14
*** wendar_ has joined #openstack-infra16:14
*** marcusvrn has quit IRC16:14
*** comstud has quit IRC16:14
*** freyes has quit IRC16:14
*** esheffield_ has quit IRC16:14
*** ttx has quit IRC16:14
*** lbragstad has quit IRC16:14
*** stevebaker has quit IRC16:14
*** odyssey4me has quit IRC16:14
*** jroll has quit IRC16:14
*** edwarnicke has quit IRC16:14
*** mpavone has quit IRC16:14
*** marcusvrn has joined #openstack-infra16:14
anteayatheir interpretation of words is not my interpretation16:14
*** AJaeger has quit IRC16:14
*** Nakato has quit IRC16:14
*** cyeoh has quit IRC16:14
*** wendar has quit IRC16:14
*** mattoliverau has quit IRC16:14
*** rainya has quit IRC16:14
*** gus has quit IRC16:14
*** russellb has quit IRC16:14
*** dolphm has quit IRC16:14
*** zigo_ has quit IRC16:14
*** zigo_ has joined #openstack-infra16:14
anteayabut through observation I have learned how they think16:14
*** asselin_ has quit IRC16:14
dstuffthashar: thanks, I'm pretty familar with what supports what ciphers browser wise, I just don't use apache2 to terminate TLS so I didn't think to check whether apache2 supported ECDHE or not16:15
fungidstufft: mrmartin: yep, that works better. now we only lose internet explorer on windows xp as originally proposed16:15
*** erikmwilson has joined #openstack-infra16:15
*** russellb has joined #openstack-infra16:15
*** d34dh0r53 has joined #openstack-infra16:15
*** markmcclain has joined #openstack-infra16:15
mrmartinok, I don't care XP, it is EOL16:15
*** jlk has joined #openstack-infra16:15
*** jlk has joined #openstack-infra16:15
*** dolphm has joined #openstack-infra16:15
*** jroll has joined #openstack-infra16:15
hashardstufft: wikimedia uses nginx as SSL terminator iirc, seems the function take care of it.16:15
*** rainya has joined #openstack-infra16:15
*** asselin_ has joined #openstack-infra16:15
*** lbragstad has joined #openstack-infra16:15
clarkbjogo: do you know where to dump the xml to define a custom cpu model?16:15
*** comstud has joined #openstack-infra16:15
fungiif people really want to run windows xp they can install a browser which isn't ie 816:15
hashardstufft: I dont have much knowledge about it though :/16:15
*** cyeoh has joined #openstack-infra16:16
clarkbjogo: I am trying to write the change to use amd64 with migration testing16:16
*** bdpayne has joined #openstack-infra16:16
ihrachyshkaanteaya, I probably miss a lot of context around 3 party people, so I still don't see any actionable items my side to improve the email. maybe you can just reply to the thread to correct anything that is wrong there?16:16
*** stevebaker has joined #openstack-infra16:16
*** Nakato has joined #openstack-infra16:16
*** gus has joined #openstack-infra16:16
clarkball of the libvirt docs I have are for the VMs not the cpu models itself16:16
*** ttx has joined #openstack-infra16:16
*** odyssey4me has joined #openstack-infra16:16
*** scheuran has quit IRC16:16
*** jvrbanac has joined #openstack-infra16:16
*** jroll has quit IRC16:16
*** jroll has joined #openstack-infra16:16
*** AJaeger has joined #openstack-infra16:16
*** AJaeger has joined #openstack-infra16:16
*** rossella_s has joined #openstack-infra16:16
*** hashar is now known as hasharOut16:16
*** freyes has joined #openstack-infra16:17
*** sdake_ has quit IRC16:17
fungidstufft: were you going to update 163228 to that string, or should i?16:17
dstufftfungi: I'll update the patch, FWIW i recommend either geting onto trusty, or using something other than apache to terminate SSL16:17
dstufftnot having ECDHE slows things not, DHE is a lot slower to handshake than ECDHE16:17
fungidstufft: yeah, both good ideas, but this is an excellent interim improvement16:18
dstufftplus you lose the guarentee of PFS with the updated cipher string16:18
anteayaihrachyshka: what I am saying is that in future please be aware that the third party folks have their own unique interpretation of how they view the world16:18
*** annegentle has quit IRC16:18
clarkbunfortunately danpb's blog also suggests throwing it on the VM directly16:18
anteayaihrachyshka: and to avoid them heading here to fix things they don't understand perhaps a brief conversation between you and infra might be helpful16:19
clarkbmaybe I am lookig in the wrong place and I need nova to put it on the VM for us16:19
*** briancurtin has joined #openstack-infra16:19
anteayaihrachyshka: fixing the email now just makes it worse, if there is any fallout I'll address it16:19
*** cody-somerville has joined #openstack-infra16:19
ihrachyshkaanteaya, ack, will do (reach infra next time016:19
anteayaihrachyshka: thank you16:20
*** rlandy has quit IRC16:20
*** Ng has joined #openstack-infra16:20
*** tnovacik has quit IRC16:20
dstufftfungi: fwiw Apache2 doesn't support configurable dh params either, which means DHE is limited to 1024 bits (it's better to have 2048 bits, but jumping to 2048 bits means you lose java 6u45, again, dunno if you care) - here's show support for ciphers when using trusty and haproxy ->
*** sarob has quit IRC16:21
dstufft(that's basically the best possible config for security that doesn't lose massives amounts of compatability)16:21
fungidstufft: excellent reference--thanks16:21
*** tnovacik has joined #openstack-infra16:22
*** edwarnicke has joined #openstack-infra16:22
*** marcusvrn1 has joined #openstack-infra16:22
*** sarob has joined #openstack-infra16:22
clarkbisn't java6 also EOL?16:22
*** marcusvrn has quit IRC16:23
openstackgerritDonald Stufft proposed openstack-infra/puppet-openstackid: Use a set of secure ciphers optimizing for security and performance
dstufftclarkb: I believe so yes, I personally don't care about support for java6, but it doesn't really matter unless you stop using apache2 to terminate TLS since apache2 doesn't support adjustable DH params16:24
*** gokrokve has joined #openstack-infra16:25
*** marcusvrn has joined #openstack-infra16:25
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set
dstufftfungi: oh, I added a SSLHonorCipherOrder onto that patch too16:26
*** esker has quit IRC16:26
*** tiswanso has quit IRC16:26
*** MarkAtwood has joined #openstack-infra16:26
dstufftfungi: that'll make it so that the servers' preference for order is used instead of the client, so that we negioate the strongest supported cipher, not just whatever the client put first16:26
clarkbjogo: sdague ^ that won't work as is because it doesn't communicate to kvm/libvirt/qemu/nova what the cpu model is but I think it has the general shape of the thing16:26
*** SumitNaiksatam has joined #openstack-infra16:27
*** tiswanso has joined #openstack-infra16:27
fungidstufft: sounds great--thanks!16:27
*** marcusvrn1 has quit IRC16:27
*** dboik_ has joined #openstack-infra16:27
*** gokrokve has quit IRC16:27
fungidstufft: the additional comment is very helpful there too16:27
dstufft(for better or worse, I know some things about TLS)16:27
*** emagana has quit IRC16:28
clarkbdstufft: sure, its just odd that so much care goes into supporting platforms which are unsupported by their authors16:28
dstufftI use HAProxy to terminate TLS on the PFS infra, <3 dynamic record size16:28
fungidstufft: yeah, i know far more than i wish i did ;)16:28
*** emagana has joined #openstack-infra16:28
dstufftstart with a small record size to optimize for TTFB, and thne it scales up the record size as data is transfered on the connection to maximize for throughput16:29
*** dboik has quit IRC16:31
*** hodos has joined #openstack-infra16:31
anteayamorning zaro16:31
*** gokrokve_ has joined #openstack-infra16:31
hodosdoes any1 have problem with nodepool cycling with message:  No current image for16:31
hodosand doing nothing16:31
*** spredzy is now known as spredzy|afk16:32
*** mtanino_away has quit IRC16:32
*** emagana has quit IRC16:33
clarkbhodos: check if you have building images `nodepool image-list`16:33
clarkbhodos: if there are no available images then no nodes can be built and you will have to wait for building to complete (and possibly fix broken image builds)16:33
*** sdake_ has joined #openstack-infra16:33
openstackgerritJames E. Blair proposed stackforge/gertty: Upgrade to requests 2.5.3
openstackgerritJames E. Blair proposed stackforge/gertty: Add a 30 second timeout for requests
openstackgerritJames E. Blair proposed stackforge/gertty: Display times in local tz
openstackgerritJames E. Blair proposed stackforge/gertty: Only sync parent commit once
dstufftfungi: oh, and the WEAK on those ciphers in ssllabs is, I beleive, entirely because the DH params are only 1024 bit. The ciphers themselves are perfectly fine.16:34
*** tonytan4ever has quit IRC16:34
fungiyeah, i figured as much16:35
hodosclarkb: yes, it says 'building' for the image i need. but the age of the state is 5 hours16:37
openstackgerritJames E. Blair proposed stackforge/gertty: Upgrade to requests 2.5.3
openstackgerritJames E. Blair proposed stackforge/gertty: Add a 30 second timeout for requests
openstackgerritJames E. Blair proposed stackforge/gertty: Display times in local tz
openstackgerritJames E. Blair proposed stackforge/gertty: Only sync parent commit once
*** sdake__ has quit IRC16:37
clarkbhodos: I would check the image build log and dig into why its not building16:38
hodosclarkb: thanks, i'll try to rebuild16:39
*** unicell has joined #openstack-infra16:40
openstackgerritJames E. Blair proposed openstack-infra/gear: Add SSL functional test
*** dizquierdo has quit IRC16:41
openstackgerritMerged openstack-infra/system-config: Enable logging of #openstack-nova
*** ayoung_afk is now known as ayoung16:44
*** hyakuhei has quit IRC16:45
*** Longgeek has joined #openstack-infra16:45
*** weshay has quit IRC16:45
jeblairsdague, fungi, clarkb: I just approved 161796; once it lands, we can land 161797 but it has no reviews16:45
*** weshay has joined #openstack-infra16:46
anteayafungi: have you had a chance to formulate any thoughts on my proposal to scale the self-nomination process for elections?16:46
*** craigbr has quit IRC16:46
anteayatristanC ttx:
*** hyakuhei has joined #openstack-infra16:47
jeblairmtreinish: i was thinking that the trove tempest job could now be reduced to running only trove-relevant tests.  do you agree?  if so, how do i go about doing that?16:47
anteayafungi: I see you are in the etherpad which is what prompted my question16:47
*** yfried|pto is now known as yfried|afk16:47
clarkbjeblair: you are removing trove from devstack enabled services on all branches and running the new test on all branches as voting. Do we expect icehouse and juno to work that way?16:48
*** asselin__ has joined #openstack-infra16:48
*** carl_baldwin has joined #openstack-infra16:48
jeblairclarkb: i believe they do now (which makes sense to me as trove has been around for a while)16:49
openstackgerritMerged openstack-infra/project-config: Fix propose-project-config-update job
*** hyakuhei has quit IRC16:50
*** hyakuhei has joined #openstack-infra16:51
*** craigbr has joined #openstack-infra16:51
*** sabeen has joined #openstack-infra16:51
mtreinishjeblair: from the jjb job definition just set the DEVSTACK_GATE_TEMPEST_REGEX="database" that will run all the tests that are tagged as using trove (with the "database" tag)16:51
fungijeblair: sdague: do we need SlickNik to acknowledge 161797 before approving?16:51
*** andreykurilin_ has joined #openstack-infra16:51
mtreinishjeblair: fwiw there are only like 5 tests16:52
*** asselin_ has quit IRC16:52
fungianteaya: i pulled it up but haven't really looked closely yet. barely treading water today (so better than the previous two days at least!)16:52
jeblairfungi: it wouldn't hurt, though i believe he's signed off on the general process so i don't think we need to block on it.  but we should keep him in the loop.16:52
*** craigbr has quit IRC16:53
fungijeblair: cool, i'll go ahead and approve16:53
jogoclarkb: thanks, not sure where to dump the xml for a custom CPU16:53
*** tqtran_ has joined #openstack-infra16:53
*** melwitt has joined #openstack-infra16:53
fungibut SlickNik ^ if you're around, keep in mind that's now separately tested16:53
*** craigbr has joined #openstack-infra16:53
*** MarkAtwood has quit IRC16:53
*** openstack has joined #openstack-infra16:56
mtreinishjeblair: heh, it's exactly 5 trove tests in tempest:
openstackgerritMerged openstack-infra/project-config: Make trove tempest job voting
*** andreykurilin_ has quit IRC16:56
mtreinishor I can't count...16:57
*** Ryan_Lane has joined #openstack-infra16:57
openstackgerritMonty Taylor proposed openstack-infra/puppet-ansible: Set default hash behavior to merge
SlickNikfungi: Thanks — yup, I'm good with 161797. jeblair and I had discussed it earlier, and I agree that it makes sense from the perspective of all the big tent discussions we've been having.16:59
*** patrickeast has quit IRC17:00
jeblairSlickNik: i'm about to propose changes to ramp-down the devstack-gate tests run in trove so it's only the trove tempest tests, and you're not runinng all the base layer tests17:00
jeblairshould speed things up for you17:00
*** thingee has joined #openstack-infra17:02
openstackgerritChris Dent proposed openstack/requirements: Update gabbi to 0.12.0
*** Longgeek has quit IRC17:02
*** mwagner_lap has quit IRC17:03
SlickNikjeblair: Awesome — thanks for your help with this!17:03
*** salv-orlando has quit IRC17:03
*** tonytan4ever has joined #openstack-infra17:05
*** thingee has quit IRC17:05
*** radez is now known as radez_g0n317:05
*** garyk has quit IRC17:06
*** dannywilson has joined #openstack-infra17:06
*** yfried|afk is now known as yfried|pto17:06
openstackgerritJames E. Blair proposed openstack-infra/project-config: Add non-voting trove tempest job
openstackgerritJames E. Blair proposed openstack-infra/project-config: Make trove-only tempest job voting
jeblairSlickNik, mtreinish: ^17:08
jeblairthat should be the end of it17:08
*** esker has joined #openstack-infra17:08
*** SumitNaiksatam has quit IRC17:09
*** tonytan4ever has quit IRC17:10
*** ivar-lazzaro has joined #openstack-infra17:10
*** ivar-lazzaro has quit IRC17:11
*** ivar-lazzaro has joined #openstack-infra17:11
*** ivar-lazzaro has quit IRC17:12
*** tonytan4ever has joined #openstack-infra17:12
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/shade: Add unit test for create_server
*** dannywilson has quit IRC17:13
*** ivar-lazzaro has joined #openstack-infra17:13
anteayawhere are we currently pointing people who could benefit from some information about how to write clear and useful git commit messages?17:13
*** bswartz has quit IRC17:13
anteayaI'm looking on the wiki, can't find the page I expect and recall vaguely a mention it was moved but I'm unsure where17:13
mtreinishanteaya: ?17:13
*** harlowja_away is now known as harlowja_17:14
mtreinishanteaya: I have it bookmarked because I leave it in review comments a lot17:14
*** dimtruck is now known as zz_dimtruck17:14
*** mgagne_PHL is now known as mgagne17:14
anteayamtreinish: thanks search didn't find it17:14
jeblairanteaya:  links to that page17:14
*** bswartz has joined #openstack-infra17:16
*** jlk has left #openstack-infra17:16
anteayajeblair: ah thank you, I had thought it was in the manual17:17
clarkbpleia2: cinerama comments on the zanata puppet change. overall looks good just a few minor things17:17
*** zz_dimtruck is now known as dimtruck17:17
*** ihrachyshka has quit IRC17:18
*** sarob has quit IRC17:18
*** e0ne is now known as e0ne_17:18
*** Somay has joined #openstack-infra17:20
*** resker has joined #openstack-infra17:20
*** Somay has quit IRC17:21
*** sarob has joined #openstack-infra17:21
*** cody-somerville has quit IRC17:21
*** chlong has quit IRC17:21
*** _shaps_ has quit IRC17:22
banixjeblair, any other infra cores who could review:
*** asselin__ has quit IRC17:22
*** markus_z has quit IRC17:23
*** esker has quit IRC17:24
clarkbit would be good if we could fix the race in new project additions17:25
*** Hefeweizen has quit IRC17:25
*** sarob has quit IRC17:26
fungiclarkb: agreed, i'll review that patch trio next17:27
*** MarkAtwood has joined #openstack-infra17:27
*** dimtruck is now known as zz_dimtruck17:27
*** sputnik13 has joined #openstack-infra17:28
*** wendar_ is now known as wendar17:28
*** e0ne_ has quit IRC17:29
*** e0ne has joined #openstack-infra17:30
*** bhuvan has quit IRC17:30
openstackgerritBharat Kumar Kobagana proposed openstack-infra/project-config: Modifying devstack plugin job template
*** tsg_ has quit IRC17:31
*** sarob has joined #openstack-infra17:32
greghaynesclarkb: Expected behavior question for this test - do we want nodepool to eventually give up on uploading an image if it keeps failing?17:32
*** SumitNaiksatam has joined #openstack-infra17:32
anteayafungi: do you have a patch number for the start of it, I can too17:32
fungijhesketh: looks good to me, so i'm going to assume the reason SergeyLukjanov didn't approve it when he +2'd is that he wasn't expecting to be around to babysit it17:33
fungianteaya: it's patches to several projects. 154300, 154305 and 15915917:33
fungithough actually 154300 doesn't look immediately related17:34
fungiclarkb: what was the relevance to that one? you mentioned those three together yesterday17:34
*** Hefeweizen has joined #openstack-infra17:35
anteayafungi: thank you17:35
mordredclarkb, fungi: I have a typo fix to push up for that - one sec17:35
*** ssam2 has quit IRC17:36
openstackgerritMonty Taylor proposed openstack-infra/system-config: Pass in a ref for the project_config repo
mriedemanyone notice this yet?
mordredclarkb, fungi ^^17:36
mordredclarkb, fungi: oh! nevermind - my bad - it's the patch in the other repo that had the typoe17:37
*** kmartin has quit IRC17:37
openstackgerritMerged openstack-infra/project-config: Add networking-ibm project to StackForge
*** Sukhdev has quit IRC17:37
fungimordred: cool, hadn't gotten to that one yet17:38
*** asselin has joined #openstack-infra17:38
openstackgerritMonty Taylor proposed openstack-infra/ansible-puppet: Add ability to run puppet with passed-in data
mordredfungi, clarkb ^^ that one17:39
fungimordred: oh, _that_ one17:39
mordred(there was a "defaults" instead of "default" that I noticed while doing something else)17:39
fungii just approved it, so will rereview now17:39
*** BharatK has quit IRC17:39
*** cdent has quit IRC17:39
fungimordred: ahh, yeah, great catch. i totally didn't spot that17:40
*** kmartin has joined #openstack-infra17:40
mordredfungi: me neither17:40
mordredfungi: I actually didn't see it until a follow up patch where I was adding another entry below it :)17:40
clarkbgreghaynes no I think it should make a new one and try try again17:40
*** zz_dimtruck is now known as dimtruck17:40
clarkbgreghaynes important thing is other provider supplies nodes despite the error17:41
greghaynesclarkb: yep, theres some bugs with that too17:41
*** e0ne has quit IRC17:41
*** annegentle has joined #openstack-infra17:41
mordredpatch bomb incoming ...17:41
openstackgerritMonty Taylor proposed openstack-infra/system-config: Move playbooks out of the puppet module
openstackgerritMonty Taylor proposed openstack-infra/system-config: Start passing in data from puppetmaster hiera
openstackgerritMonty Taylor proposed openstack-infra/ansible-puppet: Add ability to filter hiera data
openstackgerritMonty Taylor proposed openstack-infra/ansible-puppet: Add support for running puppet apply
openstackgerritMonty Taylor proposed openstack-infra/ansible-puppet: Add an option to pass in local hieradata
*** devlaps has joined #openstack-infra17:42
clarkbfungi 154300 is how puppet knows what ref to use otherwise it will still be master17:42
mordredclarkb, fungi: ^^ the above set is not done/ready - but I wanted to get some stuff out of my brain17:43
mordredI'm going to WIP the bottom commit in each thign17:43
fungiclarkb: after reviewing 154305 i now see where 154300 fits in17:43
fungiclarkb: yep17:43
*** ajmiller has joined #openstack-infra17:44
timrcSpamapS, I've changed your zuul dequeue patch quite a bit... I'm not sure if that was in a good way or a bad way... but well at least my tests pass, so assuming those are correct, maybe the dequeue cli logic is too :)
fungiand the check for undef on that fact makes it a soft dependency17:44
openstackgerritRamy Asselin proposed openstack-infra/project-config: Add puppet-openstackci project
clarkbfungi yup17:44
*** xgerman has joined #openstack-infra17:44
*** ujuc has quit IRC17:45
fungii initially got confused trying to figure out if you'd missed a crd relationship between that and one of the other patches, but looks like they can land in arbitrary order17:45
timrcI kind of want to add a cross-project-dependency dequeue test as well.  I haven't looked at the underlying dequeue tests to see if that one is already covered at a more fundamental layer.17:45
clarkbtimrc its covered in the scheduler tests17:45
timrcclarkb, Well there ya go.17:46
openstackgerritMerged openstack-infra/puppet-project_config: Provide way to specify repo ref
*** MarkAtwood has quit IRC17:46
ajmillerHi.  I have a patch to neutron-lbaas that looks like it has had all of the appropriate reviews and approvals, but it hasn't gone into the zuul queue for merge.  Can someone here have a look?  It is
ttxanteaya: reading17:46
fungiclarkb: if you want to reapprove 159159 after mordred's typo fix, i think we're all set with the others now17:46
openstackgerritMerged openstack-infra/elastic-recheck: Add fingerprint for bug 1430592
openstackbug 1430592 in devstack "testtools-1.7.0 triggering pkg_resources.VersionConflict: (unittest2 0.5.1 (/usr/lib/python2.7/dist-packages), Requirement.parse('unittest2>=1.0.0'))" [Undecided,In progress] - Assigned to Ian Wienand (iwienand)17:47
anteayattx thanks17:47
*** ryanpetrello has quit IRC17:47
clarkbfungi thanks though I wrote a decent chunk of 159159 myself17:48
anteayafungi: is patchset 4 of 159159 just do to a rebase in the stack/17:48
clarkbalso about to do another round of doctor stuff so midday is looking bad for me17:48
clarkbI can push 159159 in after17:48
*** yfried|pto is now known as yfried|afk17:48
*** bhuvan has joined #openstack-infra17:48
*** mwagner_lap has joined #openstack-infra17:48
ttxanteaya: see my comment on the etherpad17:49
anteayattx thanks17:49
fungianteaya: it's a typo fix17:49
anteayafungi: ah thanks17:49
fungianteaya: mordred removed a stray s to turn defaults into default17:49
fungiclarkb: okay, i'll just reapprove 159159 with only my +217:50
*** sarob has quit IRC17:51
anteayafungi: good eye and thanks17:51
*** ryanpetrello has joined #openstack-infra17:52
fungiajmiller: the way cross repo "depends on" works right now, i think that change didn't automatically enqueue into the gate when the changes it depends on got approved, since it was approved first. reapproving it or leaving a recheck comment will get it moving17:52
*** mtanino has joined #openstack-infra17:52
ajmillerAh, OK.  I'll do a recheck, thanks!17:52
*** jp_at_hp has quit IRC17:52
*** ominakov_ has quit IRC17:52
*** sarob has joined #openstack-infra17:53
*** ominakov_ has joined #openstack-infra17:53
*** cody-somerville has joined #openstack-infra17:53
anteayattx responded17:53
*** achanda has joined #openstack-infra17:54
ajmillerfungi.  I did the recheck, its in the check queue now, so it should be on its way to merging...  Thanks again..17:54
*** _nadya_ has quit IRC17:54
*** claudiub has quit IRC17:57
anteayafungi mordred line 94 should that be O_CREATE?17:57
anteayais that a missing E?17:57
anteayaI grepped and couldn't find another instance of it17:58
anteayafound it:
fungianteaya: it's correct18:00
*** marcusvrn has quit IRC18:00
*** radez_g0n3 is now known as radez18:00
*** otter768 has joined #openstack-infra18:01
fungianteaya: you can also test it yourself if in doubt...
SpamapSthe C gearmand has map/reduce optimizations now18:01
*** Longgeek has joined #openstack-infra18:02
fungiSpamapS: for distributing work to stuff like hadoop?18:02
SpamapSWhich I believe allow it to rapidly take results from mappers and submit them to reducers without an intermediate step of sending to the original client18:02
SpamapSfungi: no, for being a much simpler Hadoop. :)18:02
dstufftgotta save all those bytes and make identifiers super short, who cares if you can't actually read them18:02
fungiSpamapS: yeah, that was the other possibility i was considering. neat!18:02
SpamapSbut it might work well as a pub/sub mechanism18:03
*** reed has joined #openstack-infra18:03
*** reed has quit IRC18:03
*** reed has joined #openstack-infra18:03
fungidstufft: even more amusing, it's a const, so the name is optimized out anyway18:03
SpamapSseeing the operators talk about how awful rabbit is has renewed my interest in making a gearman comms backend for OpenStack. :)18:03
fungiSpamapS: even from the summary and ensuiing thread i got the impression everyone was using rabbit because it sucked the least of the available options18:04
*** salv-orlando has joined #openstack-infra18:04
*** stevemar has quit IRC18:04
SpamapSfungi: exactly18:04
SpamapSgo figure.. the one that is tested in the gate is the one that sucks the least.18:05
fungii liked the "everyone had a cron job to restart all the things"18:05
*** stevemar has joined #openstack-infra18:05
SpamapSfungi: Distributed Windows18:05
*** emagana has joined #openstack-infra18:05
*** HeOS has quit IRC18:06
fungiSpamapS: ooh! right we could switch to msmq18:06
*** otter768 has quit IRC18:06
fungii bet in comes with built-in jobs to reboot its consumers18:06
*** craigbr has quit IRC18:06
*** woodster_ has joined #openstack-infra18:07
SpamapSfungi: even better, they just reboot themselves. :)18:07
fungioh, yep18:07
*** Longgeek has quit IRC18:09
*** Ala has quit IRC18:09
*** spzala has joined #openstack-infra18:09
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Run ironicclient functional tests as STACK_USER
*** carl_baldwin has quit IRC18:10
clarkbgreghaynes: also, not sure if I was clear above, you shouldn't explicitly retry to rebuild/reupload the image. Instead the next iteration of the run loop should checkformissingimages() then try again18:14
*** salv-orlando has quit IRC18:14
clarkbgreghaynes: so I think you just need to ensure that checkformissingimages doesn't short circuit if a single image upload fails, it should move to the next image and try that and so on. Then any that failed would get tried again on the next ieration through the main run loop18:15
greghaynesclarkb: Gotcha18:15
greghaynesthat is exactly what its not doing ATM :)18:15
*** BobBall is now known as BobBall_AWOL18:16
*** marcusvrn has joined #openstack-infra18:16
*** vigneshvar has joined #openstack-infra18:16
*** gyee has joined #openstack-infra18:17
*** krisgan has joined #openstack-infra18:17
*** marcusvrn has quit IRC18:19
pc_mclarkb: anteaya: Could one of you kindly review Taking experimental job and making non-voting in check queue.18:19
clarkbgreghaynes I figured it would be a good thing to test :)18:20
clarkbgreghaynes the coverage reports point out a bunch of other exceptional cases we should be testing too18:21
clarkbpc_m so you got it to run the correct tests?18:22
pc_mclarkb: Yes, verified that current job and new job run tests in the correct directories18:23
pc_mclarkb: (common, openswan) and (common, strongswan).18:23
*** ghostpl_ has quit IRC18:23
pc_mclarkb: Also verified that devstack installed strongswan and correct device driver was selected.18:24
*** ominakov_ has quit IRC18:24
jeblairclarkb: what do you want to do with 155458 and parent?18:24
clarkbjeblair I was hoping parent would merge then I can rebase 15545818:26
jeblairclarkb: okay, i'll review testing changes first then get back to parent18:26
clarkbmostly didnt want to rebase entire thing unless necessary particularly if there are comments on 155458 but not on parent18:27
anteayaAJaeger: did you want to revisit your review on
anteayaAJaeger: you have +2's a patch for a stackforge job that uses docs.o.o?18:28
anteayaeven though your comment caught it18:28
greghaynesclarkb: haha, I found the source of my confusion - if a snapshot image is not built we build it then upload. if a dib image is not built then it is built and not uploaded and we wait to go around the polling loop again to upload18:28
anteayaAJaeger: I'll -1 in the meantime18:29
greghayneshence my confusion on "why does this even work!?" - the snapshot image always gets uploaded first and I was trying to make the dib image fail to upload first18:29
*** bknudson has quit IRC18:30
openstackgerritMerged openstack-infra/project-config: VPNaaS make new functional test non-voting
*** dhritishikhar has quit IRC18:32
*** dkranz has quit IRC18:32
*** dkranz has joined #openstack-infra18:32
*** carl_baldwin has joined #openstack-infra18:32
*** jcoufal has quit IRC18:32
*** dhritishikhar has joined #openstack-infra18:34
*** jaypipes is now known as jaypipes-afk18:34
*** marcusvrn has joined #openstack-infra18:34
*** AJaeger_ has joined #openstack-infra18:34
*** vigneshvar has quit IRC18:35
*** hdd_ has quit IRC18:36
anteayashouldn't we change the mode of project-config scripts via puppet, rather than using a patch?
anteayalooking at Make script executable.18:37
*** tsg_ has joined #openstack-infra18:37
AJaeger_anteaya: I really prefer to have it executable in git - so that I can execute it locally without problems ;)18:38
nibalizeranteaya: i agree with AJaeger_18:38
AJaeger_anteaya: for jenkins it's not a problem, we invoke with "python SCRIPT"18:39
AJaeger_But running it locally, I'm for the short form ;)18:39
nibalizerwell let me say this, if we're doing it where we bring down the repo, then run the file from the repo, then the permission bits should be in git18:39
anteayaI can understand the need to run it locally18:39
nibalizerif puppet was pulling the file from git.openstack or copying it out of the git repo and putting it somewhere (like /etc/init.d/) then I would ensure permission bits in puppet18:39
anteayaare we setting ourselves up for puppet overwriting file permissions it doesn't know about?18:40
SpamapSwow typing 'tox -egenconfig' in nova's home dir takes a long time.18:40
nibalizeri dont understand taht question18:40
SpamapSgoing on 10 minutes now18:40
anteayanibalizer: is puppet possibly going to over write file permisions taht are set in git18:41
anteayabut which are not echoed in puppet files18:41
fungianteaya: where do you see that file getting installed by puppet?18:41
anteayaI don't18:42
anteayaI'm asking18:42
nibalizerno puppet will let git be in charge18:42
anteayanibalizer: thank you18:42
openstackgerritMerged openstack-infra/gear: Add SSL functional test
fungiafaik the only automation we have clones the repo and runs stuff directly from there, without puppet doing any file manipulation18:42
*** pelix has quit IRC18:43
anteayathat was what I wanted to know, that and in a git vs puppet situation about file permissions, git wins18:43
fungiwell, there is no puppet in this case18:44
fungithat is a script run from a git checkout by a jenkins job18:44
*** salv-orlando has joined #openstack-infra18:44
nibalizeranteaya: what fungi is saying is true18:45
*** banix has quit IRC18:45
nibalizerpuppet manages the git repo, the git repo manages that file18:45
nibalizerif we had puppet and git managing that file, they would fight, and the file state would oscilate probably18:45
anteayaah great18:45
fungii'm not even sure puppet is installing that repo in the situations where we run that script18:45
nibalizerwhich is not what we want18:45
anteayathank you18:45
nibalizerproject-config ? it should be18:46
fungiit's getting pre-cached by nodepool scripts and then later cloned by job macros18:46
anteayanibalizer: yes that was the scenario I did want to avoid, which as fungi says is easy since puppet isn't doing work there18:46
*** jistr has quit IRC18:46
nibalizerfungi: ah18:47
funginibalizer: the only place it's run in automation is from
*** vigneshvar has joined #openstack-infra18:48
fungiwhich clones the repo (from the nodepool-provided cache on disk, updated to the change from zuul) and then calls tox to run the script in it18:48
fungiso... no puppet anywhere for that18:49
anteayafungi: awesome thanks18:50
fungihappy to help18:51
anteayayou do it so well18:52
*** spzala has quit IRC18:53
*** craigbr has joined #openstack-infra18:53
*** banix has joined #openstack-infra18:54
fungidid we approve a change to gerrit's javascript recently (is the worker results box missing for anyone else in the webui), or is my browser acting up?18:54
funginevermind. i think it's my browser18:54
fungino, wait, on pages where i had it, i now see it disappear on a force reload18:55
*** patrickeast has joined #openstack-infra18:56
*** fdegir has joined #openstack-infra18:56
*** dannywilson has joined #openstack-infra18:56
*** mtanino has quit IRC18:57
*** mtanino has joined #openstack-infra18:57
*** bswartz has quit IRC18:57
*** ghostpl_ has joined #openstack-infra18:58
*** marcusvrn has quit IRC18:58
anteayaworker results box18:58
anteayais that the jenkins logs links?18:58
*** e0ne has joined #openstack-infra18:58
funginext to the reviews box18:59
anteayaI see it on:
jesusaurusjeblair: i was just about to -1 162714 because im pretty sure we're double-counting ready when calculating extra_demand, can you remove your workflow+1 and read my comment?18:59
anteayato the right of the list of reviewers below the commit message18:59
fungiyeah, that works for me too, and then it disappears with a force refresh, so probably just browser wonkiness18:59
anteayaJenkins checkMar 9, 2015 12:48 PM18:59
anteayaclicking f5 still has it for me19:00
fungithanks anteaya i think it's just me. good thing i mostly use gertty anyway19:00
*** marun has quit IRC19:00
*** marcusvrn has joined #openstack-infra19:01
SpamapSWell this is just silly19:01
SpamapSit appears notifications in nova _must_ use the same url as RPC19:01
SpamapSrabbit all the things i guess. :-P19:01
openstackgerritMerged openstack-infra/project-config: Ensure that gerritbot channels are in accessbot
jeblairjesusaurus: i don't see a comment from you?19:01
*** arxcruz has quit IRC19:01
jeblairjesusaurus: feel free to go ahead and -1 it19:01
openstackgerritMerged openstack-infra/project-config: Support custom static nameserver during build
*** bswartz has joined #openstack-infra19:03
jesusaurusjeblair: gertty just synced, is it there for you?19:04
*** gokrokve_ has quit IRC19:04
jeblairjesusaurus: yep, thx, and thanks for the heads up19:04
banixanteaya: thank you for reviewing/approving my patch for our new repo.19:06
*** sdake__ has joined #openstack-infra19:08
clarkbI think the math is correct. the idea is to have min ready extra19:08
clarkbso extra demand is indeed extra19:09
*** tnovacik has quit IRC19:09
anteayabanix: you are welcome be sure to start with 4 python tests in your first patch19:10
e0nehi. does anybody know why we have so many patches in check queue?19:12
*** sdake_ has quit IRC19:12
*** bknudson has joined #openstack-infra19:13
*** mrunge_bbl is now known as mrunge19:14
*** scottda has joined #openstack-infra19:15
anteayae0ne: 160 patches is not a lot19:16
*** spzala has joined #openstack-infra19:16
anteayaover 400 would be something to question19:16
anteayaalso events19:17
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Moved preference resolution into its own resolver.
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Preference service uses private scope.
anteayaQueue lengths: 3 events, 0 results.19:17
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Safety check for preference save.
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Removed static pageSize injector.
anteayae0ne: if you ever see events reach over 500 and stay there for longer than 30 minutes, that is worth knowing about19:17
anteayaevents can spike to over 8000 for a few minutes when zuul is resetting the gate19:18
*** pblaho has quit IRC19:19
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Run some project-config jobs only if needed
*** tosky has quit IRC19:19
anteayamtreinish: in you set -eux, if haven't seen u used before and nothing I'm search is helping me find it19:19
anteayamtreinish: can I get a pointer to continue reviewing your patch?19:19
*** teran has quit IRC19:20
tsg_clarkb: need your help with bandersnatch .. the pypi cache seems to be stale (we are trying to pick up the latest pyeclib update - 1.0c2)19:20
fungitsg_: is bandersnatch hung? have you looked at its log? did it merely skip that file or has it done nothing at all?19:21
*** taron has joined #openstack-infra19:22
tsg_fungi: not sure where to look for the logs19:22
fungitsg_: on our servers it's /var/log/bandersnatch19:22
tsg_fungi: have never tried to log in to the servers .. can you tell me more19:23
anteayamtreinish: found it in my bash reference, yay books19:23
fungitsg_: oh, i thought you meant on a bandersnatch server you were running19:23
fungitsg_: i'll take a look and see what's going on19:23
anteayamtreinish: nounset, treat undefined variables as errors, not as null19:23
tsg_fungi: thanks - pypi updates don't seem to have been picked up since yesterday19:24
openstackgerritgreghaynes proposed openstack-infra/nodepool: WIP: test nodepool upload failures
greghaynesclarkb: ^ getting closer19:24
mtreinishanteaya: heh, yeah I was just going to point you to the set manpage :) Although fwiw I just copy and pasted that from another script somewhere in the repo19:24
anteayamtreinish: cargo culter!19:25
anteayamtreinish: do you want to keep that option then?19:25
fungitsg_: i see it here but i'll check the rest of our mirrors... maybe only one is stuck19:25
anteayamtreinish: I'm fine I would just like if it was intentional19:25
mtreinishanteaya: it worked fine when I tested it locally I don't think it hurts anything19:26
openstackgerritgreghaynes proposed openstack-infra/nodepool: WIP: test nodepool upload failures
anteayamtreinish: okey dokey19:26
tsg_fungi: ok19:26
fungitsg_: all our mirrors have it19:26
tsg_fungi: ok thanks - Jenkins still seems to pick 0.9.819:27
fungitsg_: are you sure you're requesting a beta version? modern pip won't install non-numeric releases without additional effort19:27
tsg_let me try another check19:27
*** hodos has quit IRC19:27
*** achanda has quit IRC19:27
*** resker has quit IRC19:27
tsg_fungi: I assume you mean non-PEP440?19:27
fungitsg_: PyECLib-1.0c2.tar.gz is a prerelease by pip's reckoning19:27
*** ominakov_ has joined #openstack-infra19:28
tsg_fungi: "pip install --upgrade" in my local testing seems to have picked it up19:28
fungiso asking pip for just PyECLib or PyECLib>=0.9.0 or something won't install 1.0c219:28
tsg_fungi: good point though .. I will consider using a numeric-only version number19:28
fungiwhen they release a 1.0 or 1.0.0 or something we'll install that19:29
*** emagana has quit IRC19:29
*** marcusvrn has quit IRC19:29
tsg_fungi: the reason I brought up bandersnatch though, is that, last few times we updated the library, it wasn't picked up until bandersnatch was kicked :)19:29
mordredtsg_: if it worked for you locally, then chances are you have an old pip19:29
anteayadougwig: did you ever take a look at mtreinish's comments on
tsg_mordred: let me double-check - I am using 14.04.219:30
mordredtsg_: yeah- pip from ubuntu is WAY too old19:30
fungitsg_: alternatively you can use something like PyECLib>=0.0.0a0 and it will see the prerelease in that spec as an indicator that prerelease versions are okay19:30
tsg_(pip 1.5.4)19:30
mordredI think they started ignoring pre-releases by default in 1.6? dstufft ^^ ?19:30
fungii thought that was 1.519:31
tsg_mordred, fungi: to avoid problems, let me push a new rev19:31
*** radez is now known as radez_g0n319:32
dougwiganteaya: no, but i just did, and it would be safe to switch to just the neutron job.  do you want that folded in that patch, or in a follow-on?19:33
*** esker has joined #openstack-infra19:33
*** timcline_ has quit IRC19:34
*** esker has quit IRC19:34
anteayamtreinish: which approach would you like dougwig to take?19:34
*** yfried|afk is now known as yfried|pto19:35
*** esker has joined #openstack-infra19:36
jesusaurusclarkb: extra demand is definitely extra, but I don't think it's enough extra. it is subtracting the ready (ready + building + test) from both the start demand and the min_ready19:36
openstackgerritMerged openstack-infra/project-config: Build wheel house during nodepool devstack caching
*** kmartin has quit IRC19:37
*** cody-somerville has quit IRC19:38
*** achanda has joined #openstack-infra19:38
openstackgerritMerged openstack-infra/ansible-puppet: Add ability to run puppet with passed-in data
jesusaurusclarkb: so we end up with (start_demand - ready) + (min_ready - ready) when we should have (start_demand + min_ready) - ready19:39
anteayadougwig: well mtreinish said he is fine with that patch so I'll approve it and please make the switch in a following patch19:39
jesusaurusassuming capacity for the extra_demand19:39
dougwiganteaya: ok, thanks.19:39
*** banix has quit IRC19:40
mtreinishanteaya: yeah I'm fine with adding the other job, but if after investigation they can be deduped that would be the best19:40
anteayadougwig: so dedupping as much as we can in that follow-up?19:41
anteayamtreinish: thanks19:41
*** dhritishikhar has quit IRC19:41
*** timcline has joined #openstack-infra19:42
openstackgerritMerged openstack-infra/project-config: add experimental ceilometer elasticsearch db jobs
anteayaI'm going to go for walk now19:42
dougwiganteaya, mtreinish - i'll submit one that drops the other grenade job, after taking one more look at the code path.  nothing in the grenade repo looks like it does anything fancy with nova-net (unlike neutron)19:42
mtreinishdougwig: well the other thing is getting feedback from the nova team on whether that code path is worth testing independently, because there might still be value in doing the upgrade test19:43
*** mpopow has joined #openstack-infra19:43
dougwigmtreinish: ok, i'll ping in channel as well.19:44
*** Somay has joined #openstack-infra19:44
*** Somay has left #openstack-infra19:44
*** HeOS has joined #openstack-infra19:45
*** johnthetubaguy is now known as zz_johnthetubagu19:45
*** baoli has quit IRC19:46
*** fdegir has quit IRC19:46
*** annegentle has quit IRC19:46
*** annegentle has joined #openstack-infra19:47
*** mfink_ has quit IRC19:47
openstackgerritMerged openstack-infra/project-config: Since grenade+neutron relies on devstack, add it to devstack's check queue
*** nelsnelson has quit IRC19:50
*** claudiub has joined #openstack-infra19:51
*** vigneshvar has quit IRC19:51
*** annegentle has quit IRC19:52
*** esker has quit IRC19:53
*** Longgeek has joined #openstack-infra19:54
*** gokrokve has joined #openstack-infra19:55
*** ominakov_ has quit IRC19:56
openstackgerritMerged openstack-infra/system-config: Pass in a ref for the project_config repo
*** kevinbenton has joined #openstack-infra19:57
openstackgerritSwaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu
clarkbjesusaurus I see so extra demand should just be min-ready?19:58
*** scottda has left #openstack-infra19:58
*** mpaolino has joined #openstack-infra19:59
jesusaurusclarkb: yeah, that's my understanding. instead of calculating extra_demand, we should just add min_ready if we have capacity19:59
*** mpaolino has quit IRC19:59
jesusaurusclarkb: does that make sense, or is my math off?19:59
*** Longgeek has quit IRC20:00
*** fdegir has joined #openstack-infra20:01
*** otter768 has joined #openstack-infra20:01
*** AJaeger_ has quit IRC20:02
*** tnovacik has joined #openstack-infra20:02
clarkbI think you may be correct but need to read through again after lunch20:04
*** praveenkumar has quit IRC20:06
*** dboik_ has quit IRC20:06
*** otter768 has quit IRC20:06
*** dboik has joined #openstack-infra20:07
*** garyh has quit IRC20:07
*** annegentle has joined #openstack-infra20:08
*** praveenkumar has joined #openstack-infra20:08
*** garyh has joined #openstack-infra20:09
*** esker has joined #openstack-infra20:09
*** hdd_ has joined #openstack-infra20:10
*** vigneshvar has joined #openstack-infra20:12
*** radez_g0n3 is now known as radez20:13
annegentlereed: fungi: reading more about the tags and user survey tie-in ideas, it feels like it's really a doc/data project20:16
annegentlelike how stackalytics really does help apply real data the more and more we gather20:16
fungiannegentle: yeah, i mostly concur20:16
annegentleeven though it seems like it could be gamed (and does get gamed) with enough data the truth (interpreted, sure) will show20:17
*** hasharOut is now known as hashar20:17
fungiannegentle: i foresee project representatives _mostly_ just proposing the tags they think apply to their projects, and there being some community review, and there's significant hope that any problems will be laid bare at that point. but for everything else, there's git revert20:18
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate: Set NOVA_ALLOW_DUPLICATE_NETWORKS=True when testing with Neutron
*** mrmartin has quit IRC20:18
annegentleI just want data.20:19
annegentleesp. for docs it's a good goal to quantify, how many bugs? how many page views? how many docs patches? Etc.20:19
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate: Set NOVA_ALLOW_DUPLICATE_NETWORKS=True when testing with Neutron
*** ihrachyshka has joined #openstack-infra20:19
fungiit seems idealistic to expect most of those tags to be clear-cut, data-driven and automatically assigned20:20
fungia lot of what's been discussed so far falls on the subjective consensus side20:21
fungii suppose we can try to find pure data analogues we can query to act as a proxy for what those tags really represent, but it feels like a bit of a stretch. i guess the best we can do is just try and see what happens20:22
*** ominakov_ has joined #openstack-infra20:23
fungifor example, contributor diversity... we can marry commit history and employer affiliation from foundation profiles to work out relative percent contributions to a project (like stackalytics and the activity dashboard do presently), but api stability i have no idea how we even begin to measure20:25
*** esp has joined #openstack-infra20:25
fungior confirming that the devstack-gate job for a project really tests that project20:26
*** yfried|pto is now known as yfried|afk20:26
*** kmartin has joined #openstack-infra20:29
*** ChuckC_ has quit IRC20:29
*** mwagner_lap has quit IRC20:30
*** _nadya_ has joined #openstack-infra20:30
openstackgerritJohn L. Villalovos proposed openstack/requirements: Avoid httpretty 0.8.8 as it can break unittests
reedannegentle, I have in mind complex tools like that are hard to game because they're multidimensional20:34
reedthe code is also free as in freedom,
reedcheck a recent example of their analysis on
*** dkranz has quit IRC20:35
*** jistr has joined #openstack-infra20:35
reedannegentle, in the specific case of OSSEval the metrics are hard to display automatically, they require 'manual' judgement though20:36
reedalso, they make sense for comparisons20:36
*** cody-somerville has joined #openstack-infra20:36
reedso we may use them to compare nova to say, barbican20:37
*** marun has joined #openstack-infra20:37
*** ominakov_ has quit IRC20:38
annegentlereed: OSSEval, tell me more? I dunno that one20:38
annegentlereed: oh wait, making the connections now.20:39
reedannegentle, it's the evaluation engine behind sosopensource.com20:39
openstackgerritJohn L. Villalovos proposed openstack/requirements: Avoid httpretty 0.8.8 as it can break unittests
* annegentle reads20:39
reedthere are other tools in this area, though, the EU has funded lots of research to evaluate quality of open source projects20:39
reedto help public procurement20:39
annegentlereed: oh neat.20:39
annegentlereed: I like this report, "While there are few books about OpenStack and some are also available about OpenNebula, there is no book about Eucalyptus."20:40
reedprof. Jesus Barahona (and his team) from Bitergia has done some research in this area20:40
reedannegentle, "books available" is a measure of the success of an open source project in fact20:40
annegentlereed: wow.20:40
reedthere is little to game with metrics like these :)20:41
annegentlereed: seems like a nice cloud writer opportunity if they know about it :)20:41
*** salv-orlando has quit IRC20:43
reedmine were just crazy ideas though, I came up with them thinking that if some tags are too hard to come up with maybe it's because they're just too hard :)20:43
tsg_fungi: assuming the jenkins jobs don't allow "sudo" in package scripts, do the jobs run "sudo pip install.."?20:43
*** jedimike has quit IRC20:44
fungitsg_: no, tox-based jobs at least are installing things from pypi into a virtualenv as the "jenkins" user20:44
tsg_fungi: ok that works :)20:45
annegentlereed: well quantified data is tough for docs, adoption, etc. for sure.20:45
*** yfried|afk is now known as yfried|pto20:45
reedindeed, annegentle, it's not simple20:45
annegentlereed: :)20:48
*** sdake__ has quit IRC20:51
*** trey has quit IRC20:51
*** sdake_ has joined #openstack-infra20:53
* mkj_wk struggles with the general python popen subprocess raw_input EOF problem20:53
*** trey has joined #openstack-infra20:55
*** achanda has quit IRC20:55
mkj_wkhaven't found a way yet to make this changed test not blow up with EOFError from the inferior git-review20:56
mkj_wkwell, I have20:56
*** mtanino has quit IRC20:56
mkj_wkbut I'm going to guess that the quick hack of time.sleep(1) wouldn't go over very well ☺20:56
*** dustins has quit IRC20:57
*** hogepodge has joined #openstack-infra20:58
ajmillerfungi I'm back with a question about which you looked at for me earlier today.  I ran recheck, and it spent all day working its way through the check queue, and finished there about 10 minutes ago.  But it didn't show up in the gate queue...20:58
fungiajmiller: that's definitely strange. i'm going to check zuul's log for any indication of why it didn't enqueue21:00
*** _nadya_ has quit IRC21:00
fungiclarkb: jeblair: ^ did any bugs emerge while i was out that account for that behavior?21:00
fungidon't want to spend too much time digging in logs if this is a known issue21:01
fungid'oh, i know what's causing that21:01
fungiit's the thing where jenkins replacing its vote doesn't emit a +1 in the event stream21:02
fungibecause gerrit already had a +121:02
* fungi sighs21:02
fungiajmiller: i'll just directly enqueue it into the gate pipeline since it should be fine at this point21:02
*** dprince has quit IRC21:02
*** esker has quit IRC21:02
ajmillerok, thanks fungi21:03
jeblairanother aprv vote would enqueue it21:03
*** bswartz has quit IRC21:03
ajmillerAh, OK, that would make sense too...21:04
*** r-daneel has quit IRC21:05
fungiyeah, for future reference if it's on that same project, any of these people,members can (re)add an approval vote to kick that21:05
*** _qb has joined #openstack-infra21:08
*** kgiusti has left #openstack-infra21:09
clarkbfungi: yes I think its because of the vote not changing21:09
clarkbfungi: zaro pushed a patch upstream to fix that21:09
clarkbnot sure how it is doing21:09
clarkbalso I am back for the day now21:09
*** eharney has quit IRC21:10
*** jistr has quit IRC21:10
*** ayoung has quit IRC21:10
dhellmannfungi, clarkb : can one of you add me to aeromancer-core and aeromancer-review in gerrit, please?,members and,members21:12
clarkbdhellmann: sure21:12
dhellmannfungi, clarkb : for reference, that's a new stackforge project:
*** aysyd has quit IRC21:12
*** andreykurilin has quit IRC21:12
jeblairdhellmann: what is that project?21:12
dhellmannjeblair: it's a little tool I've been working on for querying the openstack source base. It has commands like "which projects use this requirement" and "grep across the source of all projects for this pattern"21:13
clarkbdhellmann: yup found it in my local git log21:13
*** vponomaryov has quit IRC21:13
*** aarefiev has quit IRC21:13
clarkbdhellmann: done21:13
dhellmannclarkb: thanks!21:14
*** ldnunes has quit IRC21:14
*** aarefiev has joined #openstack-infra21:14
dhellmannclarkb: hmm, I still can't +2 -- I wonder if I messed up the ACLs file?21:15
*** teran has joined #openstack-infra21:15
*** craigbr has quit IRC21:15
clarkbdhellmann: probably the gerrit cache21:15
fungithat reminds me livegrep will be cool once we have it21:15
*** mattfarina is now known as mfer21:15
dhellmannclarkb: "gerrit cache"? Like if I wait until tomorrow it'll work?21:15
*** vponomaryov has joined #openstack-infra21:15
clarkbdhellmann:,access does look good as far as the ACLs21:15
dhellmannfungi: yeah, this just runs git grep, but it knows how to find the projects21:15
clarkbdhellmann: if it doesn't work in a few minutes I can manually purge the cache for you21:15
*** mfer has quit IRC21:16
dhellmannthe requirements queries are more interesting, and I use those for oslo to figure out which projects aren't using libs yet21:16
dhellmannclarkb: nah, I can wait, I just wanted to make sure I understood21:16
*** andreykurilin has joined #openstack-infra21:16
*** tiswanso has quit IRC21:16
_qbhello. I'm interested in setting up a third party ci instance for a large company. I've read a lot, but I can't seem to find a comprehensive diagram which shows how all these moving parts fit together. Does such a diagram exist anywhere?21:16
asselin_qb I have one21:17
_qbI'd love to see it, esp if it's up to date!21:17
_qbI can find some really old ones, but there has apparently been a lot of organic growth and lots of new components added21:18
asselinit's powerpoint...not sure how to share...let me see...21:18
asselin_qb links? I created this one myself based on how I understand it21:18
*** dimtruck is now known as zz_dimtruck21:19
*** pc_m has quit IRC21:20
_qbjust searching, I found what Freescale and IBM did21:20
_qb for example, but it's not detailed enough21:21
_qbneed to depict every component in order to get it past our security review21:21
_qb(tall order, I know)21:21
*** esker has joined #openstack-infra21:21
*** yfried|pto is now known as yfried|afk21:22
openstackgerritClark Boylan proposed openstack-infra/ansible-puppet: Its 0o not 0c to make an octal number
clarkbfungi: mordred ^21:22
fungi_qb: part of the challenge is that it's different for different deployments. we write a bag of parts which we assemble in a certain way to do our jobs upstream, but different deployments may need more or fewer of them to accomplish their goals21:23
clarkbjeblair: what do you think about jesusaurus' comment? I think I should do demand = demand + min-ready - ready if less than capacity else demand = demand - ready21:23
fungiclarkb: *sigh* i should have spotted that too21:24
_qbfungi: makes sense, but just a detailed depiction of how the upstream part works would be great21:24
clarkbfungi: its hard to catch c is so close to o21:24
clarkbfungi: also flake8 doesn't catch it21:24
zarojeblair: re mysql encoding, take a look at my comment on
mordredclarkb: I agree with your commit comment21:25
zarojeblair: do you prefer that alternative?21:25
*** matt__ is now known as mattoliverau21:25
*** yfried|afk is now known as yfried|pto21:25
jesusaurusclarkb: i think you can remove extra_demand and use label.min_ready in its place, keeping the logic the same21:26
zarofungi, clarkb : no reviews on that gerrit patch yet,
*** dkliban is now known as dkliban_afl21:27
*** dkliban_afl is now known as dkliban_afk21:27
*** movielovers has joined #openstack-infra21:27
clarkbjesusaurus: good point21:27
fungi_qb: we have this diagram floating around in some presentations
openstackgerritMerged openstack-infra/devstack-gate: Remove trove from default config
*** achanda has joined #openstack-infra21:28
zarofungi: would you be able to  help me verify
*** salv-orlando has joined #openstack-infra21:28
fungi_qb: most third-party ci deployments are the stuff to the right starting with zuul, communicating with the bits we're running upstream on the left21:28
zarofungi: look at the last comment21:29
clarkbjeblair: does that make sense to you? maybe the thing to do is just push it up and let people argue over it in gerrit :)21:29
clarkbjesusaurus: will push that up as soon as I can run tests on it21:29
jesusaurusclarkb: awesome21:29
asselinhow do you share images?21:29
clarkbasselin: imgur21:29
fungizaro: yeah, i thought jeblair was suggesting running mysql client commands to convert the charset on the tables in the current database21:30
zarofungi: yeah, i tried that but didn't help at all21:30
zarofungi: i don't think it has anything to do with the collation of the tables.  its the default connection between client and db21:31
_qbfungi: thanks, good start. I need to flesh out more detail for mine tho, depicting nodepool, puppet, jjb, components of the ci cloud, etc. I guess if it doesn't exist, maybe it'll be a useful contribution once it's done?21:31
jeblairclarkb: i was just about to sit down and do the math on that; i don't have anything further to offer yet.  but if you have a solution, feel free to push it up and hopefully by then i'll be caught up.21:31
fungizaro: got it, then yeah i expect being explicit about the encoding on the mysql server side of the connection is preferable to embedding that implementation detail in the uri21:31
*** ddieterl_ has quit IRC21:31
asselin_qb, ok, here's what I created:
jlvillalSo can I beg anyone for a review of:  The openstack/python-ironicclient is blocked because of this. :(21:32
jlvillalThis is an httpretty new release issue.21:32
_qbasselin: thanks!21:32
*** boris-42 has quit IRC21:32
fungi_qb: definitely--i think a lot of that is already being fleshed out in some contributions up for review right now so reviewing those or adding to them is always appreciated21:33
zarofungi: we can try it on review-dev.o.o first21:33
fungi_qb: asselin and krtaylor can probably point you to those faster than i can though21:33
fungizaro: yep, i'm all for trying it there and seeing what happens21:33
jeblairzaro: so after setting the default charset on the tables, and restarting gerrit, jdbc still did not choose utf8 character encoding?21:34
zarojeblair: that is correct.21:34
jesusaurusasselin: oh, nice image. thats useful21:34
*** mmedvede has quit IRC21:34
jeblairzaro: okay, let's ask SpamapS and mordred to weigh in on and the option in your comment there21:35
asselinjesusaurus, glad you like it21:35
fungiasselin: very nice diagram21:35
jeblairzaro: also, we should decide out if we _also_ want to set the table charset and collation.21:35
clarkbjeblair: ya I am pretty sure jesusaurus is correct now. Just testing the change locally before I push it up21:35
clarkbhrm no that makes tests explode21:36
clarkbjesusaurus: ^21:37
jogohave you guys ever experienced out of sync nova quotas on public clouds?21:37
clarkboh wait I think I know21:37
zarojeblair: the review-dev.o.o db has already been converted to utf8_bin if we want to do any other testing before converting review.o.o db21:37
jesusaurusclarkb: it does?21:37
*** yfried|pto is now known as yfried|afk21:38
clarkbjesusaurus: ya we don't end up building enough nodes with taht change21:38
jogoI am trying to reproduce some of the issues21:38
clarkbso the tests timeout waiting for nodes21:38
asselinfungi, thanks21:38
clarkbjogo: I don't think so. Usually when we have quota errors we have leaked actual resources that are consuming that quota21:38
jogoclarkb: good to know thanks21:39
clarkbjesusaurus: I think the reason is that we may want the middle group that the current patch provides21:39
mordredjeblair, zaro: sooo.... the current db is latin-1 and also the connection string is latin1, yeah?21:39
jesusaurusclarkb: which middle group?21:39
clarkbjesusaurus: sorry not group, ground21:40
*** radez is now known as radez_g0n321:40
clarkbjesusaurus: the change as currently written works towards satisfying min-ready without trying to reach it in one step21:40
clarkbjesusaurus: if you try to reach it in one step then you hit the capacity boundry much easier21:40
clarkbwhich is what causes the tests to fail I think21:40
*** koolhead_ has quit IRC21:41
mordredjeblair, zaro: I have no problem with the patch, and yeah, it shoudl work - however, it means that strings will be encoded strangely if we access them from not-gerrit21:42
clarkbbased on the test results I think the current change is what we want21:42
zaromordred: the character_set_server is set to "latin1"21:42
*** baoli has joined #openstack-infra21:42
jeblairclarkb: i don't think anything about this was designed to be multi-step, so that's confusing...21:42
jeblairclarkb: i'm not sure we should count on that21:42
*** HeOS has quit IRC21:42
jesusaurusclarkb: well in that case, do we want to be more explicit about what we are doing and why? and add a line that reduces extra_demand for that purpose rather than double-subtracting ready?21:42
clarkbjeblair: jesusaurus specifically I think the issue in the tests is min-ready is label specific but capacity is provider specific21:43
clarkband in the tests neither provider can satisfy min ready outright21:43
mordredif we know that we're going to be inserting data in utf8, we should really also set the character_set_server to utf8 and we should alter the table21:43
zaromordred: not sure if that answers your question but i believe setting that to utf8 fixes issue in 16310421:43
*** tonytan4ever has quit IRC21:43
mordredthat way, if a mysql command line client connects with a default value of latin1, it will know that the server is sending it utf8 and will transcode appropriately21:43
clarkbjesusaurus: jeblair: and since the tests rely on min-ready to supply nodes instead of job demand we basically schedule no jobs21:44
clarkber no nodes21:44
*** timcline has quit IRC21:44
*** ociuhandu has quit IRC21:44
*** baoli has quit IRC21:44
jeblairmordred: is that a client or server setting? (or both)21:44
mordredjeblair: both21:44
jeblairmordred: since we use trove for the server, i don't know if we can set it there.21:44
mordredjeblair: it's a setting in many places21:45
*** baoli has joined #openstack-infra21:45
jeblairmordred: so at least setting it on the gerrit server means that anything we run from there will use it, which is good.21:45
mordredjeblair: yes21:45
mordredit's likely a fine choice21:45
jeblairmordred: but connections from not-the-gerrit-server (which should be none, unless we are migrating) would not use it21:45
*** zz_dimtruck is now known as dimtruck21:46
*** sdake_ has quit IRC21:46
jeblairmordred, zaro: so sounds like config file update + alter table is the way to go21:46
jesusaurusclarkb: ah, so the min_ready of one label could potentially starve capacity of another label in the same provider...21:46
mordredjeblair: yes. although we could also set a default client charset setting in those21:46
clarkbjesusaurus: yes21:46
jeblairi'm also okay reverting this feature out and saying "reduce min-ready if it's bothering you"21:47
clarkbjeblair: which feature?21:47
jeblairclarkb: the change to ignore min ready when near capacity21:47
zarojeblair, mordred : i assume we would still be altering tables to utf8_bin ?21:47
*** sdake_ has joined #openstack-infra21:48
*** sdake_ has quit IRC21:48
*** sdake_ has joined #openstack-infra21:48
clarkbjeblair: I am going to read my orinal change again to see if that is different21:48
jeblairclarkb: i may not have been clear; the change i'm referring to was added by ianw quite a while ago21:49
jesusaurusclarkb: jeblair what are your thoughts on a larger change to consider all labels of a provider before determining how many of each label to create?21:50
clarkbjeblair: I know, but my patchset2 was different code21:50
*** amotoki has quit IRC21:50
jeblairjesusaurus: i would rather stabilize nodepool, not touch the allocator, and work on zuulv3 changes for it so we can get rid of the allocator21:50
jeblairclarkb: ok, just making sure21:50
clarkbjeblair: and that worked too then ianw suggested the current stuff21:50
*** banix has joined #openstack-infra21:51
jeblairjesusaurus: also, it is not clear to me that the allocator does not already do that :)21:51
*** dimtruck is now known as zz_dimtruck21:51
*** mrunge has quit IRC21:52
jeblair(or at least greatly simplify the allocator)21:52
*** zz_dimtruck is now known as dimtruck21:52
clarkbright I think allocator is doing the correct thing21:53
clarkbthis is determining demand to feed into the allocator that is being trouble21:53
*** mriedem is now known as mriedem_away21:55
banixanteaya: could you or any other core please add me (username on gerrit is mb-s) to the newly created project networking-ibm and networking-ibm-release21:55
*** esker has quit IRC21:55
jesusaurusoh, if the allocator is taking this demand and figuring out the correct number of nodes to allocate, then why not just always add the min_ready to the demand?21:55
clarkbhrm the capacity for loop does accomodate all providers so maybe my initial thoughts on why tests fail are wrong21:55
*** hashar has quit IRC21:56
fungijeblair: zaro: mordred: sorry, stepped away for a moment, but do we need to check whether we can set a default encoding in trove for that instance?21:56
clarkbjesusaurus: thats what it used to do prior to the change jeblair has suggested we might revert. The chagne was made because if min-ready is high for some labels you can start other labels21:56
clarkbjesusaurus: because min-ready ratios play into the allocators calculation21:56
*** HeOS has joined #openstack-infra21:56
jesusaurusright, so you can use that to your advantage to tune which labels get prioritized21:57
jeblairfungi: it might be nice to know, but considering we don't manage that in puppet right now, i worry about setting it there (since it affects the way data are stored)21:57
*** yfried|afk is now known as yfried|pto21:58
openstackgerritMathieu Gagné proposed openstack-infra/jenkins-job-builder: Add ability to configure releaseEnvVar for maven-deploy publisher
openstackgerritMerged openstack-infra/ansible-puppet: Its 0o not 0c to make an octal number
fungifair enough, that's admittedly a longer-lasting effect than the timeout config change we added21:59
jeblairfungi: yeah, if we forget to set that, we haven't lost any data21:59
SlickNikfungi / jeblair: You can use the trove config groups API to set the default encoding if you choose to do so.22:00
jeblairclarkb: i could probably get used to the behavior you have in the current patchset (4)22:00
jeblairclarkb: if you end up deciding it's the best way to go22:00
*** sdake_ has quit IRC22:00
* SlickNik just saw the conversation and is reading scrollback for context22:00
clarkbjeblair: my gut is telling me its correct, trying to quantify that though :)22:01
jeblairSlickNik: (and comments therein) should help fill you in22:01
mkj_wkclarkb: were you the one who suggested ssh -T to me earlier?22:01
fungiSlickNik: thanks. i think right now the concern is that we don't have any configuration management/orchestration around our use of trove22:01
fungiSlickNik: so it's too easy to forget to do that22:01
*** marun has quit IRC22:01
clarkbmkj_wk: yes22:02
jeblairhopefully we will be able to manage that with mordred's ansible work in the future22:02
mordredwe will22:02
*** otter768 has joined #openstack-infra22:02
mkj_wkI have another case of EOFError that it looks like the only way around it other than ugly time.sleep(1) is to include a dependency on the pexpect module22:02
mkj_wkwhich is fine for py322:03
mkj_wkbut maybe not fine for py2622:03
clarkbjesusaurus: jeblair let me try one more thing22:03
fungimkj_wk: pexpect has some odd quirks. i used to be a fan, but in more recent years have discovered that reimplementing whatever i was using it for from stdlib functions is pretty trivial and easier to debug22:04
mkj_wkfungi: ugh22:04
mkj_wkfungi: I didn't go down the pexpect path yet22:05
*** mpopow_ has joined #openstack-infra22:05
zarojeblair, mordred : so it's unclear to me what you mean when you say alter table. Does that mean change all the db configs (charset & collation) to utf8? and change the collation on the gerrit tables to 'utf8_bin'?22:05
mkj_wkeven turning off line buffering and waiting for a single byte() to be written hangs22:05
clarkbjeblair: jesusaurus I think the trouble with jesusaurus' algorithm is that max(0, start_demand -ready) removes the -ready if startdemand < ready22:05
*** annegentle has quit IRC22:05
mkj_wkI'm guessing it's related to messing around with file descriptors for logging in the test suite22:06
mkj_wkkind of frustrating; I have all the branches covered by unit tests but I really want to keep the functional test22:06
fungimkj_wk: can you interrupt it and provide the traceback? though i'm quickly reaching the point in the evening where i'm probably going to go do some not here stuff22:06
mkj_wkand with time.sleep(1) it does validate the resulting output correctly22:07
zarofungi: are we gonna test that change to mysql config?22:07
mkj_wkit's from raw_input22:07
*** mpopow has quit IRC22:07
*** otter768 has quit IRC22:07
fungizaro: sure, did you disable puppet on review-dev and edit my.cnf yet?22:07
*** baoli has quit IRC22:08
zarofungi: isn't my.cnf on the trove server?22:08
*** yfried|pto is now known as yfried|afk22:08
*** cody-somerville has quit IRC22:08
clarkbjesusaurus: jeblair ya thats it22:08
clarkbok let me write a new patch that accomodates all that I have jus learned22:08
fungizaro: not in this case no. that's what we were talking about not bothering with the server side just setting my.cnf on the gerrit server (assuming the jdbc is looking at it at all)22:08
zarofungi: i don't have ssh access to the db22:08
mkj_wkfungi: but I don't think the traceback helps much22:09
mkj_wkfungi: Even just as a backup, and for review of the changes I did due to latest review cycle, I think I'll submit the patch for review with the current ugly time.sleep()22:09
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Run ironicclient functional tests as STACK_USER
mkj_wkbe interesting, if nothing else, to see what it does under jenkins22:10
fungimkj_wk: yeah, i wonder if do_input() should just be mocked out for unit tests22:10
zarofungi: hmm, i was referring to the db side.  i didn't know there was a my.cnf on review-dev22:10
mkj_wkfungi: I *did* mock it for the unit tests22:10
mkj_wkthe unit tests are fine22:10
jeblairzaro: nobody does so i think the idea is to set my.cnf on review-dev to use the option, then see if gerrit respects it (without setting the connection string)22:10
mkj_wkit's the funtional testing where we're invoking a subprocess22:10
fungizaro: it would affect mysql client sessions run from the gerrit server22:10
fungiand potentially other things that look at that config (no idea if gerrit's process does or not)22:11
jeblairzaro: if gerrit needs the connection string set, then probably we should do it both places or revisit the idea of setting it in trove22:11
*** dboik has quit IRC22:11
*** dims has quit IRC22:11
* zaro is confused22:12
fungiultimately, i think that having our trove databases default to latin1 for _anything_ is bad form, but i understand it was a default in mysql for so long that lots of trove users probably expect it22:12
*** jedimike has joined #openstack-infra22:12
zaroreview-dev.o.o is set to use an external trove db.  why would i make changes to the my.cnf on review-dev.o.o?22:13
jeblairzaro: because mysql clients read that file22:13
fungiand ultimately most applications i've dealt with ignore what charset is reported by the database server, so gerrit strikes me as a little weird in that regard22:13
*** baoli has joined #openstack-infra22:13
openstackgerritMichael Johnson proposed openstack-infra/git-review: Choose correct branch for rebase when submitting
fungizaro: when we do maintenance, database backups and restores, et cetera, they use the my.cnf on the gerrit server22:14
zarook.  will give a try.22:14
fungibut yeah, i have doubts that gerrit itself will ever know anything about what's in that file22:15
jeblairme too, but i'd like certanties :)22:15
fungiscientific method and all that22:15
mkj_wkI don't think the rest of the patch is ugly, just that one sleep... We'll see what happens in Jenkins; I'm curious22:16
anteayafungi: could you add banix (gerrit username: mb-s) to,members and the co-responding -release group when you have a moment?22:16
*** carl_baldwin has quit IRC22:16
jrollhey, ironicclient gate is busted on a new httpretty version, someone mind getting this through?
fungimkj_wk: you said it was hanging under python 2.x but not 3.x? we seem to use raw_input() in 3.x and input() in 2.x22:17
fungianteaya: banix: done22:18
banixanteaya: fungi thanks!22:19
jeblairjroll: is that the same issue that prompted the 087 blacklist?22:19
*** craigbr has joined #openstack-infra22:19
jrolljeblair: I'm not sure, I wasn't in on that one. wouldn't be surprised22:19
jeblairjlvillal: ^22:20
jeblairi ask because the linked bug seems to have some people blacklisting 0.8.722:20
jrolljlvillal: are you thinking pin it harder?22:20
jrollerr, jeblair ^22:20
jeblairi wonder if we should continue the whack-a-mole, or change strategy22:21
jrollyeah, I was thinking the same22:21
openstackgerritClark Boylan proposed openstack-infra/nodepool: Test that dib failures don't stop node builds
openstackgerritClark Boylan proposed openstack-infra/nodepool: Accomodate ready nodes in min-ready checks
openstackgerritClark Boylan proposed openstack-infra/nodepool: Don't allocate nodes to providers without images
zarofungi, jeblair : i changed my.cnf on review-dev.o.o but it didn't work22:21
*** mmedvede has joined #openstack-infra22:21
clarkbjeblair: jesusaurus ^ I believe that to be correct and it passes tests locally22:21
clarkbit also reads a bit cleaner than patchset2 which was also correct I think22:21
*** movielovers has left #openstack-infra22:22
anteayafungi: thanks for your corrections and input on
jeblairzaro, mordred, fungi: okay so i think our choices are (a) set my.cnf AND gerrit's jdbc connection string or (b) set the value for the server in trove22:22
zarofungi, jeblair : i have tested that config on my personal mysql server and it does work for me.  but i applied it on mysql server side22:22
clarkbok puppet is much happier now with the octal settings22:24
clarkber octal fix. I do notice that the ansible task can't figure out the puppet agent state on some nodes though22:24
fungijeblair: we can likely just add it to the same custom trove config we're already applying to adjust the timeout. it strikes me as something we should consider applying to all our trove instances anyway, across the board22:24
*** ayoung has joined #openstack-infra22:24
*** hogepodge has quit IRC22:24
jeblairjroll, jlvillal: +2 with the comment that i would also +2 something stronger22:25
*** annegentle has joined #openstack-infra22:25
*** pblaho has joined #openstack-infra22:25
jeblairfungi: yeah, the complexity of (a) is leading me to think that (b) might be the better option (and try to remember not to screw it up)22:26
clarkbwow thats interesting. zm05 is running puppet222:26
jrolljeblair: thanks22:26
jeblairclarkb: i love it when that happens22:26
fungiclarkb: is it staging a revolt?22:26
clarkbfungi: must be22:26
clarkbI think 05-08 are all that way22:26
jeblairsomeday it's going to get fed up and just run ansible22:26
clarkblikely built together and all got the same initial config22:26
*** saper has quit IRC22:26
* fungi was thinking exactly the same thing22:26
clarkbthe next puppet run is the first one that will use project_config_ref22:27
clarkbin ~3 minutes22:27
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Install python-ironicclient from git when we're testing it
jlvillaljeblair: I think it is the same bug for 0.8.7 and
jlvillaljeblair: jroll: I put the link to what I think is the bug.  I'm not 100% sure though.22:28
devanandaadam_g: with ^ I think we could remove the gate-tempest-dsvm-neutron-src-python-ironicclient job ?22:29
* clarkb notices that hpcloud is mostly errors today22:29
fungijeblair: mordred: zaro: SpamapS: SlickNik: so the last question i think is, regardless of where we end up setting our encoding preference, do we need to do any sort of etl to convert text from latin1 to utf8 or is a sufficient amount of latin1 a proper subset of utf8 that it's not worth bothering (or is there some special mysql operation that happens during table conversion to take care of it for22:30
*** asettle is now known as asettle-mtg22:30
clarkbClientException: Unknown Error (HTTP 500)22:30
fungiclarkb: after seeing how the sausage is made, i pretty much just assume that errors are the steady state all clouds approach22:31
mordredclarkb: it's my favorite error22:31
clarkbmordred: OSError: [Errno 2] No such file or directory: '/etc/facter/facts.d/ansible.json'22:31
jeblairfungi: oh, i guess we should change the table regardless?22:31
clarkbmordred: so I think your ansible module needs to mkdir22:31
*** jaypipes-afk is now known as jaypipes22:31
mordredclarkb: piddle. k22:31
*** dangers is now known as dangers_away22:31
*** adalbas has quit IRC22:31
mordredclarkb: think we should just do that in the role regardless? or in the python?22:32
fungijeblair: maybe? does that make alterations to the data stored in it?22:32
clarkbmordred: I think it should be part of ansible-puppet since it intends to make use of the feature22:32
fungimy lack of mysql-savvy is showing22:32
clarkbmordred: but role vs puppet I don't know enough about ansible to say22:32
adam_g devananda maybe, still looking thru JJB yaml22:33
jeblairfungi: i don't think so22:33
clarkber sorry role vs python22:33
clarkbmordred: though maybe python because that is where we write to the dir so making the dir keeps everything together22:34
*** dboik has joined #openstack-infra22:34
clarkbmordred: keeps the code readable for understanding22:34
*** annegentle has quit IRC22:34
zarofungi: i believe it does.  but i think you can get around that by doing export then import22:34
fungijeblair: hence my question as to whether there's any etl we should be doing22:34
fungizaro: yeah, basically that22:34
fungiexport, change encoding, import22:34
*** dboik_ has joined #openstack-infra22:34
mordredclarkb: kk22:35
*** peristeri has quit IRC22:35
jeblairfungi: oh, i would expect the latin1 < utf8 characteristic to mean we don't need to etl.  keep bugging mordred for an answer though.  ;)22:35
fungijeblair: also SpamapS had described the challenges relating to mysql charset changes at one point, so cornering him might help22:36
*** cody-somerville has joined #openstack-infra22:36
*** jgrimm is now known as zz_jgrimm22:36
fungibut yeah, i gather that at least _most_ (all?) of latin1 is a subset of utf8 using the same codepoint to byte value mapping22:36
clarkbmordred: if you are taking care of that I will look into the hpcloud failures a bit more22:36
*** dboik has quit IRC22:38
fungimy jukebox has decided to start playing the original wendy carlos film score for tron, so i'm taking that as a clue to find beer and do some other things for a bit22:39
clarkbwe are seeing {"message": "No valid host was found. ", "code": 500, "created": "2015-03-11T22:32:18Z"} again22:39
SpamapSjeblair: what is the question?22:39
SpamapSI have a ridiculous amount of experience dealing with utf-8 encoding issues and mysql.22:39
jeblairfungi: END OF LINE22:40
clarkbI do not think it is related to bad images22:40
fungiSpamapS: we have a mysql database with tables containing latin1 text and want to switch to utf8. any need to convert the data?22:40
clarkbinstead the node with that error is in az2 and last time when I tried to dbeug the image I saw that az2 would fail with that error but az1 and az3 would now22:40
fungijeblair: heh. i recognize that22:40
SpamapSfungi: are you 100% certain all of the data is latin1 and none of it is high-byte?22:40
clarkbso I think that az2 is simply not able to accomodate those requests at times because it is the default az2 so likely has higher use22:40
SpamapSfungi: that is _not_ an easy question to answer. ;)22:41
clarkbwe can try to use less of az2 or maybe not use it at all22:41
clarkbor maybe poke hpcloud to add capacity there?22:41
jeblairclarkb: so you're seeing 500 unknown errors predominately in az2?22:41
*** banix has quit IRC22:41
fungiSpamapS: in this case i think it is because gerrit seems to apply some substitution to data it's storing if there is no representation in the target encoding for the codepoint provided, so stores literal ? characters in the db22:41
*** nikhil_k is now known as _pope22:41
SpamapSfungi: if the answer is yes, though, then no, "alter table foo convert to charset 'utf-8'" is safe and effective22:42
clarkbjeblair: when the theory was bad image and I was trying to reproduce that yes. I am still trying to see if that is the case with nodepool now22:42
*** gordc has quit IRC22:42
clarkbjeblair: hopefully will be able to confirm this theory soon22:42
SpamapSfungi: ah yes! gerrit probably uses iconv or some workalike22:42
fungiyeah, a java equivalent of that anyway22:42
fungiso sounds like we ought to be fine22:43
fungior at least no more broken after the conversion than before22:43
*** _pope is now known as nikhil_k22:43
mordredclarkb: yeah - I got it in just a sec22:43
clarkbalso nova show doesn't show the AZs properly :/22:43
SpamapSfungi: and yeah, thats a lossy conversion unfortunately.22:43
jeblairSpamapS: so i think the proposal is: alter the tables and set the connection encoding to utf8 in trove22:43
SpamapSfungi: but it does mean you can just convert to utf-8 and then make sure gerrit knows to expect utf-8 in/out22:43
SpamapSYeah that's the other piece. :)22:44
fungiSpamapS: mostly we want to switch to utf8 so we can stop losing from here on, but there's no hope of recovering whatever gerrit never stored in the first place obviously22:44
SpamapSthats the _stupid_ piece. :-P22:44
*** thedodd has quit IRC22:44
fungithankfully it's just review comments that are at issue. important data is in git and utf8 works there just dandy22:45
clarkbjeblair: I am seeing it in az3 nodes too22:45
clarkbjeblair: but the error message there is slightly different {"message": "No valid host was found. Exceeded max scheduling attempts 3 for instance 7321402d-0ccc-4bf3-9270-5b20583b57d2", "code": 500, "created": "2015-03-11T22:30:39Z"}22:45
*** Longgeek has joined #openstack-infra22:45
jeblairclarkb: so that seems like they ran out of capacity :(22:46
fungiclarkb: i've seen that one plenty off and on22:46
clarkbjeblair: ya that would be my guess22:46
*** dims_ has joined #openstack-infra22:46
fungisomeone (us?) called the bluff on their overcommit22:46
clarkbfungi: and throttled resources :)22:46
fungimaybe gilliard or tteggel are around and know what's going on22:47
clarkbbut ya sampling the most recent errors there is those two error messages with on affecting az2 and the other affecting az322:47
jeblairwow.  currently used nodes: rax: 253.  hpcloud: 59.22:47
fungi has a couple items for todat22:48
fungitoday too22:48
clarkbI don't think those should affect us. Horizon and account creation22:48
clarkbbut maybe fallout?22:48
*** ayoung has quit IRC22:50
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Skip the neutron client compat job for ironic, test it elsewhere
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Skip the default client compat job for ironic, test it elsewhere
*** amotoki has joined #openstack-infra22:50
adam_gdevananda, ^22:51
*** jedimike has quit IRC22:51
*** banix has joined #openstack-infra22:52
*** banix has quit IRC22:53
*** Longgeek has quit IRC22:55
clarkbfungi: any chance you can review the stack starting at 162673?22:55
fungiclarkb: sure22:55
*** sdake_ has joined #openstack-infra22:55
*** amotoki has quit IRC22:56
fungiseen those flying by but hadn't dug into them yet22:56
clarkbaz1 just had a {"message": "No valid host was found. Exceeded max scheduling attempts 3 for instance 982b5dde-c89a-4d52-aa09-c95962503f98", "code": 500, "created": "2015-03-11T22:48:41Z"}22:57
jeblairclarkb: how do you see that error message?22:57
*** sdake__ has joined #openstack-infra22:57
fungishows up in the nodepool debug log, yeah?22:58
mordredjeblair: nova show on an instance id will have that22:58
clarkbjeblair: I am grepping for 'status: ERROR' in the nodepool debug log then taking the uuid and using nova show on it22:58
openstackgerritWayne Warren proposed openstack-infra/infra-specs: (WIP) Initial work on gitub-integration spec.
fungioh, right, nova show on an instance in error starte22:58
fungithat's where i ended up finding it in the past22:58
fungiwonder how we could go about getting that into nodepoold's line of sight22:59
*** baoli has quit IRC22:59
fungii guess when it's doing the nova list and spots an instance it was waiting for go into error, it could grab nova show details before switching to delete22:59
openstackgerritgreghaynes proposed openstack-infra/nodepool: Gracefully handle upload failures
greghaynesclarkb: ^ that works from tox, unsure why I get deadlock issues when I directly23:00
*** sdake_ has quit IRC23:00
*** tnovacik has quit IRC23:02
openstackgerritMerged openstack-infra/nodepool: Add logging Popen to tests
openstackgerritMerged openstack-infra/nodepool: Use local allocation history in scheduler
*** vigneshvar has quit IRC23:03
*** dannywil_ has joined #openstack-infra23:03
*** dannywil_ has quit IRC23:03
*** craigbr has quit IRC23:03
jeblairclarkb, mordred, fungi: the last time we got more than 100 instances from hpcloud was 1600 utc (when we had 200).  it looks like we're between 50-100 since 1700 utc.23:04
clarkbjeblair: also sometimes nova show shows the az as "nova" in that case I have been looking in the nodepool list for the uuid as that shows me the az nodepool tried to schedule it in23:04
fungiclarkb: i'm cool with 162714 but ianw makes a good point with readability on the extra_demand calculation. you're doing (albeit trivial) extra work there which makes the relationship to actual_demand less obvious23:04
*** e0ne is now known as e0ne_23:04
clarkbfungi: so thats what the giant discussion was above23:04
clarkbfungi: ianw's suggestion is unfortunately not correct23:04
jeblairour graphs seem to show a problem from 1000 to 1600 utc23:05
*** ghostpl_ has quit IRC23:05
clarkbfungi: so I have incorporated some of his suggestions while writing a correct demand calculation23:05
fungioh, rereading again to see where i miscomprehended23:05
*** patrickeast_ has joined #openstack-infra23:05
clarkbfungi: specifically we can't max(0, x - y) then reuse that value in arithmetic later because the max functions breaks the maths23:05
*** dannywilson has quit IRC23:06
clarkbfungi: max(0, x - y) + z != max(0, x-y+z)23:06
*** sdake__ has quit IRC23:06
*** patrickeast has quit IRC23:06
*** patrickeast_ is now known as patrickeast23:06
fungiclarkb: oh, good point. if start_demand - ready is a negative value, then we'd be combining label.min_ready to 023:06
cody-somervilleclarkb: jeblair: Is this US East or West?23:06
clarkbfungi: yup and that speficall is what broke when I tried jesusaurus' suggestion. So the current patch is blend of jesusaurus and ianw math23:07
clarkbcody-somerville: east23:07
jeblairpeople keep asking that -- is there a west we should be using?23:07
clarkbjeblair: no, I was specifically told to not use west23:07
clarkbthough that may have changed since then23:08
fungiapparently hpcloud west is a myth used as a marketing ploy to convince customers there are multiple regions ;)23:08
clarkbin fact when I tried to add it there was a great panic23:08
*** sdake_ has joined #openstack-infra23:09
*** hugh_afk is now known as hughhalf23:09
fungiclarkb: was 162237 spurred on by an error spotted in the wild?23:11
*** hdd_ has quit IRC23:11
clarkbfungi: yes, right around when rax rebooted the world we saw that23:12
fungioh so joyous23:12
clarkbfungi: basically the exception rose up into the main event loop short circuiting it23:12
openstackgerritgreghaynes proposed openstack-infra/nodepool: Gracefully handle upload failures
clarkbjeblair wrote a fix that we got in so this is mostly CYA to make sure we don't regress again23:12
ianwclarkb: good point, that effect might be subtle enough that a extra comment to call it out is justified.23:12
fungiyep, looks like all tests and no runtime change which is why i was wondering23:13
clarkbianw: ya I should probaby put my != statement I wrote above in the change23:13
*** sdake_ has quit IRC23:14
ianwclarkb: the other option might be to just remove the whole thing all together23:15
openstackgerritClark Boylan proposed openstack-infra/nodepool: Test that dib failures don't stop node builds
openstackgerritClark Boylan proposed openstack-infra/nodepool: Accomodate ready nodes in min-ready checks
openstackgerritClark Boylan proposed openstack-infra/nodepool: Don't allocate nodes to providers without images
clarkbI am fine with it in. I think it is understood now and the comments make it pretty straightforward to understand23:16
jeblairclarkb: did you see scheduling errors in az3?23:16
*** hichihara has joined #openstack-infra23:16
openstackgerritJeremy Stanley proposed openstack-infra/nodepool: Gracefully handle upload failures
clarkbjeblair: yes23:16
fungirebased on outdated parent ^23:17
clarkbjeblair: all of them had the problems but az3 appears to be worst23:17
clarkbthough I am mostly only looking at the current errors and not historical ones (for simplicity)23:17
*** asselin_ has joined #openstack-infra23:18
clarkbI wonder if maybe this is an image interaction? we could try deleting images to see if it alleviates. We got much better reliability last time after going to the old image23:20
fungizuul has finally started making progress on the post pipeline again23:20
clarkbwhich is why I had focused on the image but its may be more complex than that23:20
*** woodster_ has quit IRC23:20
jeblairclarkb: i thought we got instances that booted, but did not boot completely23:21
rcarrillocruzclarkb: i tried your foreground fix for the devstack gate, works perfect23:21
jeblairclarkb: is there another problem i am not remembering where we would not get instances reliably?23:21
clarkbjeblair: yes that is what we got there so this does appear different23:22
clarkbjeblair: so ssh was failing on those nodes and nodepool recycled them23:22
clarkbthe tie in is when i tried to manually test the broken image I got these scheduling errors in az223:22
clarkbbooting in az1/az3 fixed them though23:22
jeblairclarkb: could we have leaked enough ports to be over-quota again?  was there a specific 'not enough ports' error?23:23
*** asselin_ has quit IRC23:23
clarkbjeblair: I think we got quota errors instead of these unknown exceptions, but worth checking how we look there23:23
jeblairwe have 605 ports23:24
jeblairwhat's our port quota?23:24
clarkbjeblair: 60023:24
clarkbI think, it should roughly be 1 port per node and we have a 600 node limit23:25
clarkbwe also have 601 floating IPs23:25
clarkbnova list shows 542 nodes23:25
clarkber -3 so 53923:25
*** sdake_ has joined #openstack-infra23:26
jeblairclarkb: so we could be hitting limits on ports and ips23:27
clarkbwe could be, but I have a hunch thats more of a symptom than a cause. Leaks would be more probable when nodepool is spinning super hard to boot stuff23:27
mordredclarkb, jeblair: do we have a current instance in that state?23:28
clarkbmordred: b741c9a5-1b1e-4028-a6ae-cf324e33203323:28
clarkbbut expect nodepool to delete it soonish23:28
mordredclarkb: can you hold it?23:28
clarkbmordred: its hard to because the delete happens immediately upon error23:29
jeblair{"message": "Connection to neutron failed: Maximum attempts reached" is another one, btw23:30
clarkbI can run my scripts to pick out the leaked ports now23:30
jeblairclarkb: ~root/ will delete them23:31
jeblairhow about i kick that off in screen?23:31
clarkbjeblair: sure, does it do the listing too?23:31
*** bknudson has quit IRC23:31
*** annegentle has joined #openstack-infra23:32
clarkbI had been doing the listing and checking it to avoid deleting dhcp and router ports23:32
jeblairclarkb: oh, take a look at it and tell me if it does what you think we should do then23:32
clarkbif thats jesusaurus' script I think it will work it will just use more api calls than necessary which probalby isn't a real problem23:32
jeblairclarkb: yep, it is23:33
jeblairit's slow but steady23:33
fungitortoises all the way down23:33
clarkbya should be fine then. it specifically greps for compute as the device_owner23:33
*** ihrachyshka has quit IRC23:34
clarkbthe faster way is the do neutron port-list -c id -c device_id -c device_owner to get those fields all at once. But then you would need to test the new script :)23:34
jeblairclarkb: running old script in screen23:34
openstackgerritQiming Teng proposed openstack-infra/project-config: new-project: stackforge/senlin
*** wenlock has quit IRC23:34
jeblairclarkb: noc tracked down a 'scheduled 3 times' error and confirmed it was due to ports23:36
*** bswartz has joined #openstack-infra23:37
clarkbI am going to run my script to delete unused floating ips23:37
clarkb`venv/bin/neutron floatingip-list | grep -v '10\.0\.' | sed -e '1,3d' -e '$d' | cut -d'|' -f 2 | xargs -n 1 -P 1 venv/bin/neutron floatingip-delete` ftr23:37
clarkbI'm weird and find grep -v, sed, and cut all useful for those specific operations rather than using only sed :)23:38
*** cody-somerville has quit IRC23:39
clarkbalso I used xargs -P 1 beacuse the last time I tried -P10 I got rate limited23:40
clarkbianw: lol python greatest hits tracebacks23:41
*** chlong has joined #openstack-infra23:41
*** annegentle has quit IRC23:43
ianwclarkb: i'm almost certain the call stack from the kernel entry-point, through the filesystem, through the i/o subsystem, to the driver is shorter than that.  i guess that isn't doing something complex like installing a package though23:43
openstackgerritMerged openstack-infra/project-config: Make manual checklang target voting
clarkbwow got rate limited with -P123:43
clarkbwill have to make a second pass at this to get those that were missed23:43
greghaynesclarkb: shouldve used -P0.523:44
clarkbgreghaynes: your test is running under tox -epy27 -- --until-failure on my local testbed23:45
clarkbgreghaynes: did you figure out the meta thing?23:45
greghaynesyes, I was doing something silly23:46
clarkbwas it the kwargs.get in the fake?23:46
greghaynessuch silly23:46
clarkbcool, and the change to the provider_manager is about what I expected23:47
clarkber nodepool.py23:47
jgriffithclarkb: wondering if you might point me in the right direction on something with unit tests and output23:47
jgriffithclarkb: I'm trying to figure out how to get rid of all the "Captured stdout" cruft in the cinder unit tests23:48
jgriffithclarkb: you had a ML post with some similar stuff a while back... thought you might have some insight23:48
clarkbjgriffith: sure, let me dig up where that happens23:48
clarkbjgriffith: we have fixtures that collect stdout and stderr and logs for each test because the tests run in parallel23:50
*** e0ne_ has quit IRC23:50
rcarrillocruzcan't quite figure out why this failure:
clarkbjgriffith: so to associate particular output with specific tests that was the solution. If you don't want captured stdout then I would make the tests not write to stdout at all23:50
rcarrillocruzcacert is None, why it gets in?23:50
rcarrillocruztests work fine locally, but i keep getting that failure in gate23:51
rcarrillocruzmordred: ^23:51
clarkbjgriffith: should be easy to identify where that happens by monkey patching stdout to raise exceptions23:51
jgriffithclarkb: hmm.. we've got *a version* of that in place.23:51
jgriffithclarkb: it's libs that seems to be the culprit23:51
*** ajmiller has quit IRC23:51
SpamapSrcarrillocruz: how are you running the tests locally btw?23:52
rcarrillocruzputting some print statements in OpenstackCloud to figure out what's passed as argument on my cloud devstack instance, can you figure at  a glance what may be?23:52
*** mtanino has joined #openstack-infra23:52
rcarrillocruzSpamapS: tox -e functional23:52
rcarrillocruzthat on a plain VM with devstack23:52
clarkbjgriffith: the other alternative is to remove the fixture and have all that write to stdout but that might be the worst thing (bceause its all interleaved and not really useable)23:52
*** bknudson has joined #openstack-infra23:52
*** funzo has quit IRC23:52
jgriffithclarkb: yeah, I gathered that from your ML post I read ;)23:52
SpamapSrcarrillocruz: maybe devstack-gate is configuring things differently.23:52
rcarrillocruzbut simulating a devstack gate test i get that same error from above ^23:53
clarkbrcarrillocruz: SpamapS perhaps os-client-config is different for some reason and providing different inputs?23:53
jgriffithclarkb: hmm... so one of the tests I run it has OS_STDOUT_CAPTURE False23:53
rcarrillocruzi'm puttint print statemnts on shade code , something is getting passed to cacert, but it  is weird to me since cacert defaults to None...23:53
SpamapSclarkb: yeah thats another possibility.23:53
SpamapSrcarrillocruz: print statements might be problematic as they'll interrupt subunit23:54
SpamapSrcarrillocruz: unless you intercept stdout.23:54
clarkbSpamapS: they should not if you have the correct base class23:54
clarkbSpamapS: see discussion with jgriffith :)23:54
jgriffithso the offenders all seem to be libs23:54
jgriffithclarkb: SpamapS hehe23:54
clarkbjgriffith: that config should pass your Falsethrough23:55
anteayaclarkb fungi whichever of you has a moment, any reason to hold off on merging: ?23:55
SpamapSah set OS_STDOUT_CAPTURE=true23:55
anteayaclarkb: it just changes a jenkins script so I don't see the harm23:56
clarkbanteaya: I don't think so, we will just want to watch it once new images are built tomorrow23:56
anteayabut I don't want to miss something23:56
clarkbanteaya: someone might want to check my math on how many seconds == 6 months23:56
anteayaokay I'll merge now then23:56
anteayaI wondered and decided to trust you23:56
anteayawill check first23:56
clarkb60 * 60 * 24 *30 * 6 = 15552000 and thats seconds minutes hours days months23:56
clarkbso I think its correct23:57
anteayawe could also do 365/2 which is 182 rather than 18023:58
anteayaif we care23:58
clarkbnah its mostly fuzzy anyways23:58
*** patrickeast has quit IRC23:58
clarkbaiui swift runs a service on its nodes that will delete at some point after that time period23:58
clarkbso they may live longer but not shorter23:58
notmynameclarkb: oh! I saw is some buffer playback this week you were asking about swift's expiring objects23:59
anteayaI confirm the accuracy of your calculation23:59
clarkbnotmyname: yes, want to review ?23:59
anteayanotmyname: context:
clarkbnotmyname: basically we are moving full steam ahead to using swift logs and one thing we realized we could do is have swift delete old ones for us23:59
clarkbnotmyname: we were told that we cannot update this attribute after we upload the object23:59

Generated by 2.14.0 by Marius Gedminas - find it at!