Friday, 2015-02-06

clarkbfungi: nice00:00
anteayawell at least we know00:00
fungithe vcsrepo docs imply that we want "present"00:01
*** zz_avozza is now known as avozza00:02
openstackgerritJeremy Stanley proposed openstack-infra/puppet-meetbot: Use present with vcsrepo bzr  https://review.openstack.org/15342700:05
fungiclarkb: morganfainberg: nibalizer: ^00:05
morganfainbergah.00:05
jeblairhow does the apply test not catch that?00:06
morganfainberggood question00:06
mordredjeblair: because puppet00:06
morganfainbergand the new version supports latest00:06
morganfainbergi'm sure00:06
jeblairfungi: i mean, it should pretty much be running the command you ran to get that error, right?00:06
clarkbI think because puppet accepts latest as a state but the provider doesn't and it isn't executing the provider00:07
morganfainberg^^00:07
clarkbjeblair: no apply test runs in noop mode00:07
mordredclarkb: o_000:07
fungiwell, except i ran --noop00:07
clarkbfungi: oh00:07
fungiand got that exact error00:07
fungii'm trying again with out --noop just to make sure it's the same00:07
mordredSpamapS: it's possible that you could exclude the devstack-cache element and still work on the problem00:07
mordredSpamapS: devstack-cache is cloning every repo in the system00:08
mordredSpamapS: on the other hand, once you do it once, the dib cache will make it nicer next time :)00:08
clarkbmordred: should 153354 be WIP?00:08
*** reed has quit IRC00:08
jeblairi will give someone a cookie if they can make the apply tests work as well as they did with puppet200:08
jeblair(this offer is not restricted to solutions that use puppet)00:09
anteayacookies and cats00:09
fungiyeah, same error without --noop too so this isn't some weird noop behavior00:09
* anteaya wonders what her currency should be00:09
pleia2anteaya: not lizards00:10
jeblairif it helps, i will bake the excellent recipe in clifford stoll's "The Cuckoo's Egg"00:10
jeblairwhich is the only book on computer hacking on my cookbook shelf00:10
anteayapleia2: ah but canada has so many00:12
anteayasnowflakes?00:12
fungipleia2: it _should_ be lizards, as their local rarity makes them a valuable commodity00:12
fungias compared to snowflakes00:12
anteayayes the value of lizards is much higher00:12
*** andreykurilin_ has joined #openstack-infra00:12
pleia2I was thinking about it the other way around :)00:13
anteayajeblair: sounds delicious00:13
*** vilobhmm has joined #openstack-infra00:13
clarkbjogo: you still ready to have nova functional voting?00:13
anteayasnowflakes are prettier00:13
jeblairyou know what they say "it takes lizards to make lizards"00:13
clarkbjogo: because its happening :)00:13
anteayado they say that?00:14
* anteaya must have missed that meeting00:14
*** dangers is now known as dangers_away00:15
*** abhirc has joined #openstack-infra00:16
anteayaclarkb: ha ha ha00:16
*** sabeen has quit IRC00:16
jogoclarkb: yes!00:17
clarkbhrm 15322700:17
clarkbI kinda wish that we would maybe have more of a discussion every time we change the way we list dependencies00:18
* pleia2 frowns at 14416600:19
pleia2not sure how anything gets merged into d-g :\00:19
pleia2for fun it fails a different one each time00:19
clarkb"it was decided" ok00:19
clarkbpleia2: if at first you don't succeed "recheck"00:20
clarkbpleia2: but ya :(00:20
openstackgerritMerged openstack-infra/puppet-meetbot: Use present with vcsrepo bzr  https://review.openstack.org/15342700:21
openstackgerritMerged openstack-infra/project-config: Make novaclient-dsvm-functional gating for novaclient  https://review.openstack.org/15333100:21
*** annegent_ has joined #openstack-infra00:22
clarkbmordred: does pbr need to grok any of what 153227 is doing?00:23
*** annegent_ has quit IRC00:24
*** _shaps__ has joined #openstack-infra00:25
*** Sukhdev has joined #openstack-infra00:26
* mordred looks00:27
*** _shaps_ has quit IRC00:27
mordredclarkb: no00:27
mordredclarkb: but if it's a feature we want - it should be something that it groks - potentially related to pip feature tags00:28
clarkbmordred: I guess the next thing is do we really want to keep doing this sort of thing and modify how these checks work?00:28
mordredor extras, rather00:28
clarkbmordred: I honestly don't understand why this is any better and think it is only going to be more prone to pip ordering issues00:28
clarkbmordred: especially if globbing returns files in different orders on different platforms (but I don't know if that is the case)00:28
fungirequirements.d ?!?00:28
clarkbfungi: yes we have so many dependencies we need an entire directory to hold them00:28
clarkbfungi: it was decided00:29
fungineat!00:29
mordredwell, and we want to avoid "unneeded" depends00:29
JayFAre pep8 jobs broken overall? I'm getting a traceback in pep8 itself on this test run: http://logs.openstack.org/17/153417/1/check/gate-ironic-python-agent-pep8/364729b/console.html00:29
fungiis requirements.d becoming a common pythonism, or are we becoming even more snowflake than when we just used regular requirements.txt files?00:29
mordredI think we've surpassed the point where we need to re-examine distro packaging00:30
clarkbfungi: I think its snowflaking hard00:30
mordredyah00:30
clarkbfungi: which is why I am somewhat sarcastically repeating that it has been decided00:30
mordredthis is re-inventing all of the complexity of dpkg without the other things00:30
*** markvoelker has quit IRC00:30
fungimordred: which other things? you mean the working?00:30
*** gondoi is now known as zz_gondoi00:31
*** markvoelker has joined #openstack-infra00:31
clarkbJayF: I havne't seen pep8 fail everywhere. stuff is merging00:31
*** zz_gondoi is now known as gondoi00:31
clarkbJayF: if I had to guess you aren't properlly pinning flake800:31
openstackgerritMerged openstack-infra/project-config: Add ACL to create branch/tags in proliantutils.  https://review.openstack.org/15313200:31
mordredI just -2d that00:31
mordredbtw00:31
JayFclarkb: hrm. I think we're using what's in global requirements00:31
*** dims__ has quit IRC00:31
mordredfungi: yah00:32
jeblairwait...00:32
JayFclarkb: We don't appear to have flake8 specifically in our requirements at all. https://github.com/openstack/ironic-python-agent/blob/master/requirements.txt https://github.com/openstack/ironic-python-agent/blob/master/test-requirements.txt00:33
jeblairpleia2: stable branches should not be voting on d-g00:33
JayFclarkb: If you can help me know the right thing; I can make it that way :)00:33
clarkbJayF: ya I am trying to figure this out00:33
JayFty00:33
jeblairpleia2: if you want to track down how that happened, we can unwind it00:33
clarkbjeblair: https://github.com/openstack/ironic-python-agent/blob/master/test-requirements.txt#L4 hacking depends on flake8 and pep800:34
pleia2jeblair: sure, having a look00:34
clarkber JayF ^00:34
JayFclarkb: I know that; Just unsure what the best way is to go forward. Should I add a flake8+pep8 pin to test-requirements.txt that matches what's in global reqs?00:34
*** wenlock has quit IRC00:34
clarkbJayF: no I don't think so00:35
*** markvoelker has quit IRC00:35
*** rfolco has joined #openstack-infra00:35
mordredclarkb JayF: hacking has the flake8/pep8 pin00:35
mordredthat's why it's listed first00:36
clarkbJayF: its also possible that your code tickles a bug in the linters00:36
JayFSo maybe this is some crazy one time failure and I should recheck?00:36
JayFif it doesn't happen again pretend it never did?00:36
clarkbJayF: I would see if passes locally00:36
clarkbJayF: just tox -repep800:36
mordredoslotest is in requirements00:36
JayFyeah that won't take long, 1s00:36
mordredis it possible it's pulling in hacking?00:36
jeblairpleia2: it may have been before clean-check00:37
*** camunoz has joined #openstack-infra00:37
openstackgerritJeremy Stanley proposed openstack-infra/puppet-meetbot: Correct source for plugins directory  https://review.openstack.org/15343100:37
clarkbmordred: the verison came out looking good though 0.8.100:37
jeblairpleia2: i think with clean-check, we may need to either make "-nv" (non-voting) versions of all those jobs, or move them to experimental00:37
fungimorganfainberg: clarkb: jeblair: clearly any resources which depend on a vcsrepo source just aren't tested in the apply test ^00:37
jeblairpleia2: probably the -nv variants would be best00:37
clarkbjeblair: I think those jobs only run against d-g00:38
mroddenhttp://logs.openstack.org/17/153417/1/check/gate-ironic-python-agent-pep8/364729b/console.html#_2015-02-05_23_07_52_835 lineno is topline of the file?00:38
clarkbjeblair: so safe to make them non voting (double check of course)00:38
clarkbpleia2: ^00:38
fungi(which is probably half our puppet)00:38
pleia2jeblair: ah, that was some time ago00:38
morganfainbergfungi, that one there is likely my bad - sorry for the copy/paste error :(00:38
jeblairclarkb: "check-tempest-dsvm-full-juno" is the one that's currently failing00:38
clarkbfungi: exciting00:38
JayFclarkb: https://gist.github.com/jayofdoom/54181f10892a278b5221 fails identically locally00:38
clarkboh tempest uses that job too00:39
clarkbJayF: my money is one your code tickling a bug in pep800:39
morganfainbergfungi, :( i really am not trying to make your life hard! I swear!00:39
mroddenprobably in one of the checks00:39
*** achanda has quit IRC00:39
JayFclarkb: added a pip freeze from the venv inside the gist00:39
fungimorganfainberg: nope, your contribution is appreciated!00:39
openstackgerritK Jonathan Harker proposed openstack-infra/jenkins-job-builder: The email publisher needs a list of recipients  https://review.openstack.org/15343200:39
mroddenif not noqa(self.lines[lineno - 1]) seems like bad code form IMO00:39
JayFpep8==1.4.5 and flake8==2.000:39
clarkbJayF: ya thats what the gate job used00:40
fungimorganfainberg: i am more annoyed by puppet and our apply job not catching seriously broken changes00:40
morganfainbergyeah but that is clearly copy/paste error from previous patchset- prior to the move to the new repo/module00:40
mordredfungi: ++00:40
morganfainbergfungi, so two fold - apply job suckage and copy/pasta issues00:40
morganfainbergi can take the blame for the latter.00:40
*** syerrapragada has quit IRC00:40
JayFclarkb: frankly it's a simple patch, idk what could be blowing it up but I'll dig a bit00:40
*** carl_baldwin has quit IRC00:41
clarkbJayF: well you get an entire traceback :)00:41
fungialso i need to learn how to install modified puppet modules into a dev env so i can test more than one layer of broken at a time00:41
clarkbJayF: it is saying lineno -1 is outside of that lists index range.00:41
*** prad has joined #openstack-infra00:42
JayFclarkb: it's from something in the commit message; jroll suggested changing it and it did the trick00:43
JayFbut there's clearly gotta be a bug in pep8 for it to completely bail vs giving reasonable output?00:44
clarkbJayF: wait commit message?00:44
*** ddieterly has joined #openstack-infra00:44
mroddenlolwut00:44
clarkbbut it isn't even running on the commit message00:44
JayFclarkb: yes, 100%00:44
JayFclarkb: Apparently not?00:44
JayFlol00:44
clarkbits running on ironic_python_agent and imagebuild00:44
JayFclarkb: the only thing I did was git commit --amend and change the commit message00:44
pleia2jeblair: so we did intentionally add these jobs https://review.openstack.org/#/c/144520/1/zuul/layout.yaml just make them nv?00:45
clarkbJayF: can we see the delta?00:45
openstackgerritjustinsb proposed openstack/requirements: Bump cassandra-driver to 2.1.4  https://review.openstack.org/15111800:45
JayFmy new commit message was literally "some message\n\nwhatever yo"00:45
JayFthe old commit message is https://review.openstack.org/#/c/153417/1 what's right there00:45
pleia2ah yes, I see tempest is using them too00:45
jrollclarkb: I know in the past, hacking has enforced rules about the commit message00:46
clarkbjroll: ya hacking but this error is in pepe800:46
jroll/shrug00:46
mroddenhacking extends pep8 IIRC00:46
clarkbunless I take that back00:46
fungipleia2: jeblair: looks like 144520 was my fault--i probably missed something when duplicating the icehouse jobs00:47
clarkbthe top of the traceback is loading the entrypoints I suppose its possible hacking ran?00:47
clarkbJayF: jroll you can test by removing hacking and just running flake8 on it00:47
JayFclarkb: self.lines is [] for the part it bailed on fwiw00:47
JayFclarkb: wdym removing hacking? go in the venv and pip uninstall hacking ?00:47
jrollyes00:47
*** gondoi is now known as zz_gondoi00:47
JayFclarkb: bingo00:48
JayFthat fixes it as well00:48
clarkbso a hacking bug00:48
clarkbyou should probably file a bug against hacking with a link to the specific patchset that break00:48
clarkb*breaks00:48
fungijeblair: pleia2: hrm, 144520 just updated to add the juno jobs, but looks like the icehouse jobs were already voting at that point00:49
JayFclarkb: /me suspects the "-"s in the commit message are responsible00:49
openstackgerritJim Rollenhagen proposed openstack-infra/project-config: Make agent_ssh job not run on stable/juno Ironic patches  https://review.openstack.org/15336300:49
jrollclarkb: fixed your comments there ^00:49
fungipleia2: might need to look back further00:49
clarkbjroll: +2 thanks00:49
jrollty!00:49
openstackgerritMerged openstack-infra/project-config: Make manila tempest jobs votable  https://review.openstack.org/15311300:49
*** zz_gondoi is now known as gondoi00:50
openstackgerritMerged openstack-infra/project-config: Changed description of EC2 API project  https://review.openstack.org/15228700:50
*** esker has joined #openstack-infra00:50
jrollfungi: when you have a moment, https://review.openstack.org/153363 unblocks a juno backport for ironic00:51
*** marun has quit IRC00:51
openstackgerritMerged openstack-infra/project-config: Automatically trigger building of docs for stackforge/tuskar-ui-extras  https://review.openstack.org/15244800:51
*** andreaf has quit IRC00:52
*** andreaf has joined #openstack-infra00:53
openstackgerritMerged openstack-infra/project-config: Make VPN repo functional tests non-voting in check queue  https://review.openstack.org/15260200:53
jrollthanks fungi :)00:53
fungiin good news, eavesdrop is now showing failed on puppetboard rather than upreported (hopefully solved once someone approves https://review.openstack.org/153431 )00:54
openstackgerritMerged openstack-infra/devstack-gate: Bump ironic build/callback timeouts  https://review.openstack.org/15320000:54
pleia2I think maybe line 536 in layout.yaml wasn't updated for juno00:57
pleia2  # tempest and d-g branchless tests need to only run on master00:57
pleia2  # (needs to be early to not impact non-voting stanzas later)00:57
pleia2  - name: ^(?!periodic-).*-icehouse$00:57
pleia2    branch: ^master.*$00:57
pleia2does that make sense?00:57
JayFclarkb: thanks; filed this here: https://bugs.launchpad.net/hacking/+bug/1418776 and getting the IPA contributor (a new co-worker of ours, clif_h, so if you see him be nice :D) to change the commit message for now.00:57
*** salv-orl_ has joined #openstack-infra00:58
clarkbpleia2: oh that could be. Though we may need to double check that we haven't set tempest's expectations in a different direction by fubaring that00:58
* pleia2 nods00:58
*** markvoelker has joined #openstack-infra01:01
*** ashleighfarnham has quit IRC01:01
*** salv-orlando has quit IRC01:01
*** salv-orl_ has quit IRC01:02
*** rfolco has quit IRC01:03
*** zul has quit IRC01:03
pleia2no, not it, we're not using periodic in the d-g jobs01:03
pleia2a few lines down though has the ones we're looking for01:03
clarkbpleia2: I think that is excluding the periodic jobs from the rule01:05
clarkb?! is the option for negative lookahead?01:05
*** dims__ has joined #openstack-infra01:05
pleia2oh, right01:05
*** markvoelker has quit IRC01:06
openstackgerritClark Boylan proposed openstack-infra/project-config: add stable compat jobs to python-barbicanclient  https://review.openstack.org/15064501:06
clarkbI just fixed sdague's change ^01:06
* mordred thinks back to the time on the first Perl Cruise when Mark-Jason Dominus did a session on the internals of Perl's regex engine ...01:06
clarkbbecause I figure we should be testing that sooner than later. Any other core want to approve that real quickly?01:06
*** mase_home has quit IRC01:07
*** gondoi is now known as zz_gondoi01:07
fungimordred: nothing makes pcre more tolerable than a tall fruity rum drink01:07
clarkbhrm I don't have any rum or fruity liquids01:08
clarkbprobably a good thing after the other week01:08
*** zz_gondoi is now known as gondoi01:08
greghaynesclarkb: then it is time to panic!01:08
*** rmcall has quit IRC01:09
mordredfungi: it was an alaska cruise01:09
mordredfungi: but there was scotch at one point01:09
*** sputnik13 has quit IRC01:09
fungimordred: i'd leave off the little paper umbrella in that case01:10
*** mtanino has quit IRC01:10
clarkbmy liquor stores are looking pretty sad. Maybe this weekend I will go stock up01:10
clarkbmordred: btw sun liquor hedgetrimmer is excellent01:10
*** dimtruck is now known as zz_dimtruck01:11
mordredclarkb: sun liquor is excellent01:11
*** andreykurilin_ has quit IRC01:11
greghaynesclarkb: have you acquired cachaca yet?01:11
clarkbgreghaynes: no01:12
* greghaynes shakes head01:12
clarkbgreghaynes: its really sad. I have no rum, no cachaca, almost no whiskey, no gin, and the bottle of brandy is disappearing quicker than it should01:12
*** nelsnelson has quit IRC01:12
mordredgrue_pm: ^^ you know - I have many pictures from that conference I could terrorize you with ...01:12
*** abhirc has quit IRC01:13
*** mase_home has joined #openstack-infra01:13
*** weshay has quit IRC01:13
*** salv-orl_ has joined #openstack-infra01:14
*** shashankhegde has joined #openstack-infra01:14
grue_pmmordred: ooh really - I have half a dozen wide photos taken with a happy snap camera01:15
*** thedodd has quit IRC01:15
mordredclarkb: dude. I have two bottles of cachaca, SEVERAL bottles of whiskey including a bottle of connemara peated I picked up last night01:15
grue_pmmordred: I distinctly remember being up all night and nearly faceplanting in an awesome talk on regex and having to leave and the instructor saying I'd gone to play bingo01:15
mordred:)01:15
anteayagrue_pm: sounds like you make an impression01:16
grue_pmtrying to mix alcohol/partying and conference didnt work well for me.01:16
*** doug-fish has left #openstack-infra01:16
clarkbmordred: I know right? but keep in mind I did have a keg at the superbowl thing full of home brew01:16
openstackgerritElizabeth K. Joseph proposed openstack-infra/project-config: Update stable branch selector for Juno  https://review.openstack.org/15343801:17
pleia2jeblair: make sense? ^^01:17
anteayapleia2: I'm missing context01:17
anteayashould I ask for some or will this be approved in the time it takes to explain it to me?01:18
pleia2anteaya: a d-g patch keeps failing to pass verification due to failed tests, jeblair noted that stable branches should not be voting on d-g01:19
*** zhiwei has joined #openstack-infra01:19
anteayaah01:19
anteayado you have the url for said patch with failing tests?01:19
pleia2https://review.openstack.org/#/c/14416601:19
pleia2we've been playing recheck all day long01:19
grue_pmmordred: where are these photos? Actuallly I think I have others too of us climbing to the top of a snow peek and seeing the danger bears no people past this point *after* we came down.01:20
grue_pmmordred: and the ones of randal's I've snarfed01:22
anteayahow do I get a git blame of one line? I tried git blame zuul/layout.yaml:538 and failed01:22
grue_pmmordred: maybe weh could have an openstack conference on a cruise liner?01:22
clarkbdoes anyone remember what mrmartin needed the puppet-solr module for? is it askbot?01:23
pleia2clarkb: yes01:23
anteayafound it git blame -L538,+1 zuul/layout.yaml01:24
*** carl_baldwin has joined #openstack-infra01:24
*** weshay has joined #openstack-infra01:25
anteayapleia2: what am I not understanding, patchset 1 voted on icehouse: https://review.openstack.org/#/c/144166 yet the code should already make the job not vote on icehouse if I am understanding correctly01:25
anteayathat line last changed last september01:26
openstackgerritMerged openstack-infra/project-config: Jenkins/Puppet: increase timeout to 60 minutes  https://review.openstack.org/15342101:26
*** shashankhegde has quit IRC01:26
openstackgerritMerged openstack-infra/puppet-meetbot: Correct source for plugins directory  https://review.openstack.org/15343101:29
openstackgerritMerged openstack-infra/project-config: Allow Tempest to install and configure for the Neutron API job  https://review.openstack.org/14864101:29
grue_pmlook mordred - pretty shiny things ;-p01:29
openstackgerritMerged openstack-infra/project-config: Install test-requirements for swiftclient functional tests  https://review.openstack.org/14571601:29
*** Sukhdev has quit IRC01:31
fungiokay, that fixed eavesdrop's puppetry, but meetbot needs a manual restart for it to take effect now01:31
*** hughhalf is now known as hugh_afk01:32
fungistill no meetings going on, so doing that now01:32
*** openstack has joined #openstack-infra01:34
*** mase_x200 has joined #openstack-infra01:36
*** melwitt has quit IRC01:37
*** tiswanso has joined #openstack-infra01:37
*** tiswanso has quit IRC01:37
fungimorganfainberg: in theory if i refer to bug 1260080 we'll get some noise back from the bot now?01:38
openstackbug 1260080 in OpenStack Security Advisory "[OSSA 2014-006] Trustee token revocations with memcache backend (CVE-2014-2237)" [High,Fix released] https://launchpad.net/bugs/1260080 - Assigned to Tristan Cacqueray (tristan-cacqueray)01:38
*** tiswanso has joined #openstack-infra01:38
fungiindeed i do!01:38
anteayawoooo01:38
morganfainbergfungi, yep01:38
anteayamorganfainberg: nice work01:38
morganfainbergyay01:38
fungithanks again morganfainberg!!!01:38
clarkbhttps://jenkins01.openstack.org/job/gate-tempest-dsvm-neutron-pg/2382/console I think that job is running on the sloewst node ever01:38
morganfainbergmy real reason: more excuses to get all openstack channels logging01:38
morganfainberg>.>01:38
morganfainberg"you get the bug bot for free **just log your channel**"01:39
morganfainbergfungi, happy to help, and glad it's a base part of our services tbh01:39
fungimorganfainberg: that's sneaky-smart01:39
* fungi transitions to standby mode01:39
clarkbmorganfainberg: fwiw I am happy to help people set up long running irc clients if they need logs in not publicly logged channels01:39
morganfainbergclarkb, i am thinking from an openstack-XXX channel it's nice, but reality it;s more because we should have the bug bot ;)01:40
morganfainbergi just got to look like i was being sneaky-smart for a moment ;)01:40
*** r1chardj0n3s is now known as r1chardj0n3s_afk01:40
anteayaI took a picture01:40
clarkbin other news that makes me happy OR and WA currently have legislation in the works to stop doing DST01:41
anteayadaylight savings time?01:41
clarkbanteaya: ya01:41
anteayayou don't like it?01:41
fungiif only other states will follow suit01:41
morganfainbergclarkb, crap.. i might need tto move to OR or WA based on that instead of NY01:41
anteayapersonally I would like us all to use utc01:41
fungiand then introduce legislation to ditch timezones01:42
morganfainbergi mean. that is a REALLY big selling point about where to move01:42
anteayaone timezone01:42
morganfainberganteaya, I agree.01:42
mordredclarkb: it's likely not going to go to a vote01:42
morganfainberganteaya, i hate doing TZ math01:42
clarkbanteaya: I have an extreme dislike for changing clocks twice a year01:42
*** tiswanso has quit IRC01:42
clarkbmordred: well the way it works in OR is it would go on our ballots in the fall01:42
anteayaclarkb: yeah that makes sense01:42
morganfainbergclarkb, i hate changing clocks when they run out of battery let alone DST01:42
anteayamorganfainberg: me too, to make my meetings on time I just changed my phone to utc01:42
morganfainbergclarkb, it's why i use NTP for everything i actually care about01:42
mordredI want us to stop DST - but I'd like to stick on DST and not on Standard01:43
anteayathen I didn't have to translate the time to $local_timezone01:43
*** tiswanso has joined #openstack-infra01:43
pleia2anteaya: hm, so something happened in early september and icehouse jobs started voting again, looking01:43
clarkbmordred: ya I think I prefer being on that side of the switch too01:43
clarkbmordred: but mostly would be happy to not switch01:43
fungiit's why i keep ntp on everything i care about and set all my clocks to utc ;)01:43
morganfainbergmordred, i don't care which side of the switch as long as it stops01:43
clarkbmorganfainberg: +101:43
mordredclarkb: this is the real reason it's not going to go anywhere right now01:43
anteayapleia2: oh okay, I'm glad I'm not just making your life difficult, I'm trying to be useful01:43
mordred"I like the long summer nights in the PNW" is the push back01:43
anteayapleia2: different success rate every day there01:43
clarkbmordred: apparently the biggest opponents are golf courses01:44
clarkbmordred: which is lolzy01:44
*** abhirc has joined #openstack-infra01:44
*** markvoelker has joined #openstack-infra01:44
morganfainbergfungi, it's more of a headache to run UTC on an iphone than it should be... but with the 1+1 i prob can make that happen easily01:44
anteayaclarkb: golf courses?01:44
clarkbanteaya: ya, they like the late evening sunlight apparently01:44
clarkbanteaya: more people golfing01:44
anteayasaskatchewan doesn't switch because it upsets the cows01:44
fungimorganfainberg: yeah, i have an ancient crap non-smart phone and it's a pain but i do manage to keep it set to utc01:44
morganfainbergmordred, i am finding how much work it's going to be to convert to android. i might be done in a month - so many things01:44
anteayaclarkb: why not just golf when you want, rather than looking at a time piece01:45
clarkbmorganfainberg: wait you do more than text, call, irc, and email on your phone?01:45
morganfainbergclarkb, i often do code review from coffeeshops01:45
clarkbanteaya: well I think most people work jobs where they are required to be in a chair between hours X and Y01:45
morganfainbergclarkb, and tethering01:45
morganfainbergclarkb, and MFA tokens01:45
clarkbmorganfainberg: tethering is easy on android01:45
clarkbso is MFA01:45
anteayaclarkb: ah, I am rapidly forgetting that world01:45
morganfainbergclarkb, it's the convert from Iphone to Andorid MFA apps01:45
anteayawould be better do make some legistation around that01:46
anteayathe golf when you want bill01:46
morganfainbergit usually requires resetting up the MFA since the original strings/QRs are now lost [due to a bad reformat]01:46
morganfainbergit's just time consuming01:46
clarkbya I guess it depends on how many places you are using them01:46
morganfainbergclarkb, i can be in my chair between 1500 and 0100 UTC if people cared about that01:47
morganfainberg:P01:47
morganfainberganteaya, ^^01:47
anteayamorganfainberg: I don't care about that01:48
anteayagolf when you want, morganfainberg01:48
morganfainbergi meant to the comment of people needing to be in a chair between X and Y01:48
anteayayeah, getting to be silly01:48
fungimorganfainberg: 1400 to 2200 utc for me (or much later, obviously)01:48
morganfainbergdoesn't really change anything just that you don't have to guess when person X wakes up01:48
anteayayou can guess or you can ask01:48
anteayayou up?01:48
morganfainbergsince they'd say "i get up at 1800" or something01:48
anteayafunny how often I get a yes to that question from mordred01:49
morganfainbergvs. "i get up at 6am".. now what timezone are they in?01:49
morganfainberghahaha01:49
anteayawaste of my time to guess for him01:49
morganfainbergin december i didn't know what TZ i was in let alone anyone else01:49
anteayaha ha ha01:49
*** zz_dimtruck is now known as dimtruck01:50
anteayatime for me to pick up my cat and move to the couch01:50
anteayashe aquired a sneeze the minute I took her to my sister's a month ago01:50
anteayaand still has it01:50
clarkbI spent 18 years without DST and it never hurt me01:51
anteayahoping it is an emotional response and will diminsh soon01:51
anteayaclarkb: you look fine to me01:51
morganfainbergarizona?01:51
clarkbbut the sun did set at just about 1830 local time every day of the year01:51
clarkbmorganfainberg: Pohnpei, Micronesia01:51
pleia2anteaya: btw, I think your git blame from earlier is a red herring, it changed in september because that's when we split out the modules (so technically the entire file changed)01:51
*** r-daneel has quit IRC01:51
anteayathe equator is reliable like that01:51
morganfainbergah01:51
anteayapleia2: ah01:51
anteayapleia2: good investigating01:51
*** bdpayne has quit IRC01:51
pleia2I've had to go back to the layout file in config to dig up more01:51
anteayapleia2: sorry for the trouble01:52
pleia2no trouble :) I was just looking for the line change you mentioned when I realized it might not actually exist01:52
clarkbfungi: its incredibly sad that joystiq is no more01:52
anteayapleia2: never trust me, always check01:52
*** gondoi is now known as zz_gondoi01:52
clarkbfungi: where will I find out about videogames I don't need01:52
* anteaya moves to fetch the cat01:52
*** salv-orl_ has quit IRC01:53
fungiclarkb: i still posit that we should hire the former joystiq'ers to be part of the openstack blog-marketing machine01:53
*** rwsu is now known as rwsu-afk01:53
fungialso one of our member companies should give werner koch a solid salary to continue supporting gnupg01:54
*** shashankhegde has joined #openstack-infra01:55
pleia2fungi: was nice to see Linux Foundation use their Core Infrastructure Initiative for that too (60k worth!)01:56
clarkbpleia2: oh wow01:56
*** zz_gondoi is now known as gondoi01:58
*** gondoi is now known as zz_gondoi01:59
*** carl_baldwin has quit IRC01:59
fungipleia2: agreed, but that's still, well, barely a living wage these days, and certainly meager compensation on its own01:59
pleia2yeah01:59
*** SumitNaiksatam has quit IRC02:00
*** Sukhdev has joined #openstack-infra02:00
fungianyway, i'm _really_ going into standby now. later all02:00
*** zz_gondoi is now known as gondoi02:00
openstackgerritMerged openstack-infra/project-config: Enable jobs to publish cross-project specs  https://review.openstack.org/15269802:01
clarkbgood night02:01
pleia2night fungi02:01
clarkbI am really hoping to check experimental the d-g multihost change once the nova shelve fix merges02:01
clarkbbut maybe dinner first02:01
openstackgerritMerged openstack-infra/project-config: Enable voting for osc-functional  https://review.openstack.org/15274902:01
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Allow independant pipelines to have a window  https://review.openstack.org/15238902:02
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for disabling bad pipelines  https://review.openstack.org/15239002:02
openstackgerritMerged openstack-infra/project-config: For project ec2-driver setting to noop-job in zuul layout  https://review.openstack.org/15281102:03
*** amitgandhinz has joined #openstack-infra02:04
*** amitgandhinz has quit IRC02:05
*** amitgandhinz has joined #openstack-infra02:05
openstackgerritMerged openstack-infra/project-config: Make  gate-git-review-docs voting  https://review.openstack.org/15265102:05
*** ccrouch has quit IRC02:06
*** mase_x200 has quit IRC02:09
*** ChuckC_ has joined #openstack-infra02:09
*** gondoi is now known as zz_gondoi02:10
*** zz_gondoi is now known as gondoi02:11
*** fandi has quit IRC02:11
*** sarob has joined #openstack-infra02:12
*** ChuckC has quit IRC02:13
*** sarob has quit IRC02:16
*** tkelsey has joined #openstack-infra02:17
*** dimtruck is now known as zz_dimtruck02:19
openstackgerritDoug Wiegley proposed openstack-infra/project-config: Add pylint jobs to service modules  https://review.openstack.org/15337302:19
pleia2alright, I'm heading off to dinner, won't be around much tomorrow or monday (catching up with home things and appointments and sleep)02:19
*** shashankhegde has quit IRC02:20
*** hdd has quit IRC02:21
*** tkelsey has quit IRC02:22
*** zz_dimtruck is now known as dimtruck02:22
*** shashankhegde has joined #openstack-infra02:23
*** baoli has joined #openstack-infra02:24
*** baoli has quit IRC02:30
*** tqtran has quit IRC02:30
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Refactor sources out of triggers  https://review.openstack.org/11899302:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add gerrit reviews into patchset approvals  https://review.openstack.org/9739002:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for negative requirements  https://review.openstack.org/10272602:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Configure triggers dynamically  https://review.openstack.org/11953402:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for 'connection' concept  https://review.openstack.org/12152802:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Call driver methods more dynamically  https://review.openstack.org/11953302:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for triggers  https://review.openstack.org/11953202:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for sources  https://review.openstack.org/11953102:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for reporters  https://review.openstack.org/11953002:31
*** wenlock has joined #openstack-infra02:32
*** esker has quit IRC02:36
*** esker has joined #openstack-infra02:37
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for 'connection' concept  https://review.openstack.org/12152802:38
*** vilobhmm has quit IRC02:40
*** esker has quit IRC02:42
*** yamahata has quit IRC02:46
openstackgerritWalter A. Boring IV (hemna) proposed openstack-infra/project-config: Add os-brick to OpenStack  https://review.openstack.org/14977802:47
*** mase_x200 has joined #openstack-infra02:48
*** harlowja is now known as harlowja_away02:48
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Configure triggers dynamically  https://review.openstack.org/11953402:52
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for 'connection' concept  https://review.openstack.org/12152802:52
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Call driver methods more dynamically  https://review.openstack.org/11953302:52
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for triggers  https://review.openstack.org/11953202:52
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for sources  https://review.openstack.org/11953102:52
*** salv-orlando has joined #openstack-infra02:53
*** dannywilson has joined #openstack-infra02:55
*** Ryan_Lane has quit IRC02:56
*** ChuckC has joined #openstack-infra02:59
*** ChuckC_ has quit IRC02:59
*** dannywilson has quit IRC03:00
*** baoli has joined #openstack-infra03:02
*** baoli has quit IRC03:08
*** r1chardj0n3s_afk is now known as r1chardj0n3s03:09
*** patrickeast has quit IRC03:10
*** shashankhegde has quit IRC03:12
*** amitgandhinz has quit IRC03:14
*** dims__ has quit IRC03:15
*** woodster_ has quit IRC03:16
*** achanda has joined #openstack-infra03:17
*** achanda has quit IRC03:23
*** hugh_afk is now known as hughhalf03:24
*** salv-orlando has quit IRC03:25
*** salv-orlando has joined #openstack-infra03:27
*** melwitt has joined #openstack-infra03:28
*** armax has quit IRC03:28
*** avozza is now known as zz_avozza03:29
*** r-daneel has joined #openstack-infra03:31
*** salv-orlando has quit IRC03:34
*** wenlock has quit IRC03:34
*** sdake_ has quit IRC03:35
*** r-daneel has quit IRC03:35
*** Longgeek has joined #openstack-infra03:36
*** zhiwei has quit IRC03:37
*** dimtruck is now known as zz_dimtruck03:39
*** mattfarina has joined #openstack-infra03:40
*** carl_baldwin has joined #openstack-infra03:45
*** koolhead17 has joined #openstack-infra03:46
*** esker has joined #openstack-infra03:49
*** r-daneel has joined #openstack-infra03:50
*** mase_x200 has quit IRC03:57
*** Sukhdev has quit IRC04:00
*** otter768 has joined #openstack-infra04:00
*** shashankhegde has joined #openstack-infra04:02
*** niska has quit IRC04:04
*** Ryan_Lane has joined #openstack-infra04:06
*** Ryan_Lane has quit IRC04:07
*** esker has quit IRC04:07
*** bdpayne has joined #openstack-infra04:10
*** bdpayne has quit IRC04:11
*** gondoi is now known as zz_gondoi04:12
mordredjhesketh: dude. that's so many zuul patches04:14
jheskethmordred: or the more for you to review ;-)04:15
mordredjhesketh: I was just thinking to myself- WOW, I really want to review more patches04:15
*** niska has joined #openstack-infra04:15
jheskethgood thing I preempted that04:16
jheskethglad to help04:16
*** BharatK has joined #openstack-infra04:16
*** carl_baldwin has quit IRC04:19
jesusaurusjhesketh: your comment on https://review.openstack.org/#/c/74943/19/nodepool/tests/test_nodepool.py confuses me, what is the test you are suggesting that i add?04:20
*** niska has quit IRC04:21
*** achanda has joined #openstack-infra04:23
*** prad has quit IRC04:24
*** armax has joined #openstack-infra04:25
*** amitgandhinz has joined #openstack-infra04:25
jheskethjesusaurus: basically testing the reverse of the test that is there04:27
*** zz_avozza is now known as avozza04:27
jheskethyou're testing that the bad-client works on a re-attempt, I'm suggesting we should also test that mutliple bad attempts raises the exception you expect04:28
*** swamireddy has joined #openstack-infra04:28
*** amitgandhinz has quit IRC04:29
jesusaurusah, hm, ok04:29
jheskethdoes that make sense?04:30
jesusaurusyeah, it does, im just not sure how I would do that04:30
*** salv-orlando has joined #openstack-infra04:31
*** yamamoto_ has joined #openstack-infra04:32
*** zz_gondoi is now known as gondoi04:33
*** gondoi is now known as zz_gondoi04:36
*** zz_gondoi is now known as gondoi04:36
*** avozza is now known as zz_avozza04:37
*** armax has quit IRC04:37
*** koolhead17 has quit IRC04:37
*** armax has joined #openstack-infra04:37
*** EmilienM|afk has quit IRC04:45
*** EmilienM has joined #openstack-infra04:46
*** shashankhegde has quit IRC04:47
*** salv-orlando has quit IRC04:48
*** koolhead17 has joined #openstack-infra04:48
*** koolhead17 has quit IRC04:48
*** koolhead17 has joined #openstack-infra04:48
*** miqui_ has joined #openstack-infra04:49
*** salv-orlando has joined #openstack-infra04:50
*** mattfarina has quit IRC04:51
*** yamahata has joined #openstack-infra04:54
*** wenlock has joined #openstack-infra04:55
*** thingee has quit IRC05:01
*** tiswanso has quit IRC05:02
*** gondoi is now known as zz_gondoi05:06
*** primeministerp has quit IRC05:07
*** bhunter71 has quit IRC05:07
*** zz_gondoi is now known as gondoi05:08
*** garyh has quit IRC05:09
*** r1chardj0n3s is now known as r1chardj0n3s_afk05:19
*** armax has quit IRC05:23
*** armax has joined #openstack-infra05:24
*** achanda has quit IRC05:25
*** primeministerp has joined #openstack-infra05:25
*** KanagarajM has joined #openstack-infra05:25
*** achanda has joined #openstack-infra05:27
*** salv-orlando has quit IRC05:29
openstackgerritKhai Do proposed stackforge/python-jenkins: Change the default server connection timeout to 15 seconds  https://review.openstack.org/15346705:29
*** garyh has joined #openstack-infra05:32
*** harlowja_at_home has joined #openstack-infra05:32
*** mrmartin has joined #openstack-infra05:34
KanagarajMcould someone review the patch https://review.openstack.org/#/c/152868/  Heat is updated with new REST API in k-205:36
*** hdd has joined #openstack-infra05:38
*** harlowja_away has quit IRC05:38
*** armax has quit IRC05:44
*** mrmartin has quit IRC05:44
*** otter768 has quit IRC05:49
*** sputnik13 has joined #openstack-infra06:04
*** woodster_ has joined #openstack-infra06:07
*** sputnik13 has joined #openstack-infra06:08
*** niska has joined #openstack-infra06:11
*** shashankhegde has joined #openstack-infra06:14
*** sputnik13 has quit IRC06:15
*** AJaeger has joined #openstack-infra06:18
*** hdd has quit IRC06:18
*** mrmartin has joined #openstack-infra06:19
*** yfried_ has joined #openstack-infra06:22
*** yfried_ is now known as yfried|afk06:22
openstackgerritMerged openstack-infra/project-config: Update chef-rake jobs to use chefdk and run on trusty  https://review.openstack.org/15219606:23
*** niska has quit IRC06:24
*** yfried has quit IRC06:25
openstackgerritMerged openstack-infra/project-config: GlusterFS: passthrough devstack config  https://review.openstack.org/15113606:26
openstackgerritMerged openstack-infra/project-config: Add gnocchi devstack tests job  https://review.openstack.org/15206406:26
openstackgerritMerged openstack-infra/project-config: Add openstack_functions.py to zuul-dev.o.o  https://review.openstack.org/15307006:26
openstackgerritMerged openstack-infra/project-config: Move puppet-tripleo to Stackforge  https://review.openstack.org/15215206:26
*** niska has joined #openstack-infra06:27
*** melwitt has quit IRC06:30
*** wenlock has quit IRC06:32
*** sdake_ has joined #openstack-infra06:33
openstackgerritMerged openstack-infra/project-config: Make agent_ssh job not run on stable/juno Ironic patches  https://review.openstack.org/15336306:34
*** openstackgerrit has quit IRC06:35
*** openstackgerrit has joined #openstack-infra06:35
*** tsufiev_ is now known as tsufiev06:37
*** salv-orlando has joined #openstack-infra06:37
*** SumitNaiksatam has joined #openstack-infra06:39
openstackgerritMerged openstack-infra/project-config: Add cloud validation (cloudv) project to stackforge  https://review.openstack.org/15269506:40
*** denis_makogon has joined #openstack-infra06:43
*** salv-orlando has quit IRC06:48
*** harlowja_at_home has quit IRC06:49
*** achanda has quit IRC06:52
*** camunoz has quit IRC06:54
*** denis_makogon has quit IRC06:56
*** mrmartin has quit IRC06:56
*** afazekas has joined #openstack-infra06:57
*** r-daneel has quit IRC06:57
*** ivar-laz_ has joined #openstack-infra07:00
*** achanda has joined #openstack-infra07:01
*** doude has quit IRC07:01
*** sputnik13 has joined #openstack-infra07:02
openstackgerritKhai Do proposed openstack-infra/jenkins-job-builder: Allow JJB to set connection timeout to jenkins server  https://review.openstack.org/15147207:02
*** ivar-lazzaro has quit IRC07:03
*** gondoi is now known as zz_gondoi07:04
*** ivar-laz_ has quit IRC07:05
*** naggappan has joined #openstack-infra07:06
*** doude has joined #openstack-infra07:06
*** ricolin has quit IRC07:06
*** vigneshvar has joined #openstack-infra07:07
openstackgerritKhai Do proposed stackforge/python-jenkins: Change the default server connection timeout to 15 seconds  https://review.openstack.org/15346707:11
*** chlong has quit IRC07:12
*** miqui_ has quit IRC07:12
*** ildikov has quit IRC07:13
*** chlong has joined #openstack-infra07:13
*** zz_gondoi is now known as gondoi07:14
*** yfried|afk has quit IRC07:16
*** yfried|afk has joined #openstack-infra07:16
*** pblaho has joined #openstack-infra07:17
*** afazekas has quit IRC07:20
*** pblaho has quit IRC07:22
*** koolhead17 has quit IRC07:22
*** scheuran has joined #openstack-infra07:25
*** KanagarajM has quit IRC07:27
*** achanda has quit IRC07:27
*** ivar-lazzaro has joined #openstack-infra07:33
*** _amrith_ is now known as amrith07:35
*** yamahata has quit IRC07:36
*** ildikov has joined #openstack-infra07:40
*** _nadya_ has joined #openstack-infra07:41
*** ivar-lazzaro has quit IRC07:45
*** _nadya_ has quit IRC07:49
*** tkelsey has joined #openstack-infra07:49
*** otter768 has joined #openstack-infra07:50
*** gondoi is now known as zz_gondoi07:50
*** zz_dimtruck is now known as dimtruck07:51
*** zz_gondoi is now known as gondoi07:51
naggappanhi could some one poin me where exactly the yamal file is present in repository for test "http://logs.openstack.org/18/153118/2/check/gate-neutron-python27/"  . I.e I need to look for "gate-neutron-python27" yamal file. I searched into openstack-infra/project-config repo but i did not get any07:51
*** ivar-lazzaro has joined #openstack-infra07:52
*** otter768 has quit IRC07:54
*** tkelsey has quit IRC07:56
*** ivar-lazzaro has quit IRC08:00
*** gondoi is now known as zz_gondoi08:00
*** belmoreira has joined #openstack-infra08:01
*** koolhead17 has joined #openstack-infra08:06
*** esker has joined #openstack-infra08:06
*** markvoelker has quit IRC08:12
*** _nadya_ has joined #openstack-infra08:13
*** markvoelker has joined #openstack-infra08:13
*** amrith is now known as _amrith_08:15
*** dimtruck is now known as zz_dimtruck08:16
*** woodster_ has quit IRC08:16
*** markvoelker has quit IRC08:18
*** andreaf has quit IRC08:20
*** jgallard__ has joined #openstack-infra08:20
*** andreaf has joined #openstack-infra08:20
tchaypojeblair: this is fun - running “gertty -v” results in no logs and it sitting at “Sync: 5"08:22
tchaypojeblair: “gertty -d” works fine08:22
tchaypoNo I tell a lie - it ran fine last time, this time it’s also sitting at “sync: 5"08:23
tchaypoah, there it goes08:23
tchaypookay, -d is working now too. Probably just the server being sad for a moment. Please ignore me.08:24
*** _nadya_ has quit IRC08:29
*** scheuran has quit IRC08:30
*** yamamoto_ has quit IRC08:31
*** luqas has joined #openstack-infra08:32
*** scheuran has joined #openstack-infra08:33
*** unicell has joined #openstack-infra08:39
*** stevemar has quit IRC08:39
*** markus_z has joined #openstack-infra08:42
*** markvoelker has joined #openstack-infra08:44
*** zz_avozza is now known as avozza08:44
*** ghostpl has joined #openstack-infra08:45
*** mpaolino has joined #openstack-infra08:47
*** scheuran has quit IRC08:50
*** alexpilotti has joined #openstack-infra08:50
*** scheuran has joined #openstack-infra08:50
nibalizertchaypo: if it makes you feel any better im trying to sync and im 1K behind and im on hotel wifi08:50
*** markvoelker has quit IRC08:51
nibalizerso thats how that goes08:51
nibalizerer you can figure out how that goes :)08:51
tchaypoIn my experience08:51
tchaypoEither fantastically well08:52
*** ssam2 has joined #openstack-infra08:52
tchaypoOr it seems as though a proxy dies after 10 requests and then nothing happens for the rest of the night08:52
*** chlong has quit IRC08:53
tchaypoI was in a hotel a few weeks ago where the AP on the ceiling outside my room stopped routing traffic.08:53
*** mrmartin has joined #openstack-infra08:54
*** koolhead17 has quit IRC08:54
*** salv-orlando has joined #openstack-infra08:56
*** yamamoto_ has joined #openstack-infra08:58
*** jpich has joined #openstack-infra08:59
*** tkelsey has joined #openstack-infra09:01
*** derekh has joined #openstack-infra09:02
*** Guest75042 has joined #openstack-infra09:03
*** yfried has joined #openstack-infra09:04
*** yfried|afk has quit IRC09:05
*** shashankhegde has quit IRC09:08
*** yamamoto_ has quit IRC09:09
*** yamamoto_ has joined #openstack-infra09:13
*** jgrimm is now known as zz_jgrimm09:14
openstackgerritCedric Brandily proposed openstack-infra/git-review: Support authentication in run_http_exc  https://review.openstack.org/12441009:15
*** _shaps__ has quit IRC09:15
*** jp_at_hp has joined #openstack-infra09:17
*** andreykurilin_ has joined #openstack-infra09:23
*** salv-orlando has quit IRC09:26
*** andreaf has quit IRC09:28
*** salv-orlando has joined #openstack-infra09:28
*** andreaf has joined #openstack-infra09:29
*** yamamoto_ has quit IRC09:30
*** alexpilotti has quit IRC09:32
*** ihrachyshka has joined #openstack-infra09:35
*** mriedem has joined #openstack-infra09:35
*** mriedem_zz has quit IRC09:36
*** abhijeetm has joined #openstack-infra09:38
*** sputnik13 has quit IRC09:39
*** e0ne has joined #openstack-infra09:41
*** dizquierdo has joined #openstack-infra09:42
*** MaxV has joined #openstack-infra09:45
*** BobBall_AWOL is now known as BobBall09:46
*** jcoufal has joined #openstack-infra09:46
*** markvoelker has joined #openstack-infra09:46
abhijeetmHi,09:48
abhijeetmcheck-grenade-dsvm job is failing on stable/juno09:48
abhijeetmfor patch : https://review.openstack.org/#/c/151998/09:48
abhijeetmgraenade log : http://logs.openstack.org/98/151998/1/check/check-grenade-dsvm/94c1693/logs/grenade.sh.txt.gz09:49
*** e0ne has quit IRC09:49
abhijeetmerror is : pkg_resources.ContextualVersionConflict: (oslo.config 1.4.0 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('oslo.config>=1.6.0'), set(['tempest-lib']))09:50
*** tmcpeak has quit IRC09:50
*** e0ne has joined #openstack-infra09:50
*** tmcpeak has joined #openstack-infra09:50
*** markvoelker has quit IRC09:51
*** otter768 has joined #openstack-infra09:51
abhijeetmIs there any bug already filed for it?09:52
*** jyuso has quit IRC09:52
*** SlickNik has quit IRC09:52
*** jyuso has joined #openstack-infra09:52
*** jpich has quit IRC09:52
*** vigneshvar has quit IRC09:52
*** AJaeger has quit IRC09:53
*** jerryz has quit IRC09:53
*** BadCub has quit IRC09:53
*** gibi_ has quit IRC09:54
*** gibi has joined #openstack-infra09:54
*** BadCub has joined #openstack-infra09:54
*** jcoufal has quit IRC09:55
*** otter768 has quit IRC09:55
*** mase_home has quit IRC09:57
*** SlickNik has joined #openstack-infra09:58
*** hichihara has quit IRC09:58
*** salv-orlando has quit IRC10:00
*** mpaolino has quit IRC10:01
*** belmoreira has quit IRC10:01
*** niska has quit IRC10:02
*** AJaeger has joined #openstack-infra10:05
*** esker has quit IRC10:05
*** vigneshvar has joined #openstack-infra10:05
*** skolekonov has joined #openstack-infra10:05
*** jpich has joined #openstack-infra10:06
*** chlong has joined #openstack-infra10:09
*** jedimike has joined #openstack-infra10:10
*** avozza is now known as zz_avozza10:13
*** dizquierdo has quit IRC10:13
AJaegernaggappan: do you still need help?10:13
*** mpaolino has joined #openstack-infra10:14
*** _nadya_ has joined #openstack-infra10:14
naggappanHi AJaeger . Thanks got it10:18
AJaegernaggappan: great!10:18
*** mika has quit IRC10:18
openstackgerritMerged openstack-infra/release-tools: Support neutron advanced services  https://review.openstack.org/15084510:19
naggappanHi AJaeger . I am actually looking for  gate-neutron-python27 job and you have mentioned about check-grenade-dsvm10:20
*** jerryz has joined #openstack-infra10:20
naggappan@AJaeger . I have done a CI setup for internal purpose. And for each checkin i need to execute neutron unit test job. So thats why i need the exact job yamal file10:21
openstackgerritMerged openstack-infra/release-tools: Be more informative when putting -2 for stable freeze  https://review.openstack.org/15318410:22
*** e0ne is now known as e0ne_10:23
*** yamamoto has joined #openstack-infra10:23
*** yamamoto has quit IRC10:24
naggappanlike it should execute unit test using tox and generate the repote in html formate10:24
*** e0ne_ is now known as e0ne10:25
*** ihrachyshka has quit IRC10:25
*** zz_johnthetubagu is now known as johnthetubaguy10:25
*** dizquierdo has joined #openstack-infra10:25
openstackgerritMerged openstack-infra/release-tools: Ignore release-tag dirs when running flake8  https://review.openstack.org/15060010:25
openstackgerritMerged openstack-infra/release-tools: allow sanity check to report multiple warnings  https://review.openstack.org/15060110:26
*** sasik has joined #openstack-infra10:29
*** sasik has left #openstack-infra10:29
*** sasik has joined #openstack-infra10:30
*** sasik has left #openstack-infra10:30
*** zz_avozza is now known as avozza10:31
*** andreykurilin_ has quit IRC10:31
*** ihrachyshka has joined #openstack-infra10:31
*** mika has joined #openstack-infra10:32
*** jcoufal has joined #openstack-infra10:33
*** skolekonov has quit IRC10:33
*** avozza is now known as zz_avozza10:34
tchaypoand now I have lovely wifi on a train. Italy is marvellous.10:35
AJaegernaggappan: This is all defined in project-config repo.10:37
AJaegerThe job-template python-jobs is defined for neutron10:37
openstackgerritMerged openstack-infra/storyboard: OAuth Storage Abstraction Removed  https://review.openstack.org/15310610:38
AJaegerIf you grep in jenkins/jobs/*yaml, you'll find the template defined in file python-jobs.yaml as 'gate-{name}-python27'10:38
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't throw exception on empty response  https://review.openstack.org/15339810:38
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't assume we get a valid response  https://review.openstack.org/15339910:38
AJaegernaggappan: hope that helps10:38
openstackgerritMerged openstack-infra/storyboard: OAuth Functional Tests  https://review.openstack.org/15306810:40
naggappanAJaeger. Yes i will look into these things now. Currenlty I use tox and run the unit test for neutron. Once its done i get the result in ".testrepository/$ID" . I have to convert this to html. You have any idea on this. I am looking for those job yamal file just to get this information only10:40
*** _nadya_ has quit IRC10:43
*** markvoelker has joined #openstack-infra10:47
*** ldnunes has joined #openstack-infra10:49
*** Guest75042 has quit IRC10:51
*** skolekonov has joined #openstack-infra10:52
*** markvoelker has quit IRC10:52
*** jgallard__ has quit IRC10:53
*** ociuhandu has joined #openstack-infra10:54
*** jgallard__ has joined #openstack-infra10:54
AJaegernaggappan: jenkins/jobs/* contains all the definitions for it and jenkins/scripts are the scripts we use.10:54
*** dims__ has joined #openstack-infra10:54
*** pblaho has joined #openstack-infra10:55
naggappanok AJaeger thanks10:55
*** amotoki has joined #openstack-infra10:56
AJaegernaggappan: read also http://ci.openstack.org/third_party.html - it explains everything on how to setup a 3rd party CI system10:56
*** jgallard__ has quit IRC10:57
*** jgallard__ has joined #openstack-infra10:57
AJaegernaggappan, there's also a weekly meeting of all people running such CI systems10:57
*** koolhead17 has joined #openstack-infra10:57
naggappanOk10:59
naggappani will attend the meeting AJAaeger.10:59
*** ihrachyshka has quit IRC11:00
AJaegernaggappan: AFAIR it's Mondays, but check the OpenStack meeting calendar for details.11:00
*** belmoreira has joined #openstack-infra11:01
*** rfolco has joined #openstack-infra11:02
*** cdent has joined #openstack-infra11:04
*** _nadya_ has joined #openstack-infra11:11
*** jgallard__ has quit IRC11:12
*** jgallard__ has joined #openstack-infra11:12
*** koolhead17 has quit IRC11:18
*** reed has joined #openstack-infra11:20
*** isviridov_away is now known as isviridov11:20
*** isviridov is now known as isviridov_launch11:21
*** rlandy has joined #openstack-infra11:21
*** yamamoto has joined #openstack-infra11:24
*** ihrachyshka has joined #openstack-infra11:25
*** yamamoto has quit IRC11:29
*** pelix has joined #openstack-infra11:30
*** denis_makogon has joined #openstack-infra11:32
*** cdent has quit IRC11:33
*** cdent has joined #openstack-infra11:35
*** koolhead17 has joined #openstack-infra11:36
*** ihrachyshka has quit IRC11:40
*** pc_m has joined #openstack-infra11:40
*** cdent_ has joined #openstack-infra11:42
*** zigo has quit IRC11:42
*** pblaho has quit IRC11:42
*** pblaho_ has joined #openstack-infra11:42
*** cdent has quit IRC11:42
*** cdent_ is now known as cdent11:42
*** markvoelker has joined #openstack-infra11:48
*** aysyd has joined #openstack-infra11:48
*** otter768 has joined #openstack-infra11:52
*** e0ne is now known as e0ne_11:54
*** markvoelker has quit IRC11:54
*** mrmartin has quit IRC11:56
*** mika has quit IRC11:56
*** mikap has joined #openstack-infra11:56
*** mikap is now known as mika11:56
*** otter768 has quit IRC11:56
*** swamireddy has quit IRC11:58
*** ameade has quit IRC11:58
*** cdent has quit IRC11:58
*** bradjones has quit IRC11:59
*** e0ne_ is now known as e0ne11:59
*** simonmcc has quit IRC11:59
*** zul has joined #openstack-infra11:59
*** ihrachyshka has joined #openstack-infra12:00
*** Guest60279 has quit IRC12:00
*** cdent has joined #openstack-infra12:00
*** zul has quit IRC12:01
*** simonmcc has joined #openstack-infra12:01
*** ameade has joined #openstack-infra12:01
*** zul has joined #openstack-infra12:01
*** Vivek has joined #openstack-infra12:01
*** Vivek is now known as Guest7197812:02
*** bradjones has joined #openstack-infra12:02
*** julim has quit IRC12:04
*** ociuhandu has quit IRC12:05
*** d34dh0r53 has quit IRC12:06
*** julim has joined #openstack-infra12:06
*** russellb has quit IRC12:06
*** jroll has quit IRC12:06
*** dizquierdo has quit IRC12:07
*** d34dh0r53 has joined #openstack-infra12:08
*** russellb has joined #openstack-infra12:09
*** ociuhandu has joined #openstack-infra12:09
*** sileht_ has joined #openstack-infra12:10
openstackgerritMerged openstack/requirements: Add newly released oslo debtcollector library  https://review.openstack.org/15341012:14
*** ihrachyshka has quit IRC12:15
*** pblaho_ has quit IRC12:15
*** ihrachyshka has joined #openstack-infra12:15
*** maurosr has quit IRC12:17
*** jgrimmn has joined #openstack-infra12:19
*** zul has quit IRC12:19
*** koolhead17 has quit IRC12:19
*** skolekonov has quit IRC12:19
*** vigneshvar has quit IRC12:19
*** BharatK has quit IRC12:19
*** esmute has quit IRC12:19
*** che-arne has quit IRC12:19
*** imcsk8 has quit IRC12:19
*** erikmwilson has quit IRC12:19
*** prad-afk has quit IRC12:19
*** therve has quit IRC12:19
*** sweston has quit IRC12:19
*** dougwig has quit IRC12:19
*** willroberts_ has quit IRC12:19
*** medberry has quit IRC12:19
*** dmsimard_away has quit IRC12:19
*** funzo has quit IRC12:19
*** jkt has quit IRC12:19
*** morgabra has quit IRC12:19
*** comstud has quit IRC12:19
*** Hunner has quit IRC12:19
*** sc68cal has quit IRC12:19
*** sileht has quit IRC12:19
*** jvrbanac has quit IRC12:19
*** msuriar has quit IRC12:19
*** wendar has quit IRC12:19
*** zz_dimtruck has quit IRC12:19
*** dtroyer has quit IRC12:19
*** cyeoh has quit IRC12:19
*** mgagne has quit IRC12:19
*** rainya has quit IRC12:19
*** rossella_s has quit IRC12:19
*** Zenet has quit IRC12:19
*** zz_jgrimm has quit IRC12:19
*** dtantsur|afk has quit IRC12:19
*** anthonyper has quit IRC12:19
*** sileht_ is now known as sileht12:19
*** maurosr has joined #openstack-infra12:19
*** pblaho__ has joined #openstack-infra12:19
*** dizquierdo has joined #openstack-infra12:21
*** jroll has joined #openstack-infra12:21
*** zul has joined #openstack-infra12:21
*** koolhead17 has joined #openstack-infra12:21
*** skolekonov has joined #openstack-infra12:21
*** vigneshvar has joined #openstack-infra12:21
*** BharatK has joined #openstack-infra12:21
*** esmute has joined #openstack-infra12:21
*** che-arne has joined #openstack-infra12:21
*** imcsk8 has joined #openstack-infra12:21
*** erikmwilson has joined #openstack-infra12:21
*** prad-afk has joined #openstack-infra12:21
*** therve has joined #openstack-infra12:21
*** sweston has joined #openstack-infra12:21
*** dougwig has joined #openstack-infra12:21
*** willroberts_ has joined #openstack-infra12:21
*** anthonyper has joined #openstack-infra12:21
*** medberry has joined #openstack-infra12:21
*** dmsimard_away has joined #openstack-infra12:21
*** funzo has joined #openstack-infra12:21
*** jkt has joined #openstack-infra12:21
*** morgabra has joined #openstack-infra12:21
*** comstud has joined #openstack-infra12:21
*** Hunner has joined #openstack-infra12:21
*** sc68cal has joined #openstack-infra12:21
*** jvrbanac has joined #openstack-infra12:21
*** msuriar has joined #openstack-infra12:21
*** wendar has joined #openstack-infra12:21
*** zz_dimtruck has joined #openstack-infra12:21
*** dtroyer has joined #openstack-infra12:21
*** cyeoh has joined #openstack-infra12:21
*** mgagne has joined #openstack-infra12:21
*** rossella_s has joined #openstack-infra12:21
*** rainya has joined #openstack-infra12:21
*** Zenet has joined #openstack-infra12:21
*** dtantsur|afk has joined #openstack-infra12:21
*** ihrachyshka has quit IRC12:24
*** baoli has joined #openstack-infra12:26
*** aysyd has quit IRC12:27
*** aysyd has joined #openstack-infra12:27
*** pblaho__ has quit IRC12:28
*** sileht has quit IRC12:28
*** sileht has joined #openstack-infra12:28
*** baoli has quit IRC12:29
*** ihrachyshka has joined #openstack-infra12:30
*** baoli has joined #openstack-infra12:30
*** jaypipes has joined #openstack-infra12:31
*** rfolco has quit IRC12:31
*** rfolco has joined #openstack-infra12:33
*** rfolco has quit IRC12:35
*** rfolco has joined #openstack-infra12:36
*** tkelsey has quit IRC12:40
*** doude has quit IRC12:40
*** doude has joined #openstack-infra12:44
*** markvoelker has joined #openstack-infra12:50
*** hashar has joined #openstack-infra12:50
*** hashar has quit IRC12:52
*** markvoelker has quit IRC12:54
openstackgerritChristian Berendt proposed openstack-dev/hacking: Add hacking check H106: Use the correct capitalization of OpenStack  https://review.openstack.org/15354712:58
*** markvoelker has joined #openstack-infra13:04
*** swamireddy has joined #openstack-infra13:05
*** hashar has joined #openstack-infra13:06
*** koolhead17 has quit IRC13:07
*** dalgaaf has joined #openstack-infra13:07
*** kgiusti has joined #openstack-infra13:08
*** mbacchi has joined #openstack-infra13:09
*** woodster_ has joined #openstack-infra13:15
*** tkelsey has joined #openstack-infra13:17
*** dims__ has quit IRC13:17
*** alexpilotti has joined #openstack-infra13:19
*** ghostpl has quit IRC13:21
*** dprince has joined #openstack-infra13:21
*** bhunter71 has joined #openstack-infra13:24
*** dkliban_afk is now known as dkliban13:25
*** abhijeetm has quit IRC13:25
*** dims_ has joined #openstack-infra13:25
naggappanto run unit test of any project, say neutron is it required to install other projects like cinder,nova,keystone,horizon etc. ?13:26
*** Guest75042 has joined #openstack-infra13:27
openstackgerritMehdi Abaakouk proposed openstack-infra/project-config: gnocchi: add swift and ceph functional  obs  https://review.openstack.org/15355513:28
*** bknudson has quit IRC13:28
silehtnaggappan, no13:28
*** bswartz has quit IRC13:28
*** ghostpl has joined #openstack-infra13:30
naggappansileht : but I just downloaded neutron repo and run ./run_test.sh in ubuntu. Which ran for long time and gave me only  2 faill and few skip. Others all went fine13:30
*** spzala has joined #openstack-infra13:39
*** hashar has quit IRC13:40
silehtnaggappan, you should use 'tox' not ./run_tests.sh13:42
*** ddieterly has quit IRC13:45
naggappansileht: Oh ok. i just that there is no difference except tox uses nose test and saves the log13:46
*** vigneshvar has quit IRC13:47
*** mwagner_lap has quit IRC13:48
*** swamireddy has quit IRC13:50
*** ihrachyshka has quit IRC13:51
naggappanSileht: just now in another channel #openstack-neutron <ihrachyshka> confirmed that to run unit test just source tree is fine13:51
*** otter768 has joined #openstack-infra13:52
*** spzala has quit IRC13:55
*** ihrachyshka has joined #openstack-infra13:55
openstackgerritMehdi Abaakouk proposed openstack-infra/project-config: gnocchi: add swift and ceph functional  obs  https://review.openstack.org/15355513:56
*** andreaf has quit IRC13:56
*** andreaf has joined #openstack-infra13:57
openstackgerritMehdi Abaakouk proposed openstack-infra/project-config: gnocchi: add swift and ceph functional jobs  https://review.openstack.org/15355513:57
*** otter768 has quit IRC13:57
silehtfungi, AJaeger sdague, I have added the bashate job in gnocchi and I got NOT_REGISTERED :( (example: https://review.openstack.org/#/c/153329/) any idea ?13:58
AJaegersileht: let me check...13:58
silehtAJaeger, thanks13:59
silehtThis error always confumes me, I have to no clue of what I can do to understand what I have done wrong in the project-config git14:00
AJaegera recheck still gets a NOT_REGISTERED.14:00
silehtAJaeger, all gnocchi jobs fail since I have added the job14:00
AJaegersileht: yeah ;(14:01
AJaegerfungi, any ideas?14:01
fungihow recently was that job added? and how soon afterward did that run?14:02
AJaegerpatch for the bashate job merged 6+ hours ago https://review.openstack.org/#/c/152064/5/jenkins/jobs/projects.yaml14:02
AJaegerand was run 3 hours afterwards - and now again14:02
AJaegerstatus.o.o/zuul still shows NOT_REGISTERED14:02
fungiokay, so it's still broken? the 153329 change sileht linked would have run moments after the job was created in zuul14:03
fungii'll check the jenkins masters and zuul logs for clues14:03
*** jroll has quit IRC14:03
*** jroll has joined #openstack-infra14:03
AJaegerthanks, fungi14:04
*** hdd has joined #openstack-infra14:04
AJaeger153329 was run 2hours afterwards AFAIU14:04
AJaeger152064/ merged at 6:26 UTC14:04
AJaegerand 153329 failed at 9:1114:04
*** mfink_ has joined #openstack-infra14:04
fungioh, you said the job creation change merged 6+ hours ago. that's definitely on the + side of 6+14:04
fungi(like nearly 8)14:05
AJaegerSorry for not beiing accurate enough - was multitasking and maths got off ;(14:05
fungino problem. i was erring on the side of "maybe this is a known issue" based on your fuzzy math ;)14:05
fungi2015-02-06 14:00:13,266 INFO zuul.Gearman: Build <gear.Job 0x7f27a91b25d0 handle: None name: build:gate-gnocchi-bashate:bare-trusty unique: 3344b63e77484d3e92ee03bb8788e95a> complete, result NOT_REGISTERED14:06
fungizuul's gearman status has no matches for gate-gnocchi-bashate14:07
*** jaypipes is now known as leakypipes14:07
openstackgerritSergey Kulanov proposed openstack-infra/project-config: Add Fuel LMA collector Plugin to Stackforge  https://review.openstack.org/15357014:07
AJaegerit does not exist on jenkins01 either ;(14:08
fungiright, no /var/lib/jenkins/jobs/gate-gnocchi-bashate directory14:08
fungipresumably the other jenkins masters are the same14:08
fungiso jjb is failing to add it14:08
fungitrying a manual run of jenkins-jobs update now to see if it's got some good errors for us14:08
*** dustins has joined #openstack-infra14:09
* sileht have no luck with jenkins-job-builder :(14:09
fungiwhee! http://paste.openstack.org/show/168126/14:09
*** mjturek1 has joined #openstack-infra14:10
fungiAttributeError: 'Jenkins' object has no attribute 'get_plugins_info'14:10
silehtoutch14:10
fungiso this is either a recent change to jjb or python-jenkins14:10
*** tiswanso has joined #openstack-infra14:10
fungitime to find out which14:10
AJaegerArgh ;(14:10
*** radez_g0n3 is now known as radez14:11
AJaegersileht: So now it's a known error, please have patience while fungi debugs...14:11
silehtAJaeger, sure14:11
silehtthanks a lot14:11
silehtI have a job in oslo.messaging stuck with the same message: https://review.openstack.org/#/c/146047/14:12
*** bswartz has joined #openstack-infra14:12
silehtI guess once you fix the issue, you rebuild all jenkins jobs ?14:12
fungilast update to stackforge/python-jenkins was 10 days ago so my money's on something hitting openstack-infra/jenkins-job-builder in the last day or so14:12
fungisileht: once we get a fix merged to jjb everything should start working for those jobs (assuming we don't have other problems hiding behind that one)14:13
silehtfungi, ok cool, thanks a lot14:13
fungijust need a few minutes to git grep, blame, revert, review, repeat ;)14:14
AJaegerfungi: I1cae480a9a341ec2f6062904c962530dfce95057 includes a "import yaml"14:14
silehtAJaeger, fungi: https://github.com/openstack-infra/jenkins-job-builder/commit/b4e5aa9b7762a8e3ea24d55bffc6928ecde0a28c14:14
silehtit's a bit old, do run master branch of jjb ?14:15
AJaegerfungi check https://review.openstack.org/#/c/132927/36/jenkins_jobs/cmd.py line 2214:15
fungisileht: yeah, my blame is taking me to the same commit14:16
fungiAJaeger: the warning you're hunting down may be unrelated to the exception14:16
*** swamireddy has joined #openstack-infra14:16
*** isviridov_launch is now known as isviridov14:16
AJaegerfungi: yeah14:16
fungisileht: yeah, https://review.openstack.org/132927 merged yesterday14:17
fungisileht: i'll try manually reverting that patch on one of the jenkins masters and see if it clears the problem14:17
*** rm_work|away is now known as rm_work14:17
AJaegerfungi: A typo, it should be get_plugin_info but is get_plugins_info.14:18
AJaegerLet me send a patch after double checking...14:18
fungiAJaeger: oh, good eye14:18
*** ddieterly has joined #openstack-infra14:19
*** joesavak has joined #openstack-infra14:20
*** ihrachyshka has quit IRC14:20
*** jlibosva has joined #openstack-infra14:21
*** skolekonov has quit IRC14:22
fungiAJaeger: the jenkins_jobs.builder.ModuleRegistry class has a get_plugin_info method, but the traceback is for jenkins.Jenkins.get_plugins_info so may not be related14:22
fungilooking in the python-jenkins source now to confirm14:22
openstackgerritSergey Kulanov proposed openstack-infra/project-config: Add Fuel LMA collector Plugin to Stackforge  https://review.openstack.org/15357014:22
silehtfungi, jenkins.Jenkins have get_plugin_info too14:23
silehtit have both14:23
fungiyep, both14:23
silehtthe code looks correct14:23
fungilooking to see if we have an old python-jenkins installed14:23
openstackgerritAndreas Jaeger proposed openstack-infra/jenkins-job-builder: Fix typo, it's get_plugin_info  https://review.openstack.org/15357414:23
*** annegent_ has joined #openstack-infra14:24
fungimaybe jjb needs to bump it's req min on that14:24
nibalizerfungi: we had to bump python-jenkins in ours this morning14:24
fungipython-jenkins==0.4.0 and jenkins-job-builder==1.1.0.35.g74c8ac414:24
nibalizerrcarrillocruz: knows the details14:24
AJaegermmh, so my patch is not needed? ;(14:24
AJaegerVery confusing...14:24
fungiAJaeger: probably not. i'm looking now to see what the earliest release is that contains that method14:25
fungisomething between 0.4.0 and 0.4.4 apparently14:25
silehtfungi, this method require >= 0.4.114:25
fungisileht: agreed according to git tag --contains b50c59aa (which i got from blame)14:26
AJaegerI see now that python-jenkins has both, so will abandon my JJB patch again14:26
fungipatch on the way14:26
rcarrillocruzwhat's up14:26
rcarrillocruzah14:26
rcarrillocruzso yeah14:26
rcarrillocruzyou need to upgrade python-jenkins14:26
rcarrillocruzcos a commit for JJB that was merged last night uses get_plugin_info, which is dependent on functionality from latest python-jenkins14:27
rcarrillocruzfungi: ^14:27
rcarrillocruz0.4.4 works for us14:27
*** yamamoto has joined #openstack-infra14:27
*** jlibosva has quit IRC14:28
openstackgerritJeremy Stanley proposed openstack-infra/jenkins-job-builder: Depend on python-jenkins 0.4.1 or later  https://review.openstack.org/15357614:28
fungircarrillocruz: nibalizer: sileht: AJaeger: ^14:28
silehtfungi, thanks14:28
* sileht hopes it solves its oslo.messaging job issues too14:29
fungipelix: zaro: mgagne: https://review.openstack.org/153576 needs +2/approval14:30
*** yamamoto_ has joined #openstack-infra14:30
fungijhesketh: SergeyLukjanov: clarkb: jeblair: mordred: ^ if any of you are around14:30
AJaegerthanks, fungi!14:30
SergeyLukjanovfungi, I'm here14:30
mordredon it14:30
fungidogpile. awesome14:31
openstackgerritGuido Günther proposed openstack-infra/jenkins-job-builder: Lazy load plugin list  https://review.openstack.org/15339614:31
* fungi goes to top up the coffee14:31
openstackgerritMerged openstack-infra/storyboard-webclient: Fix to show errors again  https://review.openstack.org/15317614:31
SergeyLukjanovfungi, +2'd14:32
*** mattfarina has joined #openstack-infra14:32
*** bswartz has quit IRC14:33
*** yamamoto has quit IRC14:34
*** ZZelle has quit IRC14:35
*** zz_gondoi is now known as gondoi14:35
*** ZZelle has joined #openstack-infra14:35
openstackgerritSergey Kulanov proposed openstack-infra/project-config: Add Fuel LMA collector Plugin to Stackforge  https://review.openstack.org/15357014:36
*** gondoi is now known as zz_gondoi14:36
*** hashar has joined #openstack-infra14:36
*** bswartz has joined #openstack-infra14:38
openstackgerritMerged openstack-infra/project-config: add stable compat jobs to python-barbicanclient  https://review.openstack.org/15064514:38
*** bknudson has joined #openstack-infra14:39
*** scheuran has quit IRC14:41
mordredfungi: I read that as "fungi goes to the top of the coffee"14:43
*** zz_gondoi is now known as gondoi14:43
* fungi is more buoyant that coffee14:45
fungier, than14:45
openstackgerritMerged openstack-infra/jenkins-job-builder: Depend on python-jenkins 0.4.1 or later  https://review.openstack.org/15357614:45
*** _amrith_ is now known as amrith14:47
*** asselin__ has joined #openstack-infra14:48
AJaegerfungi, do we need to approve now another patch for project-config to push out all changes?14:49
*** koolhead17 has joined #openstack-infra14:50
fungiAJaeger: you can if you want, though i planned to re-run jjb on the jenkins masters manually as soon as i see python-jenkins get upgraded everywhere14:51
*** rlandy has quit IRC14:51
*** ildikov has quit IRC14:52
*** ChuckC has quit IRC14:52
AJaegerfungi, I approved all patches I could this morning ;) So, now somebody else needs to review and approve14:52
AJaegerOne candidate: https://review.openstack.org/#/c/153173/14:52
fungiaww, now you're guilting me into doing reviews ;)14:53
fungimaybe i can get my stats up to top infra reviewer this way14:53
* AJaeger wasn't subtile enough, sorry14:54
fungiheh14:54
*** annegent_ has quit IRC14:55
*** zz_dimtruck is now known as dimtruck14:55
*** ddieterly has quit IRC14:55
*** markus_z has quit IRC14:55
*** ddieterly has joined #openstack-infra14:55
fungiAJaeger: that one looked fine so i approved it, but it doesn't actually create any new jobs so i don't think it would have necessarily triggered a jjb update on the jenkins masters14:57
*** ghostpl has quit IRC14:57
*** hashar has quit IRC14:57
*** hashar has joined #openstack-infra14:57
AJaegerfungi: ah, I see14:57
*** dhritishikhar_ has joined #openstack-infra14:58
*** signed8bit has joined #openstack-infra14:59
AJaegernow it gets more complicated, there're too many patches that Jenkins gave a -1...14:59
fungiwell, i should be able to start the manual jjb runs here momentarily now anyway15:00
*** signed8b_ has joined #openstack-infra15:01
*** hashar has quit IRC15:01
*** amrith is now known as _amrith_15:01
*** tonytan4ever has joined #openstack-infra15:02
*** ghostpl has joined #openstack-infra15:02
*** armax has joined #openstack-infra15:03
openstackgerritMerged openstack-infra/project-config: Add Fuel Contrail Plugin to Stackforge  https://review.openstack.org/15317315:03
*** r-daneel has joined #openstack-infra15:03
*** yamamoto_ has quit IRC15:04
*** asselin__ has quit IRC15:04
*** signed8bit has quit IRC15:04
*** dimtruck is now known as zz_dimtruck15:05
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added controller for refresh tokens  https://review.openstack.org/15359115:05
*** hdd has quit IRC15:05
*** asselin__ has joined #openstack-infra15:05
openstackgerritNikita Konovalov proposed openstack-infra/storyboard: User abort call instead of returning response  https://review.openstack.org/15251515:07
*** dansmith is now known as superdan15:08
*** nelsnelson has joined #openstack-infra15:08
*** annegent_ has joined #openstack-infra15:09
*** asselin__ has quit IRC15:12
*** yamahata has joined #openstack-infra15:13
*** GheRivero has joined #openstack-infra15:13
*** mjturek1 has quit IRC15:13
*** mjturek1 has joined #openstack-infra15:16
*** mjturek1 has left #openstack-infra15:17
*** stevemar has joined #openstack-infra15:17
*** mjturek1 has joined #openstack-infra15:17
*** swamireddy1 has joined #openstack-infra15:19
*** timcline has joined #openstack-infra15:20
*** ghostpl has quit IRC15:21
*** swamireddy has quit IRC15:21
*** jgrimmn is now known as zz_jgrimmn15:21
sdaguefungi: ... all of stackforge is cached in d-g now?15:21
mordredyah15:22
fungisdague: all of stackforge is cached on our workers in /opt/git15:22
fungisdague: not sure about d-g15:22
sdagueok15:22
mordredyah - I was answering about the workers15:22
*** signed8b_ has quit IRC15:22
fungii'd have to go digging to see whether d-g is copying all of those into the devstack cachedir or just specific ones15:22
sdagueI'm trying to provide an example for out of tree plugin testing, I can work with that I think15:22
*** yamahata has quit IRC15:23
fungiman, our ansible/puppet runs really do take a while to complete, but the flock seems to be working at least15:23
clarkbdevstack is not copying all of them15:24
clarkbbut can be found at /opt/git15:24
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added new paginate query  https://review.openstack.org/15323115:24
fungii think run_all.sh is taking >30m so we're getting one puppet apply every 45 minutes on our servers these days15:24
clarkbwow15:24
*** mtanino has joined #openstack-infra15:25
*** ghostpl has joined #openstack-infra15:25
fungiat this speed we should just run that script in a continuous loop15:25
mordredmorning clarkb15:25
*** dmsimard_away is now known as dmsimard15:25
*** zz_dimtruck is now known as dimtruck15:25
clarkbmorning15:25
jeblairweird, i think it was taking 10 mins earlier, and most of that was cacti15:25
openstackgerritMerged openstack-infra/project-config: Remove unused env variable option for tenant isolation  https://review.openstack.org/15200715:26
fungihrm, well the current run started at 1500 and is still going15:26
openstackgerritSean Dague proposed openstack-infra/devstack-gate: WIP: testing path for doing external plugins in d-g  https://review.openstack.org/15359715:27
*** tonytan4ever has quit IRC15:28
*** tiswanso has quit IRC15:30
*** wenlock has joined #openstack-infra15:31
*** tiswanso_ has joined #openstack-infra15:32
jeblairsdague: i think i see where you're going with ^ and left a clarifying comment; lemme know if we're on the same page15:32
*** amitgandhinz has joined #openstack-infra15:32
mriedemanteaya: if people running third party ci have questions in the ML, what should they tag with? [qa]?15:33
dprinceAny chance we could get this new TripleO job in today: https://review.openstack.org/#/c/153341/15:33
openstackgerritSean Dague proposed openstack-infra/devstack-gate: WIP: testing path for doing external plugins in d-g  https://review.openstack.org/15359715:33
*** hashar has joined #openstack-infra15:33
sdaguejeblair: yeh, I came to about the same conclusion at the same time :)15:34
sdaguemaybe, round 2 will be right-ish?15:34
*** tonytan4ever has joined #openstack-infra15:34
*** hashar has quit IRC15:34
sdaguethe ec2 folks pinged me privately trying to figure out how to setup jobs, and as they are actively asking good questions, I figured they might be the good base example to sort out the through path for external plugin + upstream testing15:35
*** hashar has joined #openstack-infra15:35
mriedemanteaya: looks like [ThirdPartyCI] is the tag and people mostly post to openstack-infra, so i think i've got my answer15:35
jeblairsdague: sounds cool15:35
*** hashar has quit IRC15:36
*** hashar has joined #openstack-infra15:37
*** marun has joined #openstack-infra15:39
*** dizquierdo has quit IRC15:40
*** sabeen1 has joined #openstack-infra15:40
*** dizquierdo has joined #openstack-infra15:41
*** hashar has quit IRC15:41
*** koolhead17 has quit IRC15:41
*** _nadya_ has quit IRC15:42
*** salv-orlando has joined #openstack-infra15:43
fungiokay, python-jenkins is showing at the latest release on all the jenkins masters right now (except for jenkins-dev which has some puppet issues), but i'm getting a new entrypoint related exception now: http://paste.openstack.org/show/168168/15:43
fungimordred: does ^ that look familiar?15:43
*** signed8bit has joined #openstack-infra15:43
*** dannywilson has joined #openstack-infra15:43
*** hdd has joined #openstack-infra15:44
*** MaxV has quit IRC15:44
fungiohhhh15:44
fungiwe seem to also have python-jenkins debs installed15:44
mordredfail15:44
* fungi curses15:45
openstackgerritPranav Salunke proposed openstack-infra/project-config: Updates training guides jobs  https://review.openstack.org/15211815:45
clarkbmordred denis sent mail to list about what inner requirement are. in response to your -2 (want to make sure you see that)15:45
mordredclarkb: awesome15:46
mordredclarkb: my main tl;dr is - we seem to be reinventing debian packages - so perhaps we've reached the point where we should just figure out doing that15:46
clarkbjeblair sdague when you get that sorted would be good to pass the info along to the glusterfs job15:47
mordredbecause "I only want these depends if I have these optional subcomponents" is the type of complexity that that handles very well15:47
sdagueclarkb: agreed15:47
clarkbmordred right. more importantly this change does nothing to solve that15:47
mordredyah15:49
*** adalbas has joined #openstack-infra15:49
mordredI'll follow up with an email there soo15:49
mordredsoon15:49
mordredI _do_ think that there is a feature taht's already in pip/setuptools that dstufft mentions that pbr could and should grow smarts about15:49
fungiugh, and python-setuptools deb is also installed on the jenkins masters15:49
mordredthat is related to the underlying problem they're trying to solve here15:49
mordredfungi: ew gross - what installed it?15:50
*** mfink_ has quit IRC15:50
fungimordred: i'm going to go back through the dpkg log, but probably an unattended upgrade of something depending on it15:50
dstufftmordred: setuptools extras yea15:50
mordredfungi: yay!15:50
dstufftsetuptools extras are named lists of dependencies which are opt-in and not installed by default15:50
mordreddstufft: yah. although - I don't want to generate more people thinking they're going to get more complex dpkg-style dep handling as we get more and more complicated here ... I think we're already overdoing things as it is15:51
mordred(as evidenced by the number of thigns we do that make you want to hide)15:51
dstufftHere's ane xtra: https://github.com/kennethreitz/requests/blob/master/setup.py#L65-L6715:52
dstufftyou can depend on that by using ``requests[security]``15:52
dstufftanywhere you'd normally use just ``requests]``15:52
clarkbextras doesnt fix this15:52
fungii worry that projects are trying to treat requirements lists as a substitute for install documentation15:52
*** dhritishikhar_ has quit IRC15:53
clarkbthey want some way to say if amqp option selected then install only amqp extras15:53
dstufftwhat is selecting the amqp option mean15:53
clarkbthats the tricky bit that isnt solved :)15:53
dstufftbecause you can do ``oslo.messaging[ampq]``15:53
fungiinstall oslo.messaging to work with amqp, so don't install the bits it only needs to use rabbitmq15:53
*** otter768 has joined #openstack-infra15:54
dstufftwhich will install the ampq bits15:54
*** lnxnut has joined #openstack-infra15:54
dstufftbut you won't get the ampq bits if you install ``oslo.message``15:54
clarkbdstufft sure but how as a user does that get passed into pip15:54
clarkbvia devstack or whatever calling process15:54
clarkbtheres a lot more to it than just change this string15:54
clarkbespecially since oslo.messaging is a dep of everything else15:55
*** mfink_ has joined #openstack-infra15:55
dstuffthow someone signals to devstack they want a particular extra I dunno, just saying that you don't need to invent anything new on the packaging side if that's all you need, you just need some bits to wire up devstack to installing the extras15:56
clarkbdstufft even when nova install oslo.messaging as a dep?15:56
clarkbor you make two passes?15:56
*** tiswanso_ has quit IRC15:56
clarkbI guess you could say installing oslo.messaging wont ever install a workable system. then require an explicit install of an extras set15:57
jrollas a deployer, 1) I don't want to have to specify which oslo.messaging extras to install, besides what's in the project requirements.txt, and 2) I have no problem with oslo.messaging installing a dependency that I don't use15:58
*** yamahata has joined #openstack-infra15:58
*** otter768 has quit IRC15:58
jrollclarkb: got a pointer to whatever started this?15:58
*** kgiusti1 has joined #openstack-infra15:59
*** tiswanso_ has joined #openstack-infra15:59
dstufftdo two passes or add an ``oslo.messaging[ampq]`` onto the list of things to install, or have nova support an ampq extra as well, the two pass is probably less error prone until pip has a proper dependency resolver, then adding ``oslo.messaging[ampq]`` should be equivilant15:59
openstackgerritThanh Ha proposed openstack-infra/jenkins-job-builder: Add support for new patchset-created exclude fields  https://review.openstack.org/14009515:59
*** _amrith_ is now known as amrith15:59
clarkbhttp://lists.openstack.org/pipermail/openstack-dev/2015-February/056205.html15:59
fungijroll: i have to assume it came up in an oslo weekly meeting? i haven't seen a link to the original discussion which arrived at this decision anywhere15:59
clarkbyes dont know where original discussion was16:00
clarkbanyways I no understand extras better so win :)16:00
clarkb*now16:00
dhellmannclarkb: I think we'll probably need 2 passes. We already do some custom logic for different backends. This would simplify that a bit, and put the dependency list in oslo.messaging instead of devstack, which will make it easier to keep it up to date.16:00
*** kgiusti has quit IRC16:00
fungidhellmann: is the underlying reason for this that amqp and rabbitmq conflict in some way such that you can't install the deps for both?16:01
dhellmannfungi, jroll: I saw some mention of this, and I know bnemec has been experimenting with extras in pbr, but this wasn't something we've discussed.16:01
clarkbdhellmann the precipitating change said "it was decided"16:01
dhellmannfungi: well, we don't want oslo.messaging to always depend on both as a matter of principle, though maybe the simpler solution is to reconsider that16:01
fungidhellmann: ahh, the original commit message said "it has been decided" which left us asking "where and by whom?"16:01
clarkbimplying oslo had decided to do this16:02
dhellmannclarkb: ok, I don't think I was involved in deciding16:02
dstufftat some point we want to add the ability to have "opt out" extras too, right now extras are always opt in16:02
dstufftbut idk when that'll happen16:02
*** MarkAtwood has joined #openstack-infra16:02
dhellmannclarkb, fungi : "original change"? I saw something on the dev list a few minutes ago16:02
AJaegerdhellmann: https://review.openstack.org/15322716:03
fungipersonal opinion, but people wanting to add more and more complexity to our workarounds for the lack of a dep solver in pip is to spend that time improving pip to handle these cases better instead16:03
dhellmannfungi, clarkb : "it was decided" without a reference is certainly a reason for a -116:04
dhellmannAJaeger: thanks16:04
clarkbbut I think we understand the issue now16:05
*** yamamoto has joined #openstack-infra16:05
dhellmannfungi: I agree, but I'm not sure that's where this was coming from. I think we just don't want everyone to have to install amqp, zmq, kombu, etc. if they're only going to use one of them.16:05
*** rfolco has quit IRC16:05
clarkband importantly mordred can figure out how he want pbr to deal with it16:05
bnemecThis actually came out of a discussion about a year ago: http://lists.openstack.org/pipermail/openstack-dev/2014-February/026976.html16:05
jrollI don't really see this solving any problems - are projects going to stop listing oslo.messaging in requirements.txt and require deployers to choose which "version" of oslo.messaging to install?16:05
dhellmannclarkb: yeah, I'd like to see a spec for this so we have a record of it16:05
jrollthat seems annoying at best16:05
bnemec+1 to a spec.16:06
bnemecThe original change pre-dated that whole process.16:06
AJaegercross-distro spec?16:06
dhellmannjroll: no, I think the idea is to help packagers understand what other dependencies are needed given a particular backend service choice16:06
dhellmannbnemec: right16:06
dhellmannAJaeger: if we're going to solve this in pbr, it's an oslo spec I think16:06
AJaegerat least it needs changes in infra as well;)16:07
dhellmannjroll: right now that's falling to documentation, which makes automating tooling more challenging16:07
jrolldhellmann: so how do I install, say nova, with pip, if this changes?16:07
*** isviridov is now known as isviridov_away16:07
jrolldhellmann: I don't use requirements.txt as docs, I use it to install dependencies16:07
dhellmannjroll: you might have to pip install oslo.messaging with your backend expressed first, and then install nova16:07
*** david-lyle_afk is now known as david-lyle16:07
fungidhellmann: we're not really "making" people install everything (pip is one of many deployment options), but if that's really worth avoiding then just put one backend in the requirements and people can remove/install a different one if they need?16:07
*** mpaolino has quit IRC16:07
jrolldhellmann: yeah, that seems backwards, idk16:08
*** carl_baldwin has joined #openstack-infra16:08
dhellmannjroll: I'm not sure it's completely possible to deploy nova from pip, though, and that's not something we necessarily care about is it?16:08
bnemecfungi: I think that's basically what we're doing today.16:08
dhellmannfungi: that's what we do now16:08
bnemecWe could leave that be, and then provide the alternative requirements for them to also install if they want a different backend.16:08
fungiso where did the friction against that come from?16:08
jrolldhellmann: it is, pip install -e git+https://git.openstack.org/openstack/nova#egg=nova16:08
clarkbdhellman it almost is actually. you just need libvirt and lxml deps if you use pymysql16:08
jrolldhellmann: not everyone uses distro packages16:08
dhellmannfungi: people who run non-default backends don't want other libs installed? I really don't know why we care, but that's what we're getting.16:08
fungithis goes back to my original concern that projects are trying to use requirements files as a replacement for better install documentation16:09
dhellmannbnemec: let's get a spec put together to at least explain the problem we're trying to solve16:09
jrollfungi: +116:09
jrolland +1 for a spec16:09
clarkbok some one should respond to list saying this deserves a spec in oslo (because pbr)16:09
clarkbI can do that if you like16:09
*** rfolco has joined #openstack-infra16:09
dhellmannjroll: I built a cloud based on packages like that, so I know, and I know that I had to script the installation of things that were not expressed as dependencies16:09
*** tonytan_brb has joined #openstack-infra16:09
dhellmannclarkb: I'll do that16:10
clarkbthen we can express the problem and potential solutions there16:10
clarkbkk16:10
*** tonytan4ever has quit IRC16:10
*** yamamoto has quit IRC16:10
fungiagreed. a spec would be great because it still feels to me like it's venturing into "solution searching for a problem" territory16:10
*** shashankhegde has joined #openstack-infra16:10
bnemecfungi: Isn't that just moving the requirements list from a usable place (a requirements file) to a place (documentation) that users have to install all the things manually (which they'll probably do by...creating a requirements file)?16:11
clarkbyup especially since the solution initially proposed doesnt fix the problem16:11
jrolldhellmann: right, required python dependencies that aren't in requirements.txt make me sad16:11
fungibnemec: a requirements file is a usable place. a bunch of different requirements files less so16:11
bnemecfungi: I still don't see how a bunch of different lists in documentation is better than a bunch of different requirements files. :-/16:12
bnemecBut maybe we should save this for the spec discussion.16:12
*** alaski is now known as lascii16:12
clarkbbnemec no use onereqs file16:12
fungibnemec: well, you'd still need to document the distinction between the requirements files16:12
dhellmannjroll: is zmq "required" if you're using amqp?16:12
*** salv-orlando has quit IRC16:13
dhellmannI've sent email to the list asking denis for a spec16:13
clarkbdhellmann no but installing it isnt likely to hurt (because we sort of check you can install everything together today)16:13
*** salv-orlando has joined #openstack-infra16:13
*** koolhead17 has joined #openstack-infra16:14
bnemecI think I initially suggested an install all the things strategy and there was a lot of pushback.16:14
jrolldhellmann: no, but oslo.messaging is, and if oslo.messaging isn't in requirements.txt so that I can avoid installing a zmq library...16:14
dhellmannclarkb: "install it all" and "stop adding so many dependencies" have different proponents :-/16:14
clarkbat worst its a problem for packagers and you can specify non overlapping sets in more expected to work with existing tools ways16:14
*** hdd has quit IRC16:14
*** AJaeger has quit IRC16:14
dhellmannjroll: i don't understand what you're saying there. I don't think anyone is suggesting removing oslo.messaging from your requirements list.16:14
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added milestones  https://review.openstack.org/15250016:14
*** rfolco has quit IRC16:15
dhellmannreally, I don't care about this problem, I'm just trying to explain the history. Someone else needs to be the advocate for any particular approach.16:15
jrolldhellmann: so by default, install all the oslo.messaging backends and let deployers remove them at will?16:15
*** dannywilson has quit IRC16:15
fungijroll: _if_ you're using pip16:15
fungijroll: distro packages will generally have much better inbuilt mechanisms for optional selections16:16
*** dannywilson has joined #openstack-infra16:16
jrollfungi: sure16:16
jrollI guess I don't know how many deployers use pip versus distro packages16:16
*** rfolco has joined #openstack-infra16:16
anteayamriedem: yes posting to -infra ml, in which case tagging is optional16:17
bnemecTripleO by default installs everything from source/pip to allow for CD.16:17
fungithis goes back to "we're working around lack of features in pip, why not improve pip?"16:17
dstufftjroll: fungi dhellmann fwiw you won't have to install oslo.messaging first, you could install it at the same time (but you might hit problems b/c of lack of dep solver) or install it later, also nova could also provide an extra that just installs the oslo.messaging extra (though I don't know if you can have an extra that depends on something you depend on + some extras, that might cause an error until we get the better dep solver)16:17
dstuffte.g. having install_requires=["oslo.messaging"], extras_require={"ampq": ["oslo.messaging[ampq]"]} might raise an error currently I'm not sure, but if we get a better depsolver that won't be the case for sure16:18
fungidstufft: yeah, entirely likely we'll have situations where package a uses package b and c but b also uses c and we want to specify optional dependencies in c16:18
fungiwhich i guess means install c directly first with specific options, then a16:19
fungiand hope pip dtrt with it16:19
*** jcoufal has quit IRC16:19
*** signed8b_ has joined #openstack-infra16:19
dstufftthat's something that pip _should_ handle for sure though, it might not currently though because pip doesn't handle multiple specifiers for the same thing very well16:19
clarkbfwiw I am with jroll for practical deployment reasosn. if nova + oslo.messaging support amqp and zmq I sant support for both installed so I can switch with config changes and not require redeployment16:21
*** openstackgerrit has quit IRC16:21
dstufftoslo.messaging[ampq,zmq] :D16:21
clarkbzmq and amqp are both supported16:21
clarkbtherefore their deps are required16:22
*** openstackgerrit has joined #openstack-infra16:22
fungijeblair: so the 15:00 run_all.sh took ~43 minutes to complete, then another started at 15:45 which completed at ~16:06 so it does seem somewhat variable16:22
clarkbits a bit odd for me to say upfront "I will never use X functionality so please never install it"16:22
jrollclarkb: I just think this isn't actually solving a real problem16:22
*** signed8bit has quit IRC16:23
jrollbut will wait for the spec16:23
clarkbfungi I am planning to look and see if I can narrow it down to trpuble hosts/manifests16:23
fungiclarkb: clarkb it looks like the run for git02 may have taken ~25 minutes during the 15:00 utc session16:24
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add support for volume attach/detach  https://review.openstack.org/15362316:24
fungiclarkb: so maybe this is centos 6 git remote timeout problems once again16:25
*** thedodd has joined #openstack-infra16:25
clarkbfungi: and since that runs before everything else would certainly slow down the pipeline16:25
fungiindeed16:25
fungigiven the frequency we're seeing this manifest in multiple contexts, we should probably try to narrow down what causes it16:26
*** annegent_ has quit IRC16:26
*** rmcall has joined #openstack-infra16:26
dstufftianw: btw did pip 6.0.8 fix things?16:26
fungiclarkb: i still _strongly_ suspect the glibc update16:26
fungiclarkb: i'm going to hold a centos6 nodepool node and try to reproduce16:27
clarkbfungi: it was the major change between when it didn't do this and when it started16:27
clarkbfungi: ok16:27
krtaylormriedem, actually "third-party" is the agreed topic filter tag -> http://lists.openstack.org/cgi-bin/mailman/options/openstack-dev/kurt.r.taylor@gmail.com/?VARHELP=Third+Party+CI+group16:27
fungiclarkb: but first i need to finish figuring out how to fix jjb on the jenkins masters16:27
clarkbcan I ask for reviews of the stack that starts at https://review.openstack.org/#/c/153093/1 ? at least for the first three changes there. WIll get multinode testing working for nova network again16:27
clarkband those jobs are experimental on a few projects now so will start seeing more wide use16:28
krtaylormriedem, substituting your email in url of course16:28
fungiclarkb: http://paste.openstack.org/show/168168/ ring any bells? i'm trying to unwind and currently suspect the distro packages for setuptools, pkg_resources, pip, et cetera since they're all installed on our jenkins masters16:28
clarkbfungi: anything I can do to help there. maybe we need an ansible script/command?16:28
openstackgerritSean Dague proposed openstack-infra/devstack-gate: WIP: testing path for doing external plugins in d-g  https://review.openstack.org/15359716:28
fungiclarkb: purging python-setuptools wants to tear out a ton of other distro python packages with it16:29
openstackgerrityolanda.robla proposed openstack-infra/infra-specs: Add nodepool image testing spec.  https://review.openstack.org/13959816:29
clarkbfungi: does pip freeze show python-jenkins>=0.4.1 is installed?16:29
fungiclarkb: yep16:29
*** ildikov has joined #openstack-infra16:29
fungiclarkb: however distro python-jenkins was also present according to dpkg16:29
clarkbwhat if we just remove python-jenkins the distro package?16:29
fungiclarkb: removing that and reinstalling python-jenkins via pip didn't help16:29
*** dangers_away is now known as dangers16:30
fungiclarkb: so then i tried upgrading pbr from 0.10.0 to 0.10.7 but that just got it to start similarly complaining that it couldn't find its pbr dep at that point16:30
clarkbsetuptools (0.6rc11)16:30
fungitwisty maze of package problems, all alike16:30
clarkbI am going to guess that that is part of the problem16:30
*** mjturek1 has quit IRC16:30
clarkbfungi: try a pip install -U setuptools?16:31
fungiclarkb: yep, that's from the ubuntu precise python-setuptools package16:31
*** otter768 has joined #openstack-infra16:31
clarkbthen do the things you did above again16:31
fungitrying now in hopes of sanity16:31
*** Guest75042 has quit IRC16:31
*** mjturek1 has joined #openstack-infra16:31
*** carl_baldwin has quit IRC16:31
fungiclarkb: yeah, pip install of latest setuptools and not removing the distro package seems to have worked for now16:32
fungii'll do the same on the rest of the jenkins masters16:32
clarkbcool16:33
clarkbI am guessing bug in old pkg_resources which is part of setuptools16:33
*** rmcall has quit IRC16:33
*** koolhead17 has quit IRC16:33
*** koolhead17 has joined #openstack-infra16:34
*** rmcall has joined #openstack-infra16:34
*** carl_baldwin has joined #openstack-infra16:36
*** dtantsur|afk is now known as dtantsur16:36
*** dims_ is now known as dimsum__16:39
marunjhesketh: https://review.openstack.org/#/c/146762/ now has a +2 from jeblair. I hope I've managed to satisfy your concerns as well!16:39
zxiiro2 months, 13 patchsets later I hope my patch gets merged soon :)16:40
*** mwagner_lap has joined #openstack-infra16:40
*** unicell has quit IRC16:41
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add support for volume attach/detach  https://review.openstack.org/15362316:41
clarkbzxiiro: the gerrit trigger change? still has my +2 right?16:43
zxiiroclarkb: no because I had to revise it. Waiting for Darragh to +2 though. Seems to be a very picky reviewer :)16:43
*** jgallard__ has quit IRC16:44
*** naggappan has quit IRC16:44
*** swamireddy has joined #openstack-infra16:44
openstackgerritSean Dague proposed openstack-infra/system-config: update indexer for apache logs  https://review.openstack.org/15362716:45
sdagueclarkb: ^^^16:45
fungizxiiro: he's pelix in here by the way16:45
sdaguewe aren't indexing keystone by default :(16:45
zxiirofungi: ah good to know. now I know who to bug :D16:45
fungisdague: since there are no bugs in keystone, that saves us on storage ;)16:46
*** tonytan_brb has quit IRC16:46
sdaguefungi: except, you know, this one - https://bugs.launchpad.net/keystone/+bug/141904316:46
openstackLaunchpad bug 1419043 in Keystone "Conflict on isolated credential setup" [Critical,Confirmed] - Assigned to Matthew Treinish (treinish)16:46
*** swamireddy1 has quit IRC16:46
sdaguewhich I can't see how often it happens16:46
*** tonytan4ever has joined #openstack-infra16:46
fungisdague: yeah, i figured that was precipitated by you digging into a keystone bug16:47
zxiiropelix: woohoo thanks! clarkb can you review the patch just needs you +2 again? https://review.openstack.org/140095/16:47
sdaguealso... I think we aren't indexing horizon any more for similar reasons16:47
*** prad has joined #openstack-infra16:47
pelixzxiiro: sorry about missing the little things, if I paid more attention I'd spot them all the first time :p16:47
zxiiropelix: no worries. I appreciate projects ensuring their code quality is high.16:48
clarkbzxiiro: ya let me deal with this log thing first16:48
clarkbfungi: I am +2 on sdague's log change. Both log files exist and look to be in the format sdague has them listed as16:49
openstackgerritMerged openstack-infra/project-config: Add pylint jobs to service modules  https://review.openstack.org/15337316:49
clarkbfungi: want to review that really quicky?16:49
*** signed8b_ is now known as signed8bit_ZZZzz16:49
*** carl_baldwin has quit IRC16:49
*** carl_baldwin_ has joined #openstack-infra16:49
fungisdague: clarkb: is oslofmt correct for apache logs?16:51
fungii honestly haven't compared keystone's apache logs to more typical service logs but would have thought they'd be more like typical apache log format16:51
fungiclarkb: sounds good16:51
clarkbya see sdagues comment16:51
*** marun has quit IRC16:51
clarkbkeystone uses oslo format even running in apache16:52
clarkbhorizon error log is apache though16:52
fungihuh. fun!16:52
clarkbI checked too16:52
*** dhellmann_ has quit IRC16:53
*** koolhead17 has quit IRC16:53
*** dhellmann_ has joined #openstack-infra16:54
*** signed8bit_ZZZzz is now known as signed8b_16:56
*** dhellmann_ has quit IRC16:56
*** ashleighfarnham has joined #openstack-infra16:56
*** shashankhegde has quit IRC16:56
*** annegent_ has joined #openstack-infra16:57
clarkbzxiiro: approved. Also since you're here you should check out zuul for even more awesome jenkins jobs triggered by gerrit events16:57
*** _nadya_ has joined #openstack-infra16:57
*** dhellmann_ has joined #openstack-infra16:57
clarkbzxiiro: http://ci.openstack.org/zuul/16:58
zxiiroclarkb: I've seen zuul it is pretty cool. You guys demoed some of it for us when I worked at the Eclipse Foundation.16:58
zxiiroclarkb: we're looking into installing it at The Linux Foundation in the future16:58
*** ChuckC has joined #openstack-infra16:58
*** dhellmann_ has quit IRC16:59
clarkbcool16:59
*** vigneshvar has joined #openstack-infra16:59
*** dhellmann_ has joined #openstack-infra16:59
fungizxiiro: you can see the current activity for our zuul deployment at http://status.openstack.org/zuul/ too17:00
*** tonytan4ever has quit IRC17:00
fungiit's reasonably busy at the moment17:00
*** tonytan4ever has joined #openstack-infra17:00
fungiokay, jenkins masters are all updated and jjb confirmed working on them again17:00
zxiirofungi: that's pretty neat. When it's busy seems like it's a little overwhelming to look at :)17:00
*** hdd has joined #openstack-infra17:01
*** carl_baldwin_ has quit IRC17:01
fungizxiiro: yeah, it's tracking a lot... and we've been working over time to condense that display more and more17:01
*** otter768 has quit IRC17:01
fungizxiiro: the bluish boxes are change state summaries which you can click to expand and see actual job completion details17:02
openstackgerritMerged openstack-infra/jenkins-job-builder: Add support for new patchset-created exclude fields  https://review.openstack.org/14009517:02
*** sarob has joined #openstack-infra17:02
clarkblooks like vivid finally has a cgit package17:02
clarkbthough we probably shouldn't run the mirrors on that17:03
fungiclarkb: not sure i want to hang my hopes on ubuntu 16.04 lts to fix the current issues we have ;)17:03
clarkbfungi: we could however potentially run them off of centos717:03
*** carl_baldwin has joined #openstack-infra17:03
clarkbif it comes to that should be relatively easy to swap out backends assuming that we don't have to chagne much puppet17:03
zxiirodo all of your jenkins masters have every job? or are the jobs on one master somewhere that distributes them out?17:03
clarkbzxiiro: most masters have every job, but they registers with zuul via gearman so we only distribute jobs where they are available17:04
*** annegent_ has quit IRC17:04
clarkbs/registers/register/17:04
fungiclarkb: anyway i'm switching gears now to start seeing if i can recreate and debug the recent centos 6 git remote hangs17:04
*** kgiusti1 has quit IRC17:04
clarkbfungi: let me know if I can help. Otherwise I will be spending my time this morning reviewing17:04
fungiplease review. we have a lot of changes to get through ;)17:04
*** sdake_ has quit IRC17:04
zxiiroright so where do you push your JJB configuration to?17:05
fungizxiiro: we keep it in git managed by gerrit. you can browse it at http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/17:05
*** crc32 has joined #openstack-infra17:06
fungizxiiro: it's continuously deployed onto our jenkins masters via configuration management, and jjb runs to update each jenkins master triggered from local updates to those files17:06
*** carl_baldwin has quit IRC17:06
*** thedodd has quit IRC17:06
*** annegent_ has joined #openstack-infra17:07
fungizxiiro: so basically developers can propose changes to the jjb configuration and once they're proposed they're automatically deployed, active and start running to test new changes17:07
fungier, once they're approved17:07
zxiiroah ok so I guess each jenkins master receives the jobs via a JJB script that updates them when someone pushes in a change17:07
zxiiroor do you push it to a single master which lets zuul/gearman distribute them out?17:07
fungizxiiro: more or less, yes17:07
*** talluri has joined #openstack-infra17:08
*** kgiusti has joined #openstack-infra17:08
fungizxiiro: we use ansible to trigger puppet agent on each jenkins master to retrieve the full body of configuration and then apply it locally on them if changed17:08
zxiirook, yeah that's the piece of the puzzle I've been confused about. Good to know thanks :)17:08
*** thedodd has joined #openstack-infra17:09
*** rwsu-afk is now known as rwsu17:09
fungiso all jobs get created on all masters, but then they only individually register into zuul's gearman the jobs they have workers capable of running17:09
zxiiroi wasn't clear on if only one master had all the jobs and gearman just somehow passes them along, or if you actually go to every master and push jobs to them17:09
zxiiroso that explains it :)17:09
zxiirovery cool17:10
fungifor the most part they all have representatives of all our worker types, but we do have one jenkins master which has separately privileged workers to do things that require service credentials (like uploading releases)17:10
*** markvan has quit IRC17:10
*** carl_baldwin has joined #openstack-infra17:10
*** sarob has quit IRC17:10
fungibecause the jenkins master/slave design counter-intuitively doesn't really provide any security separation and lets slaves run some things on masters17:11
fungihence our isolated master for more sensitive jobs17:11
*** annegent_ has quit IRC17:12
*** sarob has joined #openstack-infra17:12
fungiwhere none of its slaves run arbitrary proposed changes, but rather only trusted code17:12
*** Ala has quit IRC17:12
dprincejeblair: hi, any chance we could get a quick turn around on https://review.openstack.org/#/c/153341/17:12
zxiirooh i see17:12
fungizxiiro: honestly, we were all rather surprised to discover not only that jenkins masters let slaves tell them to run arbitrary things, but that it was actually an intentional design choice17:13
odyssey4mea general question - is it possible to do multi-node build tests within openstack-infra? if so, are there examples I can look at - the only reference to multinode appears to be a nodepool script17:13
dstufftfungi: jenkins is not good software17:13
*** vigneshvar has quit IRC17:14
fungidstufft: we're weaning ourselves off of it, thankfully, so we get to stop worrying about that at some point17:14
zxiirogood to know. I don't think we've realized that yet so I'll have to raise it with our infra team17:14
clarkbodyssey4me: it is possible let me get an example17:14
dstufftfungi: to turbohipster? or something else?17:14
zxiiroanyway I need to head out for some lunch. Have a wonderful day everyone :)17:14
fungidstufft: probably turbohipster, yes. there are some changes in flight to get a turbohipster gearman worker up and running17:14
clarkbodyssey4me: https://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n165717:15
*** carl_baldwin_ has joined #openstack-infra17:15
*** sarob has quit IRC17:15
*** carl_baldwin has quit IRC17:15
*** carl_baldwin_ is now known as carl_baldwin17:15
clarkbfungi: I was just reviewing one of jhesketh's changes to nodepool to better support turbohipster, not sure I fully grok how it helps yet17:15
*** salv-orlando has quit IRC17:16
clarkbodyssey4me: note that the job itself isn't all that different, just tells devstack-gate to run on multiple nodes17:16
odyssey4methanks clarkb - it looks pretty much like any other job17:16
odyssey4mewhat's the special suace that makes it go?17:16
*** abhirc has quit IRC17:17
clarkbodyssey4me: devstack-gate17:17
clarkbodyssey4me: it switches on the TOPOLOGY flag there17:17
zxiirofungi: actually before I go, do you have a link or some sort of article that explains the issue with Jenkins Master / Slaves running arbitrary code?17:18
fungizxiiro: let me dig it up17:18
*** jyuso has quit IRC17:18
*** pczesno has quit IRC17:18
*** jyuso has joined #openstack-infra17:18
*** heyongli has quit IRC17:18
fungizxiiro: the jenkins documentation has some details https://wiki.jenkins-ci.org/display/JENKINS/Slave+To+Master+Access+Control17:18
fungizxiiro: looks like they've (recently) started to introduce features to limit that behavior17:19
*** fabo has quit IRC17:19
*** koolhead17 has joined #openstack-infra17:19
openstackgerritAdam Gandelman proposed openstack-infra/shade: Adds get_network() and list_networks function  https://review.openstack.org/15064917:20
*** hdd has quit IRC17:20
*** achuprin_ has quit IRC17:20
*** rwsu has quit IRC17:20
odyssey4meclarkb: so that makes it use the 'devstack-trusty-2-node' zuul node17:21
*** fabo has joined #openstack-infra17:21
*** fabo has joined #openstack-infra17:21
fungizxiiro: the releases they mention there are ~3-4 months new17:21
*** belmoreira has quit IRC17:21
zxiirofungi: thanks!17:22
fungizxiiro: sure thing. let us know if there's anything else we can do to help!17:22
odyssey4meclarkb does the name imply that it's two nodes, so not really 'more than one node, up to x nodes'?17:22
*** ssam2 has quit IRC17:22
*** heyongli has joined #openstack-infra17:23
*** pczesno has joined #openstack-infra17:23
clarkbodyssey4me: the name implies 2 nodes let me get that node selection code17:23
morganfainbergfungi, we can move the keystone logs back to apacheformat17:23
morganfainbergit's easy17:23
*** nicodemos has joined #openstack-infra17:23
fungiodyssey4me: we have to define the node count in nodepool since the group of subnodes are built together and configured to have access to one another17:23
*** achuprin_ has joined #openstack-infra17:23
clarkbodyssey4me: https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/openstack_functions.py#n5717:23
morganfainbergbut i jumped through hoops to make it oslo format for ease of use17:23
*** Ryan_Lane has joined #openstack-infra17:23
clarkbmorganfainberg: no no no please no17:23
morganfainbergfungi, happy to help reverting that if you'd like. :)17:23
*** zz_jgrimmn is now known as jgrimm17:23
clarkbapache format is terrible for applications17:23
fungimorganfainberg: no need to change, i was just (pleasantly) surprised17:23
clarkbits good for serving files17:23
clarkbterrible for applications17:23
morganfainbergclarkb, i figured you didn't want apache format17:23
morganfainberg;)17:23
*** e0ne has quit IRC17:24
morganfainbergfungi, we pushed for a apache 2.4 [and code in devstack] for that17:24
morganfainbergfungi, trusty was a big win for us17:24
fungiindeed17:24
morganfainbergon that front17:24
fungi2.4 has some very nice features17:24
morganfainberglike configurable access log format :P17:24
*** armax has quit IRC17:25
fungijust unfortunate that it breaks a fair amount of backward compatibility from 2.217:25
morganfainbergand error log format17:25
morganfainbergor stuff17:25
clarkbmorganfainberg: I rarely care about bytes transfered unless the thing I am dealing with is files :)17:25
fungibut gotta break a few eggs to make a better webserver17:25
clarkband so on17:25
morganfainbergfungi, hehe17:25
*** carl_baldwin has quit IRC17:25
*** niska has joined #openstack-infra17:25
morganfainbergwe could probably make horizon formats like oslo in 2.417:25
clarkbjhesketh: I have reviewed 98306 without scoring it, curious to hear what you have to say about my comment there (and I realize it is your weekend so ignore me until monday)17:25
fungiand after you've "fixed" a few dozen vhost configs from 2.2 to work with 2.4, it's not so hard17:26
clarkbjeblair: you will probably be interested in 98306 as well17:26
*** koolhead17 has quit IRC17:26
*** davideagnello has quit IRC17:27
*** jpich has quit IRC17:28
*** sigmavirus24_awa is now known as sigmavirus2417:28
*** marun has joined #openstack-infra17:28
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth Authorize Scope Validation  https://review.openstack.org/15340617:29
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth tests for authorize_return  https://review.openstack.org/15340717:29
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth validates redirect_uri  https://review.openstack.org/15340517:29
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth check for no response_type  https://review.openstack.org/15310717:29
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth checks for remote data access permissions.  https://review.openstack.org/15340917:29
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth Exception on invalid oauth return data  https://review.openstack.org/15340817:29
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth check for no client_id  https://review.openstack.org/15310817:29
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth check for response_type 'code'  https://review.openstack.org/15306917:29
krotscheckAnyone want to help kick off a merge tornado? https://review.openstack.org/#/c/153069/17:30
*** thedodd has quit IRC17:31
*** thedodd has joined #openstack-infra17:31
*** shashankhegde has joined #openstack-infra17:32
fungimergenado17:33
fungiit's the git equivalent of a sharknado17:33
*** davideagnello has joined #openstack-infra17:35
mordredfungi: what if there were sharks???17:36
clarkbmordred: see comment on 12662117:36
fungimordred: then i'd want to know who committed sharks into a git repo17:37
mordredfungi: I'd want to know who _didn't_ commit sharks into a git repo17:37
fungibecause that seems patently irresponsible17:37
fungiand we're nothing but upstanding, responsible people up in here17:37
*** vigneshvar has joined #openstack-infra17:37
*** sarob has joined #openstack-infra17:37
JayF$infra folks; https://review.openstack.org/153630 is a proposed patch to fix failing Tempest tests for Ironic. Is there anything you guys can do to promote the tests on the latest patchset so we can make sure it fixes it + get the gate for Ironic unstuck?17:38
mordredclarkb: oh - well, group/groups are values that are magical to the ansible inventory17:40
mordredclarkb: that said - the image_name in the groups param is out of date and not needed17:41
fungikrotscheck: lgtm, approve at will17:41
mordredansible now knows how to just infer that17:41
mordredclarkb: I can update that17:41
JayFThe tests on that seem to be running now already, without help fwiw, ty though anyway :)17:41
clarkbmordred: well imo we don't need any of it :)17:42
*** armax has joined #openstack-infra17:42
fungiJayF: yay, we're helping!17:42
clarkbmordred: we don't actually run ansible commands across our 900 slaves ever, its usually best to rebuild the images17:42
*** ghostpl has quit IRC17:42
*** shashankhegde has quit IRC17:42
JayFfungi: Hey; it's the mechanic effect. If you want to make your car stop making the funny noise, take the mechanic for a ride, right?17:42
clarkbmordred: but it doesn't hurt I was just curious about the redundancy. Why is image_name not needed?17:42
fungiJayF: $1 for hitting it with a hammer and $499 for knowing where to hit it17:43
krotscheckfungi: Thanks!17:43
clarkbfungi: :) though I think the sum of money was much larger in the original story17:43
* krotscheck gets on the merge train!17:43
clarkbfungi: also chalk17:43
*** radez is now known as radez_g0n317:43
*** rfolco has quit IRC17:43
fungiclarkb: true17:43
openstackgerritMerged openstack-infra/storyboard: OAuth check for response_type 'code'  https://review.openstack.org/15306917:44
*** bnemec is now known as beekneemech17:44
*** ZZelle has quit IRC17:44
*** sarob has quit IRC17:44
*** rfolco has joined #openstack-infra17:44
*** SumitNaiksatam has quit IRC17:45
mordredclarkb: image_name can and is inferred by the ansible inventory logic now17:45
mordredclarkb: so - not ansible could do the same17:45
clarkbmordred: gotcha17:45
openstackgerritMerged openstack-infra/system-config: update indexer for apache logs  https://review.openstack.org/15362717:45
mordredclarkb: the main thing here is not that we do use ansible - but that there is this place we can put metadata about our servers17:45
clarkbmordred: ya17:45
mordredso that things poking around could ... OH - wait - I know why I added image_name17:45
mordredbecause that's the logical image name from nodepool17:46
*** SumitNaiksatam has joined #openstack-infra17:46
*** ChuckC has quit IRC17:46
clarkbmordred: I was also curious about the providers since that should be implicit but then realized that we map many nodepool providers onto specific cloud providers so it isn't 1:117:46
mordredwhich is not the same as the image_name that the cloud is going to know17:46
mordredyah17:46
* mordred remembers why he added those things ...17:46
clarkbya so its probably all fine as sis17:46
clarkb+217:46
mordredit's basically to put the things in to nova metadata that right now you have to run nodepool db commands to find out17:46
*** annegent_ has joined #openstack-infra17:47
fungiare we gaining renewed confidence in stability/availability of nova metadata in our providers?17:47
clarkbfungi: different metadata17:48
fungioh, not the things you query from the metadata service?17:48
clarkbfungi: there is the nova metadata service which cloud init uses to set ssh and so forth. That particular thing isn't really reliable (as multinode testing has recently shown)17:49
openstackgerritMerged openstack-infra/storyboard: OAuth check for no response_type  https://review.openstack.org/15310717:49
*** rwsu has joined #openstack-infra17:49
clarkbfungi: the other metadata is when booting a new node you can attach properties to it as metadata that is queriable with novaclient17:49
openstackgerritMerged openstack-infra/storyboard: OAuth check for no client_id  https://review.openstack.org/15310817:49
clarkbfungi: that should be reliable17:49
*** salv-orlando has joined #openstack-infra17:49
fungigot it. metadata for the nova api calls vs metadata for the instances to consume17:49
fungithough presumably one flows into the other17:50
clarkbmaybe? its the VM consumption that is particularly problematic17:50
*** ZZelle has joined #openstack-infra17:50
clarkbbecause it uses the 169.254.169.254 address which is NATed to the nova api server which doesn't seem to work right17:50
openstackgerritMerged openstack-infra/storyboard: OAuth validates redirect_uri  https://review.openstack.org/15340517:50
clarkblike its fine on one node not on two17:50
fungisince i thought you could set metadata properties on an instance via the api boot call which the instance could then later query from the metadata service to know special pieces of info17:51
fungibasically attaching metadata for consumption by the instance's operating system/userspace17:51
clarkbya it may do that (I am not sure)17:51
*** luqas has quit IRC17:51
*** achanda has joined #openstack-infra17:51
fungibut i guess we're just using it for a handy place to stick info we want to query from somewhere nottheinstance later17:52
clarkbthe whole NAT thing seems questionable though. I wonder why using the network gateway wasn't deemed a better solution to the problem17:52
clarkbits a common solution for not needing to discover where to ntp and the like17:52
fungigatewayless networks17:53
clarkbI don't think nova network does though17:53
clarkb*does those17:53
*** sarob has joined #openstack-infra17:53
clarkbbut I may be wrong17:53
fungiahh. no good guess then17:53
greghaynesclarkb: did you end up figuring out your issues with the NAT stuff?17:53
clarkbgreghaynes: nope fixed nova shelve on with nova mutlihost and have not looked back17:53
*** sarob has quit IRC17:53
clarkbgreghaynes: I asked nova for help and was told "I doubt anyone is interested in fixing that"17:54
clarkbso ya17:54
greghayneswell, anyone other than you ;)17:54
*** sarob has joined #openstack-infra17:54
openstackgerritMerged openstack-infra/storyboard: OAuth Authorize Scope Validation  https://review.openstack.org/15340617:55
openstackgerritKhai Do proposed stackforge/python-jenkins: Set the default server connection timeout to match socket default timeout  https://review.openstack.org/15346717:55
*** bhuvan has joined #openstack-infra17:55
*** denis_makogon has quit IRC17:57
openstackgerritMerged openstack-infra/storyboard: OAuth tests for authorize_return  https://review.openstack.org/15340717:57
openstackgerritMerged openstack-infra/storyboard: OAuth Exception on invalid oauth return data  https://review.openstack.org/15340817:57
openstackgerritKhai Do proposed stackforge/python-jenkins: Set the default server connection timeout to match socket default timeout  https://review.openstack.org/15346717:57
openstackgerritMerged openstack-infra/nodepool: Change sequence on actions in updateConfig  https://review.openstack.org/13081117:58
*** pelix has quit IRC17:58
*** sarob has quit IRC17:58
*** niska has quit IRC17:59
clarkbgreghaynes: the stack that starts at https://review.openstack.org/#/c/153093/ fixes nova multihost so once we get those first three changes in we should be working again18:00
*** mrmartin has joined #openstack-infra18:01
greghaynesfun fun18:01
*** ChuckC has joined #openstack-infra18:02
openstackgerritKhai Do proposed stackforge/python-jenkins: Set the server connection timeout to same as socket timeout  https://review.openstack.org/15346718:02
*** gyee has joined #openstack-infra18:03
openstackgerritMerged openstack-infra/nodepool: Caching floating ips as server list is done  https://review.openstack.org/13342218:04
clarkbmordred: for 136589 nodepool alembic migrations, can't alembic generate the migrations from your db schema? it always bugs me that we have to double account this stuff18:04
*** jp_at_hp has quit IRC18:06
*** derekh has quit IRC18:06
fungifirst experiment, i have two held bare-centos6 nodes one with the ghost update and the other without. i'm looping git remote update in a nova clone (along with a one-second sleep) and seeing if either or both get stuck18:06
*** thedodd has quit IRC18:06
greghaynesclarkb: usually you commit the alembic migration along with your schema change...18:07
clarkbgreghaynes: yes but you shouldn't have to18:07
clarkbI only bring this up because mordred's commit to add alembic appears wrong18:07
clarkbbecause it has to be double accounted so you make mistakes18:07
clarkbI am commenting on the change as I go18:08
*** dtantsur is now known as dtantsur|afk18:08
clarkbit could also be that alembic and sqlalchemy use different defaults18:09
clarkbwhich would be crazy18:09
greghaynesmaybe im missing something, but generally you write a lot of the alembic migration by hand, it just autogens the hash and such18:09
* greghaynes reads teh codes18:09
*** patrickeast has joined #openstack-infra18:09
clarkbgreghaynes: yes but if you change your python code there is sufficient info present to autogenerate a migration fromold version to new version in the general case18:09
clarkbeg I added a column18:10
clarkbor I added an index18:10
greghaynesyea, for the simple case, a lot of the time theres data mutation too though18:10
greghayneswhich is such fun18:10
clarkbright of which we have 018:11
openstackgerritBoris Pavlovic proposed openstack-infra/project-config: Add Rally job to Mistral  https://review.openstack.org/15364318:12
*** swamireddy has quit IRC18:13
anteayaoh my goodness, I finished reading backscroll18:14
*** mrmartin has quit IRC18:14
anteayait's almost time for lunch18:14
*** harlowja has joined #openstack-infra18:15
*** otter768 has joined #openstack-infra18:16
anteayaclarkb: in the parent patch of the d-g patch series for multinode is 172.24.4/23 a typo or intentional?18:17
* anteaya displays her lack of knowledge in ip address syntax conventions18:17
fungianteaya: intentional18:17
anteayathanks18:17
fungianteaya: that means the range of 412 ipv4 addresses from 172.24.4.0 to 172.24.5.25518:18
fungier, 512 addresses18:18
* fungi is suffering from a prolonged case of stupidfingers18:18
clarkbit signifies that 23 of the 32 bits in the address are fixed. The other 9 are useable18:19
anteayayour stupid fingers give me comfort18:19
clarkbanteaya: ^ if you want to use maths18:19
anteayaclarkb fungi thanks18:19
anteayait is another wonderful til day, I also learned about jenkins security in the backscroll18:20
anteayaor lack thereof18:20
*** yjiang5 has left #openstack-infra18:21
*** tkelsey has quit IRC18:21
fungianteaya: basically 2^(32-23)=51218:25
*** sarob has joined #openstack-infra18:25
*** shashankhegde has joined #openstack-infra18:25
*** otter768 has quit IRC18:26
anteayathere's some math18:26
anteayathanks18:26
*** e0ne has joined #openstack-infra18:26
anteayaclarkb: how should I interpret that the patch is failing in the experiemental pipeline?18:27
*** dizquierdo has quit IRC18:27
*** hdd has joined #openstack-infra18:27
clarkbgah sqlalchemy docs don't specify what the default was for the unique kwarg18:27
anteayathe patch being https://review.openstack.org/#/c/153093/18:28
zarofungi: was wondering if you know why gerrit user for zuul-dev.o.o is 'zuul-dev' and not 'jenkins'? it's 'jenkins' for zuul.o.o18:28
clarkbanteaya: there are three changes at the beginning of that stack, the first two will fail but the third passes. we basically need all three for it to pass18:28
zarofungi: http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n54318:28
anteayaclarkb: ah okay thanks18:28
clarkbanteaya: I kept them separate since I don't have to squash them to get it merged and it makes it more readable18:28
*** markvan has joined #openstack-infra18:28
*** markvan has quit IRC18:28
anteayaclarkb: I could barely read the first as it is, so thanks for the not squash decision18:28
*** jerryz has quit IRC18:28
zarofungi: both users are in the 'Continuous integration tools' group on review-dev.o.o18:29
*** bswartz has quit IRC18:30
*** signed8b_ is now known as signed8bit_ZZZzz18:30
clarkbmordred: I commented on 136589. +2 but with a small concern about a delta between nodedb and the migration18:30
clarkbanteaya: linux networking, its about the worst thing about linux18:30
anteayaclarkb: +1 on the parent with an explaination of what that means coming from me18:30
anteayaclarkb: networking seems to be the worst thing about everything18:30
zarofungi: zuul-dev.o.o layout is set to trigger on comments from 'jenkins' user, similar to zuul.o.o's layout.yaml18:31
zarohttp://git.openstack.org/cgit/openstack-infra/project-config/tree/dev/zuul/layout.yaml18:31
openstackgerritBoris Pavlovic proposed openstack-infra/project-config: Add Rally job to Mistral  https://review.openstack.org/15364318:32
fungizaro: not sure, but we could certainly change that easily enough18:32
mordredclarkb: nod18:32
*** carl_baldwin has joined #openstack-infra18:32
zarofungi: i'm thinking it would be better to use the same user for both zuul.o.o and zuul-dev.o.o that will make mainting the layout files easier.18:32
fungizaro: mainly i think we've just not historically put a lot of effort into getting those dev systems into a state where they're an end-to-end recreation of our toolchain18:33
fungiso there's likely to be some divergence here and there as a result, but i'm not opposed to cleaning that up18:33
zarofungi: or keep it as zuul-dev user and change the layout file to match.18:33
zarowhat would you suggest?18:34
fungizaro: i'd say let's update the account name to match production. i suspect previously when zuul-dev was connecting to review.o.o instead it used a different username so as not to conflict with the accout production zuul was using18:34
anteayawhat is q-agt in devstack's service nomenclature?18:34
anteayaI'm guessing q for quantum, no idea for agt18:35
JayFagent?18:35
*** _nadya_ has quit IRC18:35
zarofungi: cool.  so if we change zuul-dev to jenkins user would it use the same ssh rsa pub key as one for prod?18:36
JayFI honestly don't even know what quantum does; but agt has to be agent, right?18:36
anteayaJayF: quantum is the old name for neutron, devstack keeps using q to avoid clashing with nova which has n-18:36
JayFaha; and there's a neutron agent, so that fits18:36
anteayaJayF: I can buy agent for agt18:36
marcusvrn1Hi all! I'm facing some troubles in the CI jobs execution...Sometimes CI jobs starts failing, because the installed dependencies are not up-to-date. I have installed all openstack dependencies in order to optimize the jobs, but now I have some problems with that approach.. I guess the devstack script checks if the dependency was installed but not if it's the right version. Is there a way to force the job scripts (devst18:36
*** bswartz has joined #openstack-infra18:36
anteayaJayF: thank you18:37
fungizaro: no, we'd just need to update the public key for the account jenkins in the review-dev trove database so that it's the same as the zuul-dev account18:37
JayFanteaya: np; putting the hours I spend on crosswords to work :P18:37
anteayaJayF: I can't do crosswords at all, glad you can18:37
anteayamarcusvrn1: hello18:37
zarofungi: do you suggest replacing or adding an additional pub key?18:37
*** tonytan4ever has quit IRC18:37
*** dannywilson has quit IRC18:37
anteayamarcusvrn1: can you help us see the problems you are having? like perhaps by pasting some output on paste.openstack.org?18:38
*** tonytan4ever has joined #openstack-infra18:38
fungizaro: either is fine. we can always recreate the corresponding public key from a private key if we forget what it was (or just regenerate the keys entirely if we have to)18:38
*** dannywilson has joined #openstack-infra18:38
marcusvrn1anteaya: sure! one minute18:38
*** mrmartin has joined #openstack-infra18:39
*** jogo is now known as flashgordon18:40
anteayamarcusvrn1: thanks18:40
*** niska has joined #openstack-infra18:41
*** dannywilson has quit IRC18:41
*** dannywilson has joined #openstack-infra18:42
*** jedimike has quit IRC18:42
*** hdd has quit IRC18:42
zarofungi: ok, i'll add 'zuul-dev' pub key to 'jenkins' account on review-dev.o.o and change the 'gerrit_user' in site.pp18:42
fungizaro: sounds good18:43
fungii'm going to step away and go get some late lunching, but will be back later18:43
clarkbjesusaurus: 145930, nothing seems to consume the version info? I want to say this was discussed at one point but the commit itself is lacking ocntext18:44
clarkbjesusaurus: any chance you can fill ianw and I in?18:44
*** esker has joined #openstack-infra18:45
*** tonytan4ever has quit IRC18:47
*** tonytan4ever has joined #openstack-infra18:47
marcusvrn1anteaya: I'm reproducing it again, because, to fix it temporally, I have updated all dependencies manually.18:48
*** bdpayne has joined #openstack-infra18:48
anteayaokay so let's start again18:49
anteayawhat is the problem you are looking to solve?18:50
*** hdd has joined #openstack-infra18:50
openstackgerritMerged openstack/requirements: Add aioeventlet dependency  https://review.openstack.org/13875018:51
anteayamarcusvrn1: what is the problem you are looking to solve?18:52
*** abhirc has joined #openstack-infra18:52
clarkbmarcusvrn1: so you should let devstack install all of the things it wants to install. What you can do is cache the packages18:54
*** niska has quit IRC18:54
marcusvrn1anteaya: If I have a devstack dependency already installed, but not updated...for example: I have installed oslo.i18n version 0.2 and the required package is oslo.i18n version 1.0, the devstack considers it is installed, but show an error that the package version installed is not the expected one.18:57
*** imcsk8 has quit IRC18:57
marcusvrn1clarkb: but it consumes a lot of time to install each package every time that job runs18:58
jesusaurusclarkb: I was trying to figure out why "nodepool --version" gives me "error: too few arguments" and it looks like other projects have something similar to 14593018:58
*** dhellmann has quit IRC18:58
*** dhellmann_ is now known as dhellmann18:58
jesusaurusthough I think fungi was telling me that they are two separate issues18:58
*** dhellmann_ has joined #openstack-infra18:59
*** zz_avozza is now known as avozza18:59
anteayamarcusvrn1: when you set up the environment for your devstack, what did you do?18:59
*** sputnik13 has joined #openstack-infra18:59
*** ivar-lazzaro has joined #openstack-infra18:59
clarkbmarcusvrn1: if you cache the packages the time to install is reasonable18:59
*** sarob has quit IRC19:00
*** sarob has joined #openstack-infra19:00
openstackgerritSean Dague proposed openstack-infra/project-config: ec2api tempest job  https://review.openstack.org/15365919:00
sdaguejeblair: so... I *think* that does the right thing ^^^19:01
clarkbjesusaurus: https://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/cmd/nodepoolcmd.py#n319 I see that expects the info that you add19:01
*** dhellmann_ has quit IRC19:01
clarkbjesusaurus: maybe a comment with a pointer to that chunk of code?19:01
*** dhellmann_ has joined #openstack-infra19:01
sdagueclarkb: your eyes on that would be great as well19:01
jesusaurusclarkb: sure, I'll write a better commit message19:02
jeblairsdague: is the NO_CLONE or whatever it's called still effective for enable_plugin?19:02
sdaguejeblair: yep19:03
sdagueit uses the normal devstack clone paths19:03
jeblairsdague: (i ask because i agree having the url for docs purposes is cool, but want to make sure we don't accidentally clone something).19:03
jeblairsdague: cool, sounds good19:03
*** SumitNaiksatam has quit IRC19:03
*** johnthetubaguy is now known as zz_johnthetubagu19:03
*** mattfarina has quit IRC19:04
marcusvrn1antena: I looked for which packages that devstack needs and then, I installed them19:04
openstackgerritMerged openstack-infra/project-config: Add TripleO f20 puppet job  https://review.openstack.org/15334119:04
marcusvrn1anteaya: I looked for which packages that devstack needs and then, I installed them19:04
*** mattfarina has joined #openstack-infra19:04
*** sarob has quit IRC19:05
marcusvrn1clarkb: what is a reasonable time for each CI build?19:05
clarkbmarcusvrn1: are run in about an hour19:05
clarkbs/are/ours/19:05
clarkbsdague: and the plugin does all configuration itself? thats nice19:06
clarkbsdague: just looks pretty light so double checking19:06
openstackgerritK Jonathan Harker proposed openstack-infra/nodepool: Add version_info to nodepool  https://review.openstack.org/14593019:06
sdagueclarkb: yep, that's the specification for external plugins19:06
clarkbperfect19:07
jesusaurusclarkb: does that make commit message make sense to you?19:07
anteayamarcusvrn1: ah okay well that is one approach and you are indicating that is perhaps not optimal for your needs19:07
sdaguehttps://github.com/stackforge/ec2-api/tree/master/devstack - it's 2 files, a settings file that gets sourced early, then a dispatcher for various devstack phases19:07
anteayamarcusvrn1: having you tried letting devstack install what it needs itself as clarkb suggested?19:07
clarkbsdague: so I think you just need to add that job template to the project in projects.yaml to instantiate the template19:07
*** carl_baldwin has quit IRC19:07
clarkbjesusaurus: yup thanks19:08
*** yamamoto has joined #openstack-infra19:08
openstackgerritK Jonathan Harker proposed openstack-infra/nodepool: Add version_info to nodepool  https://review.openstack.org/14593019:08
sdagueclarkb: in the jobs: section?19:08
clarkbsdague: ya19:08
clarkbsdague: basically templates don't exist until you listthem in a jobs section instantiating them with a set of parameters19:09
*** ashleighfarnham has quit IRC19:09
*** dhellmann has quit IRC19:09
*** dhellmann_ is now known as dhellmann19:09
jeblairalso, if we could work out the tempest/neutron thing, we could probably make that builder a macro, or even make that a template where you instantiate it with the add-on project.  (these should be considered future refactors; don't need to do that now)19:10
jesusaurusclarkb: also, i have another trivial rebase (imports again) of 7605719:10
openstackgerritSean Dague proposed openstack-infra/project-config: ec2api tempest job  https://review.openstack.org/15365919:11
sdaguejeblair: yeh, I agree, we should make this a simpler template in the future19:11
sdagueclarkb: like that ^^^19:11
clarkbsdague: yup19:11
sdagueprojects.yaml often confuses me, so tell me where I got that wrong :)19:11
*** signed8bit_ZZZzz is now known as signed8b_19:11
*** thedodd has joined #openstack-infra19:12
openstackgerritYichen Wang proposed openstack-infra/project-config: Modify zuul jobs for stackforge/vmtp  https://review.openstack.org/15366419:12
*** alexpilotti has quit IRC19:12
marcusvrn1anteaya: clarkb: Yeah, It was set up letting devstack install what it needs...but as I was trying to optimize the build time, I installed the devstack dependencies....maybe I'm trying to optimize the Builds more than needed... mine build runs in ~45 min (with devstack dependencies installed). If the dependencies are not installed, maybe it will run in an hour19:12
clarkbsdague: its the difference between a job: and a job-template: job-templates are just containers that need to be filled19:12
*** yamamoto has quit IRC19:13
*** dustins has quit IRC19:13
anteayamarcusvrn1: sounds like you optimized yourself into a corner19:13
*** ashleighfarnham has joined #openstack-infra19:13
anteayawhy not try again letting devstack install what it needs19:13
anteayathen check what is installed and cache those packages?19:14
clarkbjesusaurus: your fix doesn't fix nodepool --version fwiw19:15
*** annegent_ has quit IRC19:15
clarkbjesusaurus: I think you also need to make the subcommand optional when using --version19:15
jeblairclarkb, sdague: double check my -1 on that... :)19:16
*** talluri has quit IRC19:16
marcusvrn1anteaya: I will do it now, after talking to you guys, I guess it's the best solution :)19:16
anteayamarcusvrn1: let us know how you make out19:16
*** talluri has joined #openstack-infra19:17
sdagueI'll let clarkb tell me the right thing there :)19:17
anteayamarcusvrn1: and we enjoy looking at stacktraces if you have an issue in future19:17
sdaguehappy to delete it19:17
openstackgerritAndrey Pavlov proposed openstack-infra/project-config: template for ec2-api gating jobs  https://review.openstack.org/15366519:17
marcusvrn1clarkb: anteaya: ok! thanks19:18
clarkbsdague: jeblair ya the job group isn't used so can just be removed19:18
clarkbjesusaurus: I am not sure how to do that with argparse off the top of my head19:18
jeblairsdague: this kitchen has a lot of cooks: https://review.openstack.org/15366519:18
sdagueyeh, I didn't realize they were running at this as well19:19
sdagueI don't know if andrey is on irc or not19:19
jesusaurusclarkb: ah19:19
anteayayay the first part of the nova-net to neutron migration spec merged!19:19
jesusaurusclarkb: that makes sense, I'll see if I can figure out how to do that19:19
sdaguehe had asked me how to do this, so I was trying to get it figured out this morning19:19
clarkbjesusaurus: you do need the version info thing too aiui so don't get rid of that19:19
* anteaya does a happy dance19:19
*** talluri has quit IRC19:21
*** dustins has joined #openstack-infra19:22
*** _buzz has joined #openstack-infra19:22
anteayaI'm going to celebrate by going for a walk19:23
*** aysyd has quit IRC19:24
sdaguejeblair: so we think we're good with getting rid of just the job group?19:24
*** aysyd has joined #openstack-infra19:24
openstackgerritSean Dague proposed openstack-infra/project-config: ec2api tempest job  https://review.openstack.org/15365919:25
*** cdent has quit IRC19:25
sdaguemy suggestion is we land that as the example, the ec2-api team can then also add their functional jobs mirroring that19:26
sdagueand it will be a good base commit to get the glusterfs folks to use19:26
sdagueclarkb: agree?19:26
clarkb+119:27
*** e0ne has quit IRC19:28
*** SumitNaiksatam has joined #openstack-infra19:29
*** nicodemos has quit IRC19:29
*** Andrey-mp has joined #openstack-infra19:29
*** alevine has joined #openstack-infra19:30
*** tqtran has joined #openstack-infra19:31
*** mrmartin has quit IRC19:32
*** vigneshvar has quit IRC19:33
*** sdake_ has joined #openstack-infra19:33
*** vigneshvar has joined #openstack-infra19:34
*** avozza is now known as zz_avozza19:34
*** thedodd has quit IRC19:34
openstackgerritMerged openstack-infra/nodepool: Revert "Pin python-daemon to <2.0"  https://review.openstack.org/15025619:35
*** tonytan4ever has quit IRC19:36
*** andreykurilin_ has joined #openstack-infra19:38
*** amrith is now known as _amrith_19:38
sdagueclarkb / jeblair - thanks19:38
sdaguealso documented this out on the ML for people - http://lists.openstack.org/pipermail/openstack-dev/2015-February/056257.html19:39
sdaguehopefully we can point more folks there19:39
SpamapShttp://paste.openstack.org/show/168273/19:42
SpamapS^^ thats where my image build failed19:42
SpamapSError: Could not update: Execution of '/usr/bin/gem install --no-rdoc --no-ri puppetlabs_spec_helper' returned 1: ERROR:  While executing gem ... (Gem::DependencyError) Unable to resolve dependencies: rspec requires rspec-core (~> 2.99.0), rspec-expectations (~> 2.99.0), rspec-mocks (~> 2.99.0)19:43
Andrey-mpsdague - thanks for example19:43
sdagueAndrey-mp: no problem19:43
sdagueAndrey-mp: happy to get you guys bootstrapped here19:44
*** achanda has quit IRC19:44
openstackgerritKhai Do proposed openstack-infra/system-config: Change the gerrit_user on zuul-dev.o.o to match zuul.o.o  https://review.openstack.org/15366819:44
*** koolhead17 has joined #openstack-infra19:44
openstackgerritYichen Wang proposed openstack-infra/project-config: Modify zuul jobs for stackforge/vmtp  https://review.openstack.org/15366419:45
*** alevine has quit IRC19:47
*** ashleighfarnham has quit IRC19:47
*** bradjones has quit IRC19:47
*** mrmartin has joined #openstack-infra19:47
*** mrmartin has quit IRC19:49
*** signed8b_ is now known as signed8bit_ZZZzz19:50
*** bradjones has joined #openstack-infra19:50
*** radez_g0n3 is now known as radez19:50
sdaguejeblair: zuul failed on the last rev of the ec2 patch - http://logs.openstack.org/59/153659/3/check/gate-project-config-layout/86c7637/console.html19:52
*** niska has joined #openstack-infra19:52
*** achanda has joined #openstack-infra19:53
jeblairthat's... weird19:54
jeblairthe cited lines are not near the changes19:55
*** sigmavirus24 is now known as sigmavirus24_awa19:55
sdagueit failed on rev2 and rev319:55
sdaguein same way19:55
jeblairoh, i see19:55
openstackgerritYichen Wang proposed openstack-infra/project-config: Modify zuul jobs for stackforge/vmtp  https://review.openstack.org/15366419:55
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Diagram the nova net multinode network setup  https://review.openstack.org/15367219:55
clarkbsdague: ^ I put it in a second commit so that I wouldn't lose your +2 <_< but how does that diagram look?19:56
jeblairsdague: commented on 15365919:56
jeblair(and the error did actually make sense, i was looking at the zuul files, not the jjb)19:57
sdagueok, cool. rev coming19:57
*** gondoi is now known as zz_gondoi19:57
openstackgerritSean Dague proposed openstack-infra/project-config: ec2api tempest job  https://review.openstack.org/15365919:57
*** signed8bit_ZZZzz is now known as signed8b_19:58
sdagueclarkb: thanks19:58
*** Andrey-mp has quit IRC20:00
*** annegent_ has joined #openstack-infra20:01
*** tiswanso_ has quit IRC20:01
*** thedodd has joined #openstack-infra20:02
*** ashleighfarnham has joined #openstack-infra20:02
*** tiswanso has joined #openstack-infra20:02
*** amitgandhinz has quit IRC20:02
*** rfolco has quit IRC20:03
*** amitgandhinz has joined #openstack-infra20:03
*** hashar has joined #openstack-infra20:03
*** signed8b_ has quit IRC20:05
*** signed8bit has joined #openstack-infra20:05
*** signed8bit has quit IRC20:06
*** signed8bit has joined #openstack-infra20:06
sdagueclarkb / jeblair - https://review.openstack.org/#/c/153659/ now with passing tests20:07
*** signed8bit has quit IRC20:07
*** signed8bit has joined #openstack-infra20:10
*** amitgandhinz has quit IRC20:10
jeblair+320:11
openstackgerritK Jonathan Harker proposed openstack-infra/nodepool: Fix nodepool --version  https://review.openstack.org/14593020:13
*** ldnunes has quit IRC20:14
*** ihrachyshka has joined #openstack-infra20:15
*** niska has quit IRC20:15
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth Validate Token Grant Types  https://review.openstack.org/15367420:17
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth tests for invalid authorization codes.  https://review.openstack.org/15367520:17
openstackgerritMerged openstack-infra/project-config: ec2api tempest job  https://review.openstack.org/15365920:18
sdaguedid something change today about how unit test nodes are setup?20:18
sdaguehttp://logs.openstack.org/59/152959/2/check/gate-devstack-unit-tests/7689185/console.html#_2015-02-06_19_12_52_31020:18
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth tests for invalid authorization codes.  https://review.openstack.org/15367520:18
sdaguewas never an issue before20:19
sdague2015-02-06 19:12:52.310 | [ERROR] /home/jenkins/workspace/gate-devstack-unit-tests/stackrc:685 Could not determine host ip address. See local.conf for suggestions on setting HOST_IP.20:19
*** achanda has quit IRC20:20
openstackgerrityolanda.robla proposed openstack-infra/nodepool: Conditionally check for targets in reconfigureManagers  https://review.openstack.org/12956020:20
sdaguejeblair: thanks!20:21
mordredsdague: not that I know of - but I do want to point out to people that sometimes machiens want to know their IP address ...20:21
SpamapSmordred: I'm of the opinion that  it is basically never the 'right' thing to care what your IP is, even if it is sometimes necessary because there are badly designed things out there.20:22
*** dustins has quit IRC20:22
sdagueSpamapS: well... devstack cares, because it's got to put that into service urls :)20:23
SpamapSsdague: right, which is what hostname is supposed to be fore. :-P20:23
SpamapSfor even20:23
openstackgerritMaru Newby proposed openstack-infra/devstack-gate: Refactor processing of testr artifacts for reuse  https://review.openstack.org/15367720:23
sdagueSpamapS: rabbit doesn't work right with hostname20:23
SpamapSsee above "badly designed things"20:23
SpamapS:)20:23
SpamapSagree20:24
SpamapShave wrangled with rabbit's hostname oddness for years20:24
sdagueok... so in perfect theory world20:24
SpamapSgoing back to people reporting lower battery life with rabbitmq installed20:24
sdagueI don't live in perfect theory world :)20:24
openstackgerritMaru Newby proposed openstack-infra/devstack-gate: Refactor processing of testr artifacts for reuse  https://review.openstack.org/15367720:24
openstackgerrityolanda.robla proposed openstack-infra/nodepool: Move image uploads out of ProviderManager  https://review.openstack.org/13740120:24
sdaguehowever, more importantly, this is new failure, that didn't fail before20:24
fungisurprisingly modest scrollback while i was out20:24
SpamapS(because it would spin forever trying to resolve and connect to its hostname.. which is rather difficult on transient wifi)20:24
openstackgerritMaru Newby proposed openstack-infra/devstack-gate: Refactor processing of testr artifacts for reuse  https://review.openstack.org/15367720:25
SpamapSsdague: nobody lives there, even the people who think they do ;)20:25
*** imcsk8 has joined #openstack-infra20:25
*** imcsk8 has quit IRC20:26
*** imcsk8 has joined #openstack-infra20:26
marunclarkb: I hope you'll be glad to know I'm refactoring d-g to avoid hackery in our hook scripts ^^ :)20:26
mtreinishSpamapS: I dunno it's all clouds, sunshine, and rainbows from my point of view. :)20:26
*** adalbas has quit IRC20:26
SpamapSThe point being, there is no actual agreed upon way to determine "this is the one address that every node I communicate with can reach me on". There are about a dozen ways to get an approximation.20:26
*** niska has joined #openstack-infra20:26
marunmtreinish: your attention on that patch would also be appreciated since I think you've worked on the code in question recently ^^20:27
*** otter768 has joined #openstack-infra20:27
sdagueso I can work around it, however, it means something changed on the base nodes that made the auto detection in devstack fail20:27
sdaguewhich... is odd20:27
*** carl_baldwin has joined #openstack-infra20:27
*** dannywilson has quit IRC20:27
mtreinishmarun: well it conflicts with: https://review.openstack.org/#/c/153017/20:28
SpamapSsdague: can you point me at the general area the autodetect is done? I've been around this bush so many times I keep a toothbrush in the bush's medicine cabinet.20:28
*** ashleighfarnham has quit IRC20:28
mtreinishmarun: actually you're dropping the interupted test stuff completely20:28
marunmtreinish: no, I'm not20:28
marunmtreinish: but I can fix the ordering as per your patch20:29
mtreinishoh nm it's there at the bottom20:29
*** davideagnello has quit IRC20:29
mtreinishI should have realized that because I noticed the conflict...20:29
*** annegentle has quit IRC20:29
SpamapSIn unrelated news, Getting fuel failures..20:29
SpamapSCaching fuel-plugin-contrail from git://git.openstack.org/stackforge/fuel-plugin-contrail.git in /home/clint/.cache/image-create/source-repositories/fuel_plugin_contrail_43829f30223c9e70974944602427351b4738130f20:29
SpamapSCloning into '/home/clint/.cache/image-create/source-repositories/fuel_plugin_contrail_43829f30223c9e70974944602427351b4738130f.tmp'...20:29
SpamapSfatal: Could not read from remote repository.20:29
marunmtreinish: are there cases where 0 will exist and so will tmp then?20:29
marunmtreinish: would you be ammenable to me fixing my patch and you abandoning yours?20:30
marunmtreinish: or is the potential review lag something that needs to be avoided?20:30
*** aysyd has quit IRC20:30
sdagueSpamapS: https://github.com/openstack-dev/devstack/blob/master/functions-common#L74620:31
*** otter768 has quit IRC20:31
*** davideagnello has joined #openstack-infra20:31
SpamapSah the old "use the first ip on the default route" trick20:31
mtreinishmarun: so 0 will always exist now, (we preseed an aggregate stream from the subunit2sql db currently) tmp will only exist if the run is interrupted20:31
marunmtreinish: ah, ok.20:31
SpamapSusually a reasonable choice ;)20:32
mtreinishmarun: which is why I had to reorder it, because the old order meant we never caught tmp20:32
*** carl_baldwin has quit IRC20:32
marunmtreinish: makes sense.  I'll submit an updated patch.20:32
mtreinishmarun: if you can think of a better way to handle pre-existing streams I'm not exactly happy with check for 0. Because that's always true and not always the desired check20:34
marunmtreinish: hmmm20:34
mtreinishmarun: especially because we'll be preseeding 10 streams instead of one in the future: https://review.openstack.org/153036 and https://review.openstack.org/15173220:34
marunmtreinish: what exactly are contained in the preseeded streams?20:35
marunare -> is20:35
jeblairsdague: logstash says that goes back to jan 28 and affects only hpcloud-b120:35
*** BharatK has quit IRC20:35
marunmtreinish: Maybe we can set a constant for the number of pre-existing streams?20:36
mtreinishmarun: currently it's pretty basic: just test_ids and average runtime between a start and stop timestamp20:36
marunmtreinish: or, should we be looking at the last stream?20:36
sdaguejeblair: interesting20:36
mtreinishmarun: after those 2 patches land the streams will be the same as from real gate runs minus the attachements20:36
fungiSpamapS: i've triggered a full replica refresh in gerrit to hopefully fix that20:36
mtreinishmarun: http://logs.openstack.org/19/149719/6/check/check-tempest-dsvm-full/4725d5f/logs/testrepository/20:36
mtreinish^^^ that's from a test patch where I save the entire testrepository from a run20:37
*** stevemar has quit IRC20:37
fungiSpamapS: stackforge/fuel-plugin-contrail was just added earlier today and we've been seeing occasional git push collisions in gerrit's replication plugin20:37
*** marcusvrn1 has quit IRC20:37
mordredAROO?20:37
*** stevemar has joined #openstack-infra20:38
marunmtreinish: so if tmp does not exist, I can imagine the subsequent check would have to be for whether the last run was a preseed or not20:38
mordredSpamapS: I'd agree with you on ips/hostnames for a lot of things, because we have this lovely thing called "the DNS"20:38
fungiSpamapS: so chances are it was an empty repo on whatever backend haproxy was directing you to20:38
*** vigneshvar has quit IRC20:38
mordredSpamapS: except there are lower level things that need to run on things sometimes20:38
*** sarob has joined #openstack-infra20:38
marunmtreinish: the only way I can see to do that would be to know how many preseeds are expected, e.g. it's set as a constant20:38
mtreinishmarun: I don't have a way to really check that though?20:38
mtreinishmarun: ah ok20:38
SpamapSmordred: we also have this /etc/hosts file that is maleable and works great for stuff like this.20:38
SpamapSfungi: I've hit the failure 3 times20:38
*** prad has quit IRC20:38
marunmtreinish: do you want me to incorporate such a check in my patch?20:38
marunmtreinish: or should that come in a follow-on?20:39
SpamapSfungi: just now in fact20:39
jeblairsdague: that includes devstack-trusty nodes which are from DIB; they are current on hpcloud-b1 and hpcloud-b2 (and possibly others).  the dib images should be the same across all of hpcloud.20:39
fungiSpamapS: coming from the same source ip address?20:39
*** dannywilson has joined #openstack-infra20:39
mtreinishmarun: I think as a followon would be fine20:39
marunmtreinish: ok20:39
fungiSpamapS: haproxy is set to persist by source address hash because of the fact that our git servers are not 100% in lockstep sync20:39
marunmtreinish: I'd appreciate landing this refactor sooner than later so that we can be assured of consistent artifact processing on the functional jobs20:39
jeblairsdague: so the remaining characteristic that ties hpcloud-b1 nodes together is they are on the same neutron net20:39
clarkbjeblair: and router20:40
*** hashar has quit IRC20:40
clarkbmarun: looking20:40
openstackgerritMaru Newby proposed openstack-infra/devstack-gate: Refactor processing of testr artifacts for reuse  https://review.openstack.org/15367720:40
fungiSpamapS: and the full replication refresh is still going (we have a lot of repos for it to check, i wanted to make sure there weren't more possibly broken besides just that one)20:41
fungiSpamapS: but should be done here in a few minutes20:41
SpamapSfungi: I shall drum my fingers in anticipation. :)20:41
fungiSpamapS: tap, tap, tap20:41
*** pcrews has quit IRC20:42
clarkbmtreinish: marun don't your two changes conflict?20:42
*** jgrimm is now known as zz_jgrimm20:42
jeblairsdague, clarkb: it's only happened 13 times in logstash, which is way too low to be every time a devstack job runs on b120:42
marunclarkb: they do20:42
marunclarkb: My latest patch incorporates his change20:42
SpamapSsdague: two things. 1) That failure message would be more debuggable with the output of the 'ip route' command.20:42
clarkbmarun: gotcha20:42
openstackgerritJohn Villalovos proposed openstack-infra/system-config: Add description of 'ironic' log files  https://review.openstack.org/15368320:42
jeblairsdague, clarkb: so i think we're looking at a highly intermittent error related to that network20:42
sdaguejeblair: it's the unit tests20:42
sdagueso that might be every time20:42
jeblairsdague: also real devstack20:42
*** ghostpl has joined #openstack-infra20:42
SpamapSsdague: 2) I think what jeblair might be getting at is it is possible the box doesn't have an IP that isn't in the fixed/floating ranges.20:42
sdaguereally?20:42
jeblairyeah, lemme try to find a log link that works :/20:43
*** pcrews has joined #openstack-infra20:44
*** dannywilson has quit IRC20:44
jeblairsdague: unit tests, neutron-dsvm-api, grenade-dsvm-neutron, grenade-dsvm-partial-ncpu and grenade-dsvm20:44
*** dannywilson has joined #openstack-infra20:45
jeblairsdague: so i guess not "vanilla" devstack, but grenade-dsvm ought to be very common, and some of the others fairly common20:45
*** bswartz has quit IRC20:45
jklarehi, can someone help me with debugging this error here (http://logs.openstack.org/99/152299/1/experimental/gate-cookbook-openstack-compute-chef-rake/fb4268d/console.html)20:45
*** ociuhandu has quit IRC20:45
*** jlvillal has joined #openstack-infra20:46
jeblairhttp://logs.openstack.org/13/146113/19/check/check-neutron-dsvm-api/cbe2cb4/console.html20:46
jeblairthe grenade ones don't have working logs :/  (from last week)20:46
jklarei tried the same thing multiple times on fresh 14.04 boxes with just a fresh build-essential installed and it worked20:46
jklareanybody any hint why its not working here?20:47
*** jlvillal has left #openstack-infra20:47
*** ghostpl has quit IRC20:48
*** ashleighfarnham has joined #openstack-infra20:48
sdaguehmmm bummer20:48
fungijklare: this looks familiar. i think the chef devs disabled this job on trusty previously because native extension building was nondeterministic20:48
*** niska has quit IRC20:48
jeblairjklare: build-essential is installed on those machines, but autoconf is not20:48
jeblairthe idea of a "base system" is apparently highly variable :/20:49
fungiahh, yep this is defintely a different one. now i see the "autoconf: not found"20:49
*** dannywilson has quit IRC20:49
sdaguejeblair: so it looks like the host was given - 10.0.0.2420:49
jeblairalso, the words "build essential" don't seem to meen what i think they mean20:49
*** shashankhegde has quit IRC20:50
sdaguewhich is inside the fixed_range that we are setting on hosts20:50
jklarefungi: yeah, its the same one we were running into before, but this should not happen anymore and i acutally can do the exact same thing on a fresh ubuntu 14.04 box with just apt-get update; apt-get install build-essential and then the scripts that were run here20:50
clarkbmarun: mtreinish that latest patchset lgtm20:50
sdaguewhich causes the error20:50
clarkbsdague: fixed should be 10.1.0.0/2020:50
fungijklare: worth noting that rackspace doesn't provide "fresh anything"20:50
clarkbsdague: that is one of the things that devstack-gate fixes20:51
fungijklare: they make their own base images, custom20:51
sdagueclarkb: well... it's FIXED_RANGE=10.0.0.0/2420:51
* mordred CAN'T WAIT to stop using cloud images20:51
sdaguehttp://logs.openstack.org/13/146113/19/check/check-neutron-dsvm-api/cbe2cb4/console.html#_2015-02-03_21_46_37_57820:51
fungijklare: so it's entirely possible that your "fresh install" had autoconf by default but rackspace's special base images omit it20:51
clarkbsdague: then that job is wrong20:51
*** andreaf has quit IRC20:51
clarkbsdague: and they shouldn't override the settings that we set20:51
sdaguehttp://logs.openstack.org/13/146113/19/check/check-neutron-dsvm-api/cbe2cb4/logs/localrc.txt.gz20:51
clarkbI mean20:51
sdagueit doesn't look like it's getting set for that job20:51
clarkbwe have done our best to give a consistent platform that works20:51
*** r1chardj0n3s_afk is now known as r1chardj0n3s20:51
*** armax has quit IRC20:52
jklarefungi: so i should install autoconf ?20:52
jeblairsdague, clarkb: maybe that's why this error doesn't show up for "normal" devstack jobs?20:52
sdaguejeblair: yep20:52
fungijklare: you can certainly apt-get install it as part of that job, yeah20:53
*** shashankhegde has joined #openstack-infra20:53
jklarefungi: do you think thats the right approach ?20:54
sdagueso I have a fix for the devstack unit tests up20:54
jklarefungi: can i get these special images somewhere?20:54
sdaguethe grenade tests should be set correctly, if we can find one that isn't with logs, we should look20:54
sdagueI actually have no idea how check-neutron-dsvm-api is configured20:54
fungijklare: by signing up for a rackspace account and booting their ubuntu 1.0.4 pvhvm image20:54
clarkbsdague: unfortunately everyone assumes 10.0.0.0/24 is theirs to use. A better strategy might be to always use 10.255.0.0/1620:55
fungier, 14.0420:55
*** _nadya_ has joined #openstack-infra20:55
jeblairsdague: they are all on jan 29 and are missing logs :(20:55
clarkbor introspecting interfaces on the local machine and picking from a range that doesn't conflict20:55
sdaguejeblair: they may be unrelated20:56
fungijklare: as soon as we can actually use rackspace's glance to upload our own working bootable images, we plan to also start making them available for download20:56
*** markvan has joined #openstack-infra20:56
clarkbfwiw we used 10.0.0.0/24 bceaues its hpcloud's default. we actually have the ability to change it20:56
jeblairclarkb: how did they get into logstash without being on static.o.o?20:56
sdagueclarkb: sure, so this is where devstack default just runs afoul, which is why it gets set20:56
clarkbjeblair: did they get renamed to hvae a .gz suffix?20:56
jeblairoooooh20:57
jklarefungi: ...kk ty, i guess i will give it a try and do a quick patch with autoconf20:57
fungijklare: but at the moment we're still roadblocked on things like the fact that they use a custom daemon to update network interface settings in instances at/after boot20:57
sdagueyeh... we need to do an apache redirect for that :)20:57
jeblairsdague: http://logs.openstack.org/16/149416/4/check/check-grenade-dsvm/2a80dde/logs/grenade.sh.txt.gz20:57
*** mwagner_lap has quit IRC20:57
sdaguejeblair: ok, that's just broken crazy pants - http://logs.openstack.org/16/149416/4/check/check-grenade-dsvm/2a80dde/logs/old/localrc.txt.gz20:58
jeblair2015-01-29 04:00:33.435 | /opt/stack/old/devstack/localrc: line 88: syntax error: unexpected end of file20:58
sdagueyep20:58
fungiwow20:59
sdagueyeh, that's just a trove grenade patch that didn't know what it was doint20:59
sdagueunrelated fail20:59
sdaguehttps://review.openstack.org/#/c/149416/ rev420:59
ekarlsois it possible with devstack to target an existing keystone ?21:00
jeblairso it's a bit of a catch-all error21:01
fungiSpamapS: btw that replication refresh looks to be well finished at this point. are you still seeing failures?21:01
*** tiswanso has quit IRC21:01
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Fix bug in template job yaml expansion.  https://review.openstack.org/15066621:01
*** yjiang5 has joined #openstack-infra21:01
stevemarekarlso, that's a good question21:01
stevemarekarlso, i want to say not at the moment21:02
clarkbekarlso: disable keystone in the service list and set the appropriate vars?21:02
dtroyerekarlso: should be, don't enable key and set the KEYSTONE_* vars appropriately21:02
clarkbdtroyer: o/21:02
stevemarekarlso, don't listen to me21:02
SpamapSfungi: failed 3 minutes ago. Restarting now.21:02
*** davideagnello has quit IRC21:02
jeblairsdague: i've confirmed the check-neutron-dsvm-api failure has "FIXED_RANGE=" in localrc21:02
*** Adri2000 has quit IRC21:02
*** melwitt has joined #openstack-infra21:03
*** Adri2000 has joined #openstack-infra21:03
fungiSpamapS: i've got a few minutes free now so i'll check for it in all the backends to make sure something's not worse wrong than usual21:03
mordredfungi: oh - the config drive support went live21:03
mordredfwiw21:03
jeblairsdague, clarkb: so is the current hypothesis: devstack-gate unit tests don't set fixed_range and we occasionally get assigned an ip that is in the default fixed_range?21:03
fungimordred: in all the regions now?21:03
clarkbjeblair: sdague note that b2 is 10.0.1.0/24 b3 is 10.0.2.0/24 and so on21:03
fungimordred: the ones we use anyway?21:03
mordredyeah - I need to test it still21:03
mordredyah21:04
fungiokay, last i heard it was just iad21:04
fungibut then again that was a while back21:04
*** melwitt_ has joined #openstack-infra21:04
mordredI was hoping to get it tested today, but then today fell apart21:04
clarkbjeblair: well b1 will always be 10.0.0.0/24 its the default that hpcloud used for a long time and was not changeable21:04
clarkbso when we switched to neutron I thought nothing of it21:04
clarkbwe could add a new network range for b121:04
jeblairclarkb: that jives with hypothesis above ^21:04
clarkbbut really if you intend on using 10/8 without introspecting network interfaces I think it is on you to set a less commonly used range like 10.255.0.0/1621:05
*** davideagnello has joined #openstack-infra21:05
openstackgerritClaudiu Belu proposed openstack-infra/project-config: Adds networking-hyperv in StackForge  https://review.openstack.org/15369121:06
*** tiswanso_ has joined #openstack-infra21:06
sdaguejeblair: yeh21:06
clarkbmy home network sits on 10.0.0.0/24 too so I am a fan of tools not using it by dfeault :021:06
clarkber :)21:06
dtroyerclarkb: I'm afraid that's a common less-commonly-used net ;)   Also, I find it nice to totally go where others aren't: 172.21.0.0/xx or something21:06
dtroyerI think changing DevStack's default might be overdue?21:06
*** bdpayne has quit IRC21:06
clarkbdtroyer: ya 172.21.0.0 would likely be better21:06
jeblairdtroyer: that might be the best most universal solution21:06
sdaguemaybe, it's whack a mole though21:07
fungiSpamapS: it's not a replication problem after all. apparently the script which precreates those empty repos for gerrit to replicate into failed to create the one for that project. diagnosing it now to see if i can tell why21:07
clarkbsdague: yes prper fix is check interfaces on host21:07
sdagueand honestly, people get confused about the 172 one21:07
clarkbdon't overlap21:07
*** armax has joined #openstack-infra21:07
dtroyerright, but fewer moles in the middle of the desert, especially when large corps all think they own 10/821:07
sdagueclarkb: or die early like we do, and make you set it21:07
*** melwitt has quit IRC21:07
sdagueclarkb: you need to set your home network to 10.42.0.0 like I did :)21:08
sdagueso, anyway, there is a unit test fix up21:08
sdaguechanging the defaults is fine as well21:08
* fungi uses 172.23.5.0/2421:08
clarkbsdague: I tell you what nova boot --flavor 42 is the best thing ever21:08
sdague:)21:08
clarkbsdague: I can always remember what flavor to use after tempest runs21:08
sdagueonly second to the heat specific flavor we had for a while... 45121:09
fungiNICE21:09
*** markmcclain has quit IRC21:09
fungicentigrade 23321:09
openstackgerritJohn Villalovos proposed openstack-infra/system-config: Add description of 'ironic' log files  https://review.openstack.org/15368321:09
sdagueoh, it's still in there21:10
sdaguelib/tempest:            nova flavor-create m1.heat 451 512 0 121:10
fungithat's marvellous21:10
SpamapSfungi: retry failed21:10
clarkbanteaya: for 136395 aren't we building for all branches and publishing to branch specific locations?21:10
clarkbanteaya: is there a problem with how that is done?21:10
fungiSpamapS: it's not a replication problem after all. apparently the script which precreates those empty repos for gerrit to replicate into failed to create the one for that project. diagnosing it now to see if i can tell why21:10
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Fix bug in template job yaml expansion.  https://review.openstack.org/15066621:10
clarkbfungi: if puppetfailed to run when git derped and you reran it then maybe the subscription for the exec wasn't triggered?21:11
fungimaybe21:11
*** davideagnello has quit IRC21:11
* mtreinish feels really slow for not noticing those flavor ids before21:11
clarkbfungi: that would happen if the event that triggers the exec was successfully executed by puppet, then it got stuck on git before runningthe exec21:11
*** daneyon_ has joined #openstack-infra21:13
*** davideagnello has joined #openstack-infra21:13
*** marun has quit IRC21:13
*** fbo has quit IRC21:14
clarkbmtreinish: for 153438 it will make those jobs stop voting21:14
daneyon_My name is Daneyon Hansen. I have had a profile on openstack.org for several years, but for some reason it is no longer appearing in search and public profile. Is their anyone who can help me troubleshoot?21:15
clarkbmtreinish: they will run but not impact jenkins votes21:15
clarkbmtreinish: so since we had made them vote for a while wanted to make sure that you are ok with the chnage (it was what we originally intended just not implemented quite right)21:15
sdake_daneyon_ you get an error right?  "accessing page doesn't exist"?21:15
mtreinishclarkb: oh, we want tempest stable jobs to be voting. That's why we run them to make sure we don't break the stable branches21:16
clarkbmtreinish: right but icehouse hasn't voted21:16
mtreinishif it's not voting I don't think anyone would pay attention to that21:16
clarkbmtreinish: and juno wasn't supposed to be but is. This was our original intention21:16
mtreinishclarkb: I believe it does: https://review.openstack.org/#/c/153462/21:16
*** kgiusti has quit IRC21:17
clarkbhrm21:17
clarkbpleia2: ^ there may be something else broken21:17
jeblair438 just sets the branch selector for those21:17
clarkbOH21:17
clarkbwow I shouldn't review things tired I guess21:17
clarkbnevermind then21:18
*** markvan has quit IRC21:18
jeblairand i think that change is not necessary because there is no juno tempest branch21:18
fungiclarkb: wow, we need to be more careful... http://paste.openstack.org/show/168306/21:18
fungiclarkb: С may look like C but it is _not_ the same codepoint21:18
clarkbwait21:18
fungiwondering how best to catch things like that21:18
clarkbdid someone intentionally do that too? I don't even21:19
fungi>>> unicodedata.name(u'\u0421')21:19
fungi'CYRILLIC CAPITAL LETTER ES'21:19
daneyon_sdake_: Sorry, it seems you were trying to access a page that doesn't exist. Please check the spelling of the URL you were trying to access and try again. URL = http://www.openstack.org/community/members/profile/139421:19
fungiclarkb: i'm guessing it's a key on a cyrillic keyboard21:19
clarkbga in vim confirms thats awesome21:19
sdake_daneyon_ not sure how that happens or why, but I'm sure some web wizard here can fix :)21:19
clarkbI wonder if daneyon_'s thing is related to membership rolls expiring based on voting practice?21:20
sdake_clarkb have any pointers on folks that can fix daneyon_'s problem?  hes submitting a talk with me and I don't want it to break the submission21:20
daneyon_calling all web wizards :)21:20
clarkbreed: ^21:20
*** achanda has joined #openstack-infra21:20
clarkbdaneyon_: the only immediate thing that comes to mind is maybe you hadn't voted enough to maintain membership qualification21:20
sdake_clarkb iiuc daneyon had two accounts registered21:20
sdake_and deleted one21:20
*** signed8bit is now known as signed8bit_ZZZzz21:21
sdake_clarkb is it too late to vote :)21:21
clarkblet me ask around21:22
sdake_clarkb thanks :)21:22
*** _nadya_ has quit IRC21:23
*** r1chardj0n3s is now known as r1chardj0n3s_afk21:24
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Replace Cyrillic capital ES with Latin capital C  https://review.openstack.org/15369821:24
fungiSpamapS: ^ there's the cause21:25
*** dprince has quit IRC21:25
*** signed8bit_ZZZzz is now known as signed8bit21:25
clarkbfungi: the diff page in gerrit is amazing for that change21:25
clarkbalso fungi ^ any ideas on daneyon_'s thing?21:25
fungii know right?21:25
openstackgerritWayne Warren proposed openstack-infra/jenkins-job-builder: Fix bug in template job yaml expansion.  https://review.openstack.org/15066621:25
*** achanda has quit IRC21:25
fungiclarkb: no ideas on daneyon_'s thing short of dragging toddmorey or wes out of hiding to dig into it, or reaching out to jpmaxman or jimmy21:26
daneyon_clarkb: I have voted every election.21:26
fungiclarkb: i'll try some alternate channels21:26
daneyon_clarkb: At some point, I think someone had used my name, Daneyon Hansen to create a 2nd profile using a different email. I deleted this 2nd account under my name.21:27
sdake_time to restore from backups!21:27
sdake_sorry oculdn't resist :)21:27
jesusaurushow do i get testr to tell me what the import errors are?21:28
clarkbjesusaurus: it should tell you which module is erroring, then import that module with python21:28
*** koolhead17 has quit IRC21:28
mordredjesusaurus: .tox/py27/bin/python -m testtools.run discover --list21:29
mordredjesusaurus: should print out a list of the available tests21:29
mordredand if there are import errors, one of them should bomb out21:29
clarkbjesusaurus: apparently something in the python test frameworks/standards/stuff don't allow the exception to bubble out properly21:29
mordredyeah - I think lifeless finally got the fix for that landed in 3.4 and is working to bubble it back out21:30
clarkboh cool21:30
jesusaurusclarkb: mordred: awesome thanks21:31
mtreinishclarkb: you should just be able to do a testr list-tests with everything up to date and it'll print the exception21:31
*** melwitt_ is now known as melwitt21:31
mtreinishthat should have all been in the latest release21:31
*** radez is now known as radez_g0n321:31
clarkbeven better21:31
clarkbfungi: clearly we should have gerrit display raw binary to us21:32
clarkbfungi: then any octet without a leading 0 loses21:32
fungiclarkb: or fix create_cgitrepos.py (assuming cgit itself deals okay with utf-8 project descriptions)21:33
fungidescriptions are one place at least where i think non-ascii printable characters should be permitted21:34
daneyon_clarkb: when i search daneyon on www.openstack.org, I find another profile number with my name associated: http://www.openstack.org/community/members/profile/8528 I get the same error as profile 1394: Sorry, it seems you were trying to access a page that doesn't exist. Please check the spelling of the URL you were trying to access and try again.21:35
*** mfink_ has quit IRC21:37
fungidaneyon_: interesting. i'm (carefully) looking around in the backend database for that since i do have access, and i see a member table entry for 1394 with profile info which looks like yours, but nothing for 852821:37
fungidaneyon_: but find no corresponding entry in the group_members table21:38
*** ChuckC has quit IRC21:38
fungi(for 1394 i mean)21:38
clarkbjeblair: any reason you didn't approve 151656?21:38
fungino, i take that back, member id 1394 is in groups 4 and 821:39
jeblairfungi: where are you looking?21:39
openstackgerritMerged openstack-infra/project-config: Replace Cyrillic capital ES with Latin capital C  https://review.openstack.org/15369821:39
fungijeblair: the silverstripe trove database, credentials are in hiera21:40
daneyon_clarkb: 1394 appears to be the one that gets loaded when i click view public profile from my profile edit page21:40
*** marun has joined #openstack-infra21:40
jeblairfungi: thx21:40
*** bswartz has joined #openstack-infra21:40
*** crc32 has quit IRC21:40
fungijeblair: openstackid_ss_mysql_.* are the hiera keys21:40
jeblairclarkb: no reason :)21:41
clarkbjeblair: ok I am apprving it then21:41
*** vigneshvar has joined #openstack-infra21:41
*** amotoki has quit IRC21:41
fungidaneyon_: yeah, i think the developers for the site have suggested previously that the account needs to be in group id 5 to count as an individual member21:41
*** leakypipes has quit IRC21:41
fungiwhich 1394 is not currently21:42
openstackgerritKhai Do proposed stackforge/python-jenkins: Set the server connection timeout to same as socket timeout  https://review.openstack.org/15346721:42
fungidaneyon_: when you browse to https://www.openstack.org/profile/ do you see buttons to the right under the "Profile" heading?21:43
*** markmcclain has joined #openstack-infra21:43
*** sigmavirus24_awa is now known as sigmavirus2421:43
fungidaneyon_: one might say something like "become a foundation member"21:43
*** signed8bit has quit IRC21:44
daneyon_fungi: ya, i see make me a foundation member. let me try that.21:44
jeblairor "membership has its rewards"21:44
*** yamahata has quit IRC21:45
daneyon_fungi: my member level was speaker. i think someone changed that for my Paris talk.. it used to be member21:45
fungidaneyon_: interesting. we'll make sure someone who manages that system looks into it21:46
*** alexpilotti has joined #openstack-infra21:47
*** dmsimard is now known as dmsimard_away21:47
jeblairfungi, mordred: make sure you are on the same page wrt to things like 15335421:49
*** ihrachyshka has quit IRC21:49
openstackgerritMerged openstack-infra/puppet-etherpad_lite: Add connection-tuning file to new Apache 2.4 conf folder structure  https://review.openstack.org/15165621:50
fungijeblair: yeah, we're both aware, that was more of a "SpamapS wanted an example because he was trying to figure out why mysql setupt would be broken under dib"21:50
jeblairok21:51
fungimordred: that should probably be a wip change, maybe indefinitely?21:51
clarkbjeblair: its ok to merge some of zaro's puppet-gerrit config changes right?21:51
*** baoli has quit IRC21:51
clarkbzaro: ^ you too21:51
jeblairclarkb: afaik21:52
openstackgerritKyle Mestery proposed openstack-infra/project-config: Add a check job for the networking-odl project  https://review.openstack.org/15370421:52
fungidaneyon_: btw i see your account in group 5 now (individual member) and http://www.openstack.org/community/members/profile/1394 is returning content now21:52
clarkb151368 is the one I am most worried about21:52
openstackgerritKyle Mestery proposed openstack-infra/devstack-gate: Add DEVSTACK_GATE_NEUTRON_ODL option to enable ODL  https://review.openstack.org/15370521:52
daneyon_fungi: works now. thank you so much.21:53
jeblairhuh, there is no number 1.  or number 2.  or number 6.21:53
daneyon_clarkb: works now. thanks a ton!21:53
clarkbdaneyon_: it was mostly fungi :) glad you got it working21:53
anteayaclarkb: for 136395 I submitted that a long time ago at andreas' request21:54
anteayaclarkb: I'll have to talk to him and refresh my memory21:54
clarkbanteaya: ok no worries21:54
anteayathanks for looking at it21:54
openstackgerritGuido Günther proposed stackforge/python-jenkins: Use empty crumb on empty resonse  https://review.openstack.org/15339821:55
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't assume we get a valid response  https://review.openstack.org/15339921:55
openstackgerritGuido Günther proposed stackforge/python-jenkins: Fix incorrect indentation  https://review.openstack.org/15370621:55
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't return None on unhandled http errors  https://review.openstack.org/15370721:55
daneyon_fungi: is their anyway to kill my 8528 account? It's the first one that shows up in a search and redirects to a busted page21:57
clarkbzaro: please see comment on 15136821:57
openstackgerritMerged openstack-infra/puppet-gerrit: Set the Gerrit change screen default to OLD_UI.  https://review.openstack.org/15135921:57
fungidaneyon_: yeah the admins for the site are looking at it right now21:58
*** amitgandhinz has joined #openstack-infra21:59
daneyon_fungi: thx21:59
fungidaneyon_: i only have some limited visibility into the backend for it at the moment and the ability to pester them ;)21:59
*** annegent_ has quit IRC22:00
*** achanda has joined #openstack-infra22:00
zaroclarkb: answered your question22:01
clarkbzaro: I don't follow. You specifically install that particular package system wide only if not on precise22:02
clarkbzaro: so if you delete the gerrit isntalled version won't that break?22:02
*** mattfarina has quit IRC22:02
zaroclarkb: ohh sorry i think i wasn't clear about that.22:03
openstackgerritJoe Gordon proposed openstack/requirements: Bump tempest-lib to > 0.2.0  https://review.openstack.org/15370822:03
openstackgerritMerged openstack-infra/puppet-gerrit: Update Gerrit Puppet module for Trusty  https://review.openstack.org/15178322:03
zaroclarkb: i removed the gerrit installed version and it did not break.22:04
clarkbzaro: so, does that mean we are using a slower non bouncy castle version of an algorithm?22:04
clarkbzaro: I think it would be good to better understand what that means to our current install22:05
*** abhirc has quit IRC22:05
daneyon_fungi: no biggie. As long as i have a working profile, then i'm OK. Killing the other profile is a nice to have22:05
fungidaneyon_: the member search at http://www.openstack.org/community/members/ is working correctly for me when i plug in your name22:05
fungidaneyon_: the general site search is "powered by google" and so it's anybody's guess what they cached22:05
*** bswartz has quit IRC22:05
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Remaining functional tests for the OAuth Endpoint  https://review.openstack.org/15275222:06
daneyon_fungi: i see, i was just using the general site search. i'm good. thx again for all the help.22:06
zaroclarkb: actually i didn't check what the difference is between the OS packaged version and the one that Gerrit gets when initializing the site.22:06
clarkbjeblair: how do I see WIP/d changes in gertty?22:06
clarkbzaro: thats not my concern, my concern is on precis ewhere we don't have the system package22:06
fungidaneyon_: hopefully that will be self-correcting with your correct profile now reachable. glad you got sorted22:06
*** salv-orl_ has joined #openstack-infra22:07
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth delete refresh tokens when access tokens deleted.  https://review.openstack.org/15371022:07
zaroclarkb: i noticed that the puppet  was setup use the os packaged version of these files and that's what i thought we wanted to do22:07
clarkbbut we should probably double check that sysem package is reasonably close to what gerrit wants22:07
*** daneyon_ has left #openstack-infra22:07
openstackgerritJoe Gordon proposed openstack/requirements: Bump tempest-lib min version to 0.2.0  https://review.openstack.org/15370822:07
clarkbzaro: yes I think using the system packages is fine22:07
clarkbzaro: the problem here is we don't have a system package on precise22:07
clarkbfungi: I think you can remove your -2 from 152217 now22:08
zaroclarkb: ohh to clarify, that lib is not needed on precise.  so it won't get setup22:08
*** salv-orlando has quit IRC22:08
clarkbzaro: why isn't it needed on precise?22:08
clarkbzaro: if you were to install gerrit 2.9 on precise you would need it right?22:08
zaroclarkb: sorry, i see now.22:08
zaroclarkb: gerrit 2.9 on precise will fail without that lib.22:09
clarkbok. we need to be a bit more careful aboutthis stuff particularly if we want these puppet modules to be reconsumable22:09
fungiclarkb: alternatively, i'm tempted to just do 153305 instead and stop caring about that repo22:09
*** bradjones has quit IRC22:09
clarkbfungi: wfm22:09
clarkbmordred: ^22:09
*** mattfarina has joined #openstack-infra22:10
*** mattfarina has quit IRC22:10
*** zigo has joined #openstack-infra22:11
*** salv-orl_ has quit IRC22:11
zaroclarkb: i think the easiest solution to this is to just let Gerrit download whatever lib it wants and use those instead of the OS packaged ones?22:12
flashgordonclarkb: just two +2 reviews away from landing https://review.openstack.org/#/c/136179/522:12
clarkbfungi: actually wait, forsimilar reasons to why I -1'd zaro's change we probably want to have a pip module22:12
flashgordonfungi: ^22:12
*** joesavak has quit IRC22:13
clarkbfungi: not everyone will have used our node install scripts so need something that gets them pip22:13
flashgordonclarkb: and getting multihost aiopcpu working22:13
*** bradjones has joined #openstack-infra22:13
clarkbflashgordon: so close22:13
*** bknudson has quit IRC22:13
*** ChuckC has joined #openstack-infra22:13
zaroclarkb: i don't see how that relates to the puppet-gerrit change22:13
*** bcourt has quit IRC22:13
clarkbzaro: just that both need to be much more careful around the idea of other people using them22:14
fungiclarkb: if so, i think it needs to be a better pip module22:15
clarkbfungi: agreed22:15
zaroclarkb: ok.  is that an answer to my suggestion?22:15
clarkbit can stash get-pip.py off in a corner somewhere and make sure it exists and has been run22:15
clarkbzaro: no22:16
clarkbzaro: I do not know why we use system packages instead ofgerrit provided packages22:16
clarkbzaro: jeblair may know22:16
*** davideagnello has quit IRC22:17
*** achanda has quit IRC22:17
zaroclarkb: might be tied to jdk on the system?22:17
*** achanda has joined #openstack-infra22:17
clarkbzaro: if I had to guess it would be so that we get package updates22:18
clarkbwhich is a good reason to use system packages22:18
*** davideagnello has joined #openstack-infra22:18
*** tkelsey has joined #openstack-infra22:19
SpamapSfungi: still broken22:19
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: OAuth now validates client ID against a configured list.  https://review.openstack.org/15371522:19
clarkbSpamapS: you could just not use that element for now if it helps22:20
* krotscheck thinks he may finally be done with auth things.22:20
jeblairclarkb, zaro: yeah, i'd love to continue to use system provided packages for things like crypto libs that gerrit uses22:20
jeblaircause i have no idea who's going to make sure that's up to date otherwise22:21
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Explicit Hook Priorities  https://review.openstack.org/15225322:21
zaroi'm not sure i see a way to run gerrit 2.9 on precise then.22:21
fungiSpamapS: yeah, the fix merged a little while ago but i may still need to manually re-run the script now22:22
SpamapSclarkb: the cache-devstack element?22:22
jeblairzaro: i don't think we have to support that22:22
zaroclarkb: ^ does that clear it up?22:22
*** baoli has joined #openstack-infra22:23
clarkbzaro: I guess. we should probably be explicit about it somewhere but ya wfm22:23
jeblairi think we should avoid breaking 2.8 on precise, and if someone wants to figure out how to do 2.9 on precise, that's fine, but otherwise, i think we can move on22:23
jeblairclarkb, zaro: i'd be okay if the module errored out and said "2.9 on precise is not supported"22:23
*** tkelsey has quit IRC22:23
clarkbI will update my review22:24
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't return None on unhandled http errors  https://review.openstack.org/15370722:24
openstackgerritGuido Günther proposed stackforge/python-jenkins: Fix incorrect indentation  https://review.openstack.org/15370622:24
openstackgerritGuido Günther proposed stackforge/python-jenkins: Use empty crumb on empty resonse  https://review.openstack.org/15339822:24
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't assume we get a valid response  https://review.openstack.org/15339922:24
openstackgerritKyle Mestery proposed openstack-infra/project-config: Add a check job for the networking-odl project  https://review.openstack.org/15370422:24
zarocool.  will update.22:24
clarkbSpamapS: yes I think thats the one22:24
clarkbSpamapS: it sets up the git repo cache thing22:24
SpamapSclarkb: ok, but I am dubious that I'll be able to test mysql ops in dib without all the pieces.22:24
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add support for volume attach/detach  https://review.openstack.org/15362322:25
* SpamapS will try it out anyway22:25
clarkbSpamapS: why? thats just a repo cache completely separate from mysql22:26
fungiSpamapS: very close now i think. the repo backends are successfully creating all the repos from the list again and i've fired off one more replication refresh to update any which were previously missing22:26
*** stevemar has quit IRC22:26
clarkbSpamapS: and if it doesn't we should probably make it so it does22:26
clarkbSpamapS: so let me know and I can work on that if it is an issue22:26
*** erlon has quit IRC22:26
SpamapSclarkb: I'm trying to replicate what breaks for you guys first. I'm not really fully aware of the whole thing, just blindly crashing down a path toward a suggested failure mode.22:26
*** stevemar has joined #openstack-infra22:27
*** otter768 has joined #openstack-infra22:27
clarkbzaro: ok reivew updated.22:28
*** signed8bit has joined #openstack-infra22:28
*** marun has quit IRC22:30
*** annegentle has joined #openstack-infra22:30
*** otter768 has quit IRC22:32
*** baoli has quit IRC22:33
*** wenlock has quit IRC22:34
*** ashleighfarnham has quit IRC22:35
*** lnxnut has quit IRC22:35
*** vigneshvar has quit IRC22:36
openstackgerritCraig Bryant proposed openstack-infra/project-config: Add maven build and repo upload for monasca-api  https://review.openstack.org/15371922:36
mordredclarkb: yah .... I want a real minimal node so we catch stuff22:36
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't return None on unhandled http errors  https://review.openstack.org/15370722:37
openstackgerritGuido Günther proposed stackforge/python-jenkins: Fix incorrect indentation  https://review.openstack.org/15370622:37
openstackgerritGuido Günther proposed stackforge/python-jenkins: Use empty crumb on empty resonse  https://review.openstack.org/15339822:37
fungiSpamapS: btw git cloning should work for all the repos again now22:38
*** abhirc has joined #openstack-infra22:39
openstackgerritKyle Mestery proposed openstack-infra/project-config: Add a check job for the networking-odl project  https://review.openstack.org/15370422:40
*** bhuvan has quit IRC22:41
*** bhuvan has joined #openstack-infra22:42
*** pc_m has quit IRC22:43
openstackgerritGuido Günther proposed stackforge/python-jenkins: fixup! Use empty crumb on empty resonse  https://review.openstack.org/15372222:43
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't return None on unhandled http errors  https://review.openstack.org/15370722:44
openstackgerritGuido Günther proposed stackforge/python-jenkins: Fix incorrect indentation  https://review.openstack.org/15370622:44
openstackgerritGuido Günther proposed stackforge/python-jenkins: Use empty crumb on empty response  https://review.openstack.org/15339822:44
*** carl_baldwin has joined #openstack-infra22:45
*** mriedem has quit IRC22:46
*** joesavak has joined #openstack-infra22:47
*** vigneshvar has joined #openstack-infra22:47
*** otter768 has joined #openstack-infra22:48
*** jsavak has joined #openstack-infra22:49
*** yamamoto has joined #openstack-infra22:50
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Pass the old entity values in the NotificationHook  https://review.openstack.org/15164522:50
openstackgerritMerged openstack-infra/puppet-storyboard: Added config.json to manifest  https://review.openstack.org/14875722:51
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: API events now include resource before/after state.  https://review.openstack.org/15231822:51
clarkbmordred: 15269122:52
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email base plugin.  https://review.openstack.org/15194822:52
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Templating Engine  https://review.openstack.org/14773422:52
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email SMTP Sender  https://review.openstack.org/14979222:52
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email configuration.  https://review.openstack.org/15194622:52
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: [WIP] Email Send Cron Worker  https://review.openstack.org/15141622:52
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Working Directory Utility Method  https://review.openstack.org/15194722:52
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Outbox  https://review.openstack.org/15141322:52
greghaynessuch email22:52
*** joesavak has quit IRC22:53
mordredclarkb: woo22:53
clarkbmordred: it timed out22:53
openstackgerritGuido Günther proposed stackforge/python-jenkins: Don't return None on unhandled http errors  https://review.openstack.org/15370722:54
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email base plugin.  https://review.openstack.org/15194822:55
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email SMTP Sender  https://review.openstack.org/14979222:55
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: [WIP] Email Send Cron Worker  https://review.openstack.org/15141622:55
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Working Directory Utility Method  https://review.openstack.org/15194722:55
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Email Outbox  https://review.openstack.org/15141322:55
mordredclarkb: I don't like that22:56
*** annegentle has quit IRC22:57
clarkbthe wheelhouse must not be working?22:57
clarkbpretty sure we set one up first thing to make lxml installs not terrible22:57
*** annegentle has joined #openstack-infra22:57
*** marun has joined #openstack-infra22:58
clarkbhrm grepping for wheelhouse shows nothing22:58
fungiyeah, that's definitely running way too long22:58
zarofungi: can you help with this?  http://paste.openstack.org/show/168354/22:58
*** signed8bit has quit IRC22:58
zarozuul looks like its using the private key to connect to gerrit and there's a mismatch.22:59
*** yamamoto has quit IRC23:00
fungizaro: if memory serves, paramiko's error message is a little misleading there, and that's it's actually just not able to read the private key23:00
fungizaro: this is supposed to be using /var/lib/zuul/ssh/id_rsa on zuul-dev right?23:00
zarofungi: so when i attempt to start zuul-server with gerrit_user='jenkins' then i gett that error, but when i start with 'zuul-dev' then no error.23:01
zarofungi: yes that what the zuul.conf is pointing to.23:01
fungizaro: i'll check whether the public key for jenkins in review-dev corresponds to the private key on zuul-dev23:02
clarkbfungi: any luck with git on centos6?23:02
fungiclarkb: no obvious signs of hangups so it's possible i need to try something more than a remote update to trigger it23:04
SpamapSfungi: ty23:05
*** amitgandhinz has quit IRC23:07
*** yamamoto has joined #openstack-infra23:07
JayFHey; fungi clarkb, can one of you bump the post job for IPA (877f668). Once it runs Ironic's gate will be fixed (finally)23:08
fungizaro: the second ssh key for username:jenkins in the review-dev database looks correct23:08
JayFthe fix is merged in already but because Ironic uses the prebuilt images, until the post job finishes Ironic is still busted23:08
* JayF has buckets full of sympothy even more for those of you troubleshooting the gate daily after losing 2/3 of a day to it23:09
*** jsavak has quit IRC23:09
fungiJayF: we have no way to reprioritize the post pipeline over the other pipelines. it runs jobs when gate and check aren't demanding new workers23:09
JayFSo that means Ironic gate will be busted for a while then, heh23:10
fungiJayF: it's a balancing act. we previously had complaints that nobody was getting check results when the post pipeline was busy23:10
JayFI mean, igi, and we're a bit of a special flower at times23:10
JayFlike I bet we're the only project where a post job would unbreak the gate for something23:10
zarofungi: yeah, i added that.  but for some reason zuul is using the private key.  which belongs to 'zuul-dev' not 'jenkins' and i think that's why the error.23:11
*** reed has quit IRC23:11
zarozuul documentation says that it uses that private key to make connections to gerrit?23:11
fungiJayF: not necessarily. we used to have some issues from time to time with projects slurping branch tarballs in jobs and a broken branch tarball would screw stuff up until another post job updated it23:11
fungizaro: `ssh-keygen -y -f /var/lib/zuul/ssh/id_rsa` on zuul-dev spits out what looks like the same string you have in the db, so if the zuul process is actually using /var/lib/zuul/ssh/id_rsa then it should work with that jenkins username23:12
*** yamamoto has quit IRC23:13
*** marun has quit IRC23:13
SpamapSfungi: thanks, confirmed its working23:13
fungizaro: how are you setting the gerrit user when starting zuul? it's currently set to user=zuul-dev in /etc/zuul/zuul.conf23:14
zaroahh so it just take the private key and generates pub key to make the connection to gerrit.  that makes sense.23:14
zarofungi: i'm manually setting it. yeah, works now because i set it back.  i was testing with the 'jenkins' user.  i'll try the test again.23:15
*** abhirc has quit IRC23:15
fungizaro: not exactly. it uses the private key to "sign" a message which the gerrit server is able to confirm was made by the private key corresponding to the public key it has listed23:16
*** sarob has quit IRC23:16
*** sigmavirus24 is now known as sigmavirus24_awa23:16
*** sarob has joined #openstack-infra23:17
zarook. well if i see the same error then there's something wonky going on there.23:18
*** annegentle has quit IRC23:18
*** ddieterly has quit IRC23:19
fungizaro: section 7 of http://www.ietf.org/rfc/rfc4252.txt describes ssh's publickey auth implementation if you're ever really, really bored23:19
*** shashankhegde has quit IRC23:20
*** dannywilson has joined #openstack-infra23:20
*** sarob has quit IRC23:21
zarono wonder why i've never gotten around to reading it :)23:21
fungiheh23:21
*** dannywilson has quit IRC23:24
*** dmsimard_away is now known as dmsimard23:24
*** dannywilson has joined #openstack-infra23:25
*** dmsimard is now known as dmsimard_away23:25
*** bknudson has joined #openstack-infra23:25
*** thedodd has quit IRC23:25
*** bswartz has joined #openstack-infra23:26
*** sarob has joined #openstack-infra23:26
zarofungi: tested again. this time didn't get the error.  not sure what happened, i'm guessing that i did a reload instead of start/stop last time and it confused zuul.  anyways sorry to bother you with that non-issue.23:28
*** salv-orlando has joined #openstack-infra23:29
*** hichtakk has joined #openstack-infra23:30
*** unicell has joined #openstack-infra23:31
zaroit's ready now: https://review.openstack.org/#/c/153668/23:31
*** ZZelle has quit IRC23:34
*** salv-orlando has quit IRC23:34
*** abhirc has joined #openstack-infra23:37
*** sdake_ has quit IRC23:37
*** dalgaaf has quit IRC23:40
*** salv-orlando has joined #openstack-infra23:43
*** salv-orlando has quit IRC23:45
*** hichtakk has quit IRC23:46
fungizaro: no problem, just glad you got it to work!23:46
openstackgerritDouglas Mendizábal proposed openstack-infra/project-config: Move barbicanclient docs jobs to post pipeline  https://review.openstack.org/15373223:47
*** timcline has quit IRC23:48
*** annegentle has joined #openstack-infra23:49
*** dkliban is now known as dkliban_afk23:50
*** marun has joined #openstack-infra23:50
*** armax has quit IRC23:55
*** stevemar has quit IRC23:56
*** abhirc has quit IRC23:56
openstackgerritMerged openstack-infra/shade: Adds get_network() and list_networks function  https://review.openstack.org/15064923:57
*** annegentle has quit IRC23:58
clarkbmordred: so I am reviewing shade changes. Do you expect me to be testing all of them? or?23:58
*** sabeen1 has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!