Wednesday, 2014-04-30

openstackgerritA change was merged to openstack-infra/config: Rename gertty ACL file to correct name
*** morgabra has quit IRC00:01
harlowjajeblair haha, u'll start love widget = widget.contents[0][0]00:02
harlowjathat part i don't like00:02
harlowjai had to do that weirdness to00:02
*** marcoemorais has quit IRC00:05
sdaguenotmyname: what defines your waiting to merge link?00:06
dimshmm, are we missing work in progress button?00:06
sdaguedims: it's now workflow -100:06
*** yamahata has joined #openstack-infra00:06
dimsah. thx00:06
derekhfungi: found it, the rackspace fedora image uses a different grub config to the default fedora cloud image, running some tests now00:08
fungiderekh: fun!00:09
fungiderekh: i've noticed that rs, in general, does not use vanilla "cloud" images in their catalog00:09
*** morgabra has joined #openstack-infra00:10
derekhfungi: yup00:13
*** atiwari has quit IRC00:14
sdaguenotmyname: looking at waiting for merge, I feel like we probably need a set of queries which are mostly house keeping. Sets of odd states that we don't think things should in00:15
*** sacharya has joined #openstack-infra00:16
*** sacharya has quit IRC00:16
sdaguejeblair: so what's your thinking around the fact that label:Code-Review>=0,self throwing an error if not logged in on that dashboard00:17
*** SumitNaiksatam has quit IRC00:18
*** nati_ueno has joined #openstack-infra00:20
*** reed has quit IRC00:21
*** reed has joined #openstack-infra00:21
*** manishg has joined #openstack-infra00:22
*** nati_uen_ has quit IRC00:23
jeblairsdague: if i click on the important changes dashboard while not logged in, it says my session is expired.  i find it curious that the error is differnt.00:24
jeblairsdague: but i think i agree with your earlier sentiment -- there's probably nothing for it and we'll just have to accept it as is.00:25
*** reed has quit IRC00:25
*** manishg_ has joined #openstack-infra00:25
*** manishg has quit IRC00:26
openstackgerritA change was merged to stackforge/gertty: Initial commit
*** yidclare has quit IRC00:28
*** thomasem has joined #openstack-infra00:29
*** talluri has joined #openstack-infra00:31
*** praneshp has quit IRC00:31
harlowjadurn just added comments to that, lol00:32
openstackgerritDerek Higgins proposed a change to openstack-infra/config: Use grub2-mkconfig if update-grub isn't present
harlowjaanyway, nothing to bad, jeblair u urwid expert now, lol00:32
jeblairharlowja: oops, sorry, i'll go check them out, thanks00:33
jeblairharlowja: and it's terrifying that you think i'm a urwid expert ;)00:33
sdaguejeblair: I'm surprised you didn't write it in elisp as an emacs extension00:33
derekhfungi: ^ that looks like it works on the rackspace fedora00:33
harlowjajeblair u seem to have found most of the urwid oddness :-P00:34
fungiderekh: oh, cool00:34
harlowjaso that makes u expert, lol00:34
harlowjaand u even got popups00:34
jeblairsdague: i thought about it... :)00:34
sdaguejeblair: I would support that :)00:35
sdagueor a server that exposes it as IMAP :)00:35
fungiderekh: oh, and yeah, i had to jump through similar hoops upgrading rackspace's debian wheezy images to jessie since they stub in an old grub. must be something about their xen domu boot setup00:35
derekhfungi: yup must be00:36
openstackgerritJoshua Harlow proposed a change to stackforge/gertty: Read using a file handle instead of read() method
harlowjaalright jeblair
fungijeblair: if seems sane to you, i'll babysit centos6 and precise image updates to make sure there are no unfortunate surprises00:43
openstackgerritJoshua Harlow proposed a change to stackforge/gertty: Allow for password prompting
*** tteggel has quit IRC00:52
*** UtahDave has quit IRC00:53
*** Adri2000 has quit IRC00:53
*** jhesketh has quit IRC00:53
openstackgerritJoshua Harlow proposed a change to stackforge/gertty: Add ordereddict requirement
*** stevebaker has quit IRC00:53
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add tool to parse puppet reports
mordredjeblair, fungi ^^00:54
mordredthat will parse a puppet yaml report file and print some basic things00:54
*** tteggel_ has joined #openstack-infra00:54
mordredbut more importantly, it will parse the file into some objects00:54
openstackgerritJoshua Harlow proposed a change to stackforge/gertty: Add ordereddict requirement
*** Adri2000 has joined #openstack-infra00:55
*** stevebaker has joined #openstack-infra01:00
*** nati_ueno has quit IRC01:01
openstackgerritDavid Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Doc: Use VersionInfo from pbr and remove openstack common modules
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Fix "delete-all"
openstackgerritMathew Odden proposed a change to openstack-infra/config: Create bash8 project in openstack-dev
*** nati_ueno has joined #openstack-infra01:04
*** jhesketh has joined #openstack-infra01:08
*** dkehn_afk is now known as dkehn01:08
derekhfungi: I gotta call it a night, I know of any other problems and will check in first thing to help deal with anything that might crop up01:11
derekh*I don't know of any ....01:13
*** derekh has quit IRC01:13
tchaypodstufft: finally have a new commit with tests - let me know if you'd prefer the two commits squashed01:16
jeblairfungi: lgtm01:16
dstuffttchaypo: nah, we don't really care about squashing01:16
tchaypodstufft: that was a quick response01:17
jeblairmordred, krotscheck: is there a way to get the task id in storyboard?01:17
openstackgerritJoshua Hesketh proposed a change to openstack-infra/config: Update zuul merger init script
openstackgerritA change was merged to openstack-infra/config: Use grub2-mkconfig if update-grub isn't present
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Fix displaying new files
*** lakshmi has joined #openstack-infra01:20
dstuffttchaypo: looks reasonable to me, did you ensure the tests fail w/o your change?01:20
openstackgerritA change was merged to stackforge/gertty: Read using a file handle instead of read() method
openstackgerritA change was merged to stackforge/gertty: Allow for password prompting
tchaypodstufft: the only one that fails without my change is  test_url_incorrect_case_file_index01:21
mordredjeblair: unsure.01:21
dstuffttchaypo: works for me01:22
mordredjeblair: how do I get help text in an xterm wthat 'helpfully' thinks F1 is for it?01:22
openstackgerritA change was merged to stackforge/gertty: Add ordereddict requirement
dstuffttchaypo: I just wanted to make sure you verified that so that we know that there is a correct test for it so this stuff doesn't break in the future :]01:22
jeblairmordred: that's apalling.01:22
mordredjeblair: I just changed the setting in the terminal01:22
tchaypoYeah. Having a test that passes after the change because it's a broken test is quite pointless :)01:23
openstackgerritA change was merged to stackforge/gertty: Fix displaying new files
*** dkehn is now known as dkehn_zzz01:23
dstuffttchaypo: ya, folks do it so I ask!01:24
dstufftI'll merge once tests pass01:24
openstackgerritJoshua Harlow proposed a change to openstack/requirements: Add wrapt library
jeblairmordred: i think HEAD may be just barely usable for you now01:24
dstuffttchaypo: uh oh01:24
dstuffttchaypo: failing tests01:24
jeblairmordred: though heads up about this bug:!/story/7401:25
dstuffttchaypo: looks like it fails on everything but 2.701:25
tchaypowell that's rude.01:25
*** talluri has quit IRC01:25
tchaypooh. coincidentally, I only tested it on 2.701:25
dstufftoh .message is deprecated in 2.x and removed in 3.x01:25
*** saschpe has quit IRC01:26
mordredsqlalchemy.exc.OperationalError: (OperationalError) database is locked u'UPDATE project SET subscribed=? WHERE project."key" = ?' (1, 157)01:27
tchaypoactually according to it's deprecated as of 2.601:27
tchayposo I'm confused about why 2.7 works01:27
dstufftprobably they screwed up the deprecation message on 2.701:28
*** saschpe has joined #openstack-infra01:30
*** david-lyle has joined #openstack-infra01:32
*** Daviey has quit IRC01:33
tchaypoon a side note - when I run "tox -epy27" I get a ton of failures. I've been ignoring everything that wasn't related to the code I was working on. Is there something I'm missing about running the tests locally?01:33
*** mfer has joined #openstack-infra01:33
*** talluri has joined #openstack-infra01:34
*** melwitt has joined #openstack-infra01:35
*** lakshmi has quit IRC01:35
*** nosnos has joined #openstack-infra01:35
*** talluri has quit IRC01:37
*** talluri has joined #openstack-infra01:37
*** Ryan_Lane has quit IRC01:37
*** talluri has quit IRC01:41
openstackgerritJoshua Hesketh proposed a change to openstack-infra/config: Install bootstrap for zuul module
*** SumitNaiksatam has joined #openstack-infra01:42
*** krtaylor has joined #openstack-infra01:42
*** sandywalsh has quit IRC01:51
*** yaguang has joined #openstack-infra01:54
portantehello folks, how do I go about adding another test run for the swift project?01:55
portanteWe recently added the ability to run our functional test suite entirely within a single process (thanks eventlet), so that now one can do: "tox -e pep8,py26,py27,func"01:56
*** david-lyle has quit IRC01:57
portantewhile we there is a devstack based functional test run as a check/gate job, I'd like to see this new in-process functional test run executed as well.01:57
portantenotmyname: do you agree?01:57
*** mfer has quit IRC01:59
openstackgerritJoshua Hesketh proposed a change to openstack-infra/config: Install bootstrap for zuul module
*** homeless has quit IRC02:05
*** zehicle_at_dell has quit IRC02:06
*** manishg_ has quit IRC02:07
*** zehicle_at_dell has joined #openstack-infra02:07
*** thomasem has quit IRC02:08
*** arnaud has quit IRC02:09
*** wenlock has joined #openstack-infra02:12
*** zhiyan_ is now known as zhiyan02:12
*** ptoohill has joined #openstack-infra02:24
ptoohillHello, following articles about error led me here. Im currently having issues submitting my first change with the following error from git-review: fatal: ICLA contributor agreement requires current contact information. I believe i may need to resign contract, but am unsure how to proceed. Any advice?02:25
*** mriedem has quit IRC02:26
ptoohillAlso, to note, when attempting to update contact info in gerrit i get a server error "Cannot store contact information"02:26
*** harlowja is now known as harlowja_away02:27
ptoohillShould i just create a new account? or maybe im asking in the wrong place02:29
*** crc32 has joined #openstack-infra02:29
*** melwitt has quit IRC02:32
*** jhesketh has quit IRC02:32
crc32Gerrit is complaining that a coworker hasen't set up their contact info and is rejecting their commits for review. This seems to be along the lines of
crc32is their a work around for this?02:32
*** jhesketh has joined #openstack-infra02:32
crc32where are the admins to contact?02:32
*** jhesketh has quit IRC02:33
mattoliverauptoohill: you are in the right place.. unfortuantly this week is the openstack off week (between releases) so many of the cores are not around.02:35
ptoohillAny known work-arounds?02:35
ptoohillthat you know of?02:35
*** melwitt has joined #openstack-infra02:36
clarkbfirst go to your gerrit settings and submit contact info as that seems to line up with your error02:36
openstackgerritA change was merged to openstack/requirements: Work around atexit bug in python < 2.7.4
mattoliverauptoohill: So you have signed the aggreement? And are the contact details up to date, do they match your lauchpad account etc?02:36
ptoohilli cannot submit as Gerrit complains02:36
ptoohillyes, i have signed the agreement and all details match up02:37
portanteclarkb: did you see the above on tox -e func for swift?02:37
ptoohillWhen attempting to submit contact changes i get server error02:37
clarkbportante no currently attemptibg vacation02:37
openstackgerritOpenStack Proposal Bot proposed a change to openstack-dev/hacking: Updated from global requirements
portanteand don't look back02:37
portanteit can wait02:37
ptoohill"Cannot store contact information"02:38
crc32it also says he already signed the ICLA.02:38
mattoliverauportante: Hmm, so the problem is gerrit details aren't correct and you can't update them then.02:38
mattoliverauptoohill: ^^^02:39
ptoohillthats what it seems like02:39
mattoliveraudamn auto correct02:39
mattoliverauportante: sorry, wrong auto correct.02:39
ptoohillim not sure if theres another issue being hidden here somehow, but yes, git review complains that i need to submit contact info, but Gerrit wont let me02:39
crc32mattoliverau: or they are correct but gerrit doesn't recognize a working config.02:39
ptoohillor that too ^02:39
portantemattoliverau: no problem02:40
ptoohillMy issue is nearly identical to this:
*** praneshp has joined #openstack-infra02:41
ptoohillSo, i assume, if Gerrit would take my attempts to add contact info the problem would be resolved. Unfortunately, Gerrit gives a server error02:42
mattoliverauptoohill, crc32: In either case, I don't have admin rights... so can't poke around.  So looks like we'll need a core to take a look. But I'll have a look around and see if the is anything else I can find we can try.02:43
ptoohillI do appreciate the help mattoliverau02:43
*** yjiang has joined #openstack-infra02:43
crc32ok thank you.02:44
crc32Will any core reviewer do?02:44
*** Shrews has joined #openstack-infra02:46
*** sweston has joined #openstack-infra02:47
*** etoews has quit IRC02:51
*** zz_gondoi is now known as gondoi02:53
*** marun has quit IRC02:54
*** gyee has quit IRC02:55
*** marun has joined #openstack-infra02:55
*** matsuhashi has quit IRC02:56
*** changbl has joined #openstack-infra02:56
*** matsuhashi has joined #openstack-infra02:57
mattoliveraucrc32: I think so, so long as they have access to gerrit to maybe see a log or too to find out what happening and to hopefully fix it.02:57
mattoliveraugerrit was updated yesterday to a later version, so we could be experiencing a bug with the new version.02:58
*** marun has quit IRC03:00
*** alexpilotti has quit IRC03:01
*** asalkeld has joined #openstack-infra03:02
*** gondoi is now known as zz_gondoi03:03
asalkeldanyone know where the WIP button has gone from gerrit? nothing about it here ->
*** alexpilotti has joined #openstack-infra03:03
mattoliverauasalkeld: its workflow -103:03
asalkeldthat's not approved?03:03
mattoliverauthats workflow +103:04
mattoliverauthere should be a label so you can't get it wrong03:04
asalkeldwhat I mean is approved == workflow +1, never approve == workflow -103:04
*** alexpilotti has quit IRC03:04
asalkeldnot exactly the same as wip03:05
asalkeldnothing telling me someone is working on it03:05
*** zz_gondoi is now known as gondoi03:05
ptoohillmattoliverau, Issue has been resolved, made my first commit! Thanks again03:06
mattoliverauasalkeld: yeah, it isn't the best, so now a big red X in workflow means don't review its a WIP.03:07
*** gondoi is now known as zz_gondoi03:07
mattoliverauptoohill: excelent, nice work!03:07
asalkeldgrumble, grumble ...03:07
*** amcrn has joined #openstack-infra03:07
*** Ryan_Lane has joined #openstack-infra03:07
*** talluri has joined #openstack-infra03:08
mattoliverauasalkeld: yeah, new gerrit gives us an API so hopefully we can do something with that.03:08
mattoliverauin the meantime grumble away :P03:08
clarkbyou can search on it too03:08
asalkeldlol so now we have create our own ui03:08
clarkbworkflow>=0 to ignore wip03:08
asalkeldthanks - later (sorry for the grumbling)03:10
*** asalkeld has left #openstack-infra03:10
*** nati_ueno has quit IRC03:10
*** marun has joined #openstack-infra03:11
*** nati_ueno has joined #openstack-infra03:11
mattoliverauclarkb: yeah good point :) oh and thanks for helping in channel even though you should be vacationing!03:12
*** crc32 has quit IRC03:13
openstackgerritA change was merged to openstack-infra/config: Bump check-rally-dsvm-{name} job timeout
*** JayF has quit IRC03:20
*** JayF has joined #openstack-infra03:20
*** manishg has joined #openstack-infra03:20
*** chandan_kumar has joined #openstack-infra03:21
*** manishg has quit IRC03:25
*** manishg has joined #openstack-infra03:26
*** ianw has quit IRC03:27
*** ianw has joined #openstack-infra03:28
*** pcrews_ has quit IRC03:28
*** resker has joined #openstack-infra03:28
fungiptoohill: was the error you were getting trying to update your contact information covered by the answer at
ptoohillfungi, i didnt see this one, nor try anything else. i just used another account :/03:30
*** kashyap has quit IRC03:30
*** manishg has quit IRC03:30
fungiptoohill: ahh, well the faq there should hopefully cover most common cla/contact-info related errors03:30
*** manishg has joined #openstack-infra03:31
ptoohillfungi: ill keep that one in mind if error comes up again, but i wanted to actually use this new email anyhow, thank you though!03:31
*** chandan_kumar has quit IRC03:32
*** esker has quit IRC03:32
fungiin other news, there are devstack-f20 nodepool nodes now, and the devstack-precise and bare-centos6 node images i updated didn't break, so we should be all set03:32
*** kashyap has joined #openstack-infra03:35
*** etoews has joined #openstack-infra03:36
*** manishg_ has joined #openstack-infra03:36
*** manishg has quit IRC03:36
*** freyes has quit IRC03:36
HenryGHi. What bug can I use to reverify ?03:38
HenryGHere is the failure:
*** etoews has quit IRC03:40
*** manishg_ has quit IRC03:40
*** ArxCruz_ has quit IRC03:40
*** manishg has joined #openstack-infra03:41
*** jhesketh_ has joined #openstack-infra03:44
*** manishg has quit IRC03:46
*** manishg has joined #openstack-infra03:46
*** nati_ueno has quit IRC03:50
*** zz_gondoi is now known as gondoi03:50
*** manishg has quit IRC03:51
*** mfer has joined #openstack-infra03:51
*** chandan_kumar has joined #openstack-infra03:51
*** gondoi is now known as zz_gondoi03:51
*** manishg has joined #openstack-infra03:52
*** Ryan_Lane has quit IRC03:54
*** zz_gondoi is now known as gondoi03:54
*** Ryan_Lane has joined #openstack-infra03:55
*** mfer has quit IRC03:56
*** manishg has quit IRC03:56
*** manishg has joined #openstack-infra03:57
*** manishg has quit IRC04:01
*** manishg has joined #openstack-infra04:02
*** gondoi is now known as zz_gondoi04:04
*** manishg has quit IRC04:07
*** manishg has joined #openstack-infra04:07
*** mattymo|home has joined #openstack-infra04:09
*** matsuhashi has quit IRC04:10
*** manishg has quit IRC04:12
*** manishg has joined #openstack-infra04:12
*** praneshp has quit IRC04:17
*** praneshp_ has joined #openstack-infra04:17
*** manishg has quit IRC04:17
*** manishg has joined #openstack-infra04:18
*** manishg has quit IRC04:22
*** manishg has joined #openstack-infra04:23
*** nati_ueno has joined #openstack-infra04:24
*** yfried has quit IRC04:25
*** nosnos has quit IRC04:26
*** manishg has quit IRC04:27
*** Longgeek has joined #openstack-infra04:27
*** manishg has joined #openstack-infra04:28
*** manishg has quit IRC04:32
*** manishg has joined #openstack-infra04:33
*** manishg_ has joined #openstack-infra04:36
*** manishg has quit IRC04:38
*** manishg_ has quit IRC04:40
*** manishg has joined #openstack-infra04:41
tchaypodstufft: so all the tests pass now.04:43
tchaypoI'm not entirely certain if the message body that is being passed back is something useful though.04:43
*** nati_ueno has quit IRC04:45
*** manishg_ has joined #openstack-infra04:46
*** manishg has quit IRC04:46
*** nati_ueno has joined #openstack-infra04:47
tchaypobut since I don'tt hink that message ever gets utilised anywhere I don't think it matters04:48
*** manishg_ has quit IRC04:50
*** alexpilotti has joined #openstack-infra04:51
*** manishg has joined #openstack-infra04:51
*** manishg has quit IRC04:56
*** manishg has joined #openstack-infra04:56
*** alexpilotti has quit IRC04:56
mattoliveraufungi: thanks for the FAQ link, I'll add it to my list of bookmarks for future reference :)04:56
*** manishg has quit IRC05:01
*** manishg has joined #openstack-infra05:02
*** ildikov has quit IRC05:03
*** matsuhashi has joined #openstack-infra05:04
*** manishg has quit IRC05:06
*** manishg has joined #openstack-infra05:07
*** UtahDave has joined #openstack-infra05:08
*** morganfainberg is now known as morganfainberg_Z05:10
*** manishg has quit IRC05:11
*** manishg has joined #openstack-infra05:12
*** nosnos has joined #openstack-infra05:16
*** manishg has quit IRC05:17
*** kashyap has quit IRC05:17
*** manishg has joined #openstack-infra05:17
*** kashyap has joined #openstack-infra05:17
ianwfungi: still around?05:19
*** zhiyan is now known as zhiyan_05:20
*** manishg has quit IRC05:21
*** nosnos has quit IRC05:22
*** manishg has joined #openstack-infra05:22
openstackgerritA change was merged to openstack/requirements: Limit psutil to <2.0.0
*** nosnos has joined #openstack-infra05:23
*** AaronGreen has joined #openstack-infra05:25
*** manishg has quit IRC05:27
*** manishg has joined #openstack-infra05:28
*** AaronGr has quit IRC05:29
*** matsuhashi has quit IRC05:30
*** matsuhashi has joined #openstack-infra05:30
*** manishg has quit IRC05:32
*** manishg has joined #openstack-infra05:33
*** Ryan_Lane has quit IRC05:33
*** Ryan_Lane has joined #openstack-infra05:34
*** matsuhashi has quit IRC05:36
*** matsuhashi has joined #openstack-infra05:36
*** etoews has joined #openstack-infra05:37
*** manishg has quit IRC05:38
*** manishg has joined #openstack-infra05:38
*** manishg has quit IRC05:43
*** yfried has joined #openstack-infra05:43
*** manishg has joined #openstack-infra05:44
*** etoews has quit IRC05:45
*** resker has quit IRC05:48
*** UtahDave has quit IRC05:49
*** manishg has quit IRC05:49
*** manishg has joined #openstack-infra05:49
*** ildikov has joined #openstack-infra05:49
*** manishg has quit IRC05:53
*** amcrn has quit IRC05:54
*** manishg has joined #openstack-infra05:54
openstackgerritAndreas Jaeger proposed a change to openstack/requirements: Update openstack-doc-tools to 0.13 to fix api-site
*** manishg has quit IRC05:59
*** zul has quit IRC05:59
*** manishg has joined #openstack-infra06:00
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Publish translated localized API documents
*** manishg has quit IRC06:04
*** manishg has joined #openstack-infra06:05
*** rcarrill` has joined #openstack-infra06:05
*** rcarrillocruz has quit IRC06:07
*** melwitt has quit IRC06:08
yfriedanyone here familiar with logstash? I have some questions about these filters
*** WaltNik has joined #openstack-infra06:10
*** manishg has quit IRC06:10
*** manishg_ has joined #openstack-infra06:10
*** afazekas has joined #openstack-infra06:14
*** zul has joined #openstack-infra06:14
yfriedmattoliverau: ^06:14
*** manishg_ has quit IRC06:15
*** manishg has joined #openstack-infra06:16
*** medieval1 has quit IRC06:17
*** viktors|afk is now known as viktors06:19
*** Ryan_Lane has quit IRC06:19
*** medieval1 has joined #openstack-infra06:20
*** Ryan_Lane has joined #openstack-infra06:20
*** manishg has quit IRC06:20
mattoliverauyfried: Not really up to speed with logstash, I plan to familarise myself with it when I get the chance as quite a few questions seem to be popping up about it. Feel free to ask, if I can't answer then they will be in scroll back for someone too when someone who knows something comes online.06:21
*** manishg has joined #openstack-infra06:22
*** jcoufal has joined #openstack-infra06:24
*** manishg has quit IRC06:26
*** e0ne has joined #openstack-infra06:26
yfriedq1: what is the multi line filter for? shouldn't each line of log be parsed independantly?06:26
*** manishg has joined #openstack-infra06:27
*** jlibosva has joined #openstack-infra06:29
*** ArxCruz has joined #openstack-infra06:30
*** _nadya_ has joined #openstack-infra06:31
*** manishg has quit IRC06:32
*** manishg has joined #openstack-infra06:32
*** ArxCruz has quit IRC06:35
*** manishg has quit IRC06:36
*** jcoufal has quit IRC06:37
*** jcoufal has joined #openstack-infra06:37
yfriedI assume it is to catch traceback messages, so that anything without timestamp is part of previous trace, but that's not how logs are built06:37
*** manishg has joined #openstack-infra06:37
clarkbyfried that is why. I dont understand your statement about how they are built06:39
*** alexpilotti has joined #openstack-infra06:39
yfriedclarkb: look for "self." to find a tracebak06:40
clarkbwell its not just tracebacks06:41
yfriedclarkb: what do you mean?06:41
clarkbits any line without a timestamp belongs to the previous line06:41
yfriedclarkb: ok06:41
mattoliverauyfried: good question.. while clark it taking a look, I'm going to take the chance to drive to the otherside of town for dinner with the inlaws.06:41
mattoliveraunight all, if I don't jump back online post dinner06:41
yfriedmattoliverau: tnx06:41
*** manishg has quit IRC06:42
clarkbyfried: there are other occurrences of multine logs06:42
*** manishg has joined #openstack-infra06:42
yfriedclarkb: but traceback in particular DO have timestamps. how would the match the previous line?06:42
yfried**they match06:43
clarkbtracebacks with the exception level are matched on log level06:43
yfriedclarkb: what do you mean?06:43
*** mkoderer has joined #openstack-infra06:44
clarkbwhen they have a timestamp they have a different log level. that log level is used to collapse them together06:44
*** alexpilotti has quit IRC06:45
yfriedclarkb: you mean loglevel = "TRACE" ?06:45
*** e0ne has quit IRC06:45
*** manishg has quit IRC06:47
yfriedclarkb: this?
*** manishg has joined #openstack-infra06:47
*** flaper87|afk is now known as flaper8706:49
yfriedclarkb: does it mean all TRACE msgs are a single muiltiline, or do we assume all lines of a single trace have the exact same timestamp and NUMBER and each pare (TS:NUM) creates a new mulline match?06:49
*** esker has joined #openstack-infra06:49
clarkbthey dont have the same timestamp and number. that is a regex06:50
clarkbit means all trace lines that havent been broken by a regex that doesnt match are a single multiline06:50
*** flaper87 is now known as flaper87|afk06:51
yfriedclarkb: oh. so if for some reasone, a non-match line appears in the middle of a trace - a new trace is created by logstash?06:51
yfriedclarkb: tnx06:51
clarkbyes anything that doesnt match breaks the event into a new event06:52
yfriedclarkb: tnx06:52
yfriedclarkb: new question:06:52
yfriedwhat's the meaning of those mutate filters?06:52
*** manishg has quit IRC06:52
yfriedclarkb: why do they remove a field?06:52
*** manishg has joined #openstack-infra06:53
clarkbthe first one replaces the message with the constructed message it doesnt remove it completely06:54
*** esker has quit IRC06:54
clarkbthe second one removes the now redundant fields06:54
*** andreykurilin_ has joined #openstack-infra06:55
*** marun has quit IRC06:55
yfriedclarkb: why is logdate redundant?06:55
clarkbbecause it is in the timestamp field06:56
clarkbthe date filter does that06:56
yfriedclarkb: ok06:56
yfriedclarkb: this is nit: since we now have logmsg and msg, why move logmsg into msg and delete logmsg? why not simply delete msg?06:57
*** manishg has quit IRC06:57
yfriedclarkb: or might there by events without logmsg?06:57
clarkbbecause logstash convention is that the message is in message06:58
yfriedclarkb: :)06:58
yfriedclarkb: tnx06:58
*** manishg has joined #openstack-infra06:58
yfriedclarkb: you helped me alot. will you be online for the following hours, or is it the end of your day?06:59
clarkbthis is end of day06:59
clarkband technically out this week soprobably not much irc until next week07:00
yfriedclarkb: that's too bad. you seem like to only logstash expert around07:00
yfriedclarkb: or do you know of anyone else?07:00
yfriedwho might also be in Europe timezone?07:01
clarkbI'm sure others can sort it out07:01
clarkbits mostly just regexes07:01
yfriedclarkb: about the multiline - shouldn't no-timestamped line somehow get detailes from the previously stamped line (it's usually a continuation of a debug msg)?07:03
*** manishg_ has joined #openstack-infra07:03
*** manishg has quit IRC07:03
*** flaper87|afk is now known as flaper8707:03
clarkbyes that is what multiline does07:04
yfriedclarkb: ok. thnx07:04
*** ArxCruz has joined #openstack-infra07:04
yfriedclarkb: why add "recieved_at"? isn't it duplication of timestamp?07:06
openstackgerritVictor Sergeyev proposed a change to openstack-infra/config: Import oslo.db repository
*** manishg_ has quit IRC07:07
*** matsuhashi has quit IRC07:08
clarkbno received at is when logstash received the log line07:08
*** manishg has joined #openstack-infra07:08
clarkbit gauges processing delay07:09
*** pblaho has joined #openstack-infra07:10
*** matsuhashi has joined #openstack-infra07:11
*** manishg has quit IRC07:13
*** manishg has joined #openstack-infra07:14
yfriedclarkb: I'm trying to ship this file:
yfriedclarkb: using this
yfriedclarkb: all 33 lines fail to ship, unless I completely disable the filter section07:17
*** Daviey has joined #openstack-infra07:17
*** matsuhashi has quit IRC07:17
*** ekarlso has joined #openstack-infra07:17
*** matsuhashi has joined #openstack-infra07:17
*** _nadya_ has quit IRC07:18
yfriedclarkb: these are the logs of the 2 tries:07:18
yfriedclarkb: with filteres -
*** manishg has quit IRC07:18
yfriedclarkb: without filter -
*** wenlock has quit IRC07:19
*** manishg has joined #openstack-infra07:19
*** talluri has quit IRC07:21
clarkbyfried: sorry internet connection is terrible I wont be able to debug that07:21
*** _nadya_ has joined #openstack-infra07:21
*** manishg_ has joined #openstack-infra07:24
*** manishg has quit IRC07:24
*** _nadya_ has quit IRC07:24
*** mattymo|home has quit IRC07:25
yfried clarkb: could you think of a way to catch all filttered out events?07:27
*** manishg_ has quit IRC07:28
clarkbthere shouldnt be any filtered out as you dont have any drops07:28
*** marun has joined #openstack-infra07:29
clarkbI would test with the stdout output though07:29
*** manishg has joined #openstack-infra07:29
*** nati_ueno has quit IRC07:29
*** _nadya_ has joined #openstack-infra07:30
clarkbthe last bit of multiline data wont flush until a non matching line arrives or a timeout is hit though07:30
*** amcrn has joined #openstack-infra07:30
*** mattymo|home has joined #openstack-infra07:30
*** andreykurilin_ has quit IRC07:30
*** manishg has quit IRC07:34
*** manishg has joined #openstack-infra07:35
*** praneshp_ has quit IRC07:38
*** manishg has quit IRC07:39
*** manishg has joined #openstack-infra07:40
*** etoews has joined #openstack-infra07:42
*** hashar has joined #openstack-infra07:43
yfriedclarkb: I disabled the multiline. now I only have 6 (instead of 33) events, and all are marked with _grokparsefaiure tag and their msg is replaced by "%{logmessage}"07:45
*** manishg has quit IRC07:45
*** manishg_ has joined #openstack-infra07:45
*** romcheg1 has joined #openstack-infra07:45
romcheg1Guys, is anyone around who can help me with the stuck review?07:46
romcheg1 it was approved but gate jobs did not start07:47
*** manishg_ has quit IRC07:49
*** etoews has quit IRC07:50
*** manishg has joined #openstack-infra07:50
*** esker has joined #openstack-infra07:50
*** ArxCruz has quit IRC07:50
*** Shivraam has joined #openstack-infra07:51
*** medieval1 has quit IRC07:51
yfriedclarkb: maybe I need the recieving server configured to catch multiline? (input codec)?07:51
*** e0ne has joined #openstack-infra07:52
*** talluri has joined #openstack-infra07:52
*** esker has quit IRC07:54
*** talluri_ has joined #openstack-infra07:55
*** manishg has quit IRC07:55
*** manishg_ has joined #openstack-infra07:55
*** zigo_ is now known as zigo07:55
*** talluri has quit IRC07:55
*** medieval1 has joined #openstack-infra07:56
*** jpich has joined #openstack-infra07:57
*** manishg_ has quit IRC07:59
*** manishg has joined #openstack-infra08:00
*** Shivraam has quit IRC08:02
*** ildikov_ has joined #openstack-infra08:04
*** cody-somerville has quit IRC08:04
*** ildikov has quit IRC08:04
*** manishg has quit IRC08:04
*** manishg has joined #openstack-infra08:05
*** e0ne has quit IRC08:05
*** darraghb has joined #openstack-infra08:08
openstackgerritCarl Loa Odin proposed a change to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality
*** manishg has quit IRC08:09
*** manishg has joined #openstack-infra08:10
*** e0ne has joined #openstack-infra08:13
*** manishg has quit IRC08:15
*** manishg has joined #openstack-infra08:15
*** mrmartin has joined #openstack-infra08:16
*** ihrachyshka has joined #openstack-infra08:17
*** cody-somerville has joined #openstack-infra08:19
*** cody-somerville has quit IRC08:19
*** cody-somerville has joined #openstack-infra08:19
*** manishg has quit IRC08:19
*** manishg has joined #openstack-infra08:20
*** derekh has joined #openstack-infra08:21
openstackgerritBenjamin Staffin proposed a change to openstack-infra/jenkins-job-builder: Tiny spelling fix: YMAL -> YAML
*** manishg_ has joined #openstack-infra08:25
*** manishg has quit IRC08:25
*** mrmartin has quit IRC08:28
*** manishg_ has quit IRC08:30
*** manishg has joined #openstack-infra08:30
*** jooools has joined #openstack-infra08:31
openstackgerritCarl Loa Odin proposed a change to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality
*** manishg has quit IRC08:35
*** manishg has joined #openstack-infra08:36
*** katyafervent is now known as katyafervent_awa08:37
*** manishg has quit IRC08:40
openstackgerritNikolay Mahotkin proposed a change to openstack-infra/config: Add mistral-dashboard project to stackforge
openstackgerritCarl Loa Odin proposed a change to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality
*** manishg has joined #openstack-infra08:42
*** katyafervent_awa is now known as katyafervent08:45
*** ArxCruz has joined #openstack-infra08:46
*** etoews has joined #openstack-infra08:46
*** manishg has quit IRC08:47
*** manishg has joined #openstack-infra08:47
*** romcheg1 has quit IRC08:49
*** ArxCruz has quit IRC08:50
*** ociuhandu has quit IRC08:51
*** etoews has quit IRC08:51
*** manishg has quit IRC08:52
*** jhesketh_ has quit IRC08:52
*** manishg has joined #openstack-infra08:52
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard-webclient: TimeLine events support
*** proffalken has quit IRC08:57
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Introducing the event timeline
*** yjiang has quit IRC08:57
*** manishg has quit IRC08:57
*** manishg has joined #openstack-infra08:58
*** Ryan_Lane has quit IRC09:01
*** manishg has quit IRC09:02
*** manishg has joined #openstack-infra09:03
openstackgerritNikolay Mahotkin proposed a change to openstack-infra/config: Add mistral-dashboard project to stackforge
*** jhesketh_ has joined #openstack-infra09:07
*** manishg has quit IRC09:07
*** manishg has joined #openstack-infra09:07
*** manishg_ has joined #openstack-infra09:12
*** manishg has quit IRC09:12
rossella_sHello all! I am having problems accessing . I am based in Italy, if I use a VPN I can load the page without any problem.09:12
BobBalltry restarting your browser session rossella_s09:12
rossella_sBobBall: I tried that and no joy09:13
rossella_sBobBall ssh is also not working...if I do ssh -p 29418 gerrit ls-projects09:14
rossella_sit hangs09:14
BobBallthat's a bigger problem :)09:14
BobBallhangs as in firewall eventually blocks it?09:14
rossella_sBobBall: yes :(09:14
*** manishg has joined #openstack-infra09:14
BobBallDid you know that the IP for has changed?  you might need to  hange your firewall config09:14
rossella_sBobBall: yes I know the IP changed09:15
BobBallAlso if you're using IPv6 I think some were having problems yesterday09:15
BobBallso try ipv409:15
openstackgerritNikolay Mahotkin proposed a change to openstack-infra/config: Add mistral-dashboard project to stackforge
rossella_sBobBall: I am using ipV409:16
*** manishg_ has quit IRC09:16
BobBallthen I'm afraid I'm out of ideas :/09:16
BobBallthat's all I've heard of as being the cause09:17
BobBallMy issues were the browser-restart one and the new IP being blocked by our corporate firewall09:17
rossella_sBobBall: my internet provider has a firewall, maybe that's where it's being blocked. How can I check that?09:18
rossella_sBobBall: thanks a lot for your patience!09:18
rossella_sBobBall: it's very unlikely that it's blocking it...anyway it's worth making sure09:18
*** manishg_ has joined #openstack-infra09:19
*** manishg has quit IRC09:19
BobBallIf you've going straight to your ISP then it's very unlikely to be blocked there since it's an outgoing connection not an incoming one09:19
BobBalland, as it worked before, they aren't blocking the port.09:19
BobBallYou'll probably have to wait for some of the infra guys who know what's changed to get more advice on this one :/09:20
BobBallone more thought09:20
BobBallwhat does resolve to?09:20
BobBallif you added it to /etc/hosts then that'll be the problem09:20
rossella_sresolves to
BobBallworth a shot09:21
BobBallthat's the right IP09:21
rossella_sI already purged the know host file09:21
rossella_sBobBall: thanks a lot for your help!09:22
*** manishg has joined #openstack-infra09:23
*** manishg_ has quit IRC09:23
*** manishg_ has joined #openstack-infra09:26
*** manishg has quit IRC09:27
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Move status stylesheets into css file
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Add in sparklines to status page pipelines
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Make zuul javascript app more modular
*** ominakov has joined #openstack-infra09:29
*** manishg_ has quit IRC09:30
*** andreaf has joined #openstack-infra09:30
*** proffalken has joined #openstack-infra09:31
*** manishg has joined #openstack-infra09:31
*** manishg_ has joined #openstack-infra09:35
*** manishg has quit IRC09:35
*** manishg has joined #openstack-infra09:38
*** manishg_ has quit IRC09:39
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Make zuul javascript app more modular
*** manishg has quit IRC09:42
*** manishg has joined #openstack-infra09:43
openstackgerritJoshua Hesketh proposed a change to openstack-infra/config: Fix zuul's app.js name
*** che-arne has joined #openstack-infra09:43
*** manishg has quit IRC09:47
*** manishg has joined #openstack-infra09:48
*** rcarrillocruz has joined #openstack-infra09:49
*** etoews has joined #openstack-infra09:50
*** esker has joined #openstack-infra09:50
*** rcarrill` has quit IRC09:50
*** yaguang has quit IRC09:51
*** manishg has quit IRC09:52
*** manishg_ has joined #openstack-infra09:52
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Fix up fetching jquery.visibility
*** etoews has quit IRC09:54
*** esker has quit IRC09:54
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Move status stylesheets into css file
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Fix up fetching jquery.visibility
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Add in sparklines to status page pipelines
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Make zuul javascript app more modular
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Fix pep8 issue with test_scheduler
*** manishg_ has quit IRC09:56
*** manishg has joined #openstack-infra09:57
jhesketh_is anybody who was involved in the gerrit upgrade around?09:57
jhesketh_fungi: ping09:58
*** manishg has quit IRC10:01
*** manishg has joined #openstack-infra10:03
*** manishg_ has joined #openstack-infra10:07
*** manishg has quit IRC10:08
*** rcarrill` has joined #openstack-infra10:09
*** alexpilotti has joined #openstack-infra10:10
*** manishg_ has quit IRC10:11
*** rcarrillocruz has quit IRC10:11
*** manishg has joined #openstack-infra10:12
BobBalldon't think so jhesketh_ - they will all be sleeping10:13
jhesketh_figured, all good :-)10:13
BobBallactually sdague may be around at some point10:13
BobBalldepends on the level of involvement you care about :)10:14
jhesketh_just curious to how a patch that fails pep8 got through the gate.. maybe we're not gating properly for example10:14
jhesketh_it's okay, I'll grab some dinner10:14
*** jp_at_hp has joined #openstack-infra10:14
BobBallCurious indeed!10:15
*** manishg has quit IRC10:16
*** dizquierdo has joined #openstack-infra10:17
*** manishg has joined #openstack-infra10:17
sdaguewhat's up?10:19
sdaguejhesketh_: pointer?10:19
BobBallsdague: I think it was a question about how the pep8 violation fixed by got merged10:19
BobBall& whether the gate is working correctly (I guess it might have been force-pushed during the upgrade)10:20
*** talluri_ has quit IRC10:20
*** nosnos has quit IRC10:21
*** manishg has quit IRC10:21
SergeyLukjanovmorning folks10:22
*** manishg has joined #openstack-infra10:22
*** ociuhandu has joined #openstack-infra10:23
sdagueBobBall: looks like fungi merged it manually10:23
sdaguethe Verified vote is not from Jenkins10:23
*** manishg has quit IRC10:27
*** manishg has joined #openstack-infra10:28
*** nosnos has joined #openstack-infra10:29
*** manishg has quit IRC10:32
*** manishg has joined #openstack-infra10:33
*** markmc has joined #openstack-infra10:34
*** manishg has quit IRC10:38
BobBallthere you fair enough10:38
BobBall-there you!10:38
jhesketh_sdague: didn't realise you could merge it through manually10:38
jhesketh_sdague: doesn't gerrit wait for --submit from zuul?10:38
*** manishg has joined #openstack-infra10:38
*** etoews has joined #openstack-infra10:39
*** talluri has joined #openstack-infra10:41
sdaguejhesketh_: infra-core can direct merge10:41
sdaguethere are times when it's needed10:41
sdaguenormal humans can't10:41
jhesketh_sdague: fair enough10:42
jhesketh_do other cores for other projects have that?10:42
*** manishg has quit IRC10:42
*** manishg has joined #openstack-infra10:43
*** etoews has quit IRC10:43
bookwarcan anyone help with gerrit authorization, here are the logs and we tried hard to ensure that the key stored is the right one, can someone check it on the server side?10:47
jhesketh_ah okay10:47
jhesketh_do infra have verify +2 rights on other projects?10:47
*** vkramskikh has joined #openstack-infra10:47
*** manishg has quit IRC10:47
*** manishg has joined #openstack-infra10:48
sdagueproject bootstrappers has all the permissions10:48
sdagueand infra core can add themselves to that group10:48
sdaguewhen needed10:48
sdaguethey don't tend to keep themselves in there10:48
jhesketh_sdague: so how do the merges get triggered.. I thought you needed to send the submit command, or is gerrit instructed to simply look for the Verified+2?10:50
*** openstackgerrit has quit IRC10:51
sdaguejhesketh_: that I'm not 100% sure on. fungi would know10:52
*** manishg has quit IRC10:52
*** manishg has joined #openstack-infra10:52
jhesketh_sure, thanks for the clarifications though :-)10:52
sdagueI think zuul is issuing the submit10:52
sdagueonce things are satisfied10:53
*** vponomaryov has quit IRC10:53
jhesketh_hmm, not sure about that, looking at the commit that went through the merge was listed under Jeremy's user, not Jenkins'10:53
*** ominakov has quit IRC10:54
*** miarmak has quit IRC10:54
*** achuprin has quit IRC10:54
*** vponomaryov has joined #openstack-infra10:56
*** yamahata has quit IRC10:56
*** ominakov has joined #openstack-infra10:56
*** miarmak has joined #openstack-infra10:56
*** manishg has quit IRC10:57
*** achuprin has joined #openstack-infra10:57
*** manishg has joined #openstack-infra10:57
*** manishg has quit IRC11:01
*** manishg has joined #openstack-infra11:02
*** manishg has quit IRC11:06
*** e0ne has quit IRC11:07
*** manishg has joined #openstack-infra11:07
*** manishg has quit IRC11:11
yfriedanyone here familiar with logstash?11:11
*** manishg has joined #openstack-infra11:11
*** manishg has quit IRC11:16
*** manishg has joined #openstack-infra11:17
*** manishg has quit IRC11:22
*** adalbas has joined #openstack-infra11:23
*** manishg has joined #openstack-infra11:23
SergeyLukjanovyfried, clarkb, but he's on vacation now11:27
*** manishg has quit IRC11:28
yfriedSergeyLukjanov: yeah. we talked earlier. anyone else?11:28
SergeyLukjanovyfried, IIRC nope11:29
*** manishg has joined #openstack-infra11:29
*** manishg has quit IRC11:33
*** manishg has joined #openstack-infra11:34
*** manishg has quit IRC11:39
*** mwagner_lap has quit IRC11:39
*** manishg has joined #openstack-infra11:40
*** manishg_ has joined #openstack-infra11:44
*** manishg has quit IRC11:44
*** rlandy has joined #openstack-infra11:46
*** manishg_ has quit IRC11:48
*** manishg has joined #openstack-infra11:50
*** esker has joined #openstack-infra11:50
*** ArxCruz has joined #openstack-infra11:50
*** manishg has quit IRC11:54
*** manishg has joined #openstack-infra11:54
*** esker has quit IRC11:55
*** Longgeek_ has joined #openstack-infra11:55
*** Longgeek has quit IRC11:58
*** manishg has quit IRC11:59
*** Longgeek_ has quit IRC11:59
*** e0ne has joined #openstack-infra12:00
*** manishg has joined #openstack-infra12:01
*** matsuhashi has quit IRC12:02
*** e0ne_ has joined #openstack-infra12:02
*** matsuhashi has joined #openstack-infra12:02
*** e0ne has quit IRC12:02
*** matsuhashi has quit IRC12:03
*** manishg has quit IRC12:05
*** manishg has joined #openstack-infra12:05
*** pdmars has joined #openstack-infra12:06
*** jgallard has joined #openstack-infra12:06
*** e0ne_ has quit IRC12:07
*** nosnos has quit IRC12:09
*** manishg has quit IRC12:10
*** manishg has joined #openstack-infra12:10
*** _nadya_ has quit IRC12:13
*** _nadya_ has joined #openstack-infra12:14
*** pblaho has quit IRC12:15
*** manishg has quit IRC12:15
*** manishg has joined #openstack-infra12:15
*** dprince has joined #openstack-infra12:16
*** skraynev_ has joined #openstack-infra12:16
*** skraynev_ has quit IRC12:16
*** sandywalsh has joined #openstack-infra12:17
*** manishg has quit IRC12:20
*** manishg has joined #openstack-infra12:21
tnurlygayanovhi there!12:22
tnurlygayanovI have a question about the tarballs automated creation for the new releases12:23
tnurlygayanovwe have the repository
tnurlygayanovand we have pushed the new tag 0.0.112:23
tnurlygayanovfor the first stable version of Mistral12:24
tnurlygayanovbut zuul creates tarball with incorrect version 0.0.0:
*** manishg has quit IRC12:25
tnurlygayanovcan anybody describe how we can create the tarbal with version 0.0.1 ?12:25
tnurlygayanovfor other repositories of Mistral zuul creates correct tarball with version number 0.0.112:26
*** skolekonov has joined #openstack-infra12:26
*** mwagner_lap has joined #openstack-infra12:27
*** manishg has joined #openstack-infra12:28
ArxCruzanteaya: the powervc team fix the issue yesterday :)12:28
*** rcarrill` is now known as rcarrillocruz12:28
*** aysyd has joined #openstack-infra12:31
*** manishg has quit IRC12:32
*** maxbit has quit IRC12:32
*** maxbit has joined #openstack-infra12:33
*** manishg has joined #openstack-infra12:33
*** maxbit has quit IRC12:33
*** maxbit has joined #openstack-infra12:36
*** weshay has joined #openstack-infra12:36
*** manishg has quit IRC12:37
*** maxbit has quit IRC12:37
*** manishg has joined #openstack-infra12:38
*** maxbit has joined #openstack-infra12:39
*** rfolco has joined #openstack-infra12:39
*** maxbit has quit IRC12:41
*** lcostantino has joined #openstack-infra12:41
*** maxbit has joined #openstack-infra12:42
*** manishg has quit IRC12:43
SergeyLukjanovtnurlygayanov, hey, do you have link to the tarball job logs?12:44
SergeyLukjanovtnurlygayanov, it looks like err in setup.cfg at least12:45
SergeyLukjanovtnurlygayanov, and is incorrect - it doesn't call pbr
tnurlygayanovSergeyLukjanov: for Mistral repo we have the same error, but tarbal was successfully created.12:46
tnurlygayanovwith correct version
SergeyLukjanovtnurlygayanov, the version is overriden by gerrit12:46
tnurlygayanovok, we will fix all small issues with setup.cfg12:46
SergeyLukjanovtnurlygayanov, but should call pbr for it12:46
SergeyLukjanovtnurlygayanov, otherwise it'll not work12:46
SergeyLukjanovtnurlygayanov, is the correct pbr init12:47
SergeyLukjanovtnurlygayanov, due to the
SergeyLukjanov    name="mistral-extra")12:47
*** esker has joined #openstack-infra12:47
*** vkramskikh has left #openstack-infra12:47
*** rakhmerov has joined #openstack-infra12:47
SergeyLukjanovversion will be 0.0.0 anyway, there is no way to override it12:47
*** julim has joined #openstack-infra12:49
tnurlygayanovSergeyLukjanov: ok, thank you!12:49
anteayaArxCruz: thank you!12:49
anteayaArxCruz: I appreciate you talking to them and following up with me, thank you12:49
ArxCruzanteaya: you're welcome :)12:50
*** mfer has joined #openstack-infra12:51
viktorshello All! Folks, could you please look at the patch to config - (Import oslo.db repository) ? Thanks!12:52
*** dkranz has joined #openstack-infra12:56
*** mbacchi has joined #openstack-infra12:59
*** eharney has joined #openstack-infra13:02
SergeyLukjanovviktors, hey, lgtm13:05
*** Ajaeger has joined #openstack-infra13:06
viktorsSergeyLukjanov: thanks a lot!13:06
*** matty_dubs|gone is now known as matty_dubs13:07
*** smarcet has joined #openstack-infra13:13
*** jgallard has quit IRC13:17
*** mriedem has joined #openstack-infra13:19
*** thomasem has joined #openstack-infra13:19
*** mwagner_lap has quit IRC13:20
*** bknudson has joined #openstack-infra13:22
*** _nadya_ has quit IRC13:28
jeblairjhesketh_: sdague: yes, that was manually merged bypassing gating during the maintenance to fix an issue with zuul itself (the removal of force-message in gerrit broke several of our pipelines).13:28
anteayajeblair: does our gerrit use a self-signed cert?
jeblairjhesketh_: sdague: fixing operational problems is about the only time we do that.13:29
*** mengxd has joined #openstack-infra13:29
jeblairanteaya: no, but review-dev does13:29
anteayajeblair: k, thanks13:29
*** signed8bit has joined #openstack-infra13:32
jeblairis there a way i can make firefox look like firefox and not chrome?  <sigh>13:33
*** dkliban_afk is now known as dkliban13:33
*** yfried has quit IRC13:34
fungijhesketh_: sdague: what was the question?13:35
*** afazekas has quit IRC13:35
jeblairfungi: how did a pep8 violation in zuul merge13:35
*** enykeev has quit IRC13:36
*** skolekonov has quit IRC13:36
jeblairfungi: i answered them above ^13:37
*** mkerrin has quit IRC13:37
jhesketh_jeblair: okay, thanks13:37
jhesketh_jeblair: so how does gerrit know when to actually do a merge?13:38
fungijeblair: yep, i see that now13:38
*** zz_gondoi is now known as gondoi13:38
fungijhesketh_: gerrit is instructed (by someone or something) to merge a change13:38
fungijhesketh_: normally zuul instructs gerrit to merge a change once all jobs succeed on it13:39
jhesketh_fungi: right, so is there some kind of force submit button that core infra people have?13:40
fungijhesketh_: for unusual circumstances (like needing to coordinate merges of things which are blocked on outside alterations, such as a gerrit upgrade) we can force a change to merge with a button, yes13:40
jeblairwe don't normally have extra acls, but we add ourselves to a group that has them when needed13:41
fungijhesketh_: normally we don't "have" that button per se. we self-add to an administrative group which has elevated access within gerrit, and then we see the submit button, verify vote options and so on13:41
jhesketh_right, makes sense :-) thanks13:41
fungiand then remove ourselves from said group once done13:41
fungito avoid... "accidents"13:41
jhesketh_heh, wise13:42
*** mengxd has quit IRC13:42
*** dkehn_ has joined #openstack-infra13:43
fungiand we really, really don't like doing it. because it can lead to issues like the pep8 failure you spotted (we were a bit rushed and not running tests manually locally when scrambling to get everything running again)13:43
jhesketh_jeblair: so with that reporter issue, should the requirement to list reporters at all just be lifted?13:43
*** plars has joined #openstack-infra13:44
jeblairjhesketh_: actually the requirement to list reporters there was lucky and let us do that -- listing a reporter already says "i want you to send a report in this manner".  that's all we want for experimental pipelines (no voting or anything).13:45
jeblairjhesketh_: but we need no arguments if all we're doing is sending a message.13:45
*** dkehn_zzz has quit IRC13:45
*** dkehn_ has quit IRC13:46
jeblairjhesketh_: we might want to find another way of doing that, because, yeah, the braces to make an empty dict are not exactly intuitive13:46
StevenKCould I get some kind souls to nudge into landing?13:46
*** dkehn_ has joined #openstack-infra13:46
*** dkehnx has quit IRC13:46
jeblairjhesketh_: but the general approach of saying "i want gerrit to report success (full stop)" and "i want gerrit to report success and pass these other arguments " seems intuitive13:47
*** dkehn_zzz has joined #openstack-infra13:48
plarsI heard that someone here might have experience with using jenkins open-id for sso along with python-jenkins for job management? We're experimenting with a similar setup and get_jobs() works if I pass username and api token for the password, but not job creation or reconfiguring13:49
*** afazekas has joined #openstack-infra13:49
jeblairi think gerritbot is dead13:49
fungijeblair: are you looking into it? if not, i'll coax it back in13:50
jeblairfungi: on it13:50
*** jgrimm has joined #openstack-infra13:50
jeblairplars: you may need to check the authorization in jenkins to make sure your user has the right perms13:51
jeblairplars: all our job management is done with jenkins-job-builder, so you can check that for examples of how we use python-jenkins to interact13:51
plarsjeblair: yeah, already checked that. The user has full administrative permissions. Even tried giving all the permissions specifically on top of that13:51
anteayaI have gertty running, and have subscribed to one repo stackforge/gertty13:52
*** _nadya_ has joined #openstack-infra13:52
plarsjeblair: I also tried jjb, which I think is using python-jenkins underneath. I still get the 403 when trying to create a job13:52
jeblairplars: do you have xsrf protection enabled in jenkins?  i think python-jenkins recently gained features to deal with that13:53
jeblairplars: but that's a possible cause13:53
*** sdake_ has quit IRC13:53
fungibookwar: has anybody helped you with your authentication problem yet? the error message you pasted looks like you're using ssh-agent but haven't done an ssh-add on that key yet13:54
jeblairplars: also, sometimes in these situations, i add a print statement to python-jenkins to print out the output returned in the response body from jenkins to see if it has any helpful info13:54
bookwarfungi: is already silved, thanks, was issue on our side )13:54
*** freyes has joined #openstack-infra13:54
fungibookwar: okay, cool13:54
jeblair  File "/usr/local/lib/python2.7/dist-packages/irc/", line 892, in send_raw13:55
jeblair    raise ServerNotConnectedError("Not connected.")13:55
plarsjeblair: looks like it's turned off at the moment13:55
*** dkehn_zzz is now known as dkehn13:56
jeblairplars: i'm out of ideas off the top of my head; i'd try getting the output from the 403 then13:56
*** openstackgerrit has joined #openstack-infra13:56
jeblairanteaya: you should have an unreviewed change now :)13:57
*** doug-fish has joined #openstack-infra13:59
anteayajeblair: thanks, it started using all my cpu so I had to shut it down, will start it again in a debugger to see if I can get some info next time14:00
jeblairanteaya: runing it with the '-d' option and tailing ~/.gertty.log might tell you what you need to know.14:01
*** zns has joined #openstack-infra14:01
*** homeless has joined #openstack-infra14:02
anteayajeblair: I'll try that14:02
openstackgerritPierre Padrixe proposed a change to openstack/requirements: Update sphinxcontrib-pecanwsme to 0.8
anteayaI was invoking with `python gertty/`14:03
*** dangers_away is now known as dangers14:03
anteayais that what I should be using?14:03
openstackgerritA change was merged to stackforge/gertty: Fix some diff comment display errors
jeblairanteaya: yep.  'python gertty/ -d' will log debug output14:04
Ajaegerinfra team, previously annegentle and myself could both +2 and approve patches in the api projects like compute-api. We noticed yesterday that we still can +2 but not approve anymore. It works fine for the normal docu projects like openstack-manuals but not for these mixed projects like compute-api.14:04
jhesketh_jeblair: (sorry, had to step away for a sec) right, but I meant what about having no reporters at all.. Should that be something we want possible?14:04
jeblairjhesketh_: oh, yes -- that's how we do the silent pipeline14:05
jhesketh_yeah, I just checked the code14:05
jhesketh_jeblair: right, I misinterpreted your commit msg14:05
jeblairAjaeger: we probably missed an acl change, i'll look14:05
fungiAjaeger: i'll take a look. you're trying to "workflow +1" them now righth? (approve has a new name)14:05
jeblairjhesketh_: sorry, heh, was under duress.  :)14:05
*** esker has quit IRC14:05
*** prad_ has joined #openstack-infra14:05
jeblairfungi: nm, i'll let you handle it if you're on it14:06
jhesketh_jeblair: heh, more likely my fault, it's after midnight here14:06
Ajaegerfungi: I have no workflow buttons at all for compute-api, they only exist for openstack-manuals etc14:06
* hashar sends jhesketh_ to bed14:06
*** esker has joined #openstack-infra14:06
fungiAjaeger: ahh, i bet we missed changing the api-projects meta project acl14:06
jhesketh_hashar: you fail to realise this is when I work best ;-)14:06
jhesketh_although my previous statement disproves that :-P14:06
*** yamahata has joined #openstack-infra14:06
hasharjhesketh_: yeah have the same issue. But have to wake up at 7am no matter what happens (kid!)14:07
jeblairjhesketh_: haha :)14:07
Ajaegerfungi: thanks for checking14:07
Ajaegerhashar: 7am? That's late ;)14:07
jhesketh_yeah, luckily I can choose my hours a bit :-)14:07
jhesketh_but there's also a reason I don't often make it to the 5am infra meetings14:07
openstackgerritAntoine Musso proposed a change to openstack-infra/zuul: cloner to easily clone dependent repositories
fungiAjaeger: yep, that was it... try again14:09
openstackgerritPierre Padrixe proposed a change to openstack/requirements: Update sphinxcontrib-pecanwsme to 0.8
Ajaegerfungi: great, netconn-api and identity-api look fine now (no review open for compute-api ;). Thanks a lot!14:10
fungiAjaeger: you're welcome. apologies for the oversight. we seem to do "special" things for acls on api projects14:11
anteayajeblair: I see the sync number counts down14:11
Ajaegerfungi: yeah, api projects are special in that both project and docu cores have access.14:11
openstackgerritAntoine Musso proposed a change to openstack-infra/zuul: cloner to easily clone dependent repositories
jeblairanteaya: yes, it shows you how many background sync tasks it has going14:12
hasharwould it make sense to have WIP -1  to stick when new patch sets are pushed ?14:12
fungiAjaeger: in theory we could solve that much with group inclusion. looks like what we're doing special in this case is overriding the default state for "Forge Author Identity" and "Forge Committer Identity" on refs/*14:12
* anteaya nods, continues to poke around14:13
jeblairhashar, YorikSar suggested that too.  unfortunately, it's a question with no right answer right now.  half the time people want it to stack, the other half they don't.14:13
boris-42fungi btw bumping timeout works woohoo=)14:13
fungiAjaeger: might be for historical reasons no longer relevant... jeblair: do you happen to remember why they're set up that way?14:13
jeblairhashar, YorikSar: s/stack/stick/.14:14
anteayajeblair: should I expect to see a link to the unreviewed change in the stackforge/gertty repo in my gertty view somewhere?14:14
hasharjeblair: not a big deal. I can just workflow -1 whenever I push a new one :] Thx!14:14
jeblairhashar, YorikSar: when you're continuously developing something, you might want it to stick; but when you say "oh, yeah, i'll fix this thing and push a new patchset" you don't.14:15
jeblairhashar, YorikSar: i think the best answer in the future might be to work on the work-in-progress plugin for gerrit14:15
fungihashar: one downside to making workflow sticky for future patchsets is that it's not a toggle for the change, it's a toggle for multiple parties individually on the change. if a core reviewer adds workflow -1 to your change, you as the change owner will be unable to remove it. currently the solution is a new patchset or hunt them down and beg14:15
anteayatoo bad we can't toggle that in preferences14:15
anteayathen individuals can choose14:15
openstackgerritA change was merged to openstack-infra/config: Import oslo.db repository
anteayasticky -1 workflow or not14:16
jeblairhashar, YorikSar: especially if the plugin can define a magic ref, then git-review could push to 'refs/wip/foo'.14:16
hasharor we could use the draft system but the change is not public14:17
jeblairhashar, YorikSar: then it wouldn't be a two-stage thing (you can push a patchset directly into wip)14:17
fungiperhaps a compromise would be to make owner-provided workflow votes sticky and non-owner workflow votes clear on new patchsets14:17
anteayafungi: that sounds like a good start14:17
jeblairhashar, YorikSar: but at the moment, there's a balance of interests in the workflow -1 system of doing wip -- if we make them sticky, then you can't remove someone else's wip vote.14:18
fungithough maybe still clear owner-added workflow +1 on new patchsets, since that could have unintended consequences for self-approving change owners14:18
anteayaso I see the title "unreviewed changes in stackforge/gertty" and the headers "Number" and "Subject" and the sync countdown and nothing else14:19
jeblairfungi: i'd rather have consistency so i can predict whether the vote will be cleared or not.  i think the best solution is to actually allow the author to express his or her intent when uploading14:19
jeblairanteaya: i merged the change while you weren't running it.  there are no open changes in gertty now.14:20
*** mwagner_lap has joined #openstack-infra14:20
jeblairanteaya: you might want to subscribe to a busier project14:20
jhesketh_jeblair: with the new gerrit I'd like to get the zuul overlay stuff happening.. if you find yourself with some spare time (heh, so sorry!) it'd be awesome to get the zuul javascript stuff merged (tail is now here:
fungijeblair: agreed. perhaps git-review could have a configurable option to check whether the current patchset in gerrit is workflow -1 prior to uploading a new patchset and then automatically reapply it if so14:20
jeblairjhesketh_: yeah, though the hold-up wasn't gerrit -- it was zuul.14:21
anteayajeblair: ah ha, it is the list page that eats up cpu14:21
*** wchrisj has joined #openstack-infra14:21
*** wchrisj has left #openstack-infra14:21
jhesketh_oh I just meant now gerrit is done it'd be a nice feature to have14:21
jhesketh_jeblair: by zuul, do you mean because you wanted to get the javascript stuff less duplicated14:22
anteayaI moved back to the list page, I had subscribed to openstack/tempest earlier and now the highest I can see on the list is python-marconiclient14:22
jeblairjhesketh_: yep -- i consider that a hard requirement before we really start on it -- because i don't want three javascript things that no one maintains.  :)  one is plenty.  :)14:22
anteayait finished giving me the full list and has calmed down again14:23
anteayanope, navigating the list with my arrow keys, up and down arrow seems to have set it off again14:23
jhesketh_jeblair: you'll have to show me how you trick people into doing javascript ;-P14:23
jhesketh_(ie me)14:23
jeblairjhesketh_: i'm so good at it i tricked myself into it.14:24
openstackgerritA change was merged to openstack-infra/zuul: Add queue-name parameter to job
jeblairjhesketh_: anyway, i'll try to get to it; i currently have 82 unreviewed changes in config though, so it might be a little bit.14:25
jhesketh_jeblair: totally understand, I realise it's pretty low priority14:26
jhesketh_and should be14:26
anteayaalso after subscribing, navigation is frozen and cpu usage goes up to around 105%14:26
jeblairfungi: can you give 63190 the once over and aprv?14:26
anteayaperhaps that is to be expected?14:26
jeblairanteaya: not too surprising -- it's not the most efficient program, yet.14:27
*** zns has quit IRC14:28
*** creiht has joined #openstack-infra14:29
creihtdid the ips of change?14:29
anteayajeblair: okay, continuing to poke around14:29
jeblaircreiht: yes14:29
anteayaso far, I am very impressed14:29
creihtI'm getting ssh errors that the signature isn't the same14:29
*** david-lyle has joined #openstack-infra14:29
jeblaircreiht: see
derekhfungi: I see the f20 nodes got built (at least the logs show no errors) thanks for helping it through, no we just have todo something useful with them14:30
creihtjeblair: since I'm here... :)14:30
creihtjeblair: did you all and notmyname get the versioning stuff figured out for swift?14:30
jhesketh_does it bother anybody else that the verified column is not leftmost in gerrit?14:30
anteayaright now we are stuck with it14:31
*** nati_ueno has joined #openstack-infra14:31
jhesketh_it bothers me enough to suggest we rename it CI and go through the pain of fixing all 3rd party systems14:31
anteayait is on the list of things we would like to address14:31
jhesketh_anteaya: upstream with gerrit?14:31
anteayathat is quite the standard of measurement for your pain14:31
jhesketh_well besides 3rd parties we can do the rest in one commit on config14:31
anteayajhesketh_: not sure, probably give upstream a try14:32
jhesketh_it's hurting my brain trying to scan what I need to review14:32
anteayajhesketh_: /me nods14:32
openstackgerritAntoine Musso proposed a change to openstack-infra/zuul: cloner to easily clone dependent repositories
*** gondoi is now known as zz_gondoi14:32
jhesketh_I fear, from what I've heard anyway, that if we went with upstream it'd take forever before it actually changed by which point I'd be used to the new order14:32
* anteaya nods14:32
anteayavery possible14:32
* jhesketh_ nearly submitted a patch to rename the column this afternoon 14:33
anteayajhesketh_: you might like to read this:
jhesketh_oh yeah, I've seen that :-)14:33
*** zns has joined #openstack-infra14:33
anteayajeblair's newest invention14:33
hasharoh men14:33
jhesketh_haven't tried it yet14:33
anteayaI'm playing with it14:33
fungicreiht: i do not believe it's been solved yet. i took a peek at the failure log for the tag merge job and for some reason it's ending up with more than just the merge commit staged (which is causing git-review to abort). i'll see if i can tell what's going on there14:33
anteayavery easy on teh eyes14:33
creihtfungi: ok cool14:33
creihtthanks for taking a look at it14:33
creihtwas just curious where it was14:34
anteayaand since jeblair wrote it, I'm sure you can make it look however you want14:34
fungiderekh: yes, i was able to get the image to build with your patch applied, and then nodes were launched from it, so should be all set for some trial jobs14:34
anteayajhesketh_: the listing and subscribing process is cpu heavy right now, and syncing takes a bit14:35
anteayajhesketh_: but it might be more satisfying than what you are currently staring at14:35
fungicreiht: for the moment, this is the situation...
derekhfungi: cool, you've seen the one I am proposing, anybody else want to take a look at  ?14:36
YorikSarhashar, jeblair, fungi: I think that the best place for WIP-stickiness is in git-review now. We shouldn't stick all -1s in Gerrit because there is (and should be) more than one person to set WIP. Creatign a new plugin for Gerrit might be a good idea but it sounds like it won't come around shortly.14:36
anteayaso what does gertty consider an unreviewed change? any status:open patch on a repo? or do I have to have subscribed to a patch for it to be considered unreviewed for gertty?14:37
jhesketh_okay, I'm off14:37
jhesketh_night all!14:37
anteayajhesketh_: night14:37
anteayasleep well14:37
jeblairYorikSar: the plugin exists, it just requires 2.9 to be fully functional14:38
jeblairfungi: huh, i could have sworn i wrote a patch to fix the merge tag thing.  1 min.14:38
*** zz_gondoi is now known as gondoi14:39
YorikSarjeblair: Oh. Good :) But I think we won't get 2.9 upgrade until Juno...14:39
jeblairfungi: the script is already fixed.14:40
*** wenlock has joined #openstack-infra14:40
fungijeblair: was it fixed after the 17th14:40
jeblairfungi: yes14:41
jeblairfungi: because of that failure14:41
fungiand just not retriggered yet i guess14:41
jeblairfungi: i think mordred was going to run the job again.14:41
openstackgerritA change was merged to openstack-dev/pbr: Make source configurable when using --coverage
fungii wonder now if it will encounter new problems since swift's milestone-proposed branch has been deleted in that time14:41
fungii'll give the retrigger a shot real quicklike14:42
*** dangers is now known as dangers_away14:43
openstackgerritA change was merged to openstack-infra/config: Create a swap partition if needed for new servers
*** tteggel_ has quit IRC14:43
anteayajeblair: emailed you a screenshot of what gertty looks like for me14:43
*** tteggel_ has joined #openstack-infra14:44
*** tteggel_ is now known as tteggel14:44
*** ildikov_ has quit IRC14:44
*** ildikov_ has joined #openstack-infra14:44
fungithough now we have an acl issue, i think, courtesy of gerrit upgrade14:45
fungiand i need to run out to a quick appointment, but will have a look at that as soon as i return14:46
anteayahappy appointment14:46
*** pcrews_ has joined #openstack-infra14:46
anteayaoh I am out for an appointment myself this afternoon, leaving in about 3 hours14:46
tnurlygayanovafter some fixes with / setup.cfg we can see that pbr want to build release
tnurlygayanovhow we can say to pbr that we want release 0.0.2?14:47
tnurlygayanovwe just run: python sdist     - and after that we can see new tarball with version
*** atiwari has joined #openstack-infra14:49
tnurlygayanovwe want to make a release of Mistral:
anteayatnurlygayanov: I know so little about pbr I am useless to you14:50
anteayahopefully someone with more knowledge will show up soon14:50
tnurlygayanovanteaya: ok, thank you, I will wait :)14:51
jeblairtnurlygayanov: you just tag 0.0.2 and push that14:53
jeblairtnurlygayanov: pbr does tag-based post-versioning by default14:53
*** _nadya_ has quit IRC14:54
openstackgerritA change was merged to openstack-infra/config: Create propose_translation_update for horizon
openstackgerritA change was merged to openstack-infra/config: Create propose_translation_update for horizon
*** andreaf has quit IRC14:56
*** rcarrill` has joined #openstack-infra14:56
openstackgerritAntoine Musso proposed a change to stackforge/gertty: Print a friendly user message when conf is missing
jeblairthat was weird14:56
hasharoh man, where is the tox.ini ! :-D14:57
*** rcarrillocruz has quit IRC14:58
*** msabramo has joined #openstack-infra14:58
openstackgerritA change was merged to stackforge/gertty: Print a friendly user message when conf is missing
jeblairi think making it installable is probably a first order requirement for that.  :)15:00
anteayaI'm going with gerritbot had a hiccup15:00
jeblairanteaya: glitch in the matrix15:00
anteayainvoke gertty gerrit15:02
anteayahashar: that gets it to run for you?15:02
jeblairfyi, i'm not really interested in adding pep8 to gertty for a while -- at least not until it settles down a bit.15:02
hasharanteaya: yeah yeah playing with it right now :-]15:03
hasharjeblair: dont you have pep8 errors reported in your text editor ?15:03
jeblair(this is a fun project for me, and hey, it's on stackforge)15:03
anteayahashar: cool15:04
anteayait is great15:04
jeblairhashar: nope!  i don't like computers bossing me around.  ;)15:04
*** Ajaeger has quit IRC15:04
*** jlibosva has quit IRC15:04
anteayaright now crtl-C doesn't kill it, it stops gertty view but logging still happens and I don't get my prompt back15:04
openstackgerritCedric Brandily proposed a change to openstack-infra/git-review: Correct test_remote to support branchs without upstream
anteayaI have to track it down with ps aux and kill it15:05
jeblairanteaya: try ctrl-\15:05
jeblairharlowja_away: pointed out some threads that should be daemonized yesterday.  also, it needs an actual quit command added.  :)15:05
*** zns has quit IRC15:06
anteayahashar: gertty gerrit doesn't work for me: how do you get it working for yourself?15:06
hasharanteaya: how it does not work yet :]15:06
*** zns has joined #openstack-infra15:06
anteayahashar: oh15:07
anteayactrl-\ works15:07
jeblairhashar: also, if you omit the name of the gerrit, it will just use the first one in the file15:07
hasharanteaya: you want:  python gertty/gertty.py15:07
anteayathat is what I am using now, just wondered if I was missing something15:07
*** esker has quit IRC15:08
anteayathe docs are ahead of the code, that doesn't happen often15:08
*** esker has joined #openstack-infra15:08
anteayahashar: what do you see in your screen when you select a view for a subscribed repo?15:09
*** dkranz has quit IRC15:09
hasharit currently tries to fetch /ra/projects/  instead of /r/projects (extra a)15:11
*** jgallard has joined #openstack-infra15:11
anteayaunsubscribing from a repo and then subscriping again to the same repo eats up cpu as well15:11
*** mkoderer has quit IRC15:12
jeblairhashar: i think the url in the config file should have a trailing slash15:12
portantecan somebody offer guidance on how to get a new swift gate/check job in place? We recently added a new tox target, "tox -e func".15:13
portantei imagine it would run like the existing swift py26 and py27 unit tests15:13
anteayaportante: do you have a link to the patch?15:13
*** ildikov_ has quit IRC15:13
hasharjeblair: and there is an extra  'a/' inserted between the url and /projects/ :D15:13
jeblairhashar: that's to make it authenticated15:14
*** signed8bit has quit IRC15:14
chmouelportante: i think it just need to be added to the envlist15:14
portanteso is that a swift tox.ini change?15:15
jeblairhashar: if the url ended with / it would be which should be correct for your setup i think15:15
portanteI thought I already added func to the tox.ini change15:15
*** signed8bit has joined #openstack-infra15:15
jeblairno, we don't run the envlist -- that's just for convenience for developers on their local workstations (so you can say "tox" and run whatever the project admins think is appropriate)15:16
hasharjeblair: ah that is the API entry point. Thanks!15:16
jeblairportante: one sec and i'll grab some links15:17
anteayahashar: so it needs to be r/a not ra/ ?15:17
jeblairportante: you'll want to add a job definition to this file:
jeblairportante: (more to come, gimme a sec)15:18
sdagueso here is a write up of some of the gerrit query changes and how to use them -
sdaguefigured that would be helpful for people to navigate the transition15:18
openstackgerritAntoine Musso proposed a change to stackforge/gertty: Properly register password from prompt
openstackgerritAntoine Musso proposed a change to stackforge/gertty: Normalize URL to ends with a slash
*** pcrews_ has quit IRC15:19
*** thedodd has joined #openstack-infra15:19
*** doddstack has joined #openstack-infra15:20
jeblairwow, we don't have any run-tox jobs right now... where did they go?15:20
anteayahashar: the importance of 91392 is lost on me15:21
anteayamissing jobs?15:21
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Modularize to enable easier testing
jeblairoh they changed to run-unittests, right15:21
hasharanteaya: the 'noop' job is the default one.  I guess James did not had the default python jobs in Jenkins and Zuul yet :]15:21
*** WaltNik has quit IRC15:22
*** talluri has quit IRC15:22
jeblairportante: strike my earlier suggestion15:22
*** che-arne has quit IRC15:22
jeblairportante: i think that job defn (gate-{name}-unittests-{env}) may do what you need...15:23
tnurlygayanovso, problem with tags and tarballs versions was fixed (need just merge all fixes and push tags)15:23
*** thedodd has quit IRC15:23
jeblairportante: add a stanza like this:
jeblairportante: to the swift project in that file, and set env = 'func'15:24
*** dkranz has joined #openstack-infra15:25
*** signed8bit has quit IRC15:25
jeblairportante: then add 'gate-swift-unittests-func' to the pipelines you want it in here:
JayFSo I just figured out a big problem with the new gerrit15:26
JayFwe're getting SPF softfails15:26
JayFwhich is why all the new mail is going into my spam folder15:26
jeblairportante: finally, you may want to add a job stanza with voting:false to this section:
portantejeblair: where is the definition of the job that results here:
portantebecause that is just doing "tox -epy27", and the new one would just be "tox -efunc"15:27
jeblairportante: descriptive text "v=spf1 ~all" <-- and it appears the new gerrit installation is sending locally, not from sendgrid15:27
JayFI'd strongly suggest dropping the SPF record or adding to it15:27
*** signed8bit has joined #openstack-infra15:28
jeblairJayF: the same was true for the old server15:28
JayFHmm. Then I still wonder why it's going to junk instead of inbox since the server change (on hosted mail, so probably everything Microsoft, since Office 365/Hotmail share incoming mail infra afaict)15:29
JayFEither way, this seems like something that's nearly trivial to fix? Do we have automation against DNS where I could file a merge request, or is it all hand-configured?15:29
*** eharney has quit IRC15:30
jeblairJayF: it's possible they incorporate reputation into softfail, and the new server has little/no reputation now.15:30
anteayahashar: so what is the purpose of what is it fixing?15:30
jeblairJayF: unfortunately, no dns automation :(15:30
JayFSo would an email to the list proposing a revised SPF record be the best route?15:30
anteayahashar: it changes the scope of the input, okay sorry15:30
*** mrodden has quit IRC15:31
hasharanteaya: if you are missing the 'password' field, it asks for the password by prompting you.  But your input is not registered in the config object :-(15:31
jeblairJayF: yeah15:31
anteayahashar: okay, i see that now, sorry15:31
anteayamakes sense15:31
JayFjeblair: will-do. Any other mail sending servers in the infra world that I might not know about?15:31
*** sdake_ has joined #openstack-infra15:32
*** sdake_ has quit IRC15:32
*** sdake_ has joined #openstack-infra15:32
jeblairJayF: lists.o.o.  also, they all send mail to me.15:32
*** _nadya_ has joined #openstack-infra15:32
jeblairi think openstackid-dev does, probably groups.15:32
JayFjeblair: for instance, I see MX for goes to Rackspace Email & Apps (woo, I helped build this), does anyone send from those accounts?15:32
jeblairJayF: yep.15:32
anteayamorning zaro15:33
jeblairJayF: honestly, it should probably be +all or ~all.15:33
JayFIt's already ~all15:33
JayFbut softfailing anything not from sendgrid seems worse than +all given that's all that's in the record15:33
jeblairJayF: yeah.  i think the common interpretation of ~all is to accept but give scrutiny -- so i think the issue is less that we're saying "anything not sendgrid bad" and more that the new server is sending a lot of mail with no reputation yet.15:34
JayFI'll take a look at it though, I've worked with mail for a while and it won't take long to assemble a better SPF record.15:34
JayFjeblair: IME, working in email, a softfail is treated much more harshly than that, especially nowadays15:34
jeblairJayF: do you git openstack-dev@lists.o.o email?15:35
JayFalthough reputation does matter quite a bit, if you pass the SPF record, you should inherit the reputation of the domain, which should be pristine15:35
JayFI get that email, and it does go to my inbox properly15:35
portantejeblair, chmouel: so is this then just as easy as modifying swift's tox.ini file to have "func" in the envlist and we're done?15:35
JayFI'm betting it doesn't softfail because lists.o.o doesn't have SPF15:35
portantecause that would be cool15:35
portanteand simple15:35
JayFAuthentication-Results:; spf=none (sender IP is
JayFvs spf=softfail for gerrit15:35
jeblairportante: no, i explained earlier why that's not the case.  see scrollback.  i also detailed what you should do to add the job.15:36
portantek, sorry I did not catch that15:36
jeblairJayF: oh, this is because of <> ?15:37
anteayaportante: timestamp 2014-04-30T15:22:5715:37
JayFyes, absolutely15:37
JayFso it's hitting the SPF record, because that's the domain in the email15:37
JayFif it were, for instance,, spf wouldn't be evaluated and it would pass15:38
JayFwell, not pass, but much better than softfail15:38
portanteanteaya: thanks15:38
JayFsomething like "v=spf1 ~all" would be better and prevent anyone from having to worry about it if gerrit servers change again15:39
JayFI'll look at openstackid-dev as well if I've ever gotten any email from it and pop a mail out to the list15:39
*** msabramo has quit IRC15:40
*** gyee has joined #openstack-infra15:40
*** vponomaryov has left #openstack-infra15:40
jeblairJayF: or we could do +all instead of whack-a-mole. :)15:42
*** che-arne has joined #openstack-infra15:42
JayFI would strongly prefer we get a good SPF record in place with the flexibility to keep things working. If you add +all, then you might as well remove the SPF record altogether :(15:43
*** msabramo has joined #openstack-infra15:43
JayFThe A++ solution would be to get all outgoing mail going through Mailgun or Sendgrid, but that's a much larger undertaking than just trying to improve the SPF record :). Besides, a reasonably setup SPF record with type entries should only have to be modified when new services that send mail come online on the domain (and not a subdomain, like lists.o.o)15:44
jeblairJayF: i don't agree at all that is a good solution.  i'm strongly opposed to paid mail providers being email gatekeepers.15:45
JayFI understand that philosophy, and that's why my actual proposal is to fix the SPF record :).15:45
JayFEasier to trust Mailgun when I can go give their whole tech staff high fives a few steps away.15:45
*** mrodden has joined #openstack-infra15:49
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Make ordereddict optional
hasharjeblair: can you exit gertty by pressing Ctrl + C ?   The app seems to terminate but I am left with a leftover Python process (might be my setup)15:52
*** serverascode has quit IRC15:52
*** UtahDave has joined #openstack-infra15:52
mugsieSergeyLukjanov: i fixed that ACL issue in - is there anything else you think I need to do?15:52
jeblairhashar: no, it needs some threads marked daemon, and also an actual exit command.  i'm using ctrl-\ for now15:52
anteayahashar: ctrl \15:53
openstackgerritA change was merged to stackforge/gertty: Make ordereddict optional
anteayaI asked the same above15:53
hasharI can't do that magic escape commands with my french keyboard :-/15:53
hasharthat gives me :15:53
*** talluri has joined #openstack-infra15:53
hasharto get backslashes I have to alt + shift + :  , doing ctrl + alt + shift + :  just yield :15:54
*** ihrachyshka has quit IRC15:54
hasharI can surely redefine it15:54
*** serverascode has joined #openstack-infra15:54
anteayajeblair: I pulled in that last change15:54
anteayanow I can't get to the list page15:55
anteayapressing l gets me no change15:55
anteayaand somehow I got subscribed to openstack/sahara-dashboard because now I am looking at the repo view15:56
anteayaand now I have the list view15:56
anteayamaybe there is lag?15:57
*** talluri has quit IRC15:58
anteayactrl + alt + shift + : wow, do you have to get help from the cat for that one?15:59
*** andreykurilin_ has joined #openstack-infra15:59
fungiJayF: jeblair: the downside to having gerrit send as something@review.o.o is that unless we also add an mx for review.o.o to somewhere which will accept bounces for that address, sender verification callouts on recipient mtas will start failing (different set of span-prevention-tripwires to deal with)15:59
*** KurtMartin is now known as kmartin15:59
JayFfungi: that's a good point. I just think we should update the SPF record and everything will be fine :)16:00
hasharanteaya: well I should get a qwerty keyboard. Or figure out how to associate a different keystroke to whatever ctrl + \  does :]16:00
anteayaor train the cat16:00
anteayathe cat invokation command16:01
*** packet has joined #openstack-infra16:01
hasharI managed to get something!16:01
hasharbut have to drop /a/ from the URL :/16:01
hasharI can't connect to (the user/pass is rejected hehe)16:02
hasharbut  works fine16:02
*** packet has quit IRC16:03
anteayawhat file did you make that change in?16:03
*** chandan_kumar has quit IRC16:03
*** sdake_ has quit IRC16:03
jeblairhashar: unauthenticated access isn't going to work for most of what gertty does; you'll want to fix that.16:03
*** dkranz has quit IRC16:04
hasharjeblair: I think I got authenticated at that address16:04
*** matty_dubs is now known as matty_dubs|lunch16:04
*** achuprin has quit IRC16:05
*** eharney has joined #openstack-infra16:05
*** afazekas has quit IRC16:05
*** packet has joined #openstack-infra16:05
*** packet has quit IRC16:05
*** packet has joined #openstack-infra16:06
*** sdake_ has joined #openstack-infra16:07
*** marcoemorais has joined #openstack-infra16:08
*** zehicle_at_dell has quit IRC16:08
hasharjeblair: ah thanks. was looking for that :]16:10
openstackgerritVictor Sergeyev proposed a change to openstack/requirements: Add oslo.db library
*** viktors has quit IRC16:12
*** signed8bit has quit IRC16:13
*** signed8bit has joined #openstack-infra16:14
fungijeblair: mordred: so the current acl situation with the tag-merge jobs is that the proposal bot account is not in a group which can push merge commits... presently only project bootstrappers has that... though looking more closely i think the release managers are also going to have a hard time pushing tags as well (right now they're allowed annotated tags but not signed tags)16:14
fungito deal with the tag-merge though, i suggest we add a specific group we can put the proposal bot account in for that access (unless you think one of the existing groups is well suited, though they don't seem like it to me)16:16
*** nati_ueno has quit IRC16:16
openstackgerritMarc Abramowitz proposed a change to openstack-dev/pbr: pbr/ Add logging
*** dkranz has joined #openstack-infra16:17
*** weshay has quit IRC16:18
*** rcarrillocruz has joined #openstack-infra16:18
fungibut separately, i'll switch the annotated tag permissions to signed tag in all-projects unless there are any objections16:18
jeblairfungi: agreed on all counts (i wonder how this was supposed to work before..)16:19
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Fix editing inline comments
fungii have a suspicion that the merge perms were previously being granted by some other permission which got stricter between 2.4 and 2.816:20
*** talluri has joined #openstack-infra16:20
*** rcarrill` has quit IRC16:20
fungifor example, the proposal bot has batch priority in global caps. not sure what that means exactly...16:21
*** hogepodge has joined #openstack-infra16:21
*** marun has quit IRC16:23
kialljeblair: Heya - Hows the tag situation? Should I bother you again tomorrow? ;)16:23
*** weshay has joined #openstack-infra16:24
*** jpich has quit IRC16:24
*** etoews has joined #openstack-infra16:24
*** hemna__ is now known as hemna16:24
*** talluri has quit IRC16:25
*** bhuvan has quit IRC16:25
*** manishg has joined #openstack-infra16:27
*** sdake_ has quit IRC16:29
jeblairkiall: i think we've got it worked out.  do you have a link to the failed job?16:29
kiallYep - It was
kiallBut the tag is unsigned16:30
kiallso needs to be `rm -rf`'d16:30
*** rakhmerov has quit IRC16:31
*** pcrews_ has joined #openstack-infra16:32
jeblairkiall: here are our options:  (a) just run the job again on the unsigned tag and publish that; (b) delete the unsigned tag so you can push a signed version; (c) ignore the unsigned tag and push a signed 2014.1.116:33
anteayajeblair: is there any http response code handling in gertty?16:33
*** markmc has quit IRC16:33
kialljeblair: (B) is ideal.. But sounds like the most painful for you?16:34
jeblairkiall: nah, it's okay.  will be just a minute.16:34
jeblairanteaya: it might log something?16:34
*** krtaylor has quit IRC16:35
jeblairkiall: done16:36
anteayajeblair: looking, I am probably not looking in the right place yet16:36
*** hashar has quit IRC16:38
anteayaI'm getting a lot of this in the logs: is that due to the auth error, hashar?16:40
*** hogepodge has quit IRC16:40
anteayaoh he left16:40
fungijeblair: mordred: notmyname: creiht:
kialljeblair: Huumm - Pushed a new signed tag, missed the zuul status page link to the job, but no tarball produced16:41
notmynamefungi: thanks16:41
*** crc32 has joined #openstack-infra16:41
*** dkliban is now known as dkliban_lunch16:41
kialljeblair: looks to be still a race somewhere?
funginotmyname: you're welcome. jeblair apparently fixed the job a while back, but it didn't get re-run and then some changes in acl behavior in new gerrit broke it again, but all working now i think16:42
kiallThat's a signed tag, so re-triggering should be enough to kick it into action16:42
*** chandan_kumar has joined #openstack-infra16:43
fungikiall: i strongly suspect the degree of staleness for hpcloud-az2 nodepool images is such that the fixed slave scripts are not present on that node16:43
*** romcheg has joined #openstack-infra16:43
* fungi checks16:43
kiallhah :)16:44
*** romcheg has left #openstack-infra16:44
fungikiall: yeah, our most recent bare-precise image in hpcloud-az2 is 348.79 hours old16:44
*** dizquierdo has quit IRC16:45
kiallNice - So failing to rebuild.. Annoying!16:45
fungicourtesy of the nova metadata server not being responsive, causing daily image updates to fail there most of the time16:45
kiallThe "new regions" metadata service is much more reliable, if that's of any value in pointing out ;)16:46
fungiapparently puppet facter wants to query it, then gives up after not getting a response, then puppet treats that as a hard failure and nodepool has to abort the image update16:46
kiall(I know, pros and cons...!)16:46
fungikiall: i can re-trigger the job and see if it gets lucky and runs on a less stale node16:47
*** jp_at_hp has quit IRC16:48
anteayajeblair: yes the http response codes are in the log file:
kiallfungi: ah - roulette, let's play ;)16:48
*** derekh has quit IRC16:48
*** matty_dubs|lunch is now known as matty_dubs16:48
jeblairfungi: this is an important problem, maybe we should remove the stale images16:49
*** bhuvan has joined #openstack-infra16:49
fungijeblair: i'm happy to do that. how old is too old? just remove bare-precise images?16:50
fungiif it's specifically the tag fixes we're concerned about, then bare-precise images prior to the script patch should cover it16:50
fungikiall: got lucky that time...,1/console.html16:51
*** mbacchi has quit IRC16:51
*** hogepodge has joined #openstack-infra16:51
fungilooks like the fix merged 04-29 20:3016:52
fungiso any bare-precise images prior to today's scheduled image update should be deleted16:53
*** annegentle has joined #openstack-infra16:53
jeblairfungi: yeah16:53
nibalizersorry i've been so afk yall16:53
nibalizer$REALJOB has moved into serious overdrive, doesn't leave much energy for OSS16:53
*** harlowja_away is now known as harlowja16:54
funginibalizer: sorry to hear that. on the other hand, the shackles of job security are often padded for comfort ;)16:55
*** etoews has quit IRC16:57
*** yidclare has joined #openstack-infra16:57
kiallfungi: I see a tarball :) Thanks!16:57
kialland jeblair - thanks :)16:57
anteayait looks like there needs to be username:password inserted in here somewhere for the url to be authorized:
*** praneshp has joined #openstack-infra16:59
fungiokay, all bare-precise nodepool images older than yesterday's ggp patch are now gone, and any nodes they begat should die off to attrition fairly rapidly as well16:59
*** mbacchi has joined #openstack-infra17:00
*** nati_ueno has joined #openstack-infra17:00
*** talluri has joined #openstack-infra17:00
*** jamielennox is now known as jamielennox|away17:00
jeblairfungi: thanks; hopefully that will let the job running robots do their jobs (so we don't have to step in for them).17:00
cody-somervilleHow can I delete a bunch of changes from a pipeline?17:00
fungijeblair: agreed17:01
*** nati_ueno has quit IRC17:01
jeblaircody-somerville: 3 options: (a) push new patchsets for all of them.  (b) stop zuul.  (c) write the 'dequeue' command line client17:01
*** praneshp_ has joined #openstack-infra17:02
jeblaircody-somerville: typically (a) is what we do around here; sometimes (b).17:02
*** _nadya_ has quit IRC17:02
*** ominakov has quit IRC17:02
fungithere are definitely times where (c) would have been great to have, since (a) and (b) are both pretty labor-intensive17:03
jeblaircody-somerville: (if they are git-dependent, you might only have to push a new patch for the first one)17:03
*** nati_ueno has joined #openstack-infra17:03
*** hashar has joined #openstack-infra17:04
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Handle "No newline at end of file" and add --no-sync option
*** praneshp has quit IRC17:04
*** praneshp_ is now known as praneshp17:04
*** hashar has quit IRC17:04
*** Ajaeger has joined #openstack-infra17:04
openstackgerritA change was merged to openstack-infra/config: Finish puppeting accessbot
cody-somervilleSo I have a bunch of jobs (~300) in the post pipeline for a bunch of deleted branches on the same repository. I also have a small number of changes unrelated in the check queue. The former seems to be making things get processed very slowly. I have a bunch of ready slaves but zuul is only slowly queuing up the -branch-tarball jobs (and sometimes -coverage jobs) for the deleted branches. It was speculated this might be be du17:06
cody-somervillee to the -branch-tarball job being configured not to run concurrently.17:06
*** chandan_kumar has quit IRC17:06
*** _david_ has joined #openstack-infra17:06
*** SumitNaiksatam has quit IRC17:06
fungicody-somerville: do you have more than one jenkins master? if so, non-concurrency isn't guaranteed anyway17:07
cody-somervilleI had about 20 -branch-tarball jobs queued up in Jenkins - and naturally it made sense they wouldn't be executing in parallel since specifically being configured not to. I thought maybe this was causing zuul to think all the slaves were busy and not dispatch more things that *could* run.17:08
fungiunless you only have one slave for the job with a single executor on it, in which case you're de-facto non-concurrent without actually configuring that option17:08
cody-somervilleSo I deleted the 20 or so -branch-tarball jobs from the jenkins queue but zuul continues to just slowly add -branch-tarball jobs again to jenkin's queue.17:08
jeblaircody-somerville: because we can't guarantee concurrency, we don't have any jobs configured in that way.  you're in untested territory.  however, i would guess the following things are happening:17:09
*** jgallard has quit IRC17:10
*** Ryan_Lane has joined #openstack-infra17:10
jeblaircody-somerville: a) jenkins gearman plugin doesn't know about the non-concurrency aspect and is scheduling jobs as fast as it can on specific nodes.  this means those nodes are unavailable to run other jobs while jenkins waits to run a job on them.17:10
*** annegentle has quit IRC17:10
jeblaircody-somerville: that explains why your check jobs are not running while you appear to have nodes available (they aren't available, they are actually "running" post jobs, they just haven't actually started yet)17:10
*** dangers_away is now known as dangers17:11
jeblaircody-somerville: b) when you remove jobs from the jenkins queue, it would not surprise me if zuul interprets that as a transient jenkins error and reschedules them17:11
harlowjajeblair do u want flake8 and stuff running on gertty?17:11
harlowjajust wondering :)17:11
jeblairharlowja: no, not for a while.  when development slows down a bit.17:12
dhellmanndid viktors ask to have the oslo-db-core team set up in gerrit? could someone add me to that group so I can add everyone else?17:12
msabramowhat is gertty?17:12
msabramogerrit on your tty?17:13
jeblairdhellmann: will do17:13
jeblairmsabramo: you win!17:13
dhellmannjeblair: thanks!17:13
harlowjajeblair sounds good17:13
msabramowoo hoo!17:13
msabramosounds like we all win17:13
harlowjamsabramo its the next best thing after :-P17:13
harlowjathat jeblair is coming up, 21st century technology17:13
dhellmannjeblair: I don't remember if there's an oslo-db-milestone or not, but I should be in that one, too :-)17:14
harlowjawith lots of new agile hotness, lol17:14
cody-somervillethe concurrency restriction on jobs was only removed recently17:14
fungimsabramo: so much gerrit it all up in yuo tty17:14
openstackfungi: Error: "!!1!eleven" is not a valid command.17:14
fungiopenstack: you're so disagreeable sometimes17:15
*** manishg has quit IRC17:15
jeblairdhellmann:,access  no oslo-db-milestone, i think we missed that in review17:15
jeblairdhellmann: i added oslo-core to oslo-db-core -- see if that's enough for you to self-manage now17:15
jeblairdhellmann: if not, i'll add you explicitly17:16
dhellmannjeblair: it's working, thanks!17:16
sdaguejeblair / mordred : thoughts on putting the new dashboard in today?17:17
dhellmannjeblair: as far as the milestone group, I'll submit a change to make that oslo-core17:17
dhellmannor oslo-db-milestone, I guess17:17
*** Alexei_987 has joined #openstack-infra17:18
jeblairsdague: can you change 'main' to 'review-inbox'?17:19
*** esker has quit IRC17:19
*** manishg has joined #openstack-infra17:19
sdaguesure, on review-dev?17:19
*** SumitNaiksatam has joined #openstack-infra17:19
_david_gerrit Workflow label confiuration: how can i propose a change for it to openstack-infra-config?17:19
jeblairsdague: yeah17:19
*** krtaylor has joined #openstack-infra17:19
sdaguejeblair: done17:20
jeblair_david_: i think it's still manually managed (mordred wants to fix that); what's the change?17:21
_david_jeblair, thanks, i see, i think we should set:  copyMinScore = true for WIP state to enforce stickeness on upload new patch sets17:22
_david_jeblair, Congrats on upgrade, well done ;-)17:22
jeblair_david_: thanks :)17:23
openstackgerritAnita Kuno proposed a change to stackforge/gertty: Adds --digest flag to url required for http auth
jeblair_david_: we can't make it sticky right now -- because the core group can -1wip someone else's change, that means they can't remove that vote, only a new patchset clears that (or they would have to pester core reviewers, and that defeats the purpose of wip)17:24
*** grantbow has joined #openstack-infra17:24
jeblair_david_: i think instead we should make it easier for devs to express whether they want a new patchset to be wip when uploaded17:24
jeblair_david_: which probably means changing git-review to support it17:24
jeblair_david_: and, i'm hoping, if we move to the wip plugin in 2.9, maybe it could support something like /refs/wip/foo so that becomes a one-step process on upload17:25
fungiand adding wip control (back) to dit-review has been on the wishlist for a while anyway17:25
jeblairfungi: i think you mean derp-review17:25
*** andreykurilin_ has quit IRC17:25
fungii SO do17:25
anteayajeblair: this change is possibly in the wrong place but that is the notion anyway:
anteayaI have to get ready to head out for an appointment, gone for several hours17:28
_david_jeblair, i thought that change owner can always remove wip state?17:28
jeblair_david_: i don't think they can remove someone else's vote though17:29
Shrewsfungi: has anyone reported the LP bug hook doesn't appear to be working?17:29
*** nati_ueno has quit IRC17:30
*** nati_ueno has joined #openstack-infra17:30
fungiShrews: you just reported it ;)17:30
fungi(in other words, first i've heard)17:30
jeblair_david_: that's not desirable -- it's a drawback of the review label approach; but it's mitigated by not having copyminscore17:30
fungii'll have a look17:30
Shrewsfirst! \o/17:30
*** amcrn has quit IRC17:31
*** nati_ueno has quit IRC17:31
*** dkliban_lunch is now known as dkliban17:31
*** nati_ueno has joined #openstack-infra17:31
Shrewsfungi: if you need an example:
fungiShrews: that helps, thanks17:31
_david_jeblair, to be fair wip plugin would clear the state with new patch set upload too17:32
fungiShrews: i'm not terribly surprised though... the gerrit hooks interface is relatively unstable and right now some of our hooks probably break when passed new options they don't recognize17:33
jeblair_david_: yeah, and i personally prefer the clear on new patchset, but at least then it becomes a discussion about what we should do vs what we must do.  :)17:33
jeblair_david_: that's why i think the best place to fix this is at patch upload -- so the developer can decide17:33
jeblairi have to run out for a few mins17:34
_david_jeblair, agreed, so let me check the options we have or can introduce upstream ...17:34
jeblair_david_: and thank you very much for your help with the upgrade.  i think some of the patches we merged recently for it were the ones you uploaded earlier in the year.  :)17:35
fungi_david_: yes, it was a HUGE help. thanks!17:36
_david_jeblair, thank you guys back for yor help LibreOffice community two years ago ;-)17:36
_david_fungi, thanks17:36
*** _david_ has quit IRC17:36
*** dripton has quit IRC17:37
*** med_ has joined #openstack-infra17:43
*** david-lyle has quit IRC17:45
*** rcarrillocruz has quit IRC17:45
*** rcarrill` has joined #openstack-infra17:46
zarojeblair: would it not make sense to give core reviewers permission to unabandone any change?17:46
fungizaro: i believe that was on our to do list now that it can be controlled with the acl17:47
*** manishg has quit IRC17:47
zarofungi: if that permission is set then i think it would make sense to set copyMinScore ?17:48
zarofungi: it seems appropriate to me to have to contact a core to unabadone a change that it's owned by yourself.17:48
fungizaro: i'm not sure what copyminscore has to do with abandoned changes17:49
fungizaro: the idea with abandonment was to allow core reviewers and change owners to abandon/unabandon changes17:49
zarofungi: ohh i meant wip all along.17:49
zarofungi: so un-wip not un-abandone17:50
*** dripton has joined #openstack-infra17:50
fungiwip is different with the workflow vote. if i as a core reviewer set your patchset to wip, you as the change owner have no ability to switch it back out of wip because you can't change my vote17:50
*** melwitt has joined #openstack-infra17:50
zarofungi: ohh of course.17:51
*** david-lyle has joined #openstack-infra17:51
openstackgerritPeter Portante proposed a change to openstack-infra/config: Add swift "in-process" functional test runs
portantechmouel: ^^^17:52
*** darraghb has quit IRC17:53
zarojeblair: i'm trying to setup gear on a VM to work with SSL certs, but I can't seem to get it to work.  I mean i can't get a connection.  would you be able to help me with that?17:53
*** thomasbiege has joined #openstack-infra17:53
*** amcrn has joined #openstack-infra17:53
*** thomasbiege has quit IRC17:56
*** che-arne has quit IRC17:56
*** doddstack has quit IRC18:00
*** dkliban is now known as dkliban_afk18:02
*** zns has quit IRC18:03
*** yfried has joined #openstack-infra18:03
yfriedasking again - is there anyone here familiar with logstash?18:04
yfriedjesusaurus: ?18:04
yfriedI'm trying to set up logstash as a deamon18:04
yfriedhow is done on the gate?18:04
*** marun has joined #openstack-infra18:05
*** jooools has quit IRC18:06
*** msabramo has left #openstack-infra18:06
fungiyfried: we have a special agent which gets zmq notification from jenkins job completion and then retrieves and injects all the log files from that job18:07
fungiso it's a batch processing loader18:07
*** nati_ueno has quit IRC18:07
fungiit's not set up like a typical logserver18:07
*** nati_ueno has joined #openstack-infra18:07
*** e0ne has joined #openstack-infra18:08
*** thomasbiege has joined #openstack-infra18:08
yfriedfungi: ok, different question:   this is how logfiles are parsed
yfriedfungi: but how do you tag logs according to job and build?18:09
fungiyfried: metadata provided in the 0mq messages coming from jenkins is interpreted by the loader18:10
yfriedfungi: how? where?18:11
*** ildikov has joined #openstack-infra18:12
yfriedfungi: tnx. I'll look into it18:13
*** timrc is now known as timrc-afk18:14
yfriedfungi: another question - are the logs being parsed during the run, or are the sent at the end of a job?18:14
*** morganfainberg_Z is now known as morganfainberg18:14
fungionce the job completes, jenkins pulls logs off the slave and publishes them to a webserver for archival, then sends a zeromq message to the logstash processor notifying of their availability. the logstash processor downloads those files from the webserver and then starts feeding them into logstash for indexing18:15
yfriedfungi: where is all of this documented? because it doesn't sound like what I've read in the short wiki for logstash.openstack18:17
yfriedfungi: yeah, that's the one I've read18:17
yfrieddoesn't seem like exactly what you described.18:19
*** e0ne has quit IRC18:19
fungiyfried: that page documents how logstash is integrated, but doesn't necessarily go into details on things happening on the jenkins side (the log archiving we're doing from that end predates any of the logstash implementation and is mostly unchanged \)18:19
*** zns has joined #openstack-infra18:19
yfriedfungi: I also was unable to find the log-pusher mentioned there18:19
fungiyfried: the links to them aren't working?18:20
*** zns_ has joined #openstack-infra18:20
fungithe urls there are lacking /cgit/ in them18:20
*** nati_ueno has quit IRC18:20
*** nati_ueno has joined #openstack-infra18:21
fungino, nevermind. perhaps they were moved and the docs not updated18:21
*** thomasbiege has quit IRC18:21
*** ominakov has joined #openstack-infra18:21
*** zns has quit IRC18:21
*** ominakov has quit IRC18:21
yfriedit speaks about a python script. I'd love to see it18:22
fungii'm busy trying to fix something else at the moment, but the puppet manifests there for the logstash server should show where they're being installed from18:22
jeblairzaro: yes, i can help -- what's up18:22
jeblairfungi, yfried i'll find the scripts and fix the docs18:22
yfriedjeblair: tnx18:22
*** hashar has joined #openstack-infra18:23
mriedemfungi: jeblair: does updating a commit message in place with new gerrit reset the review scores? assuming 'yes'18:23
fungimriedem: i assume it makes a new patchset and everything that implies18:23
jeblairfungi: mriedem yep18:23
fungibut haven't tried it myselg18:23
mriedemk, just making sure, thanks18:23
mordredmorning jeblair18:24
yfriedso, if I get this - upon finishing the job, jenkins pushes the logs via zmq to the logstash server, together with tags about build/job/etc..?18:24
* zaro is afk, bb in 2hrs18:24
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Correct log processor script links in docs
fungiyfried: jenkins uploads the logs to a webserver and sends a zmq message about them which gets interpreted by a gearman client18:25
fungisort of convoluted18:25
jeblairyfried: the 2 scripts mentioned in the docs are here:
fungiwell, interpreted by a zmq listener which in turn coordinates the indexing via gearman protocol to the cluster18:25
*** e0ne has joined #openstack-infra18:25
jeblairmordred: mornin18:26
fungiyfried: but anyway, jenkins does not send the logs to logstash. it just sticks them on a webserver, then the logstash processing scripts act on the notification and go retrieve them from the same webserver18:27
mordredjeblair: it's wet on the east coast18:28
sdagueyes, it is18:28
fungimordred: indeed. the weather radio woke me up squaking about tornadoes just a few miles north of here18:28
yfriedfungi: where is the code to make logstash retrieve the logs?18:28
fungimordred: better than an alarm clock18:28
jeblairmordred: it is 83 going on 91 at your house.18:29
mordredjeblair: *sob*18:29
*** rustlebee has left #openstack-infra18:29
jeblairand not wet.18:29
sdaguewhat, only an inch of rain since midnight here :) -
yfriedfungi: tnx. this might be a stupid question, but what exactly is gearman? I've seen it mentioned, and I now it's designed to help distribute jenkins load, but I'm not too clear18:33
fungiyfried: it's for coordinating task distribution18:34
*** thomasbiege has joined #openstack-infra18:34
*** aconrad has joined #openstack-infra18:35
fungiyfried: though we use a pure-python client+server implementation of the gearman protocol called "gear"
yfriedso gearman is the "webserver" waiting to get logs via zmq and telling logstash to parse them?18:39
yfriedfungi: ^18:39
fungiyfried: no... jobs run on a jenkins slave and generate logs. jenkins master pulls logs from the slave and scp's them to a linux server named logs.openstack.org18:40
fungithat linux server serves those raw log files via apache18:40
*** arnaud__ has joined #openstack-infra18:40
*** dkliban_afk is now known as dkliban18:41
fungiseparately, when the jenkins master is done scp'ing those logs up to logs.o.o, it also sends a zeromq message about their existence18:41
fungithe logstash cluster has a client subscribed to those zeromq notifications18:41
fungiit then communicates separately to the other logstash indexer cluster workers letting them know there's a log processing task which needs to be done18:42
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Adds insecure option to
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Adds insecure option to
fungi...via gearman protocol. one of those indexing workers accepts the job and then retrieves the corresponding files from logs.o.o18:42
*** gabriel-bezerra has joined #openstack-infra18:42
jeblairmriedem, fungi: data point -- i updated commit message on after leaving a -1 and my -1 stuck18:43
jeblairmriedem, fungi: don't know if that's because it was MY -1, or if it would do that for all -1s18:43
*** timrc-afk is now known as timrc18:44
yfriedfungi: so the script you sent me (log-gearman-worker) is for which part?18:44
*** crc32 has quit IRC18:44
fungiyfried: that was a link to the class which does that last bit (runs on the indexing cluster workers and downloads the log files from the logs.o.o webserver)18:45
*** manishg has joined #openstack-infra18:45
sdaguejeblair:*layout.yaml+NOT+label:Code-Review%253E%253D-2%252Cself,n,z - oh, how I love secondary indexes18:45
*** _nadya_ has joined #openstack-infra18:45
mordredsdague: your blog post on them is amazing18:46
sdaguemordred: thanks!18:47
mordredI need to set up one of those for, setup.cfg and tox.ini in all projects18:47
fungijeblair: mordred: not sure what's changed, but with the new bare-precise images nodepool nodes are ending at "Exception: Timeout waiting for ssh access" after quite a lot of "INFO nodepool.utils: Password auth exception. Try number X..." messages in the log18:47
fungithe images built successfully, but i wonder if our ssh key changes have screwed something up?18:47
*** nati_ueno has quit IRC18:47
*** mrodden1 has joined #openstack-infra18:48
*** nati_ueno has joined #openstack-infra18:48
mordredfungi: hrm. on both providers?18:48
fungimordred: yes, which leads me to believe it's courtesy of a recent patch (some time last night or this morning)18:49
*** nati_ueno has quit IRC18:49
mordredfungi: nod18:49
mordredfungi: we were doing something with the ssh_authorized_keys yesterda18:49
*** zns_ has quit IRC18:49
mordredfungi: I'm going to look at nodepool log to see if there's a puppet error there18:50
*** mrodden has quit IRC18:50
fungigood idea. maybe puppet didn't abort but failed to do something18:50
gabriel-bezerrahi, where can I find devstack core reviewers?18:50
gabriel-bezerraI mean, there is no #openstack-devstack channel18:50
*** nati_ueno has joined #openstack-infra18:51
fungigabriel-bezerra: here and there... #openstack-qa is where a lot of devstack discussions happen18:51
morganfainberggabriel-bezerra, i think #openstack-qa18:51
fungigabriel-bezerra: the actual list of people is,members18:51
gabriel-bezerraok. thank you. fungi and morganfainberg18:52
*** tjones has left #openstack-infra18:52
yfriedfungi: the scripts starts upon recieving zmq msg?18:52
*** e0ne has quit IRC18:54
mordredfungi: I have a patch up to fix this from yesterday18:55
fungimordred: oh!18:55
*** e0ne has joined #openstack-infra18:55
yfriedfungi: how do I get the zmq plugin to my jenkins? should it be available via jenkins plugins page?18:56
mordredfungi: but it looks landed...18:56
yfriedfungi: does it need to be DL from the githup repo?18:56
mordred2014-04-30 17:07:31,207 INFO merr: /Stage[main]/Openstack_project::Users/User::Virtual::Localuser[corvus]/Ssh_authorized_key[jeblair@operational-necessity]: Could not evaluate: Puppet::Util::FileType::FileTypeFlat could not write /home/corvus/.ssh/authorized_keys: Permission denied - /home/corvus/.ssh/authorized_key18:57
*** zns has joined #openstack-infra18:57
fungiyfried: i believe it can be installed from the jenkins plugins site. clarkb's our resident expert on this and is on vacation this week. i'm not 100% positive we're using the build on the jenkins plugin site18:57
*** ramashri has joined #openstack-infra18:57
*** david-lyle has quit IRC18:57
fungimordred: so that's the 0400 vs 0600 perms on teh a_k file18:58
*** sdake_ has joined #openstack-infra18:59
*** crc32 has joined #openstack-infra18:59
mordredfungi: that SHOULD be fixed now18:59
mordredI'm confused18:59
*** ramashri has quit IRC19:00
fungimordred: i'm manually booting an instance from one of those images now and will log in/poke around19:00
mordredfungi: like, the patch has landed - I can see the corvus_sshkey resource in the log19:00
mordredfungi: good luck - it's not going to have your ssh key19:00
mordredfungi: you may have to log in from a jenkins master as jenkins19:00
fungimordred: it will because i've told nova boot --file /root/.ssh/authorized_keys=$HOME/.ssh/id_rsa.pub19:01
fungimordred: but i can also use password login via the web virtual console in rackspace's dashboard19:01
yfriedfungi: I'm trying to build a logstash instance for our own ci jobs19:01
*** ramashri_ has joined #openstack-infra19:01
notmynameno meetbot in -meeting?19:01
notmynamejeblair: ^19:01
yfriedfungi: I tried to send a single job log data directly to logstash (via redis) and it was a very slow process, so I'm looking for a more efficient way, trying to learn from your setup19:01
notmynameoh, it's there. just lots of lag19:02
fungiyfried: yes, the complexity in our configuration evolved from multiple iterations of dealing with massive scaling problems19:02
jeblairnotmyname: yeah, seems to be lag19:02
fungiyfried: and even so, we're only able to keep about 2 weeks worth of job logs indexed (that's about a terabyte of indexes on each elasticsearch cluster member i believe, of which we currently have 6)19:03
*** sdague has quit IRC19:03
*** sdague has joined #openstack-infra19:03
*** mrodden has joined #openstack-infra19:07
*** mrodden1 has quit IRC19:07
yfriedfungi: we don't have that much volume of jobs as the gate does19:09
yfriedfungi: and I'm more concerned with stress and prolonging the job, that with storage space19:09
*** chandan_kumar has joined #openstack-infra19:09
fungiyfried: right, the system load and processing speed is why our indexing cluster is 16 rather large virtual machines each with (up to 4?) indexers running on separate cores19:10
fungii forget the exact count, but it's big19:11
fungiand that's so that we have some hope of having logs indexed within a few minutes of the job completing19:11
*** ramashri_ has quit IRC19:13
yfriedfungi: tnx. you helped my a lot. what's your timezone?19:14
fungiyfried: most of the time it's utc-4 or utc-519:14
fungiutc-4 at the moment19:15
yfriedfungi: that's US time right? too bad. do you know anyone who has similar understanding of logstash in europe timezone?19:15
fungimordred: it appears that our individual authorized_keys files are 0600 root:root and zero bytes in length19:16
fungiso perms are no longer 0400 at least, but still not owned by our accounts and have no contents19:17
fungimordred: also, ~jenkins/.ssh/authorized_keys doesn't exist at all19:18
mordredfungi: sigh19:20
mordredfungi: SO - I think ...19:20
mordredthe patch to fix the perms only works with existing files19:20
mordredso, now that we've migrated, we should revert that portion of the patch19:21
mordredand I should have just updated the perms with ansible instead19:21
fungii think most sites using puppet must do kerberos or ldap for login management ;)19:21
mordredbecause ssh_auth_key worked in nodepool19:21
mordredbefore the patch19:21
mordredso it worked without the patch on a clean machine19:21
*** e0ne has quit IRC19:22
fungimordred: do you think the file permissions change could also be responsible for the missing ~jenkins/.ssh/authorized_keys file? or is that likely a side effect of some other related key management change?19:25
*** rloo has joined #openstack-infra19:25
*** nati_ueno has quit IRC19:25
mordredI thnk it's the file perms19:26
mordredbecause it introduced a file resource with no content19:27
fungibecause the manifestation differs. for our accounts it's empty authorized_keys files owned by root. for the jenkins user it's a missing file entirely19:27
fungimordred: are you drafting the rip-out for that bit, or should i?19:28
rloohi. I wanted to get python-ironicclient dev docs up on I added jobs to jenkins/zuul and the docs are on What else do I need to do?19:29
jeblairrloo: the change to add the new jobs has merged?19:30
rloojeblair:  yes.
rloojeblair: and i can see them on docs-draft, eg
rloojeblair: what copies the stuff from docs-draft to
jeblairrloo: it looks like client docs are only published on release19:31
jeblairrloo: so it should happen the next time a release is tagged19:31
rloojeblair: hmm. is that how other client docs are done? I copy/pasted from others.19:31
jeblairrloo: yes.  i believe the intention is to avoid confusing users by publishing documentation for unreleased software19:32
rloojeblair: I assume there's a reason to do things that way. Ok, I'll discuss with deva. thx!19:32
jeblairrloo: and the clients release more frequently than the servers19:32
*** mfer has quit IRC19:32
*** david-lyle has joined #openstack-infra19:32
rloojeblair: thx again!19:33
jeblairrloo: if you are going to release soon, i'd prefer to just wait.  if you are not going to release for quite a while and otherwise have no reason to, we can try to run the job manually for the most recently released version.19:33
jeblairrloo: you're welcome.  let me know what you learn.19:33
devanandarloo: hm?19:33
rloodevananda: didn't realize you were here. ^^. our client docs won't show up until we do a release.19:34
pabelangerSo, how do global requirement gerrit reviews get created across all the projects? Is that an automated process?19:34
devanandaoh - sure. i'm happy to tag a release. lemme see what the changes are19:34
fungipabelanger: yes, it's a triggered job which runs a script which proposes them19:34
fungipabelanger: triggered off a change merging to the requirements repo19:35
rloodevananda: thx.19:35
fungipabelanger: so look in the post pipeline for the requirements project19:35
*** ihrachyshka has joined #openstack-infra19:35
dkranzfungi: In, there is no indication in the check/verify matrix of who gave the to +2/+A (I was one of them). Is that new behavior or I just never noticed it?19:36
dkranzfungi: two +2/+A19:37
fungidkranz: i think that's a side effect of the approve label going away19:38
dkranzfungi: Is that  bug or a feature?19:38
*** ihrachyshka has quit IRC19:38
*** ihrachyshka has joined #openstack-infra19:38
jeblairdkranz: misfeature, i think.  you can get the info by reading the review comments though.19:39
dkranzfungi: I would think it should show a +2 for the Approver if it is not showing Approve column19:39
jeblairDavid KranzApr 23 10:18 AM19:39
jeblairPatch Set 7: Approved19:39
dkranzjeblair: Yes, I saw that. That's how I remembered it was me :)19:39
*** thomasbiege has quit IRC19:39
sdague - mordred / SergeyLukjanov / morganfainberg - I actually think this should be in d-g, and not in devstack19:39
fungidkranz: i suspect new gerrit is unable to interpret your vote since it includes a vote on a label it doesn't think exists, so it's hiding you in that table19:39
jeblairsdague: can we make running those under apache the default?19:40
dkranzfungi: So are you saying this will only happen when looking at reviews from before the change?19:40
fungidkranz: yes19:40
sdaguejeblair: I don't know19:41
dkranzfungi: ok, then that is not so bad.19:41
fungidkranz: reviews approved after the upgrade will be workflow +1 instead19:41
dkranzfungi: thansk19:41
sdaguemorganfainberg: opinions on that?19:41
*** doddstack has joined #openstack-infra19:41
jeblairsdague: i think it's better for d-g to run as close to the default devstack as possible -- a big part of the goal is to make sure we can safely say to anyone "just run devstack, it will work"19:41
fungidkranz: we could consider retro-fixing the old changes in the db, but gerrit does some very funny things with mysql auto-timestamp fields so you have to be extremely careful to reset them to their current values. otherwise the chronology goes out the window19:42
sdaguejeblair: sure, but the proposal on the devstack side wasn't to actually change the default. It sounded like right now morganfainberg wanted a switch19:42
fungidkranz: and i'm so far unconvinced it's worth the risk/effort19:43
portantedoes anybody consume the coverage report jobs output?19:43
sdagueunrelated - - the heat team would really like to be able to work on heat adds for grenade, which they need this for19:43
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/config: Set Alpha Channel for better looking favicon
*** chandan_kumar has quit IRC19:43
pabelangerfungi, still looking but the hook script for the repo is added into zuul?19:44
dkranzfungi: I don't think it is worth it either. I just noticed the behavior and did not know it was transient.19:44
fungipabelanger: yeah, it's one of the jenkins slave scripts in our config repo19:44
pabelangerfungi, okay, think I see it19:45
fungipabelanger: there's a job in the post pipeline of the requirements project in zuul's layout which should correspond to something in our jjb config which ought to lead you to a builder macro which calls that slave script which inspects the projects.txt in the requirements repo and then proposes changes to the projects listed there19:45
*** ociuhandu has quit IRC19:45
*** mfer has joined #openstack-infra19:45
fungirube goldberg would be proud19:45
pabelangerYa, see it now: propose-requirements-updates19:47
pabelangerokay cool. Ya, I want to setup a periodic gate for something and liked the way the requirements stuff works.  Think I'll use the same logic for creating new reviews19:48
*** mattymo|home has quit IRC19:48
*** talluri has quit IRC19:51
*** talluri has joined #openstack-infra19:51
sdaguejeblair: one minor issue I see in it19:53
*** _nadya_ has quit IRC19:53
sdagueI'll admit my brain is not fully wrapped around the -2 jobs, so you'll have to tell me if those are right or not19:54
sdagueconceptually +119:54
sdaguereview feedback reported19:54
*** talluri has quit IRC19:54
*** mwagner_lap has quit IRC19:55
*** talluri has joined #openstack-infra19:55
*** rcarrillocruz has joined #openstack-infra19:56
*** rcarrill` has quit IRC19:58
jeblairsdague: nice catch.  as for -2; i'm counting on the layout check to tell us if they don't exist.  it didn't complain so i assume they do.  i belive the proposed config will run them on stable branches only.19:59
portantecoverage, coverage, ... anyone, anyone, ... bueller, bueller?19:59
*** andreykurilin_ has joined #openstack-infra20:00
jeblair<Job check-tempest-dsvm-neutron-2> ^(stable/).*$20:00
jeblairso that one will only run on stable branches20:00
jeblair<Job check-tempest-dsvm-neutron-pg-2> <Job check-tempest-dsvm-neutron-full-2> <Job check-tempest-dsvm-neutron-pg-full-2> run on all branches20:00
*** nati_ueno has joined #openstack-infra20:00
sdagueI think we want them not on stable branches?20:01
portanteI'd like to be able to access the results of the coverage jobs20:01
sdaguesalv-orlando: still up?20:01
*** Ryan_Lane1 has joined #openstack-infra20:01
jeblairportante: take the sha of the commit you're interested in; eg HEAD is 59a639c116eba2211dad722b287f9a8675a5155d20:02
jeblairportante: then go to
*** rloo has quit IRC20:03
*** markmcclain has joined #openstack-infra20:03
*** Ryan_Lane has quit IRC20:03
jeblair(i think there's a git subcommand floating around to do that automatically)20:03
Ajaegerjeblair: git os-job20:04
portantejeblair: thanks20:04
jeblairthat's it20:04
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Stop over-managing user SSH keyfiles
*** nati_ueno has quit IRC20:04
*** hogepodge has quit IRC20:05
fungimordred: jeblair: ^20:05
jogotop of gate is waiting for docs, pep8 and py27 jobs
*** Ajaeger has quit IRC20:05
fungijogo: yep. fixing20:05
jogofungi: awesome sauce20:05
*** hogepodge has joined #openstack-infra20:06
jeblairfungi: does anything pass in "old_keys"?20:06
jogosdague: have you seen this yet:
fungijeblair: nope. not that my git grep in the config repo found20:07
jogofedoraproject is giving us 403s20:07
jeblairfungi: i think old_keys might be unrelated to this -- i think it's to support key rollover20:07
fungijeblair: oh20:07
fungii'll put it back20:07
sdaguejogo: yes, that was supposed to be cached in nodepool though20:07
jeblairfungi: cool, rest looks good20:07
sdagueare our nodepool images super old?20:08
jogosdague: oh :/ is there a bug filed for this20:08
fungijeblair: i had originally asked mordred why it was there and he said it was transitional, but we might have been talking past each other20:08
*** MikeLeman has joined #openstack-infra20:08
sdaguejogo: I don't think so20:08
* jogo files an infra bug and a e-r patch for it20:09
sdagueI had thought I had that change in there long enough to get into the nodepool image, but now am realizing we might be working off of really old nodepool images now20:09
jeblairsdague: hp is so broken we only get a new image out of them every few weeks20:10
jogosdague: ohh HP :) don't you like your new corporate overlord?20:10
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Stop over-managing user SSH keyfiles
sdaguejeblair: hmmm...20:11
jeblairjogo: good point, it's all sdague's fault now.  it used to be yours but now it's his.  ;)20:11
sdaguejeblair: ok, well followup question20:12
jeblairsdague: apparently hpcloud1.1 is better at this, but we're on the fence about whether it's better at spinning up nodes once we actually have images.20:12
jeblairsdague: morded was looking into that.20:12
sdaguethe issue is fedora download sites are terrible20:12
sdaguein their uptime20:12
fungijeblair: mordred: as soon as we approve i'll force merge it (we don't currently have available bare-precise slaves to test it for style checks) and then build new images asap20:12
sdaguehowever, if we could put the image somewhere we were more sure we could pull from20:12
jogosdague jeblair:
uvirtbotLaunchpad bug 1314786 in openstack-ci "/opt/stack/new/devstack/files/Fedora-x86_64-20-20131211.1-sda.qcow2 is not  cached" [Undecided,New]20:12
sdaguemaybe we could stick it there instead20:12
jeblairfungi: lgtm; i'll let you aprv/subm20:13
sdaguebecause otherwise updating some of these base images for testing will take months to ensure that nodepool images have them20:13
jeblairsdague: the right answer is to have clouds that work20:13
openstackgerritA change was merged to openstack-infra/config: Stop over-managing user SSH keyfiles
sdagueI can't fix that part :)20:13
jeblairwe can fix it -- move to hpcloud1.1, and that's where we should put our effort20:14
fungii did at least test the patch on a manually-launched node from one of the broken images to make sure that it dtrt20:14
jogojeblair: what happened to that, I thought that already happened20:14
jeblairfungi: you have high standards20:14
jogoor at least you previously mentioned it20:14
sdaguejeblair: so what needs to happen to get us there?20:14
jeblairjogo: we're 1/3 of the way there, ask mordred for progress report20:14
jogomordred: ^20:15
fungiand it does in fact seem to be capable of adding the keyfiles and ssh dires automatically, and appears to calculate its dependence on the user objects as well20:15
*** freyes_ has joined #openstack-infra20:15
*** freyes_ has quit IRC20:15
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1314786
uvirtbotLaunchpad bug 1314786 in openstack-ci "/opt/stack/new/devstack/files/Fedora-x86_64-20-20131211.1-sda.qcow2 is not  cached" [Undecided,New]
jogosdague: ^ e-r patch20:17
*** Ryan_Lane1 has quit IRC20:17
sdaguejogo: should be tags:console20:18
sdaguebecause we'll start seeing this in grenade at some point20:18
jogoack, found another manifestion as well20:18
jogosdague: (message:"awaiting response... 403 Forbidden" OR message:"Unable to establish SSL connection.") AND tags:"console"20:20
jogoI am not sure how to include fedora in the query20:20
sdagueI don't think you can20:21
jogowith just the console tag it may be a pretty broad query, but we will see20:21
*** dprince has quit IRC20:21
sdagueany egregious hits on that which don't look like this problem?20:21
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1314786
uvirtbotLaunchpad bug 1314786 in openstack-ci "/opt/stack/new/devstack/files/Fedora-x86_64-20-20131211.1-sda.qcow2 is not  cached" [Undecided,New]
jogowhere this problem involves fedora20:21
gabriel-bezerrasomething is definitely wrong with zuul. look at how long all those builds are taking to run20:22
gabriel-bezerramore than 3h20:22
fungigabriel-bezerra: yes, it will be working again soon20:22
jogosdague: anyway next question: what are your thoughts on adding neutron grenade logs to e-r?20:22
sdagueto e-s20:23
jogoerr yes20:23
sdagueyes, we should do that20:23
fungi#status alert the gate is backed up due to broken nodepool images, fix in progress (eta 22:00 utc)20:23
openstackstatusfungi: sending alert20:23
* jogo preps a patch20:23
*** _nadya_ has joined #openstack-infra20:24
morganfainbergsdague, here now20:24
-openstackstatus- NOTICE: the gate is backed up due to broken nodepool images, fix in progress (eta 22:00 utc)20:25
morganfainbergsdague, sorry was at lunch20:25
*** ChanServ changes topic to "the gate is backed up due to broken nodepool images, fix in progress (eta 22:00 utc)"20:25
morganfainbergsdague, i'm fine with it being in either ds-g or in devstack20:25
*** SumitNaiksatam has quit IRC20:25
morganfainbergsdague, both work for me.20:25
sdaguemorganfainberg: so the question is about defaults20:25
*** nati_ueno has joined #openstack-infra20:25
morganfainbergsdague, i don't want to change any defaults at this point20:25
morganfainbergsdague, eventually we might make keystone default to an apache deploy, but we want to test it for sure20:26
sdagueok, so if we're not changing defaults, I think d-g is the right place for this20:26
*** mrodden has quit IRC20:26
morganfainbergsdague, my goal is to make a new check (non-vote) job to start with, eventually gate on it20:26
*** Ryan_Lane has joined #openstack-infra20:26
sdaguemorganfainberg: ok, so you're going to be at the test matrix session right? :)20:26
morganfainbergsdague, was planning on it. haven't done my schedule yet20:26
morganfainbergsdague :)20:26
openstackstatusfungi: finished sending alert20:26
sdaguebecause we can't just multiply out another keystone config independant of all the other configs20:27
sdagueat least not across everything20:27
*** mrodden has joined #openstack-infra20:27
morganfainbergsdague, correct. i was planning on making this a keystone-only check/gate job for now20:27
sdagueok, so lets play that game to get started. You might also want to only include keystone in the apache services list then20:28
morganfainbergsdague, ++ was just going to suggest isolating this to keystone for this change20:28
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Handle file renames in diff view
morganfainbergsdague, or well say that was my plan in this case.20:28
morganfainbergeventually i think we should have a mod_wsgi based check for anything that can run under mod_wsgi... but different argument different time20:29
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Add grenade neutron logs to ElasticSearch
morganfainbergsdague, thanks for looking at it :)20:29
jeblairyeah, if we're diverging from the default then we should put a switch in d-g -- but when it becomes the defacto default in d-g, or even the recommended way to deploy in devstack, it should become the default in devstack and eventually be removed from d-g20:30
morganfainbergjeblair, ++ 100% agree20:30
sdaguejeblair: agreed20:30
*** eharney has quit IRC20:33
*** aconrad has quit IRC20:35
*** maxbit has quit IRC20:37
openstackgerritAntoine Musso proposed a change to stackforge/python-jenkins: Wrap tests and linting with tox
sdagueanyone know who owns the ops tracks at design summit?20:37
fungisdague: the user committee i believe20:38
hasharjeblair: will you have anytime this week to review/merge the addition of tox to stackforge/python-jenkins ?  That is preventing us from adding triggers in Zuul for that repo :-)20:38
fungisdague: might ask fifieldt and/or reed when they're around20:39
*** sdake_ has quit IRC20:39
sdaguefungi: thanks20:39
*** ociuhandu has joined #openstack-infra20:39
jeblairhashar: i know!  i just saw the review to config that added the jobs.  i've gone from 96 reviews to 37 so far today.  i'm getting close.  i'll add you and mgagne to the group for python-jenkins.20:42
hasharjeblair: take your time. I know you are a busy guy :]20:42
*** jcoufal has quit IRC20:42
jeblairhashar: you should have +2 there now20:42
ArxCruzanteaya: jeblair hi, after I upgrade my zuul and layout.yaml today, I start to subscribe in all changes (although I'm not voting) is that normal from now ?20:42
hasharjeblair: you are awesome :] Can you add Khai Do as well? he is very active on jjb20:42
*** Ryan_Lane has quit IRC20:43
jeblairzaro: done ^20:43
hasharunless he objects of course20:43
jeblairheh, he can remove himself ;)20:43
jeblairArxCruz: i'm not sure i understand the question20:43
jeblairArxCruz: (specifically, i don't know what you mean by subscribe)20:43
ArxCruzIBM Power KVM is in the review list20:44
ArxCruzmy layout.yaml have gerrit: { } as you told me to do yesterday20:45
ArxCruzthis is making us receive a lot of emails for every change :)20:45
*** lcostantino has quit IRC20:46
fungiArxCruz: it looks like IBM PowerKVM Testing reported on that change20:46
fungiat 15:2420:46
*** adalbas has quit IRC20:47
*** aconrad has joined #openstack-infra20:47
ArxCruzfungi: yeah, but ibm powerkvm reported before, but never was added in the reviewer list20:47
fungiArxCruz: odd, i thought any time an account commented on a change it was automatically added to the list of reviewers for that change20:48
*** etoews has joined #openstack-infra20:48
ArxCruzfungi: nope, that's why I'm asking if that's the new behavior20:48
*** mbacchi has quit IRC20:48
fungiArxCruz: it's possible you were managing to not end up on the reviewers list for changes previously (i'm not sure how that was happening but i assume it must have been) and gerrit 2.8 perhaps behaves differently in that regard than 2.4 did20:49
ArxCruzkrtaylor: ^20:49
ArxCruzfungi: that's what I want to confirm :)20:49
jeblairArxCruz: my understanding was the same as fungi's.20:50
fungithe other possibility is that the tweaks we had to do to zuul to support non-voting changes in the 2.8 api altered that behavior subtly?20:50
ArxCruzwell, i'm using latest zuul20:51
fungiright, zuul got some patches when we upgraded gerrit, related to supporting newer gerrit20:51
*** jcoufal has joined #openstack-infra20:52
*** jcoufal has quit IRC20:53
*** Ryan_Lane has joined #openstack-infra20:55
*** zns has quit IRC20:56
ArxCruzwell, I can't investigate it right now, I'm trying to make puppetboard works correctly here, the report are not appearing20:57
ArxCruzbtw if someone have experience on it ... ;)20:57
*** marcoemorais has quit IRC20:58
*** zns has joined #openstack-infra20:58
*** arnaud__ has quit IRC20:59
*** ArxCruz has quit IRC21:00
*** pdmars has quit IRC21:01
morganfainbergpleia2, i'm looking at how to fix reviewday to handle the new WIP mode, just noticed WIP patches floated to the top again.21:03
zaroohh cool, python-jenkins made it into gerrit.21:03
*** _nadya_ has quit IRC21:03
*** sdake_ has joined #openstack-infra21:04
fungiokay, one of the bare-precise image builds has completed so far (hpcloud-az3). others are close behind i think21:04
fungi10 nodes building there so far21:04
*** thomasem has quit IRC21:04
*** etoews has quit IRC21:05
*** thomasem has joined #openstack-infra21:05
*** etoews has joined #openstack-infra21:05
*** dkliban is now known as dkliban_afk21:06
boris-42jeblair hi21:06
*** aconrad has quit IRC21:10
*** thomasem has quit IRC21:10
*** rcarrill` has joined #openstack-infra21:10
fungiand some in use now21:10
*** rcarrillocruz has quit IRC21:11
fungirax-ord has a ready image and is building nodes now as well21:11
*** markmcclain has quit IRC21:12
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Handle binary files in diffstat
openstackgerritMorgan Fainberg proposed a change to openstack-infra/devstack-gate: Allow setting services up under apache mod_wsgi
*** marcoemorais has joined #openstack-infra21:13
jeblairintervals between fixing gertty for changes it can't handle are increasing!  that was 50 minutes of nonstop reviewing!21:13
jeblairboris-42: hello21:14
boris-42jeblair so about Rally checks21:14
boris-42jeblair could I introduce you a bit?21:15
openstackgerritMorgan Fainberg proposed a change to openstack-infra/devstack-gate: Allow setting services up under apache mod_wsgi
jeblairboris-42: i'm personally interested and excited about it, but my comment wasn't based on whether i thought rally was a good idea.  it's about how we introduce new jobs like that.21:16
jeblairboris-42: i think what you are proposing should start out in the experimental pipeline so you can demonstrate that it works, and how it works21:16
boris-42jeblair so actually we already demonstrated in rally gate21:17
boris-42jeblair basically in rally we are going to use it to check that all benchmarks works21:17
boris-42jeblair as in keystone check is already merged21:18
boris-42jeblair let me just show21:18
boris-42jeblair you are putting here task configuration for rally21:18
*** aconrad has joined #openstack-infra21:18
boris-42jeblair it can be one or more benchmarks with any load in any enviroment21:18
boris-42jeblair as a result you are getting all logs collected (dsvm job) + this pretty graphic
*** markwash has quit IRC21:19
*** etoews has quit IRC21:19
boris-42jeblair ^ that shows that something is not ok with this thing under load21:19
boris-42jeblair 0 on graph == exception21:19
boris-42jeblair so there are 2 use cases21:19
fungiconfirmed i could log into one of the fresh bare-precise nodes and here's a job which ran to completion on one as well...
boris-42default task that will allow us to track performance of major functionallity21:20
fungiso i think everything should start catching back up shortly21:20
boris-42jeblair and actually working on performance21:20
fungii'm going to take a break for dinner and hopefully clear the status alert when i get back21:20
boris-42jeblair e.g. 1 patch just changes project_name.yaml and measure gates21:20
boris-42jeblair probably one time recheck to ensure21:21
boris-42jeblair and then your code with same content of project_name.yaml21:21
boris-42jeblair and probably recheck21:21
boris-42jeblair so we are getting different numbers and able to check does current patch improves performance or not21:21
boris-42jeblair in future we are going to add normalization of benchmark results + comparison with last merged patch performance21:22
boris-42jeblair so it will be even more usefully and probably we will be able to make +1/-1 decision depending of changes in perfroamnce21:22
*** dkranz has quit IRC21:22
*** eharney has joined #openstack-infra21:24
openstackgerritDerek Higgins proposed a change to openstack-infra/config: Create tripleo-specs git repo
zarojeblair: so, hope you have time now.  about gear ssl.21:27
jeblairzaro: yeah, what do you need?21:27
zarojeblair: i have setup a vm and i'm trying to test gear ssl by running server, worker, and client all on same vm21:27
zarojeblair: used this website as instructions to create my certs:
*** dangers is now known as dangers_away21:28
zarojeblair: this is how i start the server: geard -d --ssl-ca=/home/ubuntu/ca.crt --ssl-cert=/home/ubuntu/server.crt --ssl-key=/home/ubuntu/server.key21:29
zarojeblair: this is my worker connection setting: worker.addServer('localhost',4730,'/home/ubuntu/server.key','/home/ubuntu/server.crt','/home/ubuntu/ca.crt')21:30
jeblairzaro: the worker should get its own cert21:30
zarojeblair: why? it's on the same machine.21:30
*** MikeLeman has quit IRC21:31
*** rlandy is now known as rlandy|afk21:31
*** jgrimm has quit IRC21:31
jeblairzaro: it's part of how the gear acls work -- they are based on the cn in the cert, so each worker/client should get their own21:31
lifelessjeblair: - are new-project things still once a week ?21:32
jeblairlifeless: no21:32
lifelessjeblair: yay :)21:32
jeblairindeed :)21:32
lifeless(unless of ourese they are now once a month, in which case sad)21:32
zarojeblair: sorry, i'm newbie to this,  not real sure what to do now so i'm gonna ask real simple questions.21:33
zarojeblair: do you mean use the same instructions to create a seperate set of certs with the same cn name?21:33
*** hashar has quit IRC21:33
jeblairzaro: here's what i did in my testing:
jeblairzaro: so yes -- using the commands i supplied, repeat the last two for each client you want to connect.  i did it once with CLIENT='client', and another with 'worker' and another with 'server'.21:35
jeblairzaro: then i used the 'server' key for geard, the 'worker' key for my test worker, and the 'client' key when connecting with openssl on the command line.21:35
zarojeblair: ok. i will try that.  thanks.21:36
jeblairzaro: then i used this acl file with geard:
*** yamahata has quit IRC21:37
jeblairzaro: (the [worker] and [client] sections there refer to the certificate names above (they don't mean "any worker", the mean "whoever connects with the cert named 'worker'")21:37
*** yamahata has joined #openstack-infra21:40
*** rfolco has quit IRC21:40
openstackgerritA change was merged to stackforge/gertty: Properly register password from prompt
openstackgerritA change was merged to stackforge/gertty: Normalize URL to ends with a slash
*** mriedem has quit IRC21:41
openstackgerritA change was merged to stackforge/gertty: Fix editing inline comments
openstackgerritA change was merged to stackforge/gertty: Handle "No newline at end of file" and add --no-sync option
openstackgerritMarc Abramowitz proposed a change to stackforge/python-jenkins: 100% test coverage
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Support excludes when recursively processing paths
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Added recursive option
openstackgerritA change was merged to stackforge/gertty: Handle file renames in diff view
openstackgerritA change was merged to stackforge/gertty: Handle binary files in diffstat
openstackgerritMarc Abramowitz proposed a change to stackforge/python-jenkins: Add .gitignore
*** markwash has joined #openstack-infra21:47
*** rhsu has joined #openstack-infra21:50
*** matty_dubs is now known as matty_dubs|gone21:54
*** shardy is now known as shardy_afk22:01
zarojeblair: which file corresponds to the 'ssl-ca'? the .crt, .csr or .key?22:03
*** aconrad has quit IRC22:04
*** yidclare has quit IRC22:05
jeblairzaro: for the purpose of configuring gearman, ca.crt is the one you want.22:09
salv-orlandosdague: if you still need me I'm here22:09
jeblairzaro: (your geard command line from earlier looked right)22:09
*** gondoi is now known as zz_gondoi22:09
sdaguesalv-orlando: I think there was just a question of intent on the full patch22:09
sdaguebecause it made some of the jobs vote on stable only22:10
salv-orlandoah I see the intent is to run smoke jobs on stable only, the full job anywhere but voting only on master22:11
salv-orlandonot sure if my patch implements that intent however22:11
jeblairsdague: thoughts on and possible interaction with
zarojeblair: i got a connection!  thanks.22:12
jeblairzaro: yaay!22:13
mattoliverauMorning all22:14
*** nati_ueno has quit IRC22:15
*** aconrad has joined #openstack-infra22:16
fungiawesome--i've not missed much22:18
zarofungi: you might want to pickup this bug?
uvirtbotLaunchpad bug 1248794 in openstack-ci "Allow core reviewers restore auto-abandoned reviews" [Medium,In progress]22:19
*** ihrachyshka has quit IRC22:21
fungii see fairly many bare-precise nodes running jobs now, but no obvious movement in zuul status22:21
openstackgerritAlex Conrad proposed a change to openstack-infra/jenkins-job-builder: support execute-on for postbuildscript
openstackgerritAlex Conrad proposed a change to openstack-infra/jenkins-job-builder: support execute-on for postbuildscript
fungiyeah, something's not right. is running gate-nova-pep8 for 91025,1 but the zuul status page says that's still queued22:24
*** atiwari has quit IRC22:25
fungiit ran in the gate pipeline and completed reporting success...22:26
fungiand then nodepoold marked it for delete22:26
fungihuh, now zuul shows the same job running, no longer queued, but on bare-precise-rax-ord-391385522:27
*** smarcet has quit IRC22:27
fungiahh, i bet the other was queued from a gate reset22:28
*** nati_ueno has joined #openstack-infra22:28
openstackgerritK Jonathan Harker proposed a change to openstack-infra/config: Clean up some variable references
fungiand i walked in at just the wrong moment :/22:28
jesusauruscan i get some eyes on ?22:29
*** bknudson has quit IRC22:29
*** mfer has quit IRC22:29
*** yidclare has joined #openstack-infra22:30
fungijesusaurus: so 89708 mostly moves jenkins slave scripts out of the jenkins module and into the openstack_project module?22:30
fungioh, and a lot of the manifest too actually22:31
openstackgerritA change was merged to openstack-infra/config: Create bash8 project in openstack-dev
fungiokay, ^ proof things are indeed moving again22:31
jesusaurusfungi: it also moves bits of the ::jenkins::slave class into the ::openstack_projects module22:32
jesusaurussince most of that class is openstack-specific22:32
fungiyep, i see that22:32
openstackgerritA change was merged to openstack-infra/config: Update zuul merger init script
jesusaurusand i also found that the term "bare slave" is overloaded, so i renamed the two things to simple_slave and thick_slave22:33
fungithe thin/thick terminology is good, i think22:33
fungiyes, my thinking exactly22:33
openstackgerritA change was merged to openstack-infra/config: Add devstack job for Mistral project
*** Ryan_Lane1 has joined #openstack-infra22:35
mgagneat least we know didn't break XenAPI driver...22:36
*** Ryan_Lane has quit IRC22:36
*** prad_ has quit IRC22:37
*** gyee has quit IRC22:39
jeblairmgagne: yeah, i think they fixed that22:39
*** doddstack has quit IRC22:40
*** weshay has quit IRC22:42
zarojesusaurus: ++ on thick/thin22:42
*** signed8bit has quit IRC22:43
*** signed8bit has joined #openstack-infra22:43
mgagnejesusaurus: thanks for the cleanup =)22:43
*** nati_ueno has quit IRC22:43
*** nati_ueno has joined #openstack-infra22:45
jesusaurusde nada22:46
*** UtahDave has quit IRC22:47
*** arnaud__ has joined #openstack-infra22:48
*** bknudson has joined #openstack-infra22:49
*** jhesketh_ has quit IRC22:51
*** andreykurilin_ has quit IRC22:52
openstackgerritAlex Conrad proposed a change to openstack-infra/jenkins-job-builder: support execute-on for postbuildscript
*** jhesketh__ has joined #openstack-infra22:53
*** sweston has quit IRC22:53
*** bknudson has quit IRC22:54
*** jhesketh__ is now known as jhesketh22:54
*** david-lyle has quit IRC22:58
*** david-lyle has joined #openstack-infra22:58
*** david-lyle has quit IRC22:58
*** aysyd has quit IRC22:59
*** zns has quit IRC22:59
*** flaper87 is now known as flaper87|afk23:00
*** jamielennox|away is now known as jamielennox23:01
*** arnaud__ has quit IRC23:03
*** signed8bit has quit IRC23:09
*** bknudson has joined #openstack-infra23:09
openstackgerritSteve Kowalik proposed a change to openstack-infra/gitdm: Add StevenK to the HP group list
*** zns has joined #openstack-infra23:11
*** hemna is now known as hemna__23:12
jeblairphschwartz: something like 50-75% of dfw and iad nodes are failing to start now (i'm seeing lots of "ERROR" states in the logs) ord is great.23:16
phschwartzdo you have instance id's for some that went to ERROR23:17
phschwartzI want to get an admin looking right now23:17
jeblair1 sec23:17
phschwartzjeblair: ^^23:17
*** wenlock has quit IRC23:17
jeblairphschwartz: 8d320eec-6f6a-416c-a526-a14437568029  at  2014-04-30 23:17:21,51223:18
jeblairthat was bare-precise-rax-dfw-3915000.slave.openstack.org23:18
phschwartzjeblair: let me pull up the nova log23:19
*** luisg__ has joined #openstack-infra23:21
*** luisg_ has quit IRC23:21
*** mrodden has quit IRC23:31
*** sweston has joined #openstack-infra23:32
*** sweston has quit IRC23:38
harlowjajeblair how do u currently exit gertty ?? :)23:38
harlowjakill `pid`?23:38
anteayactrl \23:38
jeblairyeah ^, i need to add those daemon flags you pointed out.  :)23:39
*** sweston__ has joined #openstack-infra23:40
*** sweston__ is now known as sweston23:41
mattoliverauanteaya: Do I need to generate a gerrit http password to use gertty?23:41
anteayain settings you should see where23:42
anteayaclick the generate button23:42
anteayata da23:42
mattoliverauthought so, just wanted to make sure :)23:45
*** SumitNaiksatam has joined #openstack-infra23:45
*** mrodden has joined #openstack-infra23:45
*** nati_ueno has quit IRC23:48
*** nati_ueno has joined #openstack-infra23:48
anteayamattoliverau: sure23:48
*** mrodden has quit IRC23:49
*** manishg has quit IRC23:54
*** nati_ueno has quit IRC23:54
openstackgerritK Jonathan Harker proposed a change to openstack-infra/config: JJB can take longer than 5 minutes to run

Generated by 2.14.0 by Marius Gedminas - find it at!