Tuesday, 2014-04-29

anteayamattoliverau: do your best, if your information needs correcting someone will step in00:00
*** doddstack has quit IRC00:00
anteayamattoliverau: and thanks00:00
fungimordred: on https://review.openstack.org/#/c/90842/1/modules/openstack_project/files/gerrit/acls/openstack-dev/hacking.config what's the reasoning behind removing create on refs/*? i think that's being used by some project teams to create branches00:00
* anteaya tapers down00:00
mordredfungi: ah - basically, I removed create from things because we had it in place in support of creating tags everywhere and is not actually needed for that purpose anymore00:01
mattoliverauanteaya: thanks, all I can do it help anyone the best I can and let them know that need to take it with a grain of salt if it's something I'm not entirely sure of :) You go sleep! you deserve it!00:01
*** jhesketh has quit IRC00:01
mordredfungi: it's entirely possible that tere are legitimate reasons for that line to be places00:01
*** jhesketh has joined #openstack-infra00:02
fungimattoliverau: the easy answer is to ask people why they're doing stuff on the project vacation week ;)00:03
*** aysyd has joined #openstack-infra00:03
mattoliveraufungi: lol, true :)00:04
jeblairmordred: i just merged a change to sandbox00:04
jeblairmordred: it hasn't shown up on the apache replica yet00:05
mordredjeblair: I dont' see any issuse in the log00:05
*** SumitNaiksatam_ has joined #openstack-infra00:06
*** jhesketh has quit IRC00:07
jeblairmordred: all 5 git servers are updated but not the local replica00:07
mordredwell that's strange00:07
*** SumitNaiksatam has quit IRC00:07
*** lcostantino has joined #openstack-infra00:07
*** jhesketh_ has joined #openstack-infra00:07
jeblairmordred: becaus it's replicating to the wrong place :(00:08
jeblairpatch incoming00:08
mordredI thnk I may have just realized what you're saying and I'm placing my heands on my face00:08
*** atiwari has quit IRC00:08
fungii didn't even think about the replication config00:08
jeblairmordred: that's a good place for them.  my hands will be doing something similar soon00:08
mordredalso - by the by - jeepyb is looking at localhost for update_bug and blueprint00:08
fungiwe moved that entire tree00:08
*** asettle has joined #openstack-infra00:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Move configured gerrit replica to /opt/lib/git  https://review.openstack.org/9085600:09
*** SumitNaiksatam_ is now known as SumitNaiksatam00:09
fungimordred: update_* are looking at localhost copies of the git repos, or for the db?00:09
fungiif the latter, we probably want them to use the jeepyb.gerritdb module00:10
openstackgerritA change was merged to openstack-infra/config: Increase replication delay to 1 second  https://review.openstack.org/9085000:10
openstackgerritMonty Taylor proposed a change to openstack-infra/jeepyb: Get the db host from the gerrit config too  https://review.openstack.org/9085700:10
jeblairrestarting gerrit00:10
mordredfungi: ^^00:10
mordredjeblair: ++00:10
jeblairmordred: didn't i already do that?00:10
*** pcrews has quit IRC00:11
mordredjeblair: you did it for a different thing I thought00:11
*** loquacities has quit IRC00:11
jeblairmordred: we'll see when gerrit comes back up.  :)00:11
mordredjeblair: you did it for manage_projects00:11
*** bradm has quit IRC00:11
*** loquacities has joined #openstack-infra00:11
* mordred headdesks for good measure00:12
jeblairmordred: you are right.  that is, for some reason, different.  it doesn't seem to use jeepydb.gerritdb00:12
mordredI thnk we keep ignoring it and hoping it'll go away00:13
mordredthat said - your change isn't landed iether00:13
mordredI think I should go in and approve today's outstanding changes that at least one of us as +2'd00:13
mordreda failure00:13
mordredwith the jenkins00:13
* mordred fix jeblair patch00:14
*** gokrokve has joined #openstack-infra00:14
openstackgerritMonty Taylor proposed a change to openstack-infra/jeepyb: Use configured mysql host  https://review.openstack.org/9083800:15
fungijeblair: mordred: that's why i mentioned jeepyb.gerritdb... we added it a while back but not everything in jeepyb is going through it for db access yet (and that should be considered a bug)00:15
mordredfungi: ++00:15
mordredjeblair: jenkins broke your other thing too - I00:16
mordredI'm fixing00:16
*** SumitNaiksatam has quit IRC00:16
jeblairmordred: i'm on that one00:16
mordredjeblair: btw - thank god the problem isn't actually, you know - a fundamental issue with zuul and gerrit 2.800:16
jeblairmordred: kow00:16
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Move configured gerrit replica to /opt/lib/git  https://review.openstack.org/9085600:17
mordredfungi: https://review.openstack.org/#/c/90852/00:17
jeblairmordred, fungi: btw, i've been keeping https://etherpad.openstack.org/p/J8WNqyP2dC up to date with changes that must merge before we can return to normal00:17
mordredjeblair: excellent00:18
jeblair2014-04-29 00:18:58,457 DEBUG zuul.Gerrit: Waiting for <Change 0x7fc8c4761dd0 90858,1> to appear in git repo00:19
jeblair2014-04-29 00:19:03,547 DEBUG zuul.Gerrit: Change <Change 0x7fc8c4761dd0 90858,1> is in the git repo00:19
*** gokrokve has quit IRC00:19
jeblairwoohoo ^00:19
*** jeblair changes topic to "Gerrit upgraded to 2.8. See: https://wiki.openstack.org/wiki/GerritUpgrade Known problems with experimental pipelines and formatting of test reports."00:19
*** zehicle_at_dell has joined #openstack-infra00:20
jeblairmerging the remove force-message change00:20
openstackgerritMonty Taylor proposed a change to openstack-infra/jeepyb: Get the db host from the gerrit config too  https://review.openstack.org/9085700:20
*** sarob has quit IRC00:20
openstackgerritA change was merged to openstack-infra/config: Actually disable trivial_rebase_hook  https://review.openstack.org/9085300:21
*** gondoi is now known as zz_gondoi00:21
*** prad has quit IRC00:22
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Update acls to enforce pushing signed tags  https://review.openstack.org/9084200:24
*** bradm has joined #openstack-infra00:24
jeblairfungi: ^ that lgtm00:27
fungijeblair: tm too00:28
fungino longer removing branch creation perms00:28
openstackgerritA change was merged to openstack-infra/jeepyb: Get the db host from the gerrit config too  https://review.openstack.org/9085700:29
*** yidclare has quit IRC00:29
openstackgerritA change was merged to openstack-infra/config: Add launchpadlib to review.o.o  https://review.openstack.org/9085200:29
*** bhuvan has quit IRC00:29
openstackgerritA change was merged to openstack-infra/config: Remove force-message from zuul layout  https://review.openstack.org/9083200:29
*** lcostantino has quit IRC00:29
openstackgerritA change was merged to openstack-infra/config: Plumb mysql host through gerrit.pp  https://review.openstack.org/9083700:29
openstackgerritMonty Taylor proposed a change to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8  https://review.openstack.org/7081800:29
mordred^^ I just updated zaro's patch - the config section in it now matches what's in production00:30
*** nati_uen_ has quit IRC00:30
*** nati_ueno has joined #openstack-infra00:31
*** nati_ueno has quit IRC00:31
jeblairi ran puppet on zuul; that change is applied now and puppet in clean (except for the global authorized keys errors)00:31
fungimordred: including the runas addition for bootstrappers?00:31
jeblairi will re-enable puppet on zuul00:31
*** mburned is now known as mburned_out00:32
mordredfungi: oh! no - what does that look like again? (did you apply that to all-projects?00:32
*** jreypo has quit IRC00:32
fungimordred: (global perm i added so i could impersonate users to set workflow -1 for their wio changes earlier)00:32
openstackgerritA change was merged to openstack-infra/config: Move configured gerrit replica to /opt/lib/git  https://review.openstack.org/9085600:32
mordredfungi: cool. I'd love to add that00:32
fungiyes, to all-projects... did it through the horrible webui :/00:32
* mordred is going to get All-Projects into our config dir - doesn't really care for the manual nature of it00:33
mordredsame with teh dashboards00:33
jeblairmordred: ++00:33
openstackgerritA change was merged to openstack-infra/jeepyb: Use configured mysql host  https://review.openstack.org/9083800:33
fungimordred: agreed, we could in theory update it through m-p, just can't bootstrap a new gerrit that way obviously00:33
jeblairi believe that's all outstanding changes merged; i'll see if puppet is clean on review00:33
jeblairi thought someone had worked out the minimum perms needed to let m-p bootstrap; so we can document those00:35
mordredjeblair: I thought yolanda did00:35
mordredjeblair: if I wanted to start chasing down the ugly status breakage - any thoughts on where I'd start?00:36
fungimordred: jeblair: right, though it ended up being juju charm secret sauce for ubuntu cloud, though the concept could probably be repeated00:36
mordredfungi: I'm sure we can ansible it00:36
mordredfungi: or, you know, just write a bash script00:37
*** msabramo has joined #openstack-infra00:37
* mordred will say - working with dib at the end of last week made him really appreciate NOT having to learn a meta language00:37
jeblairfungi, mordred: couple of things missing in puppet.  patch in a sec00:37
mordredthe learning curve was "what sequence do these thigns run in, which run in the image and which run outside, and what paths are pre-set in env vars"00:38
fungithe basic takeaway is that you need a minimal but syntactically correct nugget of gerrit acl config *and* groups file written to disk with the gerrit service not yet started, enough to allow project-bootstrappers to push the rest through the api00:38
openstackgerritA change was merged to openstack-infra/config: Update acls to enforce pushing signed tags  https://review.openstack.org/9084200:39
mordredjeblair: since the acl change landed - when you re-run puppet, we may want to follow that with an m-p run00:40
*** mrodden has quit IRC00:40
mordredOR - we could trigger than just by landing the gertty patch once you've re-enabled puppet00:40
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Set github replicationDelay to match (1 sec)  https://review.openstack.org/9085900:40
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Enable run-as in gerrit  https://review.openstack.org/9086000:40
jeblairmordred, fungi: ^ those two are outstanding local-fs vs puppet changes00:41
*** mbacchi has quit IRC00:41
*** SumitNaiksatam has joined #openstack-infra00:42
fungii could have sworn i added a change identical to https://review.openstack.org/9086000:44
jeblairfungi: you did https://review.openstack.org/#/c/90814/100:44
fungiahh, i can abandon https://review.openstack.org/9081400:44
openstackgerritMonty Taylor proposed a change to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8  https://review.openstack.org/7081800:44
jeblairsorry i didn't see that00:44
*** nati_ueno has joined #openstack-infra00:44
jeblairi mean, again :)00:44
jeblairi did approve it but it conflicted or something.  possibly bogus.00:45
jeblairduring the broken times00:45
mordredfungi, jeblair ^^ docs updated to included the runAs capability00:45
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add vim to the list of explicit things on servers  https://review.openstack.org/9078600:46
*** pcrews has joined #openstack-infra00:46
mordredI just used "Cherry Pick To" on the same branch to re-parent a patch without downloading it00:46
vishyis recheck still working? https://review.openstack.org/#/c/88416/00:46
jeblairmordred: i like that you call 'vim' an 'explicit thing'00:46
jeblairvishy: we've been restarting things all day, that was likely lost; should work now00:47
mordredjeblair: :)00:48
mordredfungi: so - as a note from your time away from us ...00:48
*** jeblair changes topic to "Gerrit upgraded to 2.8. See: https://wiki.openstack.org/wiki/GerritUpgrade Known problem with formatting of test reports."00:48
mordredfungi: if you're root on puppetmaster - triggering puppet on one of the nodes is as simple as "ssh $nodename"00:48
mordredfungi: because of the captive command ssh stuff00:49
jeblairfungi: disabling puppet on a node requires running "sudo puppet agent --disable" on the node00:49
mordredand yes - that ^^00:49
mordredbecause there are now no daemons00:49
jeblairfungi: (that's in the docs, but i think we just discovered a problem with docs publishing)00:50
openstackgerritA change was merged to openstack-infra/config: Set github replicationDelay to match (1 sec)  https://review.openstack.org/9085900:50
fungijeblair: mordred: awesome. i figured out the --disable already since puppet agent obviously wasn't running as a daemon any longer. i should have expected the ssh bit following our discussion about tying shell commands to authorized keys00:51
openstackgerritA change was merged to openstack-infra/config: Enable run-as in gerrit  https://review.openstack.org/9086000:51
jeblairrunning puppet noop on review00:52
mordredjeblair: awesome00:52
jeblairokay well that's useless because of the acl change.    :)00:52
*** etoews has joined #openstack-infra00:52
jeblairrunning again and capturing output...00:52
jeblairlgtm, running for real00:53
*** ramashri has quit IRC00:54
jeblairmordred: puppet is running m-p because of the acl changes00:54
*** mrodden has joined #openstack-infra00:55
mordredjeblair: kk00:55
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Remove "Code Review" from Gerrit site header  https://review.openstack.org/9086201:03
jeblairpuppet/m-p finished01:03
jeblairrunning puppet once more01:04
jeblairerm, it seems to have removed melange and melangeclient acl files01:04
mordredjeblair: so - looking at the test results html stuff - gerrit seems to be stripping class and span from the emittted html01:05
*** sweston has quit IRC01:06
jeblairmordred: any idea about the melange thing?01:06
jeblairbecause that's triggering another run of manage-projects01:06
jeblair-rw-r--r-- 1 root    root       0 Apr 29 01:07 python-melangeclient.config01:07
*** marcoemorais has quit IRC01:07
*** jbroome has joined #openstack-infra01:08
jeblairmordred: could m-p be writing those?01:08
mordredjeblair: yes. yes it could01:08
*** jbroome has left #openstack-infra01:08
jeblairmordred: that seems weird to me; i think of that directory as input files01:08
mordredI do too - I think it may be old logic that we don't care about anymore01:09
mordredjeblair: write_acl_config01:09
openstackgerritMonty Taylor proposed a change to openstack-infra/jeepyb: Stop writing project configs  https://review.openstack.org/9086501:10
mordredjeblair: ^^01:10
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Add empty melange and python-melangeclient config files  https://review.openstack.org/9086601:11
jeblairmordred: ^^01:11
openstackgerritA change was merged to openstack-dev/pbr: README.rst: tweaks  https://review.openstack.org/8927701:12
jeblairmordred: i don't see melange files on the old server though :/01:12
*** gyee has quit IRC01:13
*** asettle has quit IRC01:13
*** gokrokve has joined #openstack-infra01:14
jeblairmordred: i don't understand what changed01:15
jeblairi don't really want to enable puppet yet because i think manage-projects will run every time01:15
jeblairmordred: do you think the 0-byte file change is safe?01:16
*** gokrokve_ has joined #openstack-infra01:17
jeblairokay, it's starting to feel like i'm the only one in the room now01:18
mordredjeblair: sorry - was in another window01:18
jeblairi guess we'll leave things as they are01:18
mordredand my notifications are b0rken01:18
mordredjeblair: I'm not sure what's changed01:18
mordredI don't think the 0-byte change is dangerous01:18
*** gokrokve has quit IRC01:18
mordredbut ... why is it trying to make an openstack/melange?01:19
mordredwe don't have that in projects.yaml any more do we?01:19
mordredoh - we still do01:19
*** sdake_ has quit IRC01:19
jeblairmordred: we added it back01:20
*** gokrokve_ has quit IRC01:20
jeblairbug 128848501:20
uvirtbotLaunchpad bug 1288485 in openstack-ci "Old projects listed in ls-projects may not be clonable" [Medium,Fix released] https://launchpad.net/bugs/128848501:20
mordredjeblair: oh - you know what?01:20
mordredjeblair: oh - neermind- you rsynced the dirs01:21
jeblairmordred: so if you like the 0 byte files change, can you approve that and we can see if that makes m-p stop running?01:21
mordredjeblair: done01:22
*** sweston has joined #openstack-infra01:23
*** asettle-bfast is now known as asettle01:23
*** nati_uen_ has joined #openstack-infra01:23
*** sdake_ has joined #openstack-infra01:24
*** sdake_ has quit IRC01:24
*** sdake_ has joined #openstack-infra01:24
openstackgerritA change was merged to openstack-infra/config: Add empty melange and python-melangeclient config files  https://review.openstack.org/9086601:26
*** gokrokve has joined #openstack-infra01:26
*** nati_ueno has quit IRC01:27
jeblairrunning puppet01:27
*** psedlak has quit IRC01:28
mordredjeblair: I'm going to muck with review-dev to try out a few things for comment links01:28
jeblairmordred: ok.  i'm assuming that something about the html changed -- the test report beautification actually matches the transformed html output that gerrit produces for list items01:29
jeblairmordred: that's probably pretty fragile01:29
*** gokrokve has quit IRC01:30
jeblairmordred: fwiw, i figured that one was less urgent and could wait until tomorrow01:30
jeblairmordred: puppet is happy now01:30
jeblairre-enabling on review01:30
mordredjeblair: yeah - I was just looking while you were running puppet01:31
jeblairSergeyLukjanov, mordred, fungi: puppet is running on all the hosts again; it should be safe to approve puppet changes01:31
mordredjeblair: my current hunch is that they're stripping the html of "unsafe" things01:31
jeblairs/running/enabled/  (habits die hard)01:32
*** yaguang has joined #openstack-infra01:32
mordredjeblair: when I open it up in developer tools - the span and class portions of the html are not there01:32
mordredbut the li is01:32
mordredoh - I see what you're saying now thought perhsp01:32
*** sandywalsh has quit IRC01:32
jeblairmordred: well, the li comes from gerrit's own transform...01:32
* mordred long day face01:32
jeblairmordred: indeed.  i think i'm going to sign off now01:33
jeblairi am becoming dangerous around production systems01:33
mordredjeblair: ok. I know what the problem is on the test results01:33
jeblairmordre: okay i can hang for another minute. :)01:33
mordredhas been added01:33
mordredwhich is not in our regex01:34
jeblairthat'll do it01:34
mordredI'll test a patch on review-dev01:34
*** gokrokve has joined #openstack-infra01:35
*** psedlak has joined #openstack-infra01:35
jeblairmordred: ok, i am going to afk for now.  thanks for finding that.01:35
mordredjeblair: have a fun night!01:36
*** gokrokve has quit IRC01:38
*** etoews has quit IRC01:40
*** gyee has joined #openstack-infra01:42
zaromordred: did you find replication docs?01:48
*** zehicle has joined #openstack-infra01:48
mordredzaro: I did not - but it turns out it was a config error on our part01:48
mordredwe moved the local replica dir and, well, we forgot to mve it in puppet01:48
zaroit's in the replication plugin, https://gerrit.googlesource.com/plugins/replication/+/stable-2.8/src/main/resources/Documentation01:48
*** etoews has joined #openstack-infra01:49
*** adalbas has quit IRC01:49
*** yjiang has joined #openstack-infra01:50
*** zehicle_at_dell has quit IRC01:51
*** zhiyan_ is now known as zhiyan01:52
mordredah - resources01:54
mordredthanks zaro01:55
*** markmcclain has quit IRC02:00
*** adalbas has joined #openstack-infra02:01
*** nati_uen_ has quit IRC02:02
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Update the URL match pattern  https://review.openstack.org/9087202:03
mordredfungi, jeblair, SergeyLukjanov: my brain cannot wrap itself around how to test this ^^02:04
mordredbut I wanted to capture it as the current state of my brain02:04
fungitaking a look02:04
fungiand then i am going to pretend that it's late here and that i need to be productive some tomorrow02:04
fungimordred: that makes sense actually. i noticed that links in gerrit 2.8 now open in a new target rather than in the current window/tab02:05
mordredfungi: I mean, it's not that destructive - we _could_ just apply it and restart gerrit and see if it fixes it02:06
fungiit's possible to test on review-dev it we want02:06
funginot so much testing the puppetry, but manually adjusting the gerrit.config in a corresponding manner02:07
mordredfungi: how do we get text in there to test ?02:07
mordredfungi: I've applied it on review-dev already02:07
mordredbut I wasn't sure what I should put in the comment02:07
mordredas input02:07
fungiahh, unordered list elements i thought02:07
fungichecking to see where i found it hard-coded into zuul previously02:08
*** smarcet has quit IRC02:09
*** sweston has quit IRC02:09
fungimordred: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/scheduler.py#n148602:10
jeblairjust leave a comment on a change in dev with the text of a comment that zuul has left in prod02:11
jeblairmordred: ^02:11
*** Sukhdev has joined #openstack-infra02:12
fungioh, i may be looking at the wrong bit of code there02:12
fungiyeah, the formatReport() method on n141002:13
Sukhdevfungi: Can I ask you a question?02:14
fungiSukhdev: you can try, though i'm fairly brain-dead at this point02:14
Sukhdevfungi: :-)02:14
Sukhdevfungi: My third party test system, when posting vote on Gerrit, it comes back and gives me a warning that remote host identification has changed02:15
fungiSukhdev: yes, we changed the gerrit service host keys today when gerrit was upgraded, as a safety precaution02:16
Sukhdevfungi: has anything changed on the gerrit server02:16
fungiSukhdev: have you been following the openstack-dev mailing list?02:17
Sukhdevfungi: Ah ha - that explains it….02:17
Sukhdevfungi: no, I was caught up with a critical issue at work02:17
Sukhdevfungi: any quick fix that I can use to get this going?02:17
*** sweston has joined #openstack-infra02:18
fungiSukhdev: http://lists.openstack.org/pipermail/openstack-dev/2014-April/033908.html02:18
fungiSukhdev: the fix will depend on the configuration of your system (particularly, what software is connecting to the gerrit api endpoint)02:20
mordredjeblair: nope. can't get it to be better :(02:20
Sukhdevfungi: I use the following command to post the vote -  ssh -p 29418 arista-test@review.openstack.org gerrit review -m ' Arista third party testing PASSED02:21
*** wenlock has joined #openstack-infra02:21
Sukhdevfungi: so, if I delete the host_id from my .ssh/known_hosts - that should do the trick, right?02:22
fungiSukhdev: yes, and then accept the new key02:22
Sukhdevfungi: how do I accept the new key?02:22
fungiSukhdev: you could try running 'ssh -p 29418 arista-test@review.openstack.org' locally as the same user which your ci runs under02:23
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Update authorship information  https://review.openstack.org/8991202:23
fungiand then when openssh prompts you about the new key fingerprint, respond "yes"02:24
Sukhdevfungi: cool….let me give it a try02:24
*** alexpilotti has quit IRC02:32
*** ramashri has joined #openstack-infra02:38
*** Ryan_Lane has quit IRC02:38
*** terryyao has joined #openstack-infra02:39
Sukhdevfungi: Thanks - I was able to get the correct keys…02:40
*** mriedem has quit IRC02:41
*** ramashri has quit IRC02:43
fungiSukhdev: you're welcome02:43
*** markwash has quit IRC02:44
*** Sukhdev has quit IRC02:47
terryyaohello, who can tell me the progress on https://review.openstack.org/9001702:53
*** bknudson has quit IRC02:54
zaroterryyao: we don't approval until a change gets two +202:54
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors  https://review.openstack.org/9087402:54
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror  https://review.openstack.org/9087502:54
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror  https://review.openstack.org/9087503:00
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors  https://review.openstack.org/9087403:00
*** yjiang has quit IRC03:01
*** yjiang|2 has joined #openstack-infra03:02
terryyaocan any core reviewer take a look at  https://review.openstack.org/90017?03:17
*** arnaud__ has quit IRC03:18
*** Longgeek has joined #openstack-infra03:22
*** mengxd has joined #openstack-infra03:27
*** gokrokve has joined #openstack-infra03:27
tjonesmattoliverau: anteaya: just letting you know i tried to push the review again and it worked just fine.03:32
*** etoews has quit IRC03:32
mengxdjeblair: ping03:34
*** msabramo has quit IRC03:35
*** msabramo has joined #openstack-infra03:36
mattoliverautjones: awesome, I was just out for lunch was going to ping you when I got back :) Glad it's working now!03:37
*** praneshp has quit IRC03:55
*** aysyd has quit IRC03:57
*** jamespage has quit IRC04:00
*** gnuoy has quit IRC04:00
jhesketh_is gerrit not accessible at the moment?04:00
*** jamespage has joined #openstack-infra04:00
*** gnuoy has joined #openstack-infra04:00
*** jamespage has quit IRC04:00
*** jamespage has joined #openstack-infra04:00
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests  https://review.openstack.org/9052404:01
*** dripton has quit IRC04:03
*** gyee has quit IRC04:04
*** marcoemorais has joined #openstack-infra04:04
*** dripton has joined #openstack-infra04:04
openstackgerritJoshua Hesketh proposed a change to openstack-infra/nodepool: Make template and node hostnames configurable  https://review.openstack.org/9088304:04
*** chandan_kumar has joined #openstack-infra04:05
*** mengxd has quit IRC04:05
*** marcoemorais has quit IRC04:08
*** sweston has quit IRC04:12
*** morganfainberg is now known as morganfainberg_Z04:20
*** praneshp has joined #openstack-infra04:21
*** gokrokve has quit IRC04:22
*** praneshp_ has joined #openstack-infra04:22
*** Ryan_Lane has joined #openstack-infra04:23
*** marcoemorais1 has joined #openstack-infra04:24
*** praneshp has quit IRC04:26
*** praneshp_ is now known as praneshp04:26
*** sweston has joined #openstack-infra04:27
*** marcoemorais1 has quit IRC04:29
*** marcoemorais has joined #openstack-infra04:30
*** ildikov has quit IRC04:33
*** sweston_ has joined #openstack-infra04:34
*** sweston has quit IRC04:34
*** mspreitz has joined #openstack-infra04:36
mspreitzI just got this complaint: The RSA host key for [review.openstack.org]:29418 has changed04:37
mspreitzDid that key really change?04:37
jhesketh_mspreitz: yes04:44
jhesketh_mspreitz: gerrit was upgraded04:45
jhesketh_mspreitz: https://wiki.openstack.org/wiki/GerritUpgrade#DNS_Spoofing_Warning04:45
mspreitzjhesketh_: thanks04:45
jhesketh_mspreitz: there was also an email to the -dev list with PGP signature confirming the new host key (if you want more confidence then review.openstack.org confirming it's own key)04:45
*** enykeev has joined #openstack-infra04:52
*** ArxCruz has quit IRC04:53
enykeevHey! Is gerrit still down? http://cl.ly/VEKQ04:54
enykeevor I'm just missing something04:55
*** talluri has joined #openstack-infra04:56
*** harlowja is now known as harlowja_away04:56
mattoliverauenykeev:gerrit is back up04:57
enykeevthen why I'm getting 503?04:57
mattoliverauenykeev: try shift+refresh04:57
enykeevtried cmd+r, no luck04:58
enykeevsame with another browser04:58
mattoliveraunot cmd+r (on a mac) acutally shift+<refresh button on browser> to refresh from cache, or try clearing your cache and then going to the site.05:00
mattoliverauenykeev: ^^ it is up, and everyone I've talked to has had the same problem.05:01
enykeevk, now it works. Thanks.05:01
mattoliverauenykeev: no problem, enjoy the new gerrit :)05:01
*** Sukhdev has joined #openstack-infra05:08
*** gokrokve has joined #openstack-infra05:14
*** yjiang|2 has quit IRC05:24
openstackgerritJoshua Hesketh proposed a change to openstack-infra/nodepool: Make template and node hostnames configurable  https://review.openstack.org/9088305:34
*** unicell has joined #openstack-infra05:34
*** ildikov has joined #openstack-infra05:35
*** pcrews has quit IRC05:43
*** Ryan_Lane1 has joined #openstack-infra05:47
*** gokrokve_ has joined #openstack-infra05:48
*** Ryan_Lane1 has quit IRC05:50
*** gokrokve has quit IRC05:52
*** sdake_ has quit IRC05:54
*** gokrokve_ has quit IRC05:57
*** chandan_kumar has quit IRC06:01
*** ashishsnigam has quit IRC06:06
*** ashishsnigam has joined #openstack-infra06:06
*** etoews has joined #openstack-infra06:09
*** praneshp has quit IRC06:12
*** etoews has quit IRC06:13
*** chandan_kumar has joined #openstack-infra06:16
*** xchu has joined #openstack-infra06:18
*** jamielennox is now known as jamielennox|away06:18
*** chandan_kumar has quit IRC06:22
*** mspreitz has quit IRC06:27
*** gokrokve has joined #openstack-infra06:27
*** mrmartin has joined #openstack-infra06:28
*** viktors|afk is now known as viktors06:29
*** gokrokve_ has joined #openstack-infra06:29
*** gokrokve has quit IRC06:32
*** gokrokve_ has quit IRC06:34
*** sweston_ has quit IRC06:38
*** ArxCruz has joined #openstack-infra06:38
*** flaper87|afk is now known as flaper8706:39
*** chandan_kumar has joined #openstack-infra06:40
*** ArxCruz has quit IRC06:40
*** e0ne has joined #openstack-infra06:42
*** che-arne has quit IRC06:51
*** d0ugal has joined #openstack-infra06:52
*** unicell has quit IRC06:54
*** gokrokve has joined #openstack-infra06:57
*** afazekas_ has joined #openstack-infra06:58
*** _nadya_ has joined #openstack-infra06:58
*** thomasbiege has joined #openstack-infra07:03
*** mrmartin has quit IRC07:03
*** jlibosva has joined #openstack-infra07:05
*** gema has joined #openstack-infra07:05
*** jcoufal has joined #openstack-infra07:06
*** Sukhdev has quit IRC07:08
*** _nadya_ has quit IRC07:08
*** thomasbiege has quit IRC07:08
*** sdake has quit IRC07:10
*** wenlock has quit IRC07:11
*** mrmartin has joined #openstack-infra07:11
*** _nadya_ has joined #openstack-infra07:13
*** ihrachyshka has joined #openstack-infra07:14
mattoliverauI'm calling it a day, have a great night/day all!07:14
*** zns has quit IRC07:16
*** thomasbiege has joined #openstack-infra07:20
*** jgallard has joined #openstack-infra07:20
*** maxbit has joined #openstack-infra07:27
*** reed has joined #openstack-infra07:27
*** sweston has joined #openstack-infra07:34
*** sweston has quit IRC07:39
*** thomasbiege has quit IRC07:41
*** gema has quit IRC07:43
*** hashar has joined #openstack-infra07:43
*** Ryan_Lane has quit IRC07:49
*** dizquierdo has joined #openstack-infra07:50
*** jpich has joined #openstack-infra07:54
*** rcarrill` has quit IRC07:54
*** skolekonov has joined #openstack-infra07:54
*** rcarrillocruz has joined #openstack-infra07:54
*** sdake has joined #openstack-infra08:01
*** ashishsnigam has quit IRC08:04
openstackgerritAntoine Musso proposed a change to openstack-infra/zuul: Pass pngcrush on status png files  https://review.openstack.org/8986508:04
*** ildikov has quit IRC08:05
openstackgerritSteven Hardy proposed a change to openstack-infra/reviewstats: Add thomasspatzier to heat-core  https://review.openstack.org/9094208:08
*** etoews has joined #openstack-infra08:11
*** yassine has joined #openstack-infra08:12
openstackgerritAntoine Musso proposed a change to stackforge/python-jenkins: Wrap tests and linting with tox  https://review.openstack.org/8952108:15
*** bauzas has joined #openstack-infra08:15
*** etoews has quit IRC08:16
*** skolekonov has left #openstack-infra08:17
*** derekh has joined #openstack-infra08:17
*** amcrn has quit IRC08:18
SergeyLukjanovmorning folks08:18
*** gema has joined #openstack-infra08:19
derekhDoes the need to be approved again https://review.openstack.org/#/c/90757/1 ? I'm guessing it didn't merge because of gerrit upgrade08:19
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Add some more Trove classifiers  https://review.openstack.org/9067408:20
derekhSergeyLukjanov: thanks08:20
SergeyLukjanovderekh, np08:20
tchaypodstufft: "write a simple test", he said08:23
tchaypoIf I'd know at the time that this was going to lead to me copying a function called find-links2 to make a new one called find-links3 that's one character different, I might have felt less cheerful..08:25
*** ildikov has joined #openstack-infra08:26
*** darraghb has joined #openstack-infra08:27
openstackgerritA change was merged to openstack-infra/config: Run check-tripleo jobs on stale changes  https://review.openstack.org/8585408:31
openstackgerritFabien Boucher proposed a change to openstack-infra/config: Fix typo in ACL.  https://review.openstack.org/9094508:33
*** sweston has joined #openstack-infra08:34
rcarrillocruzfungi: thanks for the approval on https://review.openstack.org/#/c/86382/ , just saw it...08:37
*** ihrachyshka has quit IRC08:37
*** jooools has joined #openstack-infra08:38
*** mattymo|home has joined #openstack-infra08:39
*** sweston has quit IRC08:39
*** ihrachyshka has joined #openstack-infra08:39
*** rcarrill` has joined #openstack-infra08:40
*** rcarrillocruz has quit IRC08:42
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: add tests for shell builder  https://review.openstack.org/9048008:43
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Update the URL match pattern  https://review.openstack.org/9087208:44
openstackgerritA change was merged to openstack-infra/config: Fix typo in ACL.  https://review.openstack.org/9094508:44
openstackgerritA change was merged to openstack-infra/config: Remove "Code Review" from Gerrit site header  https://review.openstack.org/9086208:46
*** mrmartin has quit IRC08:48
*** mrmartin has joined #openstack-infra08:49
openstackgerritA change was merged to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8  https://review.openstack.org/7081808:52
*** _nadya_ has quit IRC08:52
* SergeyLukjanov trying to use new screen08:54
SergeyLukjanovit's not too bad as it can be, but....08:54
*** _nadya_ has joined #openstack-infra08:56
openstackgerritA change was merged to openstack-infra/config: Update the URL match pattern  https://review.openstack.org/9087208:58
*** Mithrandir has quit IRC09:01
*** adalbas has quit IRC09:01
*** derekh has quit IRC09:03
*** gema has quit IRC09:06
SergeyLukjanovfungi, should it be approved again https://review.openstack.org/#/c/81622/?09:09
SergeyLukjanovjeblair, mordred ^^09:09
*** thomasbiege has joined #openstack-infra09:10
mrmartinHi SergeyLukjanov, git review -s doesn't work for me: http://paste.openstack.org/show/77640/ had you met with something similar?09:10
SergeyLukjanovfungi, jeblair, mordred, don't worry, it's just new screen (/me don't see the merged label)09:11
*** che-arne has joined #openstack-infra09:11
SergeyLukjanovmrmartin, have you already updated host key?09:11
mrmartinnot yet09:12
*** Mithrandir has joined #openstack-infra09:13
*** thomasbiege1 has joined #openstack-infra09:13
*** etoews has joined #openstack-infra09:13
mrmartinSergeyLukjanov, thanks works for me now!09:14
*** chandan_kumar has quit IRC09:14
*** chandan_kumar has joined #openstack-infra09:15
*** Mithrandir has quit IRC09:16
*** thomasbiege has quit IRC09:17
*** etoews has quit IRC09:18
*** bauzas has left #openstack-infra09:18
*** dangers is now known as dangers_away09:19
*** Mithrandir has joined #openstack-infra09:20
*** markmc has joined #openstack-infra09:20
*** rcarrillocruz has joined #openstack-infra09:21
*** Milind has joined #openstack-infra09:21
openstackgerritA change was merged to openstack-infra/config: Check for new files as well as changed files  https://review.openstack.org/9084309:22
*** rcarrill` has quit IRC09:22
*** jp_at_hp has joined #openstack-infra09:23
*** ominakov has joined #openstack-infra09:23
openstackgerritA change was merged to openstack-infra/config: limit the width of vote columns  https://review.openstack.org/9084409:24
openstackgerritA change was merged to openstack-infra/config: Add vim to the list of explicit things on servers  https://review.openstack.org/9078609:24
Milindtempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_stop_terminate_instance_with_tags[gate,smoke] ... FAIL09:25
Milindis this a known bug ?09:25
*** thomasbiege1 has quit IRC09:26
*** bauzas has joined #openstack-infra09:26
*** WaltNik has joined #openstack-infra09:30
ihrachyshkaany plans to revert to a nice gate jobs results view with green and red colouring?09:36
ihrachyshka(re gerrit)09:37
StevenKIs anyone getting 503s from gerrit on https?09:40
StevenKAnyone else, that is ... :-(09:40
*** ociuhandu has quit IRC09:41
BobBallAwayI'm getting them yes09:43
fifieldtany OpenStack IRC ops around?09:44
bauzasStevenK: I'm having no troubles with https09:45
fifieldtjeblair, mordred, jeremyb - had a spammer in #openstack, probably needs a ban09:45
*** terryyao has quit IRC09:47
*** marcoemorais has quit IRC09:49
StevenKbauzas: That's odd. IPv4 or v6?09:53
bauzasStevenK: IPv4 sorry09:53
bauzasStevenK: and the cert is identified09:54
* StevenK drops his IPv6 default route to see if that works09:54
bauzas(btw. this major upgrade is great, thanks mordred, fungi , jeblair and other missing)09:54
StevenKRight, IPv4 works, IPv6 gives 50309:56
*** gema has joined #openstack-infra10:01
*** ociuhandu has joined #openstack-infra10:02
*** andreaf has joined #openstack-infra10:06
*** aglarendil has joined #openstack-infra10:08
*** BobBallAway is now known as BobBall10:09
*** mrmartin has quit IRC10:09
*** alexpilotti has joined #openstack-infra10:09
*** mdenny has quit IRC10:17
*** Milind has quit IRC10:18
*** yfried has joined #openstack-infra10:19
Adri2000at https://review.openstack.org/#/c/89534/ why did jenkins decide it would recheck (and do nothing afterwards)?10:19
Adri2000is it a consequence of the gerrit upgrade?10:20
yfriedjesusaurus: ping10:20
*** _nadya_ has quit IRC10:25
*** e0ne has quit IRC10:26
*** e0ne has joined #openstack-infra10:27
yfriedcan anyone here help me with logstash?10:27
*** e0ne has quit IRC10:32
*** lucasagomes has joined #openstack-infra10:32
lucasagomeshi there, I can't push any changes to gerrit after the update, I'm getting a "fatal: Could not read from remote repository."10:32
lucasagomesanyone with the same problem?10:32
*** mrmartin has joined #openstack-infra10:32
*** _nadya_ has joined #openstack-infra10:34
lucasagomesalready fixed following https://wiki.openstack.org/wiki/GerritUpgrade10:36
*** jgallard has quit IRC10:38
openstackgerritSean Dague proposed a change to openstack-infra/config: gerrit css tweaks to make it more like old gerrit  https://review.openstack.org/9096710:46
openstackgerritNikola Dipanov proposed a change to openstack-dev/hacking: Add an import whitelist helper factory  https://review.openstack.org/8001810:57
openstackgerritNikola Dipanov proposed a change to openstack-dev/hacking: Add TestCase assertion methods for testing checks  https://review.openstack.org/8027810:57
*** dizquierdo has quit IRC10:58
*** e0ne has joined #openstack-infra11:07
*** e0ne has quit IRC11:09
*** e0ne has joined #openstack-infra11:09
*** e0ne has quit IRC11:10
*** e0ne has joined #openstack-infra11:11
*** gema has quit IRC11:11
*** lucasagomes has left #openstack-infra11:11
openstackgerritNikola Dipanov proposed a change to openstack-dev/hacking: Add an import whitelist helper factory  https://review.openstack.org/8001811:12
*** enikanorov__ is now known as enikanorov_11:13
*** mrmartin has quit IRC11:15
*** e0ne has quit IRC11:16
*** fifieldt_ has joined #openstack-infra11:16
YorikSarAdri2000: it looks like your change got into check queue just before upgrade.11:17
YorikSarAdri2000: You can recheck it manually.11:17
*** fifieldt has quit IRC11:17
openstackgerritSean Dague proposed a change to openstack-infra/config: gerrit css tweaks to make it more like old gerrit  https://review.openstack.org/9096711:28
*** e0ne has joined #openstack-infra11:29
*** aglarendil has left #openstack-infra11:29
*** yaguang has quit IRC11:30
*** mburned_out is now known as mburned11:32
*** _nadya_ has quit IRC11:32
*** julim has joined #openstack-infra11:34
*** rlandy has joined #openstack-infra11:40
SergeyLukjanovfifieldt, is op still needed?11:41
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Teams API  https://review.openstack.org/9098411:43
*** dprince has joined #openstack-infra11:47
*** sandywalsh has joined #openstack-infra11:48
SergeyLukjanovjeblair, mordred, fungi, IIRC we need to restart gerrit for https://review.openstack.org/#/c/90872/, am I right?11:49
*** che-arne has quit IRC11:50
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Teams API  https://review.openstack.org/9098411:50
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror  https://review.openstack.org/9087511:51
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors  https://review.openstack.org/9087411:51
*** ildikov has quit IRC11:54
*** mrmartin has joined #openstack-infra11:54
*** pdmars has joined #openstack-infra11:56
*** pdmars has quit IRC11:57
*** pdmars_ has joined #openstack-infra11:57
bookwarhi, i have a problem with our third-party Jenkins, it seems Gerrit Trigger gets all the changes but can't leave a comment. Has something changed in the way service accounts work? Mine is called "fuel-ci"11:58
*** pdmars_ has quit IRC11:58
bookwarfor this patchset here https://review.openstack.org/#/c/90957/ i have "INFO: Notifying BuildStarted to gerrit: gerrit approve 90957,1 --message 'Build Started http://fuel-jenkins.mirantis.com/job/stackforge-verify-fuel-web/3297/ (2/2)' --verified 0 --code-review 0" in Jenkins logs, but it is nothing in comments12:01
*** pdmars has joined #openstack-infra12:01
*** sdake has quit IRC12:01
*** prad has joined #openstack-infra12:01
*** sdake has joined #openstack-infra12:01
*** sdake has quit IRC12:01
*** sdake has joined #openstack-infra12:01
openstackgerritSean Dague proposed a change to openstack-infra/config: gerrit css tweaks to make it more like old gerrit  https://review.openstack.org/9096712:02
ihrachyshkais gerrit-to-launchpad auto-update broken?12:05
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror  https://review.openstack.org/9087512:05
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors  https://review.openstack.org/9087412:05
*** afazekas_ has quit IRC12:05
*** dkranz has joined #openstack-infra12:12
bookwarok, found it myself, as 'gerrit approve' is deprecated, we need to update our Gerrit Triiger plugin to work with new gerrit properly12:14
*** etoews has joined #openstack-infra12:16
*** afazekas_ has joined #openstack-infra12:19
*** prad has quit IRC12:19
*** weshay has joined #openstack-infra12:20
*** etoews has quit IRC12:20
*** gema has joined #openstack-infra12:22
*** rfolco has joined #openstack-infra12:23
*** zhiyan is now known as zhiyan_12:23
*** gema has left #openstack-infra12:25
*** e0ne has quit IRC12:26
*** e0ne has joined #openstack-infra12:27
*** etoews has joined #openstack-infra12:29
*** aysyd has joined #openstack-infra12:30
*** weshay has quit IRC12:31
*** jgallard has joined #openstack-infra12:32
*** dims has quit IRC12:33
*** lcostantino has joined #openstack-infra12:37
*** miqui has joined #openstack-infra12:39
*** rcarrillocruz has quit IRC12:41
*** alexpilotti has quit IRC12:46
*** rcarrillocruz has joined #openstack-infra12:47
*** dims has joined #openstack-infra12:48
*** rcarrill` has joined #openstack-infra12:51
*** rcarrillocruz has quit IRC12:53
*** dkranz has quit IRC12:53
*** rcarrillocruz has joined #openstack-infra12:54
*** adalbas has joined #openstack-infra12:56
*** rcarrill` has quit IRC12:56
*** ildikov has joined #openstack-infra12:56
*** eharney has quit IRC12:58
*** heyongli has joined #openstack-infra13:01
fungiSergeyLukjanov: you were able to work out the commentlink fix (per 90872) i take it?13:02
SergeyLukjanovfungi, morning13:02
SergeyLukjanovfungi, I've check it on a bunch of html and it's working ok13:02
*** fifieldt_ is now known as fifieldt13:03
fungioh, i see. mordred had target before href when gerrit was printing it after instead13:03
SergeyLukjanovfungi, exactly13:03
fungii was too bleary-eyed to see straight by that point last night and missed it entirely13:03
SergeyLukjanovfungi, I've updated the CR and approved it13:04
SergeyLukjanovfungi, and only after it merged I remember that we need to restart gerrit to apply it13:04
fungigreat! but yeah, it takes a gerrit restart to apply configuration changes, and since this is a cosmetic fix we may want to wait a little and batch it up with other fixes we may need to restart for anyway13:04
SergeyLukjanovfungi, sure13:05
BobBallAny chance you could workflow+1 https://review.openstack.org/#/c/90660/ again fungi?  You did so yesterday but then I guess something got restarted, so the job was lost13:06
SergeyLukjanovfungi, new screen is a bit hmmmm13:06
*** doug-fish has joined #openstack-infra13:07
sdagueBobBall: I can trigger it13:09
BobBallthanks sdague.13:09
fungiahh, yep, i see what went sideways there13:09
sdaguethough it doesn't seem like it's taking the additional approve13:09
sdaguefungi: I guess we are missing a trigger?13:10
sdagueBobBall: so actually, because we were within clean check, you actually just needed a recheck no bug to get jenkins to pay attention to it13:11
fungii added a +1 workflow to all the previously approved changes after the upgrade, adding myself to the project bootstrappers group so i'd have the necessary permissions, but when i removed myself from that group again afterward it also made my vote disappear since i no longer had permission13:11
fungithen adding myself back to that group, it appears again13:11
fungithe way gerrit applies acls is very, very bizarre13:11
BobBalloh OK13:11
*** smarcet has joined #openstack-infra13:11
BobBalloh, another question, were you aware that some ppl (such as me on one machine) can't access gerrit but others can?13:12
fungiBobBall: have they restarted their browser sessions?13:12
BobBallI get a service temporarily unavailable13:12
*** yfried has quit IRC13:12
BobBallI will testthat13:13
BobBallI hadn't :)13:13
fungii've noticed that some browsers have a tendency to not re-resolve dns unless you take more extreme measures13:13
*** weshay has joined #openstack-infra13:13
fungifirefox does that to me, and it's especially annoying when i migrate a device in and out of networks with split horizon dns13:14
*** che-arne has joined #openstack-infra13:16
*** mriedem has joined #openstack-infra13:19
*** zz_gondoi is now known as gondoi13:19
*** signed8bit has joined #openstack-infra13:20
*** flaper87 is now known as flaper87|afk13:20
*** e0ne has quit IRC13:21
*** e0ne has joined #openstack-infra13:21
*** Ryan_Lane has joined #openstack-infra13:22
*** mwagner_lap has quit IRC13:22
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Add slukjanov to users  https://review.openstack.org/9101213:23
sdaguefungi: any idea if anyone's working on adding the formatted comments back in?13:24
*** jgrimm has joined #openstack-infra13:24
*** bknudson has joined #openstack-infra13:24
sdaguefor gerrit13:24
fungisdague: for the zuul comments?13:24
fungisdague: see SergeyLukjanov's recent scrollback13:24
fungithe fix is already merged, we just need a gerrit restart13:24
SergeyLukjanovsdague, I've fixed mordred's patch and approve it already, it waits for gerrit restart and we'd like to combine it with something more important13:24
sdagueok, cool13:25
*** flaper87|afk is now known as flaper8713:25
sdaguereading tempest test returns without that is ... challenging - https://review.openstack.org/#/c/81872/1013:25
SergeyLukjanovfungi, I've decided to add my public key :)13:25
fungion the assumption we'll find something more important to restart it for anyway. it is only the morning after, after all13:25
sdaguefungi: sure13:25
sdaguefungi: I've got another css change as well - https://review.openstack.org/#/c/90967/13:26
*** Ryan_Lane has quit IRC13:26
fungisdague: saw it. that probably also needs a restart since, as you observed, gerrit compiles all the css13:26
sdaguewell, GerritSite.css isn't actually compiled in13:26
sdagueit's just inlined13:27
SergeyLukjanovfungi, do you have 5 mins to remove a bunch of branches for sahara repos?13:28
fungiSergeyLukjanov: how many is a bunch? but sure, it's quick13:28
*** jcoufal has quit IRC13:28
*** alexpilotti has joined #openstack-infra13:28
SergeyLukjanovfungi, I'd like to remove all stable/0.* from all repos13:29
SergeyLukjanovfungi, all sahara repos13:29
SergeyLukjanovfungi, so, it's about 7-8 branches13:29
*** jcoufal has joined #openstack-infra13:29
SergeyLukjanovfungi, thank you!13:29
SergeyLukjanovfungi, it's about only openstack/*sahara* repos13:31
*** mengxd has joined #openstack-infra13:31
*** mfer has joined #openstack-infra13:33
*** alexpilotti has quit IRC13:34
mengxdDoes anyone know the core member list for the openstack-infrastructure project?13:35
fungiSergeyLukjanov: but you want to leave the stable/icehouse and milestone-proposed branches?13:35
pabelangermengxd, https://review.openstack.org/#/admin/groups/306,members should get you started13:36
SergeyLukjanovfungi, yup, please13:36
*** mnaser has joined #openstack-infra13:36
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Open test results comment in new tab/window  https://review.openstack.org/9101313:37
openstackgerritIlya Sviridov proposed a change to openstack-infra/config: Added gate-magnetodb-devstack-dsvm as gate  https://review.openstack.org/9101413:37
*** matty_dubs|gone is now known as matty_dubs13:37
fungiSergeyLukjanov: so just deleting these... http://paste.openstack.org/show/77706/13:38
SergeyLukjanovfungi, yup13:40
*** markmcclain has joined #openstack-infra13:40
*** mengxd has quit IRC13:41
fungiSergeyLukjanov: done. double-check that i didn't screw anything up13:41
fungimengxd: if you want to bring something up in the openstack infra team meeting later today, you may want to get it on the agenda ahead of time... https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting13:42
anteayafungi: I can read backscroll or ask, so I'll ask first, is there anything useful I can do to help you/someone else?13:42
SergeyLukjanovfungi, everything is correct, thank you!13:42
fungianteaya: so far not much seems to have come up. however i'm 2+ weeks behind on e-mail so digging myself out may unearth some pending tasks13:43
*** mbacchi has joined #openstack-infra13:43
anteayafungi: okay will start to read backscroll, let me know if there is something you unearth13:43
*** markmcclain has quit IRC13:45
*** homeless has joined #openstack-infra13:47
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests  https://review.openstack.org/9052413:53
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Project Groups API  https://review.openstack.org/9073613:53
*** flaper87 is now known as flaper87|afk13:53
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Favicon Whitelabel  https://review.openstack.org/9016713:55
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Removed logo images  https://review.openstack.org/9016913:56
*** flaper87|afk is now known as flaper8713:56
*** prad has joined #openstack-infra13:58
*** mrodden has quit IRC13:59
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Updated alignment on detail page.  https://review.openstack.org/9050613:59
*** pcrews has joined #openstack-infra14:00
*** heyongli has quit IRC14:00
*** dkranz has joined #openstack-infra14:01
openstackgerritA change was merged to openstack-infra/storyboard: Users can be filtered with LIKE clauses  https://review.openstack.org/9021714:02
openstackgerritA change was merged to openstack-infra/storyboard: Projects can be filtered with LIKE clauses  https://review.openstack.org/9047514:03
*** derekh has joined #openstack-infra14:03
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Added test for migrations  https://review.openstack.org/9004714:03
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Users and projects may now be selected using typeahead search  https://review.openstack.org/9050214:09
*** thomasem has joined #openstack-infra14:09
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Project Groups API  https://review.openstack.org/9073614:09
*** yfried has joined #openstack-infra14:11
*** mrodden has joined #openstack-infra14:12
*** pblaho has joined #openstack-infra14:13
*** msabramo has quit IRC14:16
*** ashishsnigam has joined #openstack-infra14:20
mordredfungi: think we should restart gerrit to pick up SergeyLukjanov's fix?14:22
*** david-lyle_ has joined #openstack-infra14:22
*** zhiyan_ is now known as zhiyan14:23
SergeyLukjanovmordred, we think that we should wait for some more important stuff14:24
fungimordred: yes, though just wanted to make sure if other issues turn up which also need a restart, that we lump them in rather than having more than one restart today (if possible)14:24
*** mdenny has joined #openstack-infra14:24
jeblairgood morning all14:25
fungimordred: sdague also had a css patch which i believe requires a restart as well14:25
SergeyLukjanovjeblair, morning14:25
jeblairthat's sdague's css fix which i just +2d (thanks)14:25
*** markmcclain has joined #openstack-infra14:25
*** david-lyle_ is now known as david-lyle14:25
jeblair(i hesitated a moment about the nonvotable background thing -- but i think with our mix of code review and workflow acls, it's actually not useful information for us)14:26
*** blamar has joined #openstack-infra14:26
fungiagreed (and approved)14:26
SergeyLukjanovand I approved https://review.openstack.org/#/c/90862/ to remove "Code Review" from header14:27
jeblair(it would pretty much tell you that most people can't vote verified, and some other people are _either_ the owner or core)14:27
SergeyLukjanovjeblair, yup, it's not very useful IMO14:27
*** changbl has quit IRC14:27
jeblairzaro: is there a way to remove the "Monitoring" menu entry?  i don't think we'll ever use it (we can always directly navigate to it), and it takes up a lot of space14:28
*** dizquierdo has joined #openstack-infra14:28
*** thedodd has joined #openstack-infra14:29
jeblair(it's... unfortunate... that javamelody monitoring gets a top level menu item and project dashboards can't get one at all anywhere)14:29
* SergeyLukjanov don't see monitoring menu item :(14:30
jeblairSergeyLukjanov: we have it configured for admin-only.  we should probably consider making you a gerrit admin.  :)14:30
SergeyLukjanovjeblair, oh, got it :)14:30
openstackgerritA change was merged to openstack-infra/config: gerrit css tweaks to make it more like old gerrit  https://review.openstack.org/9096714:31
gabriel-bezerraHi, I've got 2 +1s on this review, but Jenkins didn't give the check or verify. How can I fix this? https://review.openstack.org/#/c/90771/14:31
*** mwagner_lap has joined #openstack-infra14:31
jeblairgabriel-bezerra: "recheck no bug"14:31
rcarrillocruzrecheck no bug i guess14:31
jeblairgabriel-bezerra: we restarted zuul a lot yesterday for the gerrit upgrade, the jobs were probably lost during that14:31
*** ArxCruz has joined #openstack-infra14:32
gabriel-bezerraoh, good.. I was looking for some dumb bug created just to start new rechecks..14:32
SergeyLukjanovSergeyLukjanov, gerrit admin could be probably useful - I can help with stuff like removing branches, adding to groups, etc. // if it's in admin permissions :)14:33
SergeyLukjanovheh, it's not for myself, jeblair  :_14:33
gabriel-bezerrathank you very much14:33
openstackgerritIlya Sviridov proposed a change to openstack-infra/config: Added new python-magnetodbclient project to Stackforge  https://review.openstack.org/9105014:33
*** msabramo has joined #openstack-infra14:33
*** wenlock has joined #openstack-infra14:34
mordredjeblair: yeah - I really wish we could re-work the top menu14:34
sdagueok, it's taking me a while to sort out my custom queries again in the new language. I'll do an updated blog post later with new recipes14:35
mordredsdague: if you get some things sorted out - we could potentially add them as custom dashboards14:36
anteayaright now our known problem is formatting of test reports, what is the issue with test report formatting?14:36
sdaguemordred: sure14:36
sdagueis there a way for me to register such things?14:36
mordredanteaya: SergeyLukjanov landed a patch to fix it14:36
mordredsdague: not yet - at the moment it takesa  core14:36
anteayawell done SergeyLukjanov14:36
sdagueI've basically got a number of queries that I use to take slices of things14:36
mordredsdague: but I'd very much like to make them files in git14:37
sdaguebecause my total reviewable list is too long to manage14:37
*** e0ne has quit IRC14:37
sdaguemordred: ok, just let me know how to contribute them14:37
*** bnemec has quit IRC14:39
anteayaSergeyLukjanov: is this the patch in question? https://review.openstack.org/#/c/91013/114:39
mordredsdague: it's really easy to make them - all you need is the gerrit queries- this is the important changes dashboard: http://paste.openstack.org/show/77720/14:39
mordredsdague: which is here: https://review.openstack.org/#/projects/All-Projects,dashboards/important-changes:important-changes-dashboard if you haven't foudn it14:39
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Add Gertty project  https://review.openstack.org/8945714:41
jeblairmy new projects have sat so long their configuration is out of date :(14:41
*** andreaf has quit IRC14:41
jeblairSergeyLukjanov: you may want to look at that change and check out the ACL changes between the last two patchsets14:42
*** pblaho has quit IRC14:42
jeblairSergeyLukjanov: that's what those items should look like for all projects now14:42
openstackgerritIlya Sviridov proposed a change to openstack-infra/config: Added gate-magnetodb-devstack-dsvm as gate  https://review.openstack.org/9101414:42
sdaguemordred: is there a way to apply that to something other than All-Projects?14:42
*** bnemec has joined #openstack-infra14:42
jeblairsdague: works for any project14:42
sdaguewhat's the url syntax to apply to a specific project14:42
sdagueor a list14:42
*** markmcclain has quit IRC14:43
jeblairsdague: afaik what project it's attached to just determines when/where it shows up in the ui14:43
SergeyLukjanovanteaya, https://review.openstack.org/#/c/90872/ fix for formatting14:43
jeblairsdague: i think if you wanted to filter by project you'd just put it in the query14:43
SergeyLukjanovanteaya, and https://review.openstack.org/#/c/91013/1 is just a possible improvement14:43
sdaguefor instance - https://review.openstack.org/#/projects/openstack-dev/grenade,dashboards/important-changes:important-changes-dashboard14:43
sdaguedoesn't seem to do what I'd think14:43
SergeyLukjanovjeblair, /me looking14:44
anteayaSergeyLukjanov: ah, was looking for it under your name, thanks for the url14:44
SergeyLukjanovjeblair, I'll check all other open new-project requests14:44
jeblairsdague: yeah, that's just grenade inheriting the ability to display the important changes dashboard because it is a child of all-projects14:44
*** markmcclain has joined #openstack-infra14:44
sdaguejeblair: right, but doesn't filter based on the project14:44
jeblairsdague: if grenade had a grenade-specific dashboard, it would be available at https://review.openstack.org/#/projects/openstack-dev/grenade,dashboards/important-changes:foo14:45
jeblairsdague: but would not be available at https://review.openstack.org/#/projects/openstack-dev/somethingnotgrenade,dashboards/important-changes:foo14:45
jeblairsdague: yeah, and afaik (but i haven't really looked) there's no way to use $current_project_url_context as a search variable14:45
*** viktors is now known as viktors|afk14:45
anteayawhat are the chances I can specify in my preferences that comments are collapse all for me?14:45
anteayaas a personal default14:46
sdagueok, let me etherpad up a couple of things that I think would be useful, and we'll see if they are doable14:46
*** zns has joined #openstack-infra14:46
anteayait is already there, I found it14:46
*** ihrachyshka has quit IRC14:46
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Create infra-manual project  https://review.openstack.org/8809914:47
jeblairsdague: https://review.openstack.org/Documentation/user-dashboards.html#_token_code_project_code14:48
jeblairsdague: i was completely wrong you can totally use the current project context in a dashboard query.  :)14:48
sdaguethat will make this much better :)14:49
fungiwhile i was out, mriedem requested membership in openstack-ci-bugs on lp (presumably to assist with triage). any objections? should i ask again at the meeting, or just approve?14:49
anteayaSergeyLukjanov: so does 91013 open in a pop-up window?14:49
jeblairfungi: just approve; i don't really know why that's not open14:49
fungijeblair: approved14:50
openstackgerritAndreas Jaeger proposed a change to openstack/requirements: Update openstack-doc-tools to 0.13  https://review.openstack.org/9074614:50
mriedemfungi: jeblair: muwahahaha14:52
phschwartzMorning infra14:52
SergeyLukjanovanteaya, yup14:52
anteayaSergeyLukjanov: but only if you click on it, right?14:52
mordredjeblair: oh!14:52
mordredIt is possible to define a default dashboard for a project in the projects project.config file in the refs/meta/config branch:14:53
openstackgerritA change was merged to openstack-infra/config: Fix minor non-important error in cache_devstack  https://review.openstack.org/9077514:53
anteayaSergeyLukjanov: is there an option to have it not open in a new window?14:53
jeblairmordred: what does 'default' mean?14:53
*** bhuvan has joined #openstack-infra14:53
jeblairphschwartz: good morning14:53
mordredjeblair: I'm not 100% sure - but it certainly seems like something that might be useful given how undiscoverable these things are14:54
SergeyLukjanovanteaya, not add _blank target :)14:54
fungii personally find that behavior very annoying14:55
*** ildikov_ has joined #openstack-infra14:55
anteayaSergeyLukjanov: sorry I don't understand14:56
SergeyLukjanovfungi, opening in new tab?14:56
fungibrowsers implement ways for users to control whether links open in the same place or a new window/tab/whatever. having the application decide that for you is silly14:56
fungiand disruptive14:56
SergeyLukjanovanteaya, I mean that if we're adding target=_blank than link will be opened in new tab on click14:56
SergeyLukjanovfungi, it sounds fair14:56
*** ildikov has quit IRC14:58
anteayaokay, yeah that was what I was trying to get at, and fungi summarized my sense of it nicely14:58
SergeyLukjanovfungi, anteaya, heh, that's funny, but I agree with it and don't know why I've proposed this change :)14:59
anteayamy browser needs to have the last word on how something is opened for me14:59
* SergeyLukjanov going to abandon14:59
anteayaSergeyLukjanov: I've had those moments14:59
jeblairSergeyLukjanov: what change?14:59
*** ildikov_ has quit IRC14:59
jeblairanteaya: see https://review.openstack.org/#/settings/preferences  for comment visibility15:00
*** derekh has quit IRC15:00
anteayajeblair: https://review.openstack.org/#/c/91013/1/modules/openstack_project/manifests/gerrit.pp15:00
anteayajeblair: yes, found it after I asked about it, thanks15:01
*** Longgeek_ has joined #openstack-infra15:01
anteayaone of these days I will look _before_ I ask15:01
mordredwhere's teh fun in that15:01
*** zns has quit IRC15:02
mordredjeblair: so - puppet seems to thnk it needs to add our ssh keys every time15:02
mordredjeblair: http://puppetdb.openstack.org/report/jenkins07.openstack.org/5cf45a24153e74f58f94c2fbf7ae389747e6cb5615:03
anteayaenjoying a nice dinner with jeblair and a few others, which you pay for (thanks monty), that's the fun15:03
*** Longgeek has quit IRC15:03
mordredjeblair: I'll look in to it a little more, but if I can't figure it out, I'll be proposing we revert the use of ssh_authorized_key there15:03
*** rcarrillocruz has quit IRC15:03
SergeyLukjanovmordred, yeah, I've already seen it when approved this change, but I was thinking that it's ok15:03
dkranzfungi: My review bookmarks broke it seems due to (at least) from the query "CodeReview" changing to "Code-Review". Is there some advice on fixing these?15:04
SergeyLukjanovmordred, jeblair, btw it's about https://review.openstack.org/#/c/87340/15:04
*** e0ne has joined #openstack-infra15:05
*** zns has joined #openstack-infra15:05
*** zns has quit IRC15:06
msabramoAny pbr folks? Could use feedback on https://review.openstack.org/8920515:06
sdaguedkranz: I'm working on that one15:06
*** zns has joined #openstack-infra15:06
dkranzsdague: Thanks! I created them from your so useful email about it.15:06
sdaguedkranz: https://review.openstack.org/Documentation/user-search.html is the new docs if you don't want to wait15:07
dkranzsdague: cook thanks15:07
sdaguethough, based on the previous conversation, we might be able to turn some of these into custom dashboards15:07
sdagueso they would be baked into gerrit15:07
sdaguewhich would be super nice15:07
*** bauzas has quit IRC15:07
*** bauzas has joined #openstack-infra15:07
mordredoh. zomg15:08
mordredpuppet is useless15:08
mordredor at least puppetboard is15:08
mordred(actually, I think it's puppet's fault)15:08
*** rcarrillocruz has joined #openstack-infra15:08
mordrederr: /Stage[main]/Openstack_project::Users/User::Virtual::Localuser[clarkb]/Ssh_authorized_key[clarkb_keys]: Could not evaluate: Puppet::Util::FileType::FileTypeFlat could not write /home/clarkb/.ssh/authorized_keys: Permission denied - /home/clarkb/.ssh/authorized_keys15:08
mordredthat is the error it's not showing us15:08
mordredbecause, you know, why bother showing an error in the dashboard15:09
anteayawhich node?15:09
mordredall of them15:09
* mordred hitting with angry gorilla power15:09
anteayathat must be a site15:10
*** UtahDave has joined #openstack-infra15:10
anteayasight even15:10
SergeyLukjanovfungi, thanks for the comment on https://review.openstack.org/#/c/91012/ - I understand that it doesn't add my acc to any servers, just would like to push my public key in15:12
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Fix broken ant example  https://review.openstack.org/9106115:12
SergeyLukjanovfungi, I've commented re agreement with guideline15:12
SergeyLukjanovmordred, oh... I've checked puppetboard after this changes was applied and thought that everything is ok...15:14
*** ashishsnigam has quit IRC15:14
openstackgerritA change was merged to openstack-infra/config: Add Gertty project  https://review.openstack.org/8945715:15
sdagueinteresting bit - https://review.openstack.org/#/c/44714/15:15
sdaguemerge conflicts no longer trigger a V-215:16
*** ihrachyshka has joined #openstack-infra15:16
*** gokrokve_ has joined #openstack-infra15:18
openstackgerritGraham Hayes proposed a change to openstack-infra/config: Add stackforge/designate-specs  https://review.openstack.org/8754015:18
anteayasdague: :/15:20
anteayait triggers a V+215:21
*** gokrokve has quit IRC15:22
BobBallWhat was gerrit's old IP address?15:23
*** Ryan_Lane has joined #openstack-infra15:23
anteayaBobBall: second line on https://etherpad.openstack.org/p/J8WNqyP2dC15:24
BobBallthanks anteaya15:24
BobBallthat was the magic etherpad I was looking for!15:24
openstackgerritA change was merged to openstack-infra/config: Create infra-manual project  https://review.openstack.org/8809915:24
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Fix ssh key error and also remove duplicates  https://review.openstack.org/9106615:25
mordredjeblair, SergeyLukjanov: ^^ that patch itself isn't tested - but the concepts in it have checked out - I'm going to do a quick test run of it on a host in a mordred env15:25
*** ildikov has joined #openstack-infra15:27
*** Ryan_Lane has quit IRC15:27
*** ihrachyshka has quit IRC15:28
jeblairoh new projects merged!15:28
kialljeblair: got a minute to rekick that failed tag job I mentioned last night for me?15:30
jeblairmordred: i don't understand "${title}_keyfile"15:30
*** ihrachyshka has joined #openstack-infra15:30
*** zhiyan is now known as zhiyan_15:31
jeblairkiall: i'm not certain things are working yet -- i pushed a tag yesterday and it didn't show up; we should wait until i get to the bottom of that15:32
*** zehicle has quit IRC15:32
kialljeblair: No problem, I'll ping you again at some point tomorrow then?15:32
jeblairkiall: sounds good15:32
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Fix ssh key error and also remove duplicates  https://review.openstack.org/9106615:33
mordredjeblair: ok- ^^ that one works15:33
mordredjeblair: so - ${title}_keyfile is there to ensure that the file is more 600 and not mode 400 which is what the files our there are15:34
jeblairmordred: okay now "${title}_keyfile" makes sense.  :)15:34
jeblairmordred: the lack of name was throwing me off in the old patchset15:34
mordredit was  -- not great :)15:34
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Fix broken builders-from example  https://review.openstack.org/9107915:34
jeblairi can imagine for once puppet and i were confused by the same thing15:35
mordredthis should get us back down to one key each and also give us a way to delete old keys when we cycle them15:35
mordredjeblair: given an inifinte number of  ...15:35
*** dizquierdo has quit IRC15:35
jeblairmordred: (we could delete old keys before, we just removed them from the big string)15:35
mordredI mean, this gets us feature parity to the old code15:36
mordredwith extra compexity15:36
jeblairmordred: how do we have more than one key each?15:36
*** talluri has quit IRC15:36
mordredwe revert the patch that introduced this and go back to the old code15:36
anteayamordred: did you just make the key id's up?15:37
mordredanteaya: I did not - those were the existing key ids that were originally there15:37
*** talluri has joined #openstack-infra15:37
jeblairmordred: so rolling keys is a little awkward, but i guess we can live with it15:37
anteayaoh okay, in which case I revoke my -115:37
anteayawonder why russellb would give himself only one L in his name15:39
mordredjeblair: honestly - I find everything about puppet's primitives to manager users and keys to be so ridiculous I think I've given up having an opinion15:39
*** msabramo has quit IRC15:39
mordredthat our virtualuser class exists at all should be seen as a giant glaring deficiency15:40
jeblairhere's the gerritbot tag error15:40
jeblairmordred: yeah15:40
mordredjeblair: so - the tag didn't replicate?15:40
jeblairyeah, checking zm02's logs now15:40
*** talluri has quit IRC15:41
*** zhiyan_ is now known as zhiyan15:42
*** dangers_away is now known as dangers15:45
*** afazekas_ has quit IRC15:47
jeblairmordred: hrm, i'm not getting it.  zm02 updated the repo before that job ran, it should have updated from gerrit directly over ssh, and that git fetch command works for me now.15:48
*** ominakov has quit IRC15:48
jeblairmordred: the local git repo on zm02 has the tag, and zuul hasn't touched it since.15:48
*** Longgeek_ has quit IRC15:49
*** Longgeek has joined #openstack-infra15:50
*** sweston has joined #openstack-infra15:50
*** ihrachyshka has quit IRC15:51
*** msabramo has joined #openstack-infra15:52
*** jcoufal has quit IRC15:52
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Add a release job for sandbox  https://review.openstack.org/9108415:52
mordredjeblair: that makes me less happy15:52
jeblairmordred, fungi, SergeyLukjanov: can we go ahead and merge that so i can test pushing lots of tags? ^15:52
mordredjeblair: yah15:53
fungijeblair: lgtm, though you might want it in pre-release too for later testing of regexes?15:54
jeblairsure 1 sec15:54
*** matty_dubs is now known as matty_dubs|lunch15:54
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Add a release job for sandbox  https://review.openstack.org/9108415:54
*** julim has quit IRC15:55
*** andreaf has joined #openstack-infra15:55
*** WaltNik has quit IRC15:58
*** bauzas has left #openstack-infra15:59
*** msabramo has quit IRC16:02
Alex_GaynorOnly the devstack machines are managed by nodepool right?16:02
*** gokrokve_ has quit IRC16:02
fungiAlex_Gaynor: most of our jenkins slaves16:03
openstackgerritA change was merged to openstack-infra/gear: Add access control  https://review.openstack.org/8487316:03
Alex_Gaynorfungi: does that include the centos ones?16:03
fungiAlex_Gaynor: yes. the bare-centos6 slaves are nodepool-managed16:03
fungiAlex_Gaynor: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/nodepool/nodepool.yaml.erb#n2116:04
Alex_Gaynorfungi: hmm, there seems to be way fewer centos nodes right now then are needed (like every build is blocked on the python26 builder)16:04
fungiAlex_Gaynor: thanks, i'll see if we're having trouble adding new centos6 nodes for some reason16:04
*** Longgeek has quit IRC16:04
*** ihrachyshka has joined #openstack-infra16:04
*** markmcclain has quit IRC16:05
jeblairfungi: mordred made a quick script in /var/log/nodepool to count success/failure16:06
SlickNikhey guys.16:07
fungiAlex_Gaynor: looks like we're having centos6 trouble in hpcloud-region-b (about 70 building and 70 deleting, which accounts for the majority of the centos6 nodes at the moment). i'll dig deeper16:07
*** msabramo has joined #openstack-infra16:07
*** yassine has quit IRC16:07
anteayaSlickNik: hey there16:07
Alex_Gaynorfungi: ah, glad I said something!16:08
*** derekh has joined #openstack-infra16:08
fungiAlex_Gaynor: me too. you're our early-warning system ;)16:08
SlickNikThe trove "reddwarf" user hasn't been able to post on trove gerrit reviews since the upgrade. It's using the gerrit review plugin. Are there any known issues with this?16:08
*** alexpilotti has joined #openstack-infra16:09
*** e0ne has quit IRC16:09
fungiSlickNik: check your local logs, but you're probably getting stuck on http://lists.openstack.org/pipermail/openstack-dev/2014-April/033908.html16:09
anteayaSlickNik: have you updated the host keys for the reddwarf user: https://wiki.openstack.org/wiki/GerritUpgrade#DNS_Spoofing_Warning16:09
anteayalike fungi said16:10
SlickNikanteaya / fungi: Nope I haven't done that yet, so that's probably what it is.16:10
SlickNikWill try that and see if it fixes it. Thanks! :)16:10
*** doug-fish has quit IRC16:11
*** zhiyan is now known as zhiyan_16:11
mordredfungi: you may also want to check out the script just so that you know it's there16:11
fungijeblair: mordred: great--thanks!16:12
mordredthere's processing yesterday16:12
mordredbtw- lookie there- hpcloud 1.1 is doing well!16:12
mordredand WOW - rax is doing very very very badly16:12
mordredis that related to yesterday's resolv.conf issue?16:13
*** jgallard has quit IRC16:13
fungimordred: yes, almost certainly16:13
mordredyup. rax numbers in the current log are back to normal16:14
mordredalthough dfw is very unhappy16:14
*** jlibosva has quit IRC16:14
*** doug-fish has joined #openstack-infra16:14
mordredthat's 1100 succeded, 681 failed since the log rolled over at 00:00 UTC16:14
phschwartzmordred: Is that 681 builds failed?16:14
mordredthat's 681 node creations failed16:15
mordredphschwartz: so, 681 we tried to spin up a node and something in the process failed before we put it into the pool as an available slave16:15
phschwartzmordred: Let me see if I can get one of our cloud ops to take a look. That is a disturbing number16:15
fungii'm going to see if i can coax the script into breakdown by provider+label since i have a feeling most/all bare-centos6 nodes in hpcloud-region-b are failing to build16:15
phschwartzmordred: Are there stored logs as to what went wrong? Ie was it an node creation/build issue or configuration inside of the node16:16
*** signed8bit has quit IRC16:16
mordredphschwartz: we test that tons of things work before we consider it "good" - dns has been a recent beast we've been battling16:16
mordredphschwartz: could be either in that count16:16
fungimordred: one possibility is that one or more of the images failed to update in rax-dfw and still has the chattr +i /etc/resolv.conf going on16:16
mordredphschwartz: so from rax perspective, those could look like "success"16:16
mordredfungi: ++16:16
fungicheck and see if all labels in rax-dfw have corresponding images <24 hours old16:17
mordredphschwartz: we _do_ have logs, I don't think we've audited them fully to make sure they're safe to publish to a location you could get them from16:17
phschwartzmordred: I will err on the worst case and see if I can have a cloud ops person take a look.16:17
mordredfungi: oldest in rax-dfw is 19 hours16:18
mordredfungi: hpcloud-region-b nodes are mostly over 200 hours old16:18
mordredfungi: actually ALL of hp images are - 1.0 regions are in the 400 hours range16:19
fungioh, right, i deleted them all out of rax yesterday since they were broken, so these failures in rax-dfw very well may not be our doing16:19
*** alexpilotti has quit IRC16:19
*** chandan_kumar has quit IRC16:19
fungihplcoud 1.0 images mostly don't succeed because of the nova metadata server timeouts, last i saw16:20
fungican't remember if region b has the same issue16:20
*** markmcclain has joined #openstack-infra16:20
phschwartzmordred, fungi: I have asked our admins to look at our logs and see if they see anything funky in our regions that might cause an issue. As to the image failures, let me know if it looks like there is an actual snapshotting issue with glance for us and I will get some people involved to see what the issue might be.16:23
mordredphschwartz: cool, thanks!16:23
mordredphschwartz: we'd trying to figure out good ways we can provide data to you guys on an ongoing basis16:23
mordredI mean, given that we use a ton of resources quite actively - any way in which we can also be a data collection vehcile is awesome16:24
*** yidclare has joined #openstack-infra16:24
*** dangers is now known as dangers_away16:24
phschwartzmordred: That would be a huge help. Let me know anything you need to help facilitate that, even if I have to work in my spare time to write a log submission api + datastore for you guys to hit.16:24
*** BadCub has joined #openstack-infra16:24
*** sweston has quit IRC16:24
jeblairssh root@elasticsearch06.openstack.org timeout -s 9 30m puppet agent --onetime --ignorecache --no-daemonize --no-usecacheonfailure --no-splay16:25
jeblairPermission denied (publickey).16:25
jeblairmordred: ^16:25
*** talluri has joined #openstack-infra16:25
mordredjeblair: the key on es06 looks right16:26
jeblairssh root@jenkins-dev.openstack.org timeout -s 9 30m puppet agent --onetime --ignorecache --no-daemonize --no-usecacheonfailure --no-splay16:27
jeblairPermission denied (publickey).16:27
mordredjeblair: where are you seeing that?16:27
mordredjeblair: I believe jenkins-dev puppet is still broken and we expect jenkins-dev to not work16:28
*** salv-orlando_ has joined #openstack-infra16:29
jeblairmordred: i'm looking into why it's failing on git01 and saw those other errors16:29
*** mriedem has quit IRC16:29
mordredjeblair: but I've just tested both that command line _and_ just ssh elasticsearch06.openstack.org from ci-puppetmaster and both worked16:29
derekhHi, is anybody willing to pull the trigger on devstack-f20 nodepool instances ? https://review.openstack.org/#/c/86842/  I'll be gone shortly but can be online later to help with any problems building the templates16:29
*** salv-orlando has left #openstack-infra16:30
*** salv-orlando has quit IRC16:30
*** salv-orlando_ is now known as salv-orlando16:30
openstackgerritA change was merged to openstack-infra/config: Add a release job for sandbox  https://review.openstack.org/9108416:30
jeblairerr: Failed to apply catalog: Parameter key failed on Ssh_authorized_key[gerrit-replication-2014-04-25]: Key must not contain whitespace: AAA....16:30
jeblairmordred: that's why git01 is failing16:30
fungimordred: the centos6 build failures in hpcloud-region-b look like ssh timeouts after the build completes16:30
*** mriedem has joined #openstack-infra16:30
jeblairfungi, mordred: do we need to bump up our ssh timeouts again?16:30
*** marcoemorais has joined #openstack-infra16:31
*** GheRivero has quit IRC16:31
fungiwell, we're also failing to build centos6 images in rackspace at all, so i'm going to start diagnosing that as well16:31
mordredjeblair: I'm very confused - there is no space in the key ...16:31
jeblairmordred: i agree16:32
tjonesfolks - on this patch turbo hipster gave a -1 and then a +1 but the stats still show -1 https://review.openstack.org/#/c/86443/  can someone take a look?16:33
dimstjones, i am hoping a "recheck migrations" will help. but yes, need to figure out why the -1 is sticky16:34
fungitjones: that's very, very odd...16:34
tjonesthe 2nd migration was ok so i am surprised it did not update the -116:34
mordredjeblair: also, the key is on git01.o.o16:34
*** ildikov has quit IRC16:35
mordredjeblair: I wonder - do we need to put quotes around $git_gerrit_ssh_key ?16:35
*** Ajaeger has joined #openstack-infra16:36
*** matty_dubs|lunch is now known as matty_dubs16:36
AjaegerHi jeblair, I see on https://review.openstack.org/#/c/82176 "Patch Set 1: (1 inline comment)" - but don't see a comment from you. Any idea what happened?16:37
jeblairAjaeger: i don't recall leaving an inline comment :/16:37
fungii think the centos6 image build failures look like they're because curl isn't following a redirect from https://raw.github.com/pypa/pip/master/contrib/get-pip.py to https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py16:37
*** pcrews_ has joined #openstack-infra16:38
jeblairfungi: i thought we updated that?16:38
*** pcrews has quit IRC16:38
mordredfungi: yes. there is a patch at least in flight to fix that16:38
Ajaegerjeblair: I don't see you voting either ;) This is strange16:39
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Remove accumulated python cruft  https://review.openstack.org/9057316:39
fungimordred: fixing http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/nodepool/scripts/fix_pip.sh#n26 or elsewhere?16:39
mordredfungi: ^^ that's not the one that came in yesterday, but also contains the fix16:39
openstackgerritZane Bitter proposed a change to openstack-infra/reviewstats: Add Thomas Spatzier to heat-core  https://review.openstack.org/9109416:39
fungioh, great16:40
mordredjeblair: have you been able to use REST api against our gerrit? I tried last night (granted I was tired) and got bumpkiss16:40
jeblairmordred: yep.  i have a gertty running right now.16:40
mordredjeblair: awesome. so, am I just dumb for thinking that "curl https://review.openstack.org/groups" should do something?16:41
jeblairmordred: you want https://review.openstack.org/groups/16:42
jeblairmind that trailing /16:42
mordredok. so that needs auth - but that's fine - I'll just look at gertty to see how you do that16:42
derekhbtw, this hasn't merged yet so may be still causing problems https://review.openstack.org/#/c/90757/16:42
fungimordred: and you probably need to set yourself up with an http password in your preferences16:43
bnemecAny idea why Jenkins doesn't want to recheck https://review.openstack.org/#/c/85224/ ?16:43
derekhfungi: mordred ^ is that what ye are talking about in the get-pip.py url16:43
fungiderekh: oh, that's the other one i thought i saw for that. wonder why it hasn't merged...16:43
*** markmcclain has quit IRC16:44
derekhfungi: beats me16:44
fungiderekh: i think its initial check job got lost in yesterday's restarts, so the approval with no existing verify vote didn't trigger anything16:46
fungiderekh: i just did a recheck no bug on it and it's got jobs runnin gnow16:46
fungifingers crossed16:46
mordredderekh: yes16:46
derekhfungi: ok, thanks16:46
fungimordred: your 90573 will be much smaller rebased on that16:47
*** dangers_away is now known as dangers16:47
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Stories can be filtered with LIKE clauses  https://review.openstack.org/9047816:47
*** vhoward has left #openstack-infra16:48
*** GheRivero has joined #openstack-infra16:48
*** sweston has joined #openstack-infra16:49
anteayajeblair mordred so is gertty somewhere that others can consume yet?16:50
anteayabecause I am very curious16:51
jeblairanteaya: i'm trying to figure out why puppet isn't running on the git servers, which is required in order to create the repo for it16:51
jeblair(and any other project)16:51
anteayajeblair: ah16:51
zarojeblair, mordred : i don't know if you can remove the monitoring menu.  i believe it's configured from the javamelody plugin.16:51
*** atiwari has joined #openstack-infra16:52
zarojeblair, mordred : i believe there is a plugin that will allow you to customize the top menues. real simple plugin16:52
*** weshay has quit IRC16:52
mordredjeblair: mind if I poke on git03 while you're poking on git01?16:53
* mordred doens't want to step on jeblair work16:54
zaromordred, jeblair : https://github.com/dluksza/gerrit-top-menu-extension-example16:54
jeblairmordred: go for it16:55
*** SumitNaiksatam has quit IRC16:55
*** mrmartin has quit IRC16:55
*** krtaylor has quit IRC16:56
*** julim has joined #openstack-infra16:57
zarojeblair, mordred : would it make sense to clean the db for review-dev?  I've noticed that it somehow got into a state where the schema on review-dev db is slightly different than what's on review db16:57
*** julim has quit IRC16:58
jeblairmordred: the value passed to git_backend has a newline16:58
mordredputting strip() around it makes it work - so yeah, I agree16:58
mordrednow where is the newline coming from?16:58
jeblairmordred: i think because of the "|" thing in yaml16:59
*** harlowja_away is now known as harlowja16:59
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror  https://review.openstack.org/9087516:59
openstackgerritDan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors  https://review.openstack.org/9087416:59
*** etoews has quit IRC16:59
mordredjeblair: I was justa bout to say the same thing16:59
anteayazaro: is the data in review-dev db a copy of other data?16:59
mordredjeblair: you wanna take it out?16:59
jeblairmordred: done17:00
*** julim has joined #openstack-infra17:00
jeblairmordred: git01 --noop seems to be looking good17:01
mordredsame on git0317:01
zaroanteaya: no, data is completely difference.  schema is not the same either.  we should make the schema the same so it's more like prod gerrit.17:01
*** derekh has quit IRC17:02
*** hashar has quit IRC17:02
*** zns has quit IRC17:02
jeblairi think we just missed the 1700 run; when it finishes, i'll kick run_all again17:02
*** amcrn has joined #openstack-infra17:02
mriedemdo the images that are used to run devstack/tempest in community have net.core.somaxconn set to something besides the default 128?17:02
anteayazaro: yes, since the purpose of review-dev is staging for review, the db schema on review-dev should mirror the db schema on review17:03
anteayaany other opinons?17:04
mordredjeblair: awesome17:04
*** andreykurilin_ has joined #openstack-infra17:04
*** flaper87 is now known as flaper87|afk17:05
*** Ryan_Lane has joined #openstack-infra17:06
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use storyboard for gertty  https://review.openstack.org/9110217:07
openstackgerritA change was merged to openstack-infra/config: Replace raw.github.com with raw.githubusercontent.com  https://review.openstack.org/9075717:08
jesusaurusyfried: pong17:08
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Fix ssh key error and also remove duplicates  https://review.openstack.org/9106617:08
mordredpuppet-lint ^^17:08
*** flaper87|afk is now known as flaper8717:08
*** zehicle_at_dell has joined #openstack-infra17:08
*** SumitNaiksatam has joined #openstack-infra17:09
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Remove accumulated python cruft  https://review.openstack.org/9057317:10
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Fix install_puppet.sh for trusty  https://review.openstack.org/9057417:10
mordredfungi: both of those ^^ are tiny little changes - now rebased on top of the github url content change17:11
*** mkoderer has quit IRC17:12
lcostantinoqq, xenserver is still failing with security-groups issues right? or it's sporadic?17:12
*** signed8bit has joined #openstack-infra17:12
jeblairmordred: https://review.openstack.org/#/c/88554/17:14
jeblairthat is really impressive...17:14
jeblairi think gertty has managed to upload a review that says it has inline comments (without any), and a review that somehow was merged with zuul's review17:15
tjonesanother gerrit oddity today.  When i search for status:open message:"refactor - phase 1" i get nothing, but when i search for status:open message:"phase 1" i get a bunch of stuff (including what I am looking for).  This search used to give me exactly what i wanted.17:15
*** rcarrillocruz has quit IRC17:15
mordredjeblair: I am very impressed by your technical skills and would like to subscribe to your newsletter17:16
anteayatjones: does this doc about searching help explain your results? https://review.openstack.org/Documentation/user-search.html17:16
*** jpich has quit IRC17:16
*** praneshp has joined #openstack-infra17:17
*** morganfainberg_Z is now known as morganfainberg17:18
anteayatjones: looks like this library is used for regex matches: http://www.brics.dk/automaton/17:18
anteayaI wonder if that lib is the same version as we were using before17:18
tjonesanteaya: not really.  I would still expect message="refactor - phase 1" to narrow down from message="phase 1"17:18
*** jooools has quit IRC17:19
*** sacharya has joined #openstack-infra17:19
mordredtjones: I would too - but I often expect sanity17:19
anteayathere is something different with how double quotes are used in searches17:19
anteayawhat do you get if you try message='refactor - phase 1'17:20
openstackgerritA change was merged to openstack-infra/config: Use storyboard for gertty  https://review.openstack.org/9110217:20
*** chandan_kumar has joined #openstack-infra17:20
tjonesha just tried that.  got an error - status:open message:"refactor - phase 1"17:20
tjonesthe error was no viable alternative at character '''17:21
*** nati_ueno has joined #openstack-infra17:21
*** rcarrillocruz has joined #openstack-infra17:21
jeblairinfra-manual and gertty projects exist now17:21
mordredjeblair: w00t!17:21
* mordred goes to add them to his watch list17:21
*** arnaud has joined #openstack-infra17:22
tjonesi wonder if the commit title is not part of the "commit body" any more17:22
jeblairi might not upload gertty just yet, until i can make sure it's not uploading very crappy data to gerrit17:22
* anteaya nods17:22
anteayaand yay for new repos17:22
*** gyee has joined #openstack-infra17:23
anteayatjones: can you post the url you are hoping your search locates?17:23
* jeblair switches back to the tag problem17:23
mordredjeblair: uhm17:23
mordredjeblair: I do not see openstack-infra/infra-manual in gerrit17:23
*** changbl has joined #openstack-infra17:23
jeblairoh, they exist on git0*17:24
tjonesanteaya: here they are https://review.openstack.org/#/q/status:open+message:%22spawn+refactor+-+phase+1%22,n,z  https://review.openstack.org/#/q/status:open+message:%22phase+1%22,n,z17:24
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests  https://review.openstack.org/9052417:24
mordredoh - well, that's good at least17:24
mordredjeblair: you switch back to tag problem - I'll look at the projects on review17:24
jeblairoh no they don't17:24
jeblairpuppet on git03.openstack.org failed with return code: 117:24
jeblairmordred: ^17:24
mordredjeblair: how about I look at the whole thing17:24
jeblairmordred: git01 and 02 worked but not 0317:24
*** sabari has quit IRC17:25
jeblairmordred: ++ lemme know if you need anything, i'll work on tags17:25
mordredjeblair: doh17:25
mordredI had left puppet disabled on git03 - my bad17:25
anteayatjones: okay I get it you are searching the commit message17:25
mordredgood to know the system is actually working as designed!17:25
*** zns has joined #openstack-infra17:25
tjonesanteaya: yes - i am really looking for things with the top line of the commit message (akak title) that match "spawn refactor - phase 1"17:26
* anteaya nods17:26
anteayacatching up to you17:26
asselin__Hi, I'm trying to do a check bug here for multiple bugs, but it didn't kick off a new check. Is this related to the new gerrit or is my comment not correct? https://review.openstack.org/#/c/90449/17:26
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Allow infra-core to push tags to sandbox  https://review.openstack.org/9110817:27
jeblairmordred, fungi: ^ whoops17:27
mordredjeblair: +@ - fyi - you no longer need the create permission in refs/tags/*17:28
mordredpushSignedTag by itself is all you need17:28
jeblairmordred: true.  i suppose i've also given us perms to create branches.  that's probably not a bad thing.17:28
mordredbut it also will not break anything, and it's for sandbox17:28
*** weshay has joined #openstack-infra17:28
mordredjeblair: well, if it was refs/* yeah17:28
jeblairoh, i'll fix then17:29
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Allow infra-core to push tags to sandbox  https://review.openstack.org/9110817:29
anteayatjones: the second search gets you any commit message with the word phase/Phase anywhere in the message17:29
anteayatjones: it doesn't seem to be doing anything with the 117:29
tjonesanteaya: ugh17:29
tjonesanteaya: so it really does not like multiple words any more i am guessing17:30
mordredwow. that's ... great17:32
jeblairtjones: you might want to ask over in #gerrit17:32
*** openstackgerrit has quit IRC17:32
mordredfungi: can I get some love on https://review.openstack.org/#/c/91066/ ?17:33
*** openstackgerrit has joined #openstack-infra17:33
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests  https://review.openstack.org/9052417:34
jeblair(now i'm trying to figure out why the sandbox-tag job wasn't created)17:34
anteayatjones: you seem to be correct: https://review.openstack.org/#/q/status:+open+message:%22spawn%22+AND+message:%22refactor%22+AND+message:%22phase%22+AND+message:%221%22,n,z17:35
anteayatjones: though I have no idea why https://review.openstack.org/#/c/85387/ gets caught in the search results17:35
dimshmm..."recheck migrations" is not re-triggering turbo-hipster17:36
mordrederr: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid parameter soft_write_failure at /opt/config/production/modules/openstack_project/manifests/puppetmaster.pp:108 on node ci-puppetmaster.openstack.org17:37
mordredI continue to want errors reported to puppetboard17:37
mordredAaronGr, Hunner: ^^ is there any easy/quick fix in our config so that an agent will actually report back if it gets bogus manifests/17:38
tjonesanteaya: i guess i was just lucky before that it worked17:39
*** SumitNaiksatam has quit IRC17:39
anteayaI wouldn't call being able to search for a string lukc17:39
anteayaI would call it common sense17:39
fungimordred: the "${title}_keys" block in your change to localuser.pp is just transitional, to clkear out the old entries added by the previous iteration?17:39
jeblairmordred: i feel like this report is missing something too: http://puppetdb.openstack.org/report/jenkins01.openstack.org/d20b16cf72665c41d0bf2442bc487cf0931051fb17:39
anteayathe question I have is why do you have to have one word strings for message now?17:39
jeblairmordred: like, i dunno, running jenkins-jobs?17:39
*** ativelkov_away is now known as ativelkov17:40
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Fix soft_write_failure parameter name.  https://review.openstack.org/9110917:40
tjonesyes that is a good question17:40
mordredfungi: yes17:41
mordredjeblair: did it run jenkins jobs and fail?17:41
*** SumitNaiksatam has joined #openstack-infra17:41
openstackgerritA change was merged to openstack-infra/config: Allow infra-core to push tags to sandbox  https://review.openstack.org/9110817:42
fungimordred: funny that they refer to the comment on a key as a key id... it's really just a freeform comment area at the end of the line17:42
mordredfungi, jeblair ^^ above change fixes the fact that puppet doesn't run on puppetmaster currently17:42
jeblairmordred: dunno, i'm currently translating PDT to UTC so i can find the right report yaml file on disk17:42
mordredjeblair: lovely17:42
jeblairmordred: i dare you to ask me how i feel about that being the process to find errors.17:42
mordredjeblair: hey man! how do you feel about the need to translate PDT to UTC to that you can find a file on disk to find an error??? **bunny rabbits**17:43
anteayatjones: one word messages with no quotes nets the same result as one word messages with double quotes: https://review.openstack.org/#/q/status:open+message:spawn+AND+message:refactor++AND+message:phase+AND+message:1,n,z17:43
anteayatjones: I am suspecting something is going on with the quotes17:43
anteayabut I don't know what yet17:43
openstackgerritA change was merged to openstack-infra/config: Fix ssh key error and also remove duplicates  https://review.openstack.org/9106617:44
jeblair      message: "Failed to call refresh: Command exceeded timeout at /opt/config/production/modules/jenkins/manifests/job_builder.pp:50"17:44
tjonesanteaya: that query gives 1 (the 1st one) that does not even have the work spawn in it.  something very strange is going on17:45
mordredjeblair: spectacular17:45
jeblairmordred: look, an error! ^17:45
mordredwhy would that not be reported to puppetdb?17:45
mordredI mean - it's not puppet failing to compile17:45
mordred(which I disagree with not being reported, but I could at least understand how operationally something might short circuit)17:45
jeblairmordred: i should have known there would be an error because no error was reported to puppetdb and that is how we tell there's an error.17:45
jeblairmordred: istr that we found that some things were not making it to puppetdb, and no one could figure out why.17:46
jeblairmordred: i think this may have been one of them.17:46
anteayahere is the double quotes in a username bug: http://code.google.com/p/gerrit/issues/detail?id=260217:46
anteayatjones: yes, that one patch keeps showing up in every query I try, don't know why17:47
*** SumitNaiksatam has quit IRC17:48
*** chandan_kumar has quit IRC17:49
openstackgerritSalvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingeprint for bug 1314313  https://review.openstack.org/9111217:50
uvirtbotLaunchpad bug 1314313 in neutron "Firewall fails to become active within 300 seconds" [Critical,New] https://launchpad.net/bugs/131431317:50
*** mrmartin has joined #openstack-infra17:51
jeblairoh, the jenkins-jobs runs from earlier are still running17:51
jeblairfor about 80 minutes now17:51
jeblairi'd really love to know what they are doing17:52
openstackgerritA change was merged to openstack-infra/config: Fix soft_write_failure parameter name.  https://review.openstack.org/9110917:53
*** ativelkov has left #openstack-infra17:53
anteayatjones: not exact but it does give some info on message: http://code.google.com/p/gerrit/issues/detail?id=1441&q=search%20message%3A&colspec=ID%20Type%20Stars%20Milestone%20Status%20Priority%20Owner%20Summary17:54
mordredjeblair: that seems like a long time17:54
mordredjeblair: btw - gertty and infra-manuals are now in gerrit17:54
*** sarob has joined #openstack-infra17:55
*** etoews has joined #openstack-infra17:56
tjonesanteaya: i tried topic but that does not work either :-(17:56
Hunnermordred: The agent *should* send a report with the failure messages... is it not?17:57
*** markwash has joined #openstack-infra17:57
anteayatjones: https://review.openstack.org/#/q/status:open+topic:%255Ebp/vmware-spawn.*,n,z17:58
anteayaapparently strings are very upsetting to search, where as regex gets through: status:open topic:^bp/vmware-spawn.*17:59
*** darraghb has quit IRC17:59
anteayatjones: is that enough to keep you moving forward?17:59
tjonesanteaya: ok i see now.  nice.  thanks for your help!!!  searching strings upsetting….  who knew ;-)17:59
mordredHunner: yes. the agent sends that report18:00
anteayayeah, uh, wow is all I can say18:00
mordredit's happily on the puppetmaster in the yaml reports dir18:00
mordredHunner: but it most certainly does not show up in puppetboard18:00
anteayatjones: note the . in the .* is part of the regex18:00
tjonesanteaya: yeah got it!18:00
mordredHunner: in a couple of different scenarios we've hit today18:00
*** jp_at_hp has quit IRC18:01
*** etoews has quit IRC18:01
*** _nadya_ has joined #openstack-infra18:01
*** ildikov has joined #openstack-infra18:02
anteayathere don't seem to be any bug reports about string searching failing with other people's gerrits18:02
anteayadid we have do create any config around searching for our upgrade?18:03
mordredHunner: one example is:18:03
mordred17:37:24          mordred | err: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid parameter soft_write_failure at18:03
mordred                          | /opt/config/production/modules/openstack_project/manifests/puppetmaster.pp:108 on node ci-puppetmaster.openstack.org18:03
*** jamielennox|away is now known as jamielennox18:03
mordredthat manifested as the agent just not reporting at all, which is easy enough to find in puppetboard- "why is that server grey?" - although it does require ability to shell to the server and run commands to figure out why18:03
*** mrmartin has quit IRC18:04
*** mengxd has joined #openstack-infra18:05
HunnerPerhaps puppetboard just doesn't render those reports? If they're submitted to the master, then they should be sent to the puppetdb too... I'll ask18:05
jeblairhrm. jjb gets each job 3 times before posting it18:05
mordredHunner: there were also 2 other things we ran in to where there was an error reported on teh command line but it didn't show up in puppetboard18:06
*** jaypipes has joined #openstack-infra18:06
salv-orlandothere is a rise in neutron unclassified gate failures. A change in a test uncovered a bug, which is now causing failures in the gate:18:06
*** jgrimm has quit IRC18:06
mordredHunner: and in those cases, puppetboard showed content - it got the report - and even worse, showed that the puppet run was successful18:06
salv-orlandoBug: https://bugs.launchpad.net/neutron/+bug/1314313; e-r query: https://review.openstack.org/#/c/90798/18:06
uvirtbotLaunchpad bug 1314313 in neutron "Firewall fails to become active within 300 seconds" [Critical,New]18:06
*** BadCub has quit IRC18:06
*** che-arne has quit IRC18:07
jeblairjjb appears to be ignoring its cache18:07
jeblairzaro: ^18:07
Hunnermordred: Do you have an example of that error?18:07
Hunner(the one that did show up and the run was reported successful)18:08
mordredHunner: yup18:08
jeblairzaro, mgagne: ^ our jjb processes are updating all of the jobs every time they run (even with no changes)18:08
mordredHunner: look at this: http://puppetdb.openstack.org/report/jenkins01.openstack.org/075bb6a3c6a6593e525a92716e852f62cd27d9c118:08
*** melwitt has joined #openstack-infra18:08
mordredHunner: now, in the report.yaml file, you'll see this:       message: "Could not evaluate: Puppet::Util::FileType::FileTypeFlat could not write /home/corvus/.ssh/authorized_keys: Permission denied - /home/corvus/.ssh/authorized_keys"18:08
mgagnejeblair: we cannot be sure often those jobs got updated :D18:08
mgagnejeblair: any idea on the root cause?18:08
jeblairmgagne: not yet18:09
anteayasalv-orlando: thank you for reporting18:09
jeblair(btw, we have 2289 jenkins jobs now)18:09
*** mengxd has quit IRC18:11
jeblairphschwartz: did you have this problem recently?18:12
phschwartzjeblair: Yes I have been having the same issue with the latest jjb18:12
jeblairphschwartz: make any headway?18:12
*** etoews has joined #openstack-infra18:12
*** ramashri has joined #openstack-infra18:13
phschwartzjeblair: I think I haven't narrowed down an actual issue as the source points to a valid location for the cache and it is there.18:13
*** BadCub01 has joined #openstack-infra18:13
phschwartz(or I should say was there after I fix perms on the dir)18:14
jeblairphschwartz: ok, thanks.  i'll see what i can come up with18:14
phschwartzI think my issue was something ran as sudo that changed the perms on my cache dir18:14
phschwartzI am doing another run of jjb in a bit so I would have more info18:14
*** msabramo has quit IRC18:16
*** etoews has quit IRC18:17
fungimordred: hpcloud region a is "west" and region b is "east" right?18:17
mordredfungi: not _quite_18:18
mordredfungi: or - maybe it is18:18
*** etoews has joined #openstack-infra18:18
mordredyeah - region-a is west - and we don't have it enabled because they told us to stay off it for a while18:18
zarojeblair: i'll do some testing with jjb locally to see if i can repro.18:19
*** talluri has quit IRC18:19
*** _nadya_ has quit IRC18:19
fungimordred: well, 1.0 uses the region a endpoint as well18:19
fungimordred: i guess they share a keystone?18:20
openstackgerritSalvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313794  https://review.openstack.org/9079818:20
uvirtbotLaunchpad bug 1313794 in neutron "lock_wait timeout caused by plug_vip_port" [High,New] https://launchpad.net/bugs/131379418:20
mordredfungi: /me waves hands in the air and then hides18:20
*** alexpilotti has joined #openstack-infra18:21
*** jgrimm has joined #openstack-infra18:24
anteayajust joined the #gerrit channel, channel topic identifies 2.718:24
zarophschwartz, jeblair : this one merged a while ago but might be the cause? https://review.openstack.org/#/c/59515/18:25
phschwartzzaro: I was just looking at that commit directly to see if it is causing an issue18:26
zarophschwartz, jeblair : this one as well because it updates how the config file gets loaded. https://review.openstack.org/#/c/5208018:26
mordredzaro, phschwartz that LOOKS like would wind up with ignore_cache being false in most cases appropriately18:27
jeblairzaro: we don't set that, and it looks like it defaults to false18:27
zarothat one merged just last week.18:27
phschwartzYeah, after just stepping through a jjb run in pdb, it is set to false and it is trying to load the cache so I am even more stumped.18:27
*** hashar has joined #openstack-infra18:28
anteayajust when through the gerrit irc logs for april, noone has used the word "string" that I could find18:28
mordredI could imagine a world in which description mismatches would cause it to think it's changed18:29
mordredI actually re-read that patch and it also looks sane to me18:29
*** manishg has joined #openstack-infra18:30
*** zns has quit IRC18:31
zarophschwartz: i assume you are using jjb from master?18:31
phschwartzlastest from pip18:31
phschwartzand master18:31
*** pdmars has quit IRC18:33
phschwartzok, so now I can see after a few tests. My issue was purely due to dir perms where my user couldn't write to the dir. It is working from cache with no issue after the perms fix.18:33
mordredjeblair: ^^ any chance we have a perms issue?18:34
yfriedjesusaurus: Hi, I'm shipping files to logstash during debug18:35
phschwartzmordred: my perm issue was the dir I was trying to use as cache was owned by root due to another app that I ran with sudo18:35
*** manishg_ has joined #openstack-infra18:35
yfriedjesusaurus: and when I'm trying to reship some files it ignores it, even though I've specificly deleted all sincedb files on the shipper machine18:36
*** talluri has joined #openstack-infra18:36
*** msabramo has joined #openstack-infra18:36
*** manishg has quit IRC18:37
*** manishg_ is now known as manishg18:37
jesusaurusyfried: are you setting start_position? by default it starts at the end so you will need to append lines to the file before anything goes through logstash18:38
manishgI'm getting 503 when accessing review.openstack.org since yesterday (when using firefox) - it started yesterday (was working fine earlier).  works fine with safari.18:38
yfriedjesusaurus: I am18:39
sdaguemanishg: did you close your browser?18:39
mordredSergeyLukjanov: you know stackalytics seems to be down?18:39
*** melwitt has quit IRC18:40
DinaBelovamordred, it is ~30 mins18:40
manishgsdague: yeah I closed firefox, I flushed the cache too (yesterday).  Let me do it again.18:40
*** marcoemorais1 has joined #openstack-infra18:40
yfriedthey were shipped ok once. and again after deleting the sincedb files. the third time doesn't work18:40
mordredDinaBelova: ah - ok. cool18:40
mordredjust making sure18:40
*** melwitt has joined #openstack-infra18:41
manishgsdague: closed the window but it was still running in background!  killed that process and restarted.  works fine now.  thanks.18:41
yfriedjesusaurus: any suggestion?18:42
*** AaronGr has quit IRC18:42
*** marcoemorais has quit IRC18:42
sdaguemanishg: no prob. now just pay it forward to others asking the question :)18:43
manishg:) sdague:  will do!18:43
openstackgerritJames E. Blair proposed a change to openstack-infra/jenkins-job-builder: Treat ignore_cache as a boolean  https://review.openstack.org/9112418:45
jeblairzaro, mgagne, phschwartz, mordred: ^18:45
jesusaurusyfried: you could add a file output to see if logstash is reading and shipping the events but elasticsearch is failing to index them18:45
*** gabriel-bezerra has left #openstack-infra18:45
anteayaokay we are 15 minutes to -infra meeting, going to try to go for my walk after the meeting then18:45
yfriedjesusaurus: it is18:45
yfriedjesusaurus: as I said - it worked twice before18:46
jesusaurusand you stopped logstash before deleting the sincedb file?18:47
phschwartzjeblair: Not sure if that is needed, but can't hurt.18:47
jesusaurusyfried: sorry, im not really sure what could be happening18:48
yfriedjesusaurus: tnx anyway18:48
jesusaurusyfried: if the logstash process isnt logging any errors and elasticsearch isnt logging any errors, its pretty hard to tell what might be going wrong :/18:48
mgagnejeblair: would it be a "good idea" to use oslo.config instead?18:50
jeblairphschwartz: it's required -- that's the reason why our runs were not using the cache18:50
*** AaronGr has joined #openstack-infra18:50
jeblairphschwartz: my guess is that if your runs were using the cache, you were probably setting it explicitly on the command line18:51
yfriedjesusaurus: it has to do with going over files for the second time18:51
jeblairmgagne: i'm not sure -- i don't really want to add unnecessary dependencies, but if we have mary more command-line-or-config-file options it might be a good idea18:55
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add additional yaml/xml tests, literalinclude them  https://review.openstack.org/9113118:57
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Publish translated localized API documents  https://review.openstack.org/9113218:57
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Treat ignore_cache as a boolean  https://review.openstack.org/9112418:58
jeblairmeeting time in #openstack-meeting18:58
*** hashar is now known as hasharTechTalk18:58
*** pblaho has joined #openstack-infra18:59
jeblairfungi: around for meeting?18:59
*** derekh has joined #openstack-infra19:00
SergeyLukjanovmordred, yup, looks like stackalytics is down now, maintainers already know about it19:00
*** morganfainberg is now known as morganfainberg_Z19:01
*** julim_ has joined #openstack-infra19:01
*** ominakov has joined #openstack-infra19:01
*** ominakov has quit IRC19:01
dansmithmordred: gerrit (via git-review) is still definitely faster.. dunno if it's gerrit performance or the new box, but... +2 on that from me :)19:03
*** signed8bit has quit IRC19:03
dansmithgit-review completes on the same day as I start it now, which is super nice19:03
*** julim has quit IRC19:04
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Return after successfull auth  https://review.openstack.org/9030819:04
*** vhoward has joined #openstack-infra19:05
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Add slukjanov to users  https://review.openstack.org/9101219:05
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Some tweaks to get closer to Python 3 compat  https://review.openstack.org/8781019:07
krotschecksdague: When you get a chance, can you look at storyboard.openstack.org, try to edit a task, and see if that typeahead functionality is more like what you were trying to do with the chosen patch?19:07
*** vhoward has left #openstack-infra19:07
openstackgerritIvan Melnikov proposed a change to openstack-dev/hacking: Consider sys.real_prefix for _get_import_type  https://review.openstack.org/8249519:07
sdaguekrotscheck: well, at least in chrome, it's not very clear about what's going in that field19:08
krotschecksdague: Oh, point. i should add an icon indicator...19:08
sdagueyeh, the chosen UI is good in that it shows what you type and the search list19:09
sdaguethat seems to be jumping to diffent items, but as I don't see my typing, it's pretty blind19:09
*** talluri has quit IRC19:09
*** signed8bit has joined #openstack-infra19:16
*** andreykurilin_ has quit IRC19:17
*** SumitNaiksatam has joined #openstack-infra19:18
*** zns has joined #openstack-infra19:18
DinaBelovamordred - stackalytics is up19:18
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Some tweaks to get closer to Python 3 compat  https://review.openstack.org/8781019:25
sdagueanteaya: http://lucene.apache.org/core/3_0_3/api/all/org/apache/lucene/analysis/standard/StandardTokenizer.html - some details on what lucene does to text before indexing it19:28
*** SumitNaiksatam has quit IRC19:28
sdagueso basically, not being able to find '-' is probably expected19:28
anteayasdague: thanks19:29
*** eharney has joined #openstack-infra19:30
*** mark0n has joined #openstack-infra19:31
*** ociuhandu has quit IRC19:32
notmynamemordred: what's the status of the m-p merge job? it doesn't look like it ever ran19:38
*** ramashri has quit IRC19:38
notmynamemordred: is there something I can do on my end?19:38
*** BadCub01 has quit IRC19:38
zaroanteaya: i'm not about redoing schema, wanted to see if others wanted to do it first.19:39
anteayanotmyname: when you type m-p I think manage projects since we have been working on that as well19:39
anteayanotmyname: you mean milestone-proposed, yeah?19:39
anteayazaro: oh19:39
zarofungi, mordred, jeblair do you guys think it would be a good idea to sync up review-dev.o.o db schema with one from review.o.o?  i think they are out of sync.19:40
jeblairzaro: go for it19:40
*** ramashri has joined #openstack-infra19:40
*** nati_ueno has quit IRC19:42
zarocan someone export the review.o.o schema for me?  or should i just let gerrit create fresh one using the gerrit.war?19:42
jeblairzaro: you need to keep the data19:43
*** etoews has quit IRC19:43
zarojeblair: ohh well, i'm not sure how to sync shema without blowing away data.19:43
*** etoews has joined #openstack-infra19:44
zarois that even possible?19:44
jeblairzaro: with alter table19:45
notmynamejeblair: do you know anything about the milestone-proposed merge jobs?19:45
*** ihrachyshka has quit IRC19:45
jeblairnotmyname: not off hand; is it urgent such that i should try to catch myself up or can it wait for mordred?19:46
notmynamejeblair: it can probably wait until mordred gets back, but pbr versions are not correct until it happens19:47
funginotmyname: talking about the failure at https://jenkins.openstack.org/job/swift-merge-release-tags/1/console19:48
openstackgerritDoug Hellmann proposed a change to openstack-infra/config: Import oslo.i18n  https://review.openstack.org/9114919:48
zarojeblair: ok, whichever way don't i still need an export of the schema from review.o.o ?19:50
jeblairzaro: how do you know it's out of sync?19:50
zarojeblair: because there's a table on review-dev db that wasn't in the review db.  that's why we didn't have you that TRUNCATE workaround in the upgrade db script.19:51
zarohave you/ have to use19:52
*** morganfainberg_Z is now known as morganfainberg19:52
jeblairzaro: i won't be able to get to it for a bit19:53
zarojeblair: ok. no hurry i guess. whenever.19:53
zarofungi, jeblair: what needs to happen for review-dev to use the same version of gerrit as review.o.o?  i think puppet files have been updated, is it just a restart?19:57
*** ociuhandu has joined #openstack-infra19:58
*** krtaylor has joined #openstack-infra19:58
fungizaro: probably...?19:59
*** hasharTechTalk is now known as hashar19:59
*** ramashri has quit IRC20:00
openstackgerritDoug Hellmann proposed a change to openstack-infra/config: Import oslo.i18n  https://review.openstack.org/9114920:01
zarofungi: how can i check whether puppet has (or has not) updated review-dev?20:01
*** pblaho has quit IRC20:02
notmynamefungi: yes, that one20:02
*** derekh has quit IRC20:05
anteayatjones: https://review.openstack.org/#/q/message:a_test_title_to.see.what.I.can,n,z20:08
*** fifieldt has quit IRC20:08
*** fifieldt has joined #openstack-infra20:09
tjonesanteaya: nice!20:09
anteayatjones: so it seems to consider words joined by underscores or . as one token20:09
*** sarob has quit IRC20:09
anteayait really hates the ! even in regexs20:09
*** nati_ueno has joined #openstack-infra20:09
anteayaand it treats - like whitespace20:10
*** etoews_ has joined #openstack-infra20:10
anteayain terms of splitting on it20:11
JayFHmm. Ever since the gerrit upgrade, all emails from gerrit are going to my spam folder20:11
*** sdake_ has joined #openstack-infra20:11
openstackgerritA change was merged to openstack-dev/hacking: Consider sys.real_prefix for _get_import_type  https://review.openstack.org/8249520:12
*** Ajaeger has quit IRC20:13
anteayaArxCruz krtaylor do either of you know who IBM powerVC is? They need to comment once per patchset, right now they are commenting twice: https://review.openstack.org/#/c/91154/20:13
*** etoews has quit IRC20:13
*** nati_uen_ has joined #openstack-infra20:13
tjonesanteaya: thanks for looking into that20:13
anteayatjones: np, so the tl;dr is that searching for strings with multiple words won't work anymore20:13
ArxCruzanteaya: I know, but they will be on only in a few hours20:14
anteayaso creative use of specialized tokens, regexs and topic look like it might help you20:14
anteayaArxCruz: k, I'll leave it with you, thanks20:14
*** mattymo|home has quit IRC20:16
*** nati_ueno has quit IRC20:17
ArxCruzanteaya: already sent an email to the responsible, but they will be available only in 6-7 hours20:17
anteayaArxCruz: not a big deal, they are not voting on anything, they are just on the sandbox repo right now that I can see20:18
anteayaso it is good they are staying there for now20:18
anteayaArxCruz: https://review.openstack.org/#/q/owner:pwrdrvci%2540cn.ibm.com+status:open,n,z20:19
asselin__anyone run into this error when creating a new image? INFO:nodepool.image.build.local.d-p-c:Parameter key failed on Ssh_authorized_key[jenkins-master-2014-04-24]: Key must not contain whitespace:20:19
ArxCruzanteaya: by the way, I'm noticing my user are not posting back the results20:19
ArxCruzanteaya: in my zuul log I see this:20:19
asselin__(this is my own local setup)20:19
ArxCruz2014-04-29 20:01:17,503 DEBUG zuul.reporter.gerrit.Reporter: Report change <Change 0x7f7d10ac2310 91146,1>, params {'force-message': True}, message: Build failed. For rechecking only on the IBM PowerKVM CI, add a review comment with recheck-pkvm20:20
ArxCruzbut I'm not seeing in review.openstack.org20:20
anteayaArxCruz: hmmm, you updated host keys20:20
jeblairArxCruz: I7836860f9e62bcb169bc13c5e3b5b5ecf6d325a120:20
jeblairArxCruz: sorry, rather this one: I7d1abe590dc79edde4d3ccf998ede59fccc4844a20:21
jeblairArxCruz: you'll need zuul master as well20:21
ArxCruzneed to update zuul?20:21
ArxCruzlast time I did was so much pain...20:21
jeblairArxCruz: yes, tip of zuul master plus that change to your layout20:21
*** jlk has quit IRC20:23
*** wendar has quit IRC20:24
*** wendar has joined #openstack-infra20:24
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Drop the ref from the tag fetch in GGP  https://review.openstack.org/9116420:25
ArxCruzjeblair: okay, thanks20:25
jeblairmordred, fungi: ^ that should take care of the tag problem20:25
anteayaasselin__: so in the backscroll jeblair encountered an issue with a newline in a key, he wrapped the value in strip() to remove it, I don't think there was a patch20:25
anteayaasselin__: but I could be wrong20:25
fungiokay, well i seem to be able to rebuild bare-centos6 images successfully now, and the count seems to be better distributed across providers again20:25
jeblairmordred, fungi: the problem likely occurred only while we were having replication issues.  the fix above will make it so that it should work regardless of replication status20:26
jeblairmordred, fungi: moreover, i think very rarely we have seen that failure in normal operation -- those cases were likely replication delays which this should fix20:26
jeblairmordred, fungi: (i tested by cloning a copy of sandbox from the old gerrit server; that was helpful)20:27
anteayaasselin__: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-29.log timestamp 2014-04-29T16:58:2620:27
jeblairfungi: what was the contos problem?20:27
fungijeblair: oh, that's a tricky one. so if you fetch a specific tag it needs to be one your remote update found, but if you fetch all tags it will get them even when remote update failed?20:27
jeblairfungi: (btw, i think you and i are the only cores around if you want to go aheand and aprv https://review.openstack.org/91164)20:28
jeblairfungi: even better -- it only fails the first time.  a subsequent fetch will work20:28
ArxCruzjeblair: so gerrit: { } means post a message failing or not, but don't vote ?20:28
jeblairfungi: (because the first fetch got the info needed for the second fetch to work)20:28
*** jooools has joined #openstack-infra20:28
jeblairArxCruz: yeah, it's basically "use this reporter with no arguments"20:28
fungijeblair: the centos6 image build issue was curl not following the github redirect to the new url, so that outstanding patch definitely solved it20:28
asselin__anteaya, thanks...not sure if it's the same...but I will try it20:29
jeblairfungi: what caused the 70 building/deleting nodes?20:29
anteayaasselin__: that is all I have on that one20:29
fungijeblair: that i'm less sure about. from the logs it looked like we were timing out ssh access to them in hpclolud-region-b and it was going into a tight rebuild loop20:30
*** sabari has joined #openstack-infra20:30
openstackgerritA change was merged to openstack-infra/config: Drop the ref from the tag fetch in GGP  https://review.openstack.org/9116420:30
fungijeblair: it subsided on its own before i could pin it down, so not really sure20:32
fungiasselin__: https://review.openstack.org/9106620:32
asselin__fungi, I saw that and tried it but it didn't solve the issue for me...I noticed in another commit the ssh keys had their whitespace and comment section removed.20:33
SlickNikfungi / anteaya: Fixed the issue with the gerrit-trigger plugin. Turns out that I had already updated the host key fingerprint and that wasn't the issue.20:34
*** pdmars has joined #openstack-infra20:34
anteayaSlickNik: what was the issue?20:34
SlickNikfungi / anteaya: The deprecated 'approve' alias for the 'review' SSH command has been removed as of gerrit 2.8.20:34
SlickNikAnd gerrit trigger plugin was using this alias.20:35
anteayahow did you fix it?20:35
asselin__fungi, mordred do you know the reason why that change is needed? i.e. a change in puppet?20:35
SlickNik(by default)20:35
anteayaSlickNik: just remove the alias?20:35
SlickNikTHey let you override the commands in the  global configuration for the plugin though.20:35
anteayaSlickNik: oh you changed it there, cool20:36
SlickNikSo I just changed the commands to use 'review' instead of 'approve'20:36
SlickNikYup, that seems to be working well.20:36
anteayathanks for letting us know20:36
anteayayour's won't be the last system tripping on this20:36
fungiasselin__: ahh, yep, https://review.openstack.org/8734020:36
SlickNikNot a problem. Now you know. :)20:37
SlickNikFYI: The release notes that talk about the removal of the deprecated 'approve' alias - https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNotes-2.8.html20:37
fungiasselin__: to support being able to run puppet agent on demand via ssh from the master we implemented the ssh_authorized_key type, so that change started using it everywhere for consistency20:38
*** jlk has joined #openstack-infra20:38
SlickNikThe helpful part is under the 'Schema Change' section20:38
fungiSlickNik: oh, yep, someone else posted a similar note to either the dev or infra ml, i forget which20:38
*** yidclare has quit IRC20:39
SlickNikfungi: Cool, :) Just wanted to give you guys a heads up in case you get hit with a similar question from other folks.20:40
asselin__fungi, ok so perhaps that broke the instructions in nodepool readme: export NODEPOOL_SSH_KEY=`cat ~/.ssh/id_rsa.pub`20:40
anteayaSlickNik: https://wiki.openstack.org/wiki/GerritUpgrade#Gerrit_Trigger_Plugin20:40
asselin__fungi, since the public key can have spaces...I'll take a look and see if that is it20:40
anteayaSlickNik: do add anything you think is missing20:40
mordredasselin__: aha!20:40
mordredasselin__: let me go look ... it's possible we broke that20:41
asselin__mordred, thanks!20:41
SlickNikanteaya: Thanks! That looks good.20:42
anteayaArxCruz had posted to the infra ml about changing the host key: http://lists.openstack.org/pipermail/openstack-infra/2014-April/001201.html20:42
anteayaSlickNik: thanks20:42
fungiasselin__: as a workaround, you might try switching to NODEPOOL_SSH_KEY=`cut -d' ' -f-2 ~/.ssh/id_rsa.pub`20:42
*** julim_ has quit IRC20:42
asselin__fungi, thanks will try now20:42
mordredfungi: I think we might want to do that in prepare_node.sh20:42
fungimordred: we probably need to split it there and pass in the separate vars, agreed20:43
mordredfungi: we could test to see if the ssh key string has ssh-rsa in it, and if so , split it20:45
mordredjeblair: so - apparently we've grown instances of gliffy, the code review cuckoo?20:46
jeblairmordred: 'diffy'20:46
*** msabramo has quit IRC20:47
mordredoh. right. sorry20:47
jeblairmordred: and it appears to be hardcoded in the gwt side of things; completely ignoring the server side avatar configuration20:47
jeblairtop notch this thing.20:47
anteayaha ha ha20:47
anteayathe bird is mandatory20:47
jeblairanteaya: indeed, i feel like the gerrit devs are insisting on giving us the bird.20:48
anteayaha ha ha20:48
mordredjeblair: well - I betcha we could javascript that bird out of existence20:48
anteayathey can't get the ux right20:49
anteayabut they hard code a troll20:49
mordredwhich I'm kind of tempted to waste mental energy on just out of general principle20:49
jeblairmordred: yeah; it's not obvious to me at the moment how to do that20:49
sdaguejeblair: https://etherpad.openstack.org/p/gerrit-dashboards what do you think of this for a dashboard set20:49
fungiwe should totally replace diffy with gliffy the code review troll20:50
mordredfungi: or a picture of zuul20:50
mordredthere is no diffy, only zuul20:50
anteayathe dragon in the tutu?20:50
jeblairwhile i'm ranting -- i checked in on project renames: https://gerrit-review.googlesource.com/#/c/42247/20:51
jeblair"Updated 8 months ago"20:51
mordredsdague: what does foreach do?20:51
sdaguemeans it can apply to projects as a token20:51
sdaguemordred: https://review.openstack.org/Documentation/user-dashboards.html#_token_code_project_code20:52
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/config: Gerrit - increase the width of the comments  https://review.openstack.org/9117020:52
anteayasdague: Lost changes makes me thing of patches that have logs that are lost, due to a random zuul error20:52
mordredsdague: those look like good things20:52
sdagueso then if you applied it to nova, you'd only get nova code, for instance20:52
anteayasdague: do you care that I think that?20:52
jeblairsdague: i don't read it as being required in order to use the project token20:53
*** adalbas has quit IRC20:53
jeblairsdague: i read it as shorthand for "append to all queries"20:53
sdaguejeblair: ok, yeh, I'm not sure20:53
jeblair(which i worry would make it harder to write non project-specific queries)20:53
sdagueanteaya: if you have a better name suggestion, I'm all for it20:53
sdaguejeblair: it would be good to see it in operation to understand it's limitations20:54
*** _nadya_ has joined #openstack-infra20:54
mordredso - right now, important changes is a file called "important-changes-dashboard" in the ref refs/dashboards/important-changes on All-Projects20:54
anteayaadrift, lacking, astray, wayward20:54
anteayaI love a trip to the thesaurus20:55
anteayalike any of them?20:55
mordredwhich is to say - any dashboard needs a category (refs/dashboards/important-changes) and a name "important-changes-dashboard"20:55
mordredI'm not 100% sure how to delete a dashboard ref, btw20:55
mordredbut inside of that t's just files20:55
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313851  https://review.openstack.org/9079420:56
mordredfor testing, since deleting a file is easy - I could push a new additional file to the important-changes ref ?20:56
uvirtbotLaunchpad bug 1313851 in neutron "lock wait timeout in create_dhcp_port" [Medium,New] https://launchpad.net/bugs/131385120:56
sdaguemordred: that seems like an undoable thing to do20:57
fungimordred: might be able to push a null ref to it. doesn't look like the ui treats those as actual branches though20:57
sdagueso if we do it badly it can be wiped out20:57
*** e0ne has joined #openstack-infra20:57
mordredthat's my main concern right now20:57
sdaguealso, who has permissions to do refs/ manip? is that just the manage-projects script?20:58
jeblairsdague, mordred: i think as long as sdague's queries are going in a different file, the foreach should be fine -- if they get combined with the existing important changes dashboard file, i think the project token needs to distribute down to those 3 queries and the foreach should be removed20:58
fungimordred: since it looks like a branchish ref in git, and it's possible to push :remotename to delete branches, i'd say it's worth trying on review-dev20:58
mordredand project bootstrappers20:58
sdaguejeblair: yeh, I think this should be a different file20:58
mordredjeblair: I agree20:58
asselin__BTW, I have a change that's still needs review. What's the best way to get it reviewed? I'd like some initial feedback before adjusting the merge conflicts. https://review.openstack.org/#/c/87384/20:58
mordredjeblair, fungi: so - you guys fine with me pushing that dashboard to test it out?20:59
jeblairmordred: yes20:59
jeblairmordred: i think you should undo it; i get a 50021:00
anteayascant, wanting, missing, scarce21:00
*** yidclare has joined #openstack-infra21:00
*** SumitNaiksatam has joined #openstack-infra21:00
mordredwow. that was exciting21:01
sdagueany indication what went wrong?21:01
fungiwow, that's... fun21:01
*** melwitt has quit IRC21:01
mordrednope. I got no errors on push21:01
jeblairi'm thinking review-dev might be a good place to push that next.21:01
mordredjeblair: yup21:01
mordredjeblair: might be worth giving sdague project bootstrappers there so he can just push to his heart's content for a bit?21:01
jeblairmordred: wfm21:01
anteayasdague: so far I like wayward the best21:01
sdagueanteaya: ++ wayward sounds good21:02
mordredsdague: I'm going to put you in the project bootstrappers group on review-dev21:02
sdaguemordred: sounds good21:02
jeblairmaybe section names21:02
*** _nadya_ has quit IRC21:02
sdaguedoes review-dev have enough data to make this useful?21:02
mordredsdague: https://etherpad.openstack.org/p/gerrit_upgrade_script has a section on adding a new dashboard21:02
*** melwitt has joined #openstack-infra21:03
jeblairsdague: yeah, and you can always add more data21:03
mordredsdague: I think we're going for "functions at all without breaking things" before we try pushing to review again21:03
jeblairmordred: ++21:03
zarojeblair: there's an upcoming change to allow hiding monitoring menu: https://gerrit-review.googlesource.com/5659821:03
anteayamordred: can I be added to project bootstrappers on review-dev as well? I would like to play with dashboards too21:03
*** marcoemorais1 has quit IRC21:03
*** msabramo has joined #openstack-infra21:03
sdaguemordred: I'm good with that21:03
mordredsdague: done21:03
*** aysyd has quit IRC21:03
sdaguethis sounds like a reasonable plan, I'm happy to be a part of it :)21:03
openstackgerritBen Nemec proposed a change to openstack-infra/reviewstats: Update for Gerrit 2.8  https://review.openstack.org/9117321:04
jeblairzaro: cool.  i'm quite okay with the default of falso, btw.21:04
openstackgerritA change was merged to openstack-infra/config: update jenkins-dev.o.o with same plugins as jenkins.o.o  https://review.openstack.org/7704021:04
*** marcoemorais has joined #openstack-infra21:04
mordredjeblair: can more things default to falso21:05
fungimordred: not sure if you saw, but notmyname was asking about https://jenkins.openstack.org/job/swift-merge-release-tags/1/console (looks like maybe git-review wasn't expecting to find another outstanding commit besides the merge commit?)21:05
notmynamemordred: fungi: just wondering about the current status. versions on master are wrong until m-p gets merged.21:06
*** marcoemorais has quit IRC21:06
*** andreaf has quit IRC21:06
sdaguemordred: is there a way to actually check out the existing refs/meta/... ?21:07
*** marcoemorais has joined #openstack-infra21:08
zarosdague: git fetch origin refs/meta/config;git checkout FETCH_HEAD21:10
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313800  https://review.openstack.org/9079721:10
uvirtbotLaunchpad bug 1313800 in neutron "lock wait timeout while updating LB pool" [Medium,New] https://launchpad.net/bugs/131380021:10
*** dangers is now known as dangers_away21:10
bknudsonit's not going to break anything if we use the "New" view in gerrit?21:11
mordredbknudson: your sanity21:12
bknudsonmight be too late for that already21:12
sdaguezaro: thanks21:13
*** SumitNaiksatam has quit IRC21:14
*** dizquierdo has joined #openstack-infra21:15
*** SumitNaiksatam has joined #openstack-infra21:15
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313794  https://review.openstack.org/9079821:17
uvirtbotLaunchpad bug 1313794 in neutron "lock_wait timeout caused by plug_vip_port" [High,New] https://launchpad.net/bugs/131379421:17
*** dkliban is now known as dkliban_afk21:18
*** krtaylor has quit IRC21:18
*** eharney has quit IRC21:19
openstackgerritDavid Ostrovsky proposed a change to openstack-infra/config: Add configuration option to disable top level menu for javamelody plugin  https://review.openstack.org/9117621:20
*** dprince has quit IRC21:20
*** eharney has joined #openstack-infra21:21
*** morganfainberg is now known as morganfainberg_Z21:22
*** hashar has quit IRC21:23
*** morganfainberg_Z is now known as morganfainberg21:25
*** matty_dubs is now known as matty_dubs|gone21:26
sdaguewhat was the magic exploding url?21:28
mordredsdague: https://review.openstack.org/#/admin/projects/All-Projects,dashboards21:28
*** rfolco has quit IRC21:29
sdagueyep, still 50021:29
sdaguecan someone see if there is anything useful on the server?21:29
sdaguehttps://review-dev.openstack.org/#/admin/projects/All-Projects,dashboards I mean is 50021:30
openstackgerritA change was merged to openstack-infra/config: Add fedora devstack instances to nodepool  https://review.openstack.org/8684221:30
*** BadCub has joined #openstack-infra21:30
jeblairsdague: ack21:30
jeblairsdague: not really: http://paste.openstack.org/show/77809/21:32
*** sdake_ has quit IRC21:32
*** morganfainberg is now known as morganfainberg_Z21:33
sdagueoh, description is apparently mandatory21:34
*** thomasem has quit IRC21:34
sdaguegood to know21:34
*** morganfainberg_Z is now known as morganfainberg21:34
openstackgerritA change was merged to openstack-infra/config: Add check-rally-dsvm-glance job  https://review.openstack.org/8980121:36
jeblairi'm going to restart gerrit to pick up those changes21:36
*** mbacchi has quit IRC21:37
jeblairdims: ping21:38
*** mwagner_lap has quit IRC21:39
sdagueit's a thing21:40
sdagueI had to adjust it because jenkins isn't a user there21:41
sdaguebut it seems to do the thing we want21:41
sdaguejeblair: https://etherpad.openstack.org/p/gerrit-dashboards (the part under Version 2 is the working one)21:42
sdaguefeel free to change title or description, not wedded to either21:43
*** dizquierdo has quit IRC21:43
*** sdake_ has joined #openstack-infra21:48
*** sdake_ has quit IRC21:48
*** sdake_ has joined #openstack-infra21:48
*** lcostantino has quit IRC21:48
jeblairrestarting gerrit now21:49
*** praneshp has quit IRC21:50
jeblairdone, and the zuul commentlinks look correct21:50
*** jeblair changes topic to "Gerrit upgraded to 2.8. See: https://wiki.openstack.org/wiki/GerritUpgrade"21:50
fungiwhy yes... yes they do!21:50
sdaguejeblair: nice, the css seems to have taken correctly (I can disable stylish on it now)21:51
*** smarcet has quit IRC21:51
fungii still think the current gerrit design is waaaay to liberal with whitespace (particularly line spacing), but it's usable21:51
*** praneshp has joined #openstack-infra21:51
jeblairfungi: agreed21:51
fungithey've injected tons of extra padding for reasons i can't fathom21:52
jeblairfungi: ipad?21:52
fungimaybe they thought the interface was too dense21:52
openstackgerritBoris Pavlovic proposed a change to openstack-infra/config: Bump check-rally-dsvm-{name} job timeout  https://review.openstack.org/9063021:52
fungijeblair: heh, so they can finger-paint their links more easily. gotta be it! ;)21:53
*** weshay has quit IRC21:56
notmynameah. thank you for making the jenkins comments in gerrit readable again21:57
*** dims has quit IRC21:57
jeblairnotmyname: ++21:58
jeblairsdague: "Error in operator label:Code-Review>=0,self" if you aren't logged in21:58
jeblairsdague: that makes sense of course21:58
*** mfer has quit IRC21:59
sdagueoh, interesting21:59
sdagueI wonder if there is any mitigation for that21:59
notmynamejeblair: I was literally looking through the code to see how to change it. didn't find anything (or actually I think I pulled in the patch before you restarted gerrit and therefore didn't see anything that made sense). then I glanced in here and saw y'all had just fixed it21:59
sdaguegot to hit dinner, but will be back on after22:00
notmynamesdague: so i take it you are creating generic dashboards for everyone?22:00
jeblairnotmyname: yeah, we merged the fix this morning but needed a restart to fix it up22:00
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingeprint for bug 1314313  https://review.openstack.org/9111222:00
uvirtbotLaunchpad bug 1314313 in neutron "Firewall fails to become active within 300 seconds" [Critical,New] https://launchpad.net/bugs/131431322:00
funginotmyname: the fix was merged ~13 hours ago in https://review.openstack.org/9087222:01
jeblairsdague: maybe s/main/review-inbox/ ?22:01
notmynamefungi: well that explains why what I found didn't match the DOM I was seeing :-)22:01
funginotmyname: exactly ;)22:01
jeblairnotmyname: yeah, sdague is writing some "zero inbox" style dashboards22:02
notmynamejeblair: nice.22:02
jeblair"Needs final +2", "Passed Jenkins, No Negative Feedback", "Wayward Changes" (basically, changes older than 2d no one has reviewed)22:02
jeblairnotmyname: ^22:02
fungiianw: devstack-f20 nodes are building in nodepool currently. i'll let you know if they succeed or need additional troubleshooting22:03
notmynameif my vote counts at all, my (now broken) bookmarks are: "all swift reviews", "patches waiting to merge", "patches with one core reviewer", "open patches I have reviewed", "stuff I haven't reviewed", "patches with no core reviews"22:03
*** dosaboy has quit IRC22:03
funginotmyname: yeah i too have a query i use for "has at least one +2 and isn't approved yet"22:04
*** dosaboy has joined #openstack-infra22:04
notmynamelooks like some of those are being handled already. the "waiting to merge" is pretty usefil, I think22:04
jeblairnotmyname: maybe add some notes to: https://etherpad.openstack.org/p/gerrit-dashboards  ?22:05
notmynamealso, the wayward changes is good, but 2 days is pretty short.22:05
jeblairnotmyname: (especially if you can paste in your old queries, and double especially if you can fix them :)22:05
mattoliverauMorning everyone22:06
jeblairmattoliverau: good morning22:07
*** gondoi is now known as zz_gondoi22:08
fungijhesketh_: how has turbo-hipster been holding up since yesterday's gerrit upgrade? i think dims had a patch he added a recheck migrations comment on which t-h never seemed to pick up (though now it has a new patchset, so the -1 is gone anyway)22:09
*** dkranz has quit IRC22:09
*** rlandy has quit IRC22:09
*** msabramo has left #openstack-infra22:09
*** dims has joined #openstack-infra22:10
dimsjeblair, pong22:11
dimsjeblair, i was staring at http://i.imgur.com/kBCaZhy.png?1 and filed that review. did not realize it was broken urls :) we should be good22:11
*** nati_uen_ has quit IRC22:12
dimsjeblair, not sure if this other one is a known problem - comments leak to the right - http://i.imgur.com/XoY1tse.png22:12
*** sacharya has quit IRC22:17
jeblairdims: cool, i was hoping that might be it... as for the other one, i think the old version of gerrit did that too -- i think it may have decided to go into "<pre>", but for that comment, i don't know why :/22:18
*** mfer has joined #openstack-infra22:18
*** afazekas has quit IRC22:18
dimsjeblair, thanks22:18
zaroarrg! gerrit line numbers on right margin is only on old screen, new screen has it on left margin. ohh well.22:19
*** mfer has quit IRC22:20
fungizaro: maybe they misunderstood the definition of the word "old"22:23
zarojeblair: ok, new gerrit javamelody plugin build will allow you to hide it's menu.  should be available tommorow on tarballs.o.o.  I believe you will need to manually install it.22:23
*** saschpe has quit IRC22:23
jeblairzaro: plugin installation isn't under puppet?22:24
zarojeblair: no, it's not.  only the installation of gerrit core + core plugins is in puppet22:25
fungiianw: image build is failing. full log is at http://nodepool.openstack.org/image.log but i'll see if i can get you some specifics22:25
mgagneCan I say that I feel lost with the "new" screen? :O22:25
jeblairzaro: can you work on fixing that?  i thought your puppet changes for plugins supported that.22:26
*** saschpe has joined #openstack-infra22:26
jeblairmgagne: you are in good company here.22:27
* mgagne feels better22:27
*** alexpilotti has quit IRC22:27
zarojeblair: yeah, i can take a look but i believe gerrit will require a manual restart at least.22:27
notmynameI have a patch that doesn't seem to have been picked up by zuul: https://review.openstack.org/#/c/90380/22:27
fungimgagne: i'm going to start referring to them as "new screen" and "screen classic"22:28
*** prad has quit IRC22:28
funginotmyname: saw the same on another change from that vintage earlier. add a recheck no bug comment22:28
notmynamefungi: ok, thanks22:29
notmynamefungi: that did it. thanks22:29
*** etoews_ has quit IRC22:29
funginotmyname: one of our restarts interrupted things between when jenkins cleared its vote and would have reported, so it has no verify vote at all (causing the approval trigger not to fire)22:29
*** zigo_ has joined #openstack-infra22:35
*** persia__ has joined #openstack-infra22:35
*** markvoelker has quit IRC22:37
*** mriedem has quit IRC22:37
*** bhuvan has quit IRC22:37
*** zigo has quit IRC22:37
*** greghaynes has quit IRC22:37
*** andreykurilin_ has joined #openstack-infra22:37
*** jgrimm has quit IRC22:37
*** nati_ueno has joined #openstack-infra22:37
*** bhuvan has joined #openstack-infra22:37
*** mriedem has joined #openstack-infra22:37
*** mark0n has quit IRC22:37
openstackgerritMarc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Pass argv to main  https://review.openstack.org/9054222:37
*** changbl has quit IRC22:37
*** nati_ueno is now known as 7JTAAM1ZL22:37
*** nati_ueno has joined #openstack-infra22:37
*** eharney has quit IRC22:37
*** _dguerri has quit IRC22:37
*** bhuvan has quit IRC22:37
*** bhuvan has joined #openstack-infra22:37
jeremybfifieldt: i think you were looking for someone else22:37
*** persia has quit IRC22:37
*** thedodd has quit IRC22:37
*** greghaynes_ has joined #openstack-infra22:37
*** proffalken has quit IRC22:37
*** Shrews has quit IRC22:37
*** 7JTAAM1ZL has quit IRC22:38
fungijeremyb: fifieldt: maybe me?22:38
*** kiall has quit IRC22:38
*** doug-fish has left #openstack-infra22:38
jeremybfungi: ohai jeremys :)22:38
fungi(fifieldt was /msg'ing me earlier, though i expect he's asleep now)22:39
fungiohai other jeremy ;)22:39
*** BadCub has quit IRC22:39
jeremybfungi: 29 09:45:22 < fifieldt> jebl air, mor dred, jeremyb - had a spammer in #openstack, probably needs a ban22:39
fungithere are far too many of us, which is why i'm fungi instead22:40
jeremybi find the jeremy space isn't too bad. but too many jonathan, david, michael, rachel22:40
*** mriedem has quit IRC22:43
*** e0ne has quit IRC22:46
*** e0ne has joined #openstack-infra22:46
*** mriedem has joined #openstack-infra22:57
*** flaper87 is now known as flaper87|afk22:57
adam_gwhats the easiest way to get results of a specific jenkins job in aggregate across multiple project check pipelines?22:58
*** signed8bit has quit IRC22:58
*** david-lyle has quit IRC22:59
*** greghaynes_ is now known as greghaynes22:59
*** jhesketh_ is now known as jhesketh22:59
fungiadam_g: can you be a little more specific about what it is you're wanting? a better use case description might help23:00
*** mwagner_lap has joined #openstack-infra23:01
*** etoews has joined #openstack-infra23:02
adam_gfungi, pass/fail over the last N hours for job $foo, which is configured to run as part of the check pipeline for multiple projects23:02
fungiadam_g: aha, you want statistics on job result states23:03
fungiadam_g: have you had a poke around in http://graphite.openstack.org/ at all?23:03
adam_gfungi,  not yet. was wondering where the best place to start was,  will take a look there now23:04
*** e0ne has quit IRC23:05
*** e0ne has joined #openstack-infra23:06
*** yamahata has quit IRC23:07
*** markmc has quit IRC23:07
*** e0ne has quit IRC23:08
fungiadam_g: http://graphite.openstack.org/render/?width=586&height=308&_salt=1398812938.615&target=stats_counts.zuul.pipeline.check.job.check-tempest-dsvm-full.SUCCESS&target=stats_counts.zuul.pipeline.check.job.check-tempest-dsvm-full.FAILURE23:09
fungiadam_g: something like that?23:09
*** derekh has joined #openstack-infra23:10
adam_gfungi, yeah. reading up on graphite now :)23:12
*** dkehn is now known as dkehn_afk23:14
openstackgerritDerek Higgins proposed a change to openstack-infra/config: Skip running update-grub if it doesn't exist  https://review.openstack.org/9118823:14
derekhfungi: thanks for merging the F20 nodes, looks like they are failing to build at the very end in restrict_memory,sh23:14
derekhfungi: so how would you feel about skipping it until I find a proper alternative ? ^23:15
fungiderekh: yeah, i see where it's mainly failing trying to run update-grub in restrict_memory.sh23:16
fungiderekh: ideas on how to adjust the kernel command line in f20?23:16
*** andreykurilin_ has quit IRC23:17
fungiderekh: if you can just work out how to pass mem=8G to the kernel at boot, we can solve it the right way23:17
derekhfungi: ok, I'm trying to figure out how that image got the legacy grub config file in the first place, the one I am testing on has /boot/grub/grub.conf23:18
fungiderekh: hmm... perhaps it has both?23:18
*** alexpilotti has joined #openstack-infra23:19
fungian old (but unused) /etc/default/grub file and also a /boot/grub/grub.conf?23:19
fungiif so, then maybe merely swapping the order in that conditional would suffice23:19
fungii assume /etc/default/grub is merely being used here to directly modify grub 1.x /boot/grub/menu.lst files23:20
derekhfungi: perhaps, its not present on the default fedora cloud image, I'll see if I can figure out where it is coming from23:21
derekhfungi: hopy will have a new patch in a bit23:21
fungiderekh: okay, cool. i'm semi-around for a few more hours to give it a spin if so23:22
derekhfungi: ok, thanks23:22
*** bknudson has quit IRC23:23
*** melwitt has quit IRC23:23
*** zns has quit IRC23:23
*** manishg has quit IRC23:25
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Initial commit  https://review.openstack.org/9119123:27
* jeblair is amused to note that gertty is unabled to display the diff at all for thet because of https://storyboard.openstack.org/#!/story/7023:29
jeblairmordred, fungi: https://review.openstack.org/gitweb?p=stackforge/gertty.git;a=blob;f=project.config;hb=36b9e169bc6451baaf1ce57983c199e421d7363d23:30
*** hogepodge has quit IRC23:30
jeblairmordred, fungi: i don't think m-p did the right thing there23:30
jeblairinfra-manual worked, however23:31
*** ArxCruz has quit IRC23:31
jeblairGithubException: 403 {u'documentation_url': u'https://developer.github.com/v3/#rate-limiting', u'message': u'API rate limit exceeded for openstack-project-creator.'}23:31
*** ArxCruz_ has joined #openstack-infra23:31
jeblairalso ERROR:manage_projects:Failed to add groups file for project: stackforge/gertty23:32
jeblairmordred: that seems really really familiar ^23:32
harlowjajeblair for https://github.com/stackforge/gertty have u tried https://pypi.python.org/pypi/gerrit-view :)23:34
harlowjai saw gertty pop up :-P23:36
jeblairharlowja: gertty is to be a complete replacement for the gerrit ui, that supports an email/news reading like workflow, which i think is a little different from cgerrit... however --23:36
*** amcrn has quit IRC23:36
*** mrodden has quit IRC23:36
jeblairharlowja: i was thinking of you when i started it, and it uses urwid.23:36
jeblairharlowja: i'm hoping you'll find it interesting.  :)23:37
harlowjacould just suck over cgerrit, i'm fine with that :-P23:37
harlowjathen have 1 repo for all23:37
harlowjaor just all of https://pypi.python.org/pypi/gerrit-view23:37
harlowjai'm fine with that to :)23:38
harlowjau have my permission ;)23:38
*** matsuhashi has joined #openstack-infra23:38
*** sweston has quit IRC23:39
*** wenlock has quit IRC23:39
jeblairharlowja: cool; worth thinking about -- it's my first urwid program, and i had a lot of things i needed to explore from scratch to start with (the new rest api, offline syncing, and just learning to use urwid)23:39
harlowjajeblair ya, urwid is a tough one to figure out :)23:39
harlowjaespecially customizing it, lol23:39
jeblairtell me about it :)  i'm just now understanding how to make not-insane reusable widgets23:40
harlowjaya, i'm not still that good at it :-P23:40
jeblairharlowja: but that commit has all the fundamentals working23:40
jeblairif i can figure out the manage-projects issue, i should be able to land it with gertty23:41
openstackgerritKhai Do proposed a change to openstack-infra/config: deploy gerrit plugins into folder with plugin name  https://review.openstack.org/9119223:41
openstackgerritKhai Do proposed a change to openstack-infra/config: puppetize installation of gerrit third party plugins  https://review.openstack.org/9119323:41
zarojeblair: ^23:41
jeblairzaro: cool, thanks23:41
harlowjajeblair checking it out23:41
harlowjawill try to instill what i learned about urwid :-P23:42
jeblairharlowja: that would be awesome23:42
*** jooools has quit IRC23:43
jeblairDEBUG:manage_projects:Executing command: git --git-dir=/opt/lib/jeepyb/stackforge/gertty/.git --work-tree=/opt/lib/jeepyb/stackforge/gertty checkout -B master origin/master23:43
jeblairINFO:manage_projects:Command said: fatal: git checkout: updating paths is incompatible with switching branches.23:43
jeblairmordred: ^23:43
*** etoews has quit IRC23:44
*** kiall has joined #openstack-infra23:47
*** pdmars has quit IRC23:50
jeblairhrm, /home/gerrit2/acls/stackforge/gertty.config is empty23:50
*** mrodden has joined #openstack-infra23:50
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Some project ui/ux updates  https://review.openstack.org/9119623:51
*** persia__ is now known as persia23:51
*** persia has quit IRC23:51
*** persia has joined #openstack-infra23:51
jeblairfungi: ping23:52
*** etoews has joined #openstack-infra23:52
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Rename gertty ACL file to correct name  https://review.openstack.org/9119723:52
fungijeblair: heya23:53
jeblairfungi: i'm going to self-approve that23:53
jeblairfungi: oh, cool you're here23:53
*** nati_uen_ has joined #openstack-infra23:53
jeblairfungi: maybe you could approve it then :)23:53
fungiapproved, yes23:54
fungithat explains the odd error, for sure23:55
openstackgerritKhai Do proposed a change to openstack-infra/config: deploy gerrit plugins into folder with plugin name  https://review.openstack.org/9119223:55
openstackgerritKhai Do proposed a change to openstack-infra/config: puppetize installation of gerrit third party plugins  https://review.openstack.org/9119323:55
*** dguerri has joined #openstack-infra23:57
*** nati_ueno has quit IRC23:57
*** proffalken has joined #openstack-infra23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!