Tuesday, 2014-10-21

*** gokrokve_ has quit IRC00:01
*** msabramo has quit IRC00:03
*** todd_dsm has quit IRC00:03
*** xchu has joined #openstack-infra00:04
*** unicell has joined #openstack-infra00:04
*** xchu has quit IRC00:04
*** kirscht has joined #openstack-infra00:04
*** kgiusti has left #openstack-infra00:04
*** kgiusti has quit IRC00:04
*** zaphod42 has quit IRC00:05
*** zaphod42 has joined #openstack-infra00:05
*** zigo has quit IRC00:07
*** zaphod42 has quit IRC00:10
mordredanteaya: you _don't_ need auth to do anythign with it, actually00:11
mordredanteaya: you can totally access resources marked as available for system:anyuser00:11
anteayathe documentation talked about a puppet password00:11
anteayaI don't yet understand how that fits in with the flow00:12
anteayabut I'll work on figuring out the pieces00:12
mordredin fact, right now, I'm pretty sure if you did "sudo apt-get install openafs-client ; ls /afs/openstack.org/user" you'd see three files ... but there would be no use or purpose in you doing that00:12
*** david-lyle has quit IRC00:12
mordredanteaya: so, practically, yeah, there isn't much point in people installing the client broadly00:12
*** alexandra has joined #openstack-infra00:12
mordredso I don't expect much of it00:13
*** todd_dsm has joined #openstack-infra00:13
*** annegent_ has quit IRC00:13
*** emagana_ has quit IRC00:13
*** zigo has joined #openstack-infra00:13
mordredanteaya: I mention that mainly for the purposes of sharing information :)00:14
*** emagana has joined #openstack-infra00:14
*** asettle has quit IRC00:15
anteayathank you00:15
anteayai appreciate the shared information00:15
openstackgerritMichael Tupitsyn proposed a change to openstack-infra/jenkins-job-builder: Support additional events for HipChat plugin  https://review.openstack.org/12976000:15
*** david-lyle has joined #openstack-infra00:15
*** emagana has quit IRC00:18
*** dimsum_ has joined #openstack-infra00:21
*** Ryan_Lane has quit IRC00:22
*** weshay has joined #openstack-infra00:22
openstackgerritAngus Lees proposed a change to openstack-infra/project-config: Promote gate-neutron-pylint check to voting  https://review.openstack.org/12976200:25
*** Ryan_Lane has joined #openstack-infra00:25
*** Ryan_Lane has quit IRC00:27
*** tonytan4ever has quit IRC00:27
*** annegent_ has joined #openstack-infra00:29
*** annegent_ has quit IRC00:30
*** bdpayne has quit IRC00:31
*** nelsnels_ has joined #openstack-infra00:36
*** andreaf has quit IRC00:37
*** nelsnelson has quit IRC00:38
*** andreaf has joined #openstack-infra00:38
*** Ryan_Lane has joined #openstack-infra00:40
*** gokrokve has joined #openstack-infra00:42
*** markvan_ has quit IRC00:43
*** markvan has joined #openstack-infra00:44
asselinnibalizer, jesusaurus what do you think of this? https://github.com/rasselin/puppet-jenkins00:44
*** baoli has joined #openstack-infra00:45
*** jerryz has joined #openstack-infra00:48
*** dangers is now known as dangers_away00:49
openstackgerritChristopher Yeoh proposed a change to openstack-infra/project-config: Add API Working Group project  https://review.openstack.org/12846600:49
*** mmedvede has quit IRC00:51
*** aysyd has quit IRC00:51
*** hdd has quit IRC00:57
*** arnaud has quit IRC00:57
*** arnaud__ has quit IRC00:57
*** alexandra is now known as asettle00:57
*** mmaglana has quit IRC00:57
*** patrickeast has quit IRC01:00
*** sdake_ has joined #openstack-infra01:01
*** gokrokve has quit IRC01:02
*** r1chardj0n3s_afk is now known as r1chardj0n3s01:02
*** gokrokve has joined #openstack-infra01:03
cyeohanyone know if something weird is going on with jenkins? I had a bunch of changesets fail jenkins with merge conflict errors this morning, but when I got to rebase them there is no actual merge conflict. Its been around 5 or 6 patches across different projects01:03
*** amotoki has joined #openstack-infra01:04
*** yamahata has joined #openstack-infra01:04
anteayacyeoh: hi chris01:06
anteayawe were doing work01:06
cyeohanteaya: hi!01:06
anteayaand the mass rebase happened as a result01:06
anteayajust a recheck should do it01:06
anteayawe tried to get all affected patches but might have missed some01:07
anteayasorry not a mass rebase01:07
*** msabramo has joined #openstack-infra01:07
cyeohanteaya: ok, thanks!01:07
anteayaa mass -1 with a merge conflict message01:07
anteayawith no actual merge conflict01:07
*** gokrokve has quit IRC01:07
anteayacyeoh: how are you doing?01:07
anteayabeen a long time since we chatted01:07
anteayaany idea when you can travel again?01:07
cyeohanteaya: am doing pretty well! the new chemo is much easier to handle so its generally only a few days after that I'm affected really badly. And I only have it every 3 weeks.01:08
anteayaoh thank goodness01:08
cyeohso I'm going to try to get to the nova mid-cycle - was a bit too late to organise for Paris01:09
anteayaI'm glad you have found something taht is gentler on your system01:09
anteayaoh yay!!01:09
cyeohand given the distance will be bringing along Kelly just in case.01:09
anteayathat would inspire me to try to go01:09
anteayagood idea01:09
*** yaguang has joined #openstack-infra01:10
cyeohit'll be good to be at a face to face meeting again. Has been too long!01:10
anteayawould you bring your daughter too?01:10
anteayaor do you have a grandmother looking forward to taking her?01:10
anteayacyeoh: no kidding01:10
anteayaI can't speak for everyone but *I've* been missing you01:11
cyeohAlyssa would be coming too :-) Would probably have to do a stop over at disneyland ;-)01:11
anteayagood idea01:11
anteayathat sound awesome01:11
cyeohanteaya: thanks - I've really missed not being able to attend the summits and mid-cycle01:11
anteayaman I feel ya01:11
*** sdake_ has quit IRC01:11
anteayaso glad you have some good things happening now01:11
anteayaand the nova mid-cycle to look forward to01:12
cyeohyea its good :-) And Kira is having kittens!01:12
* anteaya hopes Kira is a cat01:12
cyeohour Bengal kitten01:12
anteayaoh nice01:12
anteayawhen are you expecting the sharp little pockets of fuzzy love?01:13
cyeohthe kittens belong to the original breeder (its the deal that was made when we got her), but if there is a snow bengal we'll probably keep one01:13
*** yaguang has quit IRC01:13
anteayawhat is a snow bengal?01:13
cyeohnot sure, but probably 5-6 weeks away.01:13
* anteaya hopes for pictures01:13
*** weshay has quit IRC01:13
*** yaguang has joined #openstack-infra01:13
anteayaI hope she delivers safely01:13
*** todd_dsm has quit IRC01:13
cyeohso this is what they can look like: http://www.google.com.au/imgres?imgurl=http%3A%2F%2Fwww.snowbengalkittensforsale.com%2Fimages%2FslpSide.jpg&imgrefurl=http%3A%2F%2Fwww.snowbengalkittensforsale.com%2F&h=300&w=400&tbnid=SbzJLk5iy2MEkM%3A&zoom=1&docid=yiVr2cuzIDhLdM&ei=TrNFVKyqGcHDmQWKxYG4AQ&tbm=isch&ved=0CCUQMygEMAQ&iact=rc&uact=3&dur=3108&page=1&start=0&ndsp=3001:14
cyeohKira is a brown bengal - https://pbs.twimg.com/media/B0bNrahCIAAGqSz.jpg:large - that's her with her pregnant belly01:15
anteayashe looks like she is doing pretty well01:16
*** mfer has joined #openstack-infra01:16
anteayashe still has 6 weeks to go though eh?01:16
cyeohyep. So she should get quite a bit bigger. Hasn't stopped her from hunting the other cats at night yet though01:17
*** otter768 has joined #openstack-infra01:18
anteayabet the other cats are really happy about that01:20
StevenKThey might be able to hear her coming up on them, then ...01:21
cyeohheh yea :-) And it does slow her down a bit. The other two cats are old grumpy things and just want to sleep though while she just wants to play all the time01:21
*** Longgeek has joined #openstack-infra01:21
anteayathe swish of the big belly01:21
anteayathose young cats01:22
anteayaso playful01:22
cyeohyea and Bengals remain very playful for years.01:22
*** dane_leblanc has joined #openstack-infra01:23
anteayacyeoh: yeah so I'm -1 on the api-wg repo mostly since it doesn't fit the structure of the creation of other openstack/ repos01:23
anteayaand the tc hasn't had a chance to chat about it01:23
anteayasince not everyone can create an openstack/ repo01:23
anteayawhat do you envision going into the repo?01:23
cyeohanteaya: yea that's fine - I agree we need the tc to approve it first01:23
anteayadocs mostly?01:23
anteayacyeoh: thanks01:23
anteayaactually it appears there are more repos taht fit this model01:24
anteayaso this is just one more example to demonstrate the conversation needs to happen01:24
*** radez_g0n3 is now known as radez01:24
cyeohanteaya: yes, only documentation - process for how the API WG works and and then the API guidelines themsleves01:24
*** rushiagr_away is now known as rushiagr01:24
anteayacyeoh: cool01:24
anteayaso we will let the tc have a chew01:24
anteayaand then we will have a process *hopefully*01:24
anteayaso I can know how to address more patches of this nature01:25
cyeohyep, I think Thierry said they would talk about it at the next TC meeting01:25
anteayaso hopefully by your tomorrow we can have some sort of guidance01:25
anteayaand go from there01:25
cyeohyep! I'm getting called to go to lunch so I'd better head off. Been great talking to you again :-)01:25
anteayasame here01:26
anteayamiss you01:26
anteayaglad to see you around01:26
anteayaenjoy lunch01:26
anteayaand say hi to kelly when you get a chance01:26
*** rushiagr is now known as rushiagr_away01:26
cyeohwill do - good to be around more again :-)01:26
*** andreaf has quit IRC01:27
*** Longgeek has quit IRC01:27
*** andreaf has joined #openstack-infra01:27
*** todd_dsm has joined #openstack-infra01:28
*** dane_leblanc has quit IRC01:29
*** gokrokve has joined #openstack-infra01:29
*** dane_leblanc has joined #openstack-infra01:30
openstackgerritRamy Asselin proposed a change to openstack-infra/infra-specs: Update puppet module git subtree instructions  https://review.openstack.org/12976801:30
*** gokrokve has quit IRC01:31
*** gokrokve has joined #openstack-infra01:31
asselinnibalizer, jesusaurus ^^ I updated the spec with the steps I took to do the git subtree here: https://github.com/rasselin/puppet-jenkins01:32
anteayaasselin: why the move to # instead of numbers?01:33
asselinb/c it's easier to add steps :)01:33
asselinthey get generated with the actual 1,2,301:34
anteayaoh do they01:34
* anteaya waits for the check docs to return01:34
anteayaall this .rst magic that I don't know01:34
asselinyes, i tested it locally01:34
asselinI'm learning it too01:34
anteayayou are doing a good job01:35
anteayakeep it up01:35
asselingoing for dinner. see you tomorrow anteaya01:35
anteayaasselin: enjoy dinner, thanks for your hard work01:36
jeblaircyeoh: oh hi! :)  sorry about the -1s (my fault); the rechecks for all the errors will still probably take another 18 hours from now (it's a queue, so 0-18 hours for any given change)01:36
anteayasee you tomorrow01:36
jeblairanteaya: that's some rough math based on the zuul job queue graph ^01:37
anteayajeblair: thanks01:40
*** otter768 has quit IRC01:40
*** sdake_ has joined #openstack-infra01:40
anteayaI hadn't looked before01:40
anteayabut the line just goes straight up01:41
anteayawith 607 still in check01:41
anteayaso yeah, having the 18 hour guesstimate is helpful for answering tomorrow's questions too01:41
*** otter768 has joined #openstack-infra01:42
*** SumitNaiksatam has joined #openstack-infra01:42
jeblairclarkb, fungi, mordred, anteaya: http://paste.openstack.org/show/122421/01:45
jeblairthat's an afs client install/config on a new precise vm01:45
mordredjeblair: ++01:46
jeblairoh, and this is the coolest thing about afs: http://paste.openstack.org/show/122422/01:46
mordredjeblair: we could pre-seed the two questions01:46
jeblairmordred: well, our puppet config will answer them; i mostly documented it for folks who want to install the client locally01:47
*** chianingwang has joined #openstack-infra01:47
mordredjeblair: sure. I'm just saing I think we could write a 3 line non-interactive shell script :)01:47
jeblairmordred: oh, ok cool.01:47
jeblairmordred: wasn't sure how hard that was :)01:47
clarkbyou should use expect01:48
jeblairfungi, clarkb, mordred, SergeyLukjanov: so afstest.openstack.org exists and has the client running if you want to log in and poke around01:48
clarkb>_> <_> <_<01:48
*** hdd has joined #openstack-infra01:48
anteayajeblair: awesome01:48
jeblairi figure we can do some tests with it tomorrow.  i think now i will eat some brunswick stew.01:49
krotscheckUrm… ok, so when mordred said “AFS == Andrew File System” he was actually telling the truth?01:49
clarkbjeblair: that sounds good01:49
clarkbkrotscheck: yes01:49
* krotscheck never really knows when to take him seriously.01:49
*** SumitNaiksatam has quit IRC01:49
krotscheckTalk about a blast from the past for me. Go go Alma Mater :)01:50
jeblairkrotscheck: it's named after BOTH andrews! :)01:50
mordredjeblair: did I mention krotscheck went to CMU?01:50
mordredjeblair: http://paste.openstack.org/show/12242401:50
mordredjeblair: I believe that's what you need to do01:50
jeblairmordred: i don't recall hearing that, but he just told me :)01:51
*** esker has joined #openstack-infra01:51
mordredfor the preseed stuff01:51
jeblaircmu is awesome01:51
jeblairafs _and_ self-driving cars01:51
mordredkrotscheck: jeblair and I went to NCSU, which was also an AFS place01:51
clarkbmordred: that doesn't use expect01:51
mordredclarkb: no need. turns out this stuff just works :)01:51
krotscheckjeblair: I loved it so much I stayed there for FIVE YEARS!01:52
mordredclarkb: in fact, I KEEP meaning to make more of our puppet stuff just drive debconf where it's possible01:52
mordredclarkb: but then I don't01:52
clarkbmordred: I mean debconf is cool but expect is so much more 1990s like AFS01:52
mordredclarkb: you're trying to troll, by making fun of the age of AFS01:54
mordredclarkb: I see what you're doing there01:54
anteayaasselin: the split repo appears to have all the files and directories I would expect01:54
anteayaasselin: good job01:54
jeblairoops, forgot 'apt-get install krb5-user'01:54
jeblairjust ran that01:54
mordredjeblair: oops :)01:54
jeblairthis, however, is not 1990s: http://paste.openstack.org/show/122426/01:55
*** VijayTripathi has joined #openstack-infra01:55
mordredjeblair: thats so awesome01:55
jeblairfungi: this is the first afs cell i've used with an aes token!01:55
jeblairokay, brunswick stew for realz01:56
clarkbmordred: yes sorry :P I figured you would enjoy that slight troll01:56
mordredclarkb: I did01:56
anteayatroll lite01:56
mordredclarkb: you should test my shell script snipped by running it as root on your laptop01:56
*** VijayTripathi has quit IRC01:57
mordredI'd test it - but I have afs already installed and working01:57
mordredON A PLANE01:57
clarkbprobably not tonight. I am about to find dinner then find something other than work to do01:57
mordredyou should do that01:57
clarkbdon't I need to add my principal somewhere too?01:57
mordredI mean01:58
mordredyou don't need that to install AFS or verify you can see the openstack stuff01:58
dstufftclarkb: lifeless wat01:58
clarkboh do I get read only in that case?01:58
dstufftI got highlights but too old to see them01:58
mordredwe've got /afs/openstack.org set rl for system:anyuser01:58
clarkbdstufft: so get-pip.py if you have two indexes in your pip.conf or passed via -i01:58
lifelessdstufft: get-pip.py failing when the main index doesn't contain pip but secondary indices do.01:58
clarkbdstufft: and the first one is a dead index then the secondary index is completely unused01:58
clarkband pip fails01:58
lifelessclarkb: its not a dead index01:59
dstufftwell you can only pass one index via -i01:59
dstufft-i replaces the default index01:59
dstufftit's not additive01:59
lifelessdstufft: oh so my test was broken.01:59
dstufft--extra-index-url adds an extra index01:59
lifelessdstufft: but still - we have two indexes, one local.01:59
clarkblifeless: in greghaynes' case there was no file there01:59
dstufftor --find-links adds a flat directory01:59
clarkblifeless: index was file:///tmp/pypi but that dir did not exist02:00
dstufftlifeless: can you tell me exactly what you're invoking02:00
clarkbat least that is how I parsed02:00
lifelessclarkb: http://paste.openstack.org/show/122432/ does indeed behave differently02:01
lifelessdstufft: ^02:01
dstufftfile://$(pwd)/foo is nont existent yes?02:01
*** mfer has quit IRC02:02
lifelessdstufft: yes02:02
lifelessdstufft: but I just tried02:02
dstufftso yea, ``pip install testtools -i file://$(pwd)/foo`` is setting the primary index to file://$(pwd)/foo, overiding the default or any config file/env var02:02
lifelessdstufft: http://paste.openstack.org/show/122434/02:02
*** nosnos has joined #openstack-infra02:02
lifelessdstufft: this is an artificial test02:02
lifelessdstufft: greghaynes has more detail02:03
*** alexandra has joined #openstack-infra02:04
*** todd_dsm has quit IRC02:04
dstufftI don't think I understand the problem02:05
dstufftI would expect pip install testtools -i file://$(pwd)/foo to fail02:05
lifelessdstufft: yes, but the second pip invocation fails too02:05
*** Ryan_Lane has quit IRC02:05
*** mfer has joined #openstack-infra02:06
crinklethere seem to be a lot of stuck jobs in zuul.openstack.org, is that related to the mass -1's that I see mentioned in scrollback?02:06
mordredjeblair: http://paste.openstack.org/show/122433 does the krb5-config default realm t02:06
dstufftlifeless: the second one probably fails becasue the URL is wrong02:06
dstufftlifeless: try https://pypi.python.org/simple/02:06
lifelessdstufft: its not that, since I had -i there before and it works02:07
*** asettle has quit IRC02:07
lifelessdstufft: confirmed - http://paste.openstack.org/show/122435/02:07
clarkbcrinkle: yup, zuul is currently chewing through ~500 changes to update those -1s02:08
dstufftlifeless: name's still wrong02:08
dstufftimple != simple02:08
clarkbcrinkle: and we are basically maxed out on the number of nodes we have right now02:08
lifelessthat snuck in somewhere02:08
crinkleokie i'll wait, thanks clarkb02:08
lifelessdstufft: http://paste.openstack.org/show/122436/ still fails02:08
dstufftlifeless: what's the error?02:08
lifelessOSError: [Errno 2] No such file or directory: '/home/robertc/personal/testrepository/foo/testtools/'02:09
lifelessdstufft: it appears to be not consulting the extra index02:09
lifelessdstufft: its possible this is the file:// urls don't raise 404 that tchaypo fixed - has that actually gotten into a release?02:09
*** koolhead17 has joined #openstack-infra02:09
dstufftI see02:10
dstufftyea that's what it is02:10
dstufftand no it hasn't02:10
tchaypoyo ho ho02:10
dstufftpip install --upgrade git+https://github.com/pypa/pip.git@develop02:10
dstufftfixes it02:10
lifelessdstufft: please sir can I have the fix released please please please02:10
lifelessdstufft: please02:10
tchaypoand introduces 3 other bugs :)02:10
dstufftthere are new bugs with the develop branch?02:10
lifelessdstufft: tchaypo was teasing.02:11
tchaypo(I have no knowledge of specific bugs, I'm just teasing)02:11
tchayposorry dstufft02:11
lifelessdstufft: So seriously, what blocks a release of pip ?02:11
dstufftI mean it wouldn't surprise me02:11
lifelessdstufft: its what 6 months ago we fixed this?02:11
dstufftlifeless: I wanted to get PEP 440 into the next release is the primary blocker, let me see how much more I need to do on that and see if it makes sense to just pushj that off into the next next release02:12
* greghaynes reads backscroll02:16
greghaynesSo, using -i multiple times wasnt the actual issue, the issue is when we specify multiple indexes in .pip/pip.conf02:17
lifelessgreghaynes: no02:17
lifelessgreghaynes: the issue is that the index is a file index02:17
lifelessgreghaynes: not an http index02:17
lifelessgreghaynes: we either need a pip release (easy, if we can convince dstufft to do it, like today)02:18
greghaynesoh? so that works over http?02:18
lifelessgreghaynes: or we need to export the local index over http02:18
tchaypogreghaynes: when the index is http://, it catches a 404 and tries variant spellings02:18
greghayneswell, devpi solves this :)02:18
lifelessgreghaynes: get-pip might be broken some other way, but pip itself is broken02:18
tchaypogreghaynes: when the index is file:// it just does sys.exit() instead02:18
dstufftthe problem is when we switched to using requests I didn't catch all the exceptions we needed02:18
dstufftwhen I implemented file:// support02:18
dstufftso blame me02:18
dstufftI'm bad at software what can I say02:19
*** bdpayne has joined #openstack-infra02:19
tchaypogreghaynes: my patch catches the exception and creates a fake 404 that then bubbles up and triggers the try-harder code02:19
greghaynesI have never seen a bug around not catching all the exceptions from requests ;)02:19
lifelessdstufft: well, file:/// should be in requests, no ?02:19
greghaynes(not really)02:19
dstufftlifeless: no, requests is a HTTP library not a access random files on your filesystem library02:19
lifelessdstufft: blah :)02:20
dstufftso I wrote a connection adapter that mounts a thing that adds that particular thing to requests for file:// urls02:20
dstufftI just didn't write it good enough02:20
*** VijayTripathi has joined #openstack-infra02:21
dstufftA full release isn't going to hapen today, I might be able to cut a RC once I take a minute to look at the current state of things02:21
greghaynesyep, so sounds like I should poke at devpi for now02:21
greghayneswhich I was considering anyways02:21
*** signed8bit_ZZZzz is now known as signed8bit02:21
*** signed8b_ has joined #openstack-infra02:23
dstufftor ``twistd -n web --path $(pwd)/foo``02:23
dstufftora ny other web server02:23
lifelessSimpleServer can do it02:23
dstufftdoes simpleserver create index.html's automatically?02:23
dstufftI can't remember02:23
dstufftthen just use that if you want02:24
greghaynesyes, we just already have some echo "blah" > index.html so I figure if were going to also run a http server why not use the tool that does all of this02:24
greghaynesoh, wat02:24
greghaynesit does auto index02:24
clarkbdoes for me02:24
*** stevemar has quit IRC02:25
greghaynesok, SGTM :)02:25
dstufftI hate the simple protocol, but the one really good thing about it02:25
*** stevemar has joined #openstack-infra02:25
dstufftit makes it really easy to stand up a custom repo02:25
*** signed8bit has quit IRC02:26
*** sdake_ has quit IRC02:32
*** signed8b_ has quit IRC02:34
*** pcm_ has quit IRC02:35
*** salv-orlando has quit IRC02:40
*** Ryan_Lane has joined #openstack-infra02:42
*** david-lyle_ has joined #openstack-infra02:42
*** david-lyle has quit IRC02:44
*** yaguang has quit IRC02:47
*** yaguang has joined #openstack-infra02:47
*** dimsum_ has quit IRC02:58
*** dimsum_ has joined #openstack-infra02:59
*** otherwiseguy has joined #openstack-infra03:00
*** radez is now known as radez_g0n303:01
*** amotoki_ has joined #openstack-infra03:01
*** dimsum_ has quit IRC03:03
*** mmaglana has joined #openstack-infra03:03
openstackgerritDavid Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Fix command to build docs in installation documentation  https://review.openstack.org/12949803:04
openstackgerritDavid Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Ignore testr_subunit_log  https://review.openstack.org/12950303:04
*** dane_leblanc has quit IRC03:06
*** sdake_ has joined #openstack-infra03:06
*** mfer has quit IRC03:07
*** VijayTripathi has quit IRC03:07
*** amotoki has quit IRC03:09
*** amotoki_ is now known as amotoki03:10
*** Longgeek has joined #openstack-infra03:10
*** yjiang5 has quit IRC03:13
*** r1chardj0n3s is now known as r1chardj0n3s_afk03:13
*** KanagarajM has joined #openstack-infra03:14
*** yjiang5 has joined #openstack-infra03:15
*** Longgeek has quit IRC03:15
*** greghaynes has quit IRC03:15
*** greghaynes has joined #openstack-infra03:18
*** ivar-laz_ has joined #openstack-infra03:21
*** otter768 has quit IRC03:22
*** otter768 has joined #openstack-infra03:23
*** pcrews has quit IRC03:23
openstackgerritDavid Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Add support for "Override Build Parameters" option in EnvInject  https://review.openstack.org/12918903:24
openstackgerritDavid Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Add tests for properties inject  https://review.openstack.org/12950203:24
*** esker has quit IRC03:25
*** ivar-lazzaro has quit IRC03:25
*** ivar-laz_ has quit IRC03:26
*** nosnos has quit IRC03:26
*** bdpayne has quit IRC03:27
*** nosnos has joined #openstack-infra03:27
*** yaguang has quit IRC03:28
*** VijayTripathi has joined #openstack-infra03:29
*** armax has quit IRC03:30
*** nosnos has quit IRC03:32
*** otter768 has quit IRC03:33
*** ayoung has quit IRC03:35
*** ChuckC has quit IRC03:35
*** spzala has quit IRC03:36
*** koolhead17 has quit IRC03:40
*** mmaglana has quit IRC03:42
*** mmaglana has joined #openstack-infra03:42
*** hdd has quit IRC03:53
*** r1chardj0n3s_afk is now known as r1chardj0n3s03:53
*** gokrokve_ has joined #openstack-infra04:02
*** jamespd has quit IRC04:04
*** jamespd has joined #openstack-infra04:04
*** bradjones has quit IRC04:05
*** alexandra is now known as asettle04:05
*** gokrokve has quit IRC04:06
*** gokrokve_ has quit IRC04:07
*** bradjones has joined #openstack-infra04:07
*** garyh has quit IRC04:09
*** nosnos has joined #openstack-infra04:16
*** emagana has joined #openstack-infra04:18
openstackgerritIan Wienand proposed a change to openstack-infra/system-config: Add HP Cloud Centos7 images  https://review.openstack.org/12849804:29
*** mbacchi has quit IRC04:32
*** otherwiseguy has quit IRC04:40
*** garyh has joined #openstack-infra04:40
*** timrc is now known as timrc-afk04:49
*** changbl has joined #openstack-infra04:49
*** armax has joined #openstack-infra04:50
*** jerryz has quit IRC04:54
*** jerryz has joined #openstack-infra05:01
*** wendar_ is now known as wendar05:03
*** yfried has joined #openstack-infra05:16
*** chandankumar has joined #openstack-infra05:18
*** yfried is now known as yfried|afk05:24
*** gokrokve has joined #openstack-infra05:25
*** yfried has joined #openstack-infra05:27
*** yfried|afk has quit IRC05:29
*** emagana has quit IRC05:29
*** emagana has joined #openstack-infra05:30
*** Sukhdev has joined #openstack-infra05:30
*** harlowja is now known as harlowja_away05:30
*** otherwiseguy has joined #openstack-infra05:31
*** yfried_ has joined #openstack-infra05:32
*** yfried_ is now known as yfried|afk05:32
*** yfried has quit IRC05:32
*** gokrokve has quit IRC05:34
*** gokrokve has joined #openstack-infra05:34
*** emagana has quit IRC05:34
*** ildikov has joined #openstack-infra05:37
*** BharatK has joined #openstack-infra05:38
*** Longgeek has joined #openstack-infra05:41
*** Longgeek has quit IRC05:47
*** reed has quit IRC05:47
*** david-lyle_ has quit IRC05:48
*** k4n0 has joined #openstack-infra05:50
*** ryanpetrello has quit IRC05:55
*** ryanpetrello has joined #openstack-infra05:55
*** gokrokve has quit IRC05:57
*** gokrokve has joined #openstack-infra05:58
*** andreaf has quit IRC05:58
*** andreaf has joined #openstack-infra05:58
*** yfried|afk is now known as yfried_06:02
*** gokrokve has quit IRC06:02
*** otherwiseguy has quit IRC06:02
*** maxbit has joined #openstack-infra06:13
*** r1chardj0n3s is now known as r1chardj0n3s_afk06:14
*** yolanda has joined #openstack-infra06:15
*** Longgeek has joined #openstack-infra06:15
*** chandankumar has quit IRC06:15
*** bo_sh has joined #openstack-infra06:19
*** oramraz has joined #openstack-infra06:19
openstackgerritA change was merged to openstack-infra/system-config: Fix one missed afs->openafs rename  https://review.openstack.org/12972906:19
*** sileht has quit IRC06:19
*** armax has quit IRC06:19
*** gokrokve has joined #openstack-infra06:28
*** yfried_ has quit IRC06:29
openstackgerritSpencer Krum proposed a change to openstack-infra/project-config: Use beaker tests on puppet storyboard  https://review.openstack.org/12609306:29
*** gokrokve has quit IRC06:33
*** yfried_ has joined #openstack-infra06:36
*** chandankumar has joined #openstack-infra06:38
*** flaper87|afk is now known as flaper8706:39
*** VijayTripathi has quit IRC06:40
*** Adri2000 has quit IRC06:54
*** _nadya_ has joined #openstack-infra06:56
*** _nadya_ has quit IRC06:59
*** _nadya_ has joined #openstack-infra07:00
*** _nadya_ has quit IRC07:00
*** Adri2000 has joined #openstack-infra07:01
*** hashar has joined #openstack-infra07:02
*** luqas has joined #openstack-infra07:02
*** ihrachyshka has joined #openstack-infra07:06
*** Sukhdev has quit IRC07:08
*** _nadya_ has joined #openstack-infra07:08
*** dizquierdo has joined #openstack-infra07:08
*** andreykurilin_ has joined #openstack-infra07:11
*** e0ne has joined #openstack-infra07:12
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Unbreak translation proposals  https://review.openstack.org/12982707:13
*** ihrachyshka has quit IRC07:13
openstackgerritDenis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Ability to restrict where job can be executed.  https://review.openstack.org/12982807:13
*** jyuso has joined #openstack-infra07:16
*** jyuso has quit IRC07:17
*** HeOS has quit IRC07:18
*** amcrn has quit IRC07:20
*** MaxV has joined #openstack-infra07:23
*** changbl has quit IRC07:24
*** mmaglana has quit IRC07:26
*** Guest59036 has joined #openstack-infra07:27
*** Ryan_Lane has quit IRC07:28
openstackgerritDenis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Review issues fixed.  https://review.openstack.org/12928707:28
*** gokrokve has joined #openstack-infra07:28
*** jgallard has joined #openstack-infra07:28
*** skolekonov has joined #openstack-infra07:30
*** mrmartin has joined #openstack-infra07:32
StevenKdstufft: When you're around, I've found that my bandersnatch mirror is not quite consistent. There are a number of files that are referenced in simple/*/index.html that are not under packages at all. Is there any way I can tell bandersnatch, "You're not done with this packages, try harder" ?07:32
dstufftStevenK: you can reset the state of the bandersnatch mirror completely and it'll do a full sync again instead of a partial sync07:33
dstufftit won't redownload files it already has though07:33
dstufftso it's not super bad07:33
StevenKls: cannot access ../../packages/py2/o/os-collect-config/os_collect_config-0.1.29-py2-none-any.whl: No such file or directory07:33
*** gokrokve has quit IRC07:33
StevenKls: cannot access ../../packages/source/c/crossbar/crossbar-0.9.9.zip: No such file or directory07:33
StevenKThings like that07:33
dstufftdelete the state file and rerun sync07:34
StevenKdstufft: As in <pypi root>/status ?07:34
dstufftand the todo file07:34
StevenKNo todo file07:34
StevenKdstufft: Neither state nor todo exist in /srv/pypi07:35
StevenKThe cloest I have is status07:35
StevenKsteven@thrashed:~$ cat /srv/pypi/status ; echo07:36
dstuffttry that then07:36
StevenKdstufft: Oh, the other thing I wanted to whine about, is can you have bandersnatch log the filesize when it's downloading something07:37
*** dimsum_ has joined #openstack-infra07:37
StevenKThen you can get some idea of the churn per day/time period07:37
*** afazekas has joined #openstack-infra07:38
dstufftStevenK: I don't actually have commit or much to do with bandersantch :) I'm sure Christian would be happy to have a PR for that though07:38
*** andreaf has quit IRC07:38
* StevenK stabs Bitbucket for not allowing new openid signups07:41
*** andreykurilin_ has quit IRC07:41
*** dimsum_ has quit IRC07:41
openstackgerrityolanda.robla proposed a change to openstack-infra/system-config: Install puppet modules in right order  https://review.openstack.org/12213207:42
*** mgrygoriev has joined #openstack-infra07:47
*** yfried_ is now known as yfried|afk07:48
*** stevemar has quit IRC07:50
*** zz_johnthetubagu is now known as johnthetubaguy07:50
*** jcoufal has joined #openstack-infra07:50
*** yfried|afk is now known as yfried_07:52
*** johnthetubaguy is now known as zz_johnthetubagu07:53
*** Hal has joined #openstack-infra07:54
*** Hal is now known as Guest5793907:54
*** Guest59036 has quit IRC07:54
*** alexpilotti has joined #openstack-infra07:54
BharatKI am following https://raw.github.com/jaypipes/os-ext-testing/master/puppet/install_master.sh script to install Jenkins Mater Node07:55
BharatKBut facing so many issues with puppet07:55
BharatKIs that the correct link?07:56
*** yamamoto has quit IRC07:56
*** jpich has joined #openstack-infra07:56
*** Guest57939 has quit IRC07:57
*** Hal_ has joined #openstack-infra08:00
*** jistr has joined #openstack-infra08:02
*** jedimike has joined #openstack-infra08:03
*** zz_johnthetubagu is now known as johnthetubaguy08:04
*** mpaolino has joined #openstack-infra08:09
openstackgerritSpencer Krum proposed a change to openstack-infra/system-config: Test failure of apply test  https://review.openstack.org/12983708:09
*** berendt has joined #openstack-infra08:10
*** arxcruz has joined #openstack-infra08:11
*** Hal_ has quit IRC08:13
*** mancdaz_away is now known as mancdaz08:14
*** jp_at_hp has joined #openstack-infra08:18
*** yamahata has quit IRC08:21
openstackgerrityolanda.robla proposed a change to openstack-infra/storyboard-webclient: Allow assigning tasks without expanding the edition box  https://review.openstack.org/12564308:24
openstackgerritA change was merged to openstack-infra/project-config: Remove confusing access "refs/meta/config" ACL def  https://review.openstack.org/12975408:24
*** koolhead17 has joined #openstack-infra08:27
*** mmaglana has joined #openstack-infra08:27
*** jlibosva has joined #openstack-infra08:27
openstackgerritA change was merged to openstack-infra/project-config: Adding puppet-pip as split out module  https://review.openstack.org/12609608:27
*** koolhead_ has joined #openstack-infra08:27
*** koolhead17 has quit IRC08:27
*** _nadya_ has quit IRC08:28
*** gokrokve has joined #openstack-infra08:28
*** camunoz has quit IRC08:28
*** e0ne has quit IRC08:30
*** gokrokve has quit IRC08:33
*** lttrl has quit IRC08:33
*** openstackgerrit has quit IRC08:34
*** openstackgerrit has joined #openstack-infra08:35
*** sileht has joined #openstack-infra08:35
*** vdo has joined #openstack-infra08:37
*** Hal_ has joined #openstack-infra08:38
*** mrmartin has quit IRC08:41
*** HeOS has joined #openstack-infra08:42
*** talluri has joined #openstack-infra08:42
*** davidhadas has quit IRC08:43
*** cdent has joined #openstack-infra08:44
*** Hal_ has quit IRC08:46
*** hashar has quit IRC08:47
*** Hal_ has joined #openstack-infra08:51
*** ihrachyshka has joined #openstack-infra08:51
*** bradjones_ has joined #openstack-infra08:53
*** bradjones has quit IRC08:53
*** bradjones_ is now known as bradjones08:53
*** bradjones has quit IRC08:53
*** bradjones has joined #openstack-infra08:53
*** Hal_ has quit IRC08:56
*** lttrl has joined #openstack-infra09:00
*** koolhead_ has quit IRC09:08
*** pelix has joined #openstack-infra09:09
*** yamamoto has joined #openstack-infra09:09
*** andreaf_ is now known as andreaf09:10
openstackgerritA change was merged to openstack-infra/project-config: puppet-sahara: update Gerrit permissions  https://review.openstack.org/12965509:13
*** yamamoto has quit IRC09:15
*** hashar has joined #openstack-infra09:16
*** mpaolino has quit IRC09:21
*** rlandy has joined #openstack-infra09:21
hasharhello! any idea how I can have a change in Gerrit to be linked with a launchpad bug?   I proposed https://review.openstack.org/#/c/128921/ with the magic header 'Closes-Bug: #1358517'   but I had to manually update the bug on launchpad https://bugs.launchpad.net/zuul/+bug/135851709:26
uvirtbotLaunchpad bug 1358517 in zuul "Change stuck in queue when Gearman errors during merge:merge submit" [Undecided,In progress]09:27
hasharI guess I am doing something wrong :D09:27
*** talluri has quit IRC09:27
*** mugsie has quit IRC09:27
rcarrillocruzdo you still use Launchpad for filing bugs for infra?09:28
rcarrillocruzor should I check storyboard?09:28
*** gokrokve has joined #openstack-infra09:28
*** mugsie has joined #openstack-infra09:29
rcarrillocruzAJaeger: morning :-), if you are around ^09:29
AJaegerrcarrillocruz, use launchpad09:29
*** salv-orlando has joined #openstack-infra09:32
*** gokrokve has quit IRC09:33
*** e0ne has joined #openstack-infra09:37
*** KanagarajM has quit IRC09:38
*** 1JTAAQZNO has joined #openstack-infra09:40
*** pblaho has joined #openstack-infra09:40
*** teran has quit IRC09:42
*** kingia has joined #openstack-infra09:51
*** kingia is now known as iain_king09:51
*** talluri has joined #openstack-infra09:52
*** yamamoto has joined #openstack-infra09:54
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Fix command to build docs in installation documentation  https://review.openstack.org/12949809:56
*** hashar_ has joined #openstack-infra09:56
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Ignore testr_subunit_log  https://review.openstack.org/12950309:58
*** hashar has quit IRC09:59
*** teran has joined #openstack-infra10:00
*** unicell has quit IRC10:01
*** nosnos has quit IRC10:02
*** nosnos has joined #openstack-infra10:03
yolandahere, can i ask a question about how pypi publish works? i'm interested in picking a change i did for gerritlib, but i see that the latest version published in the mirror is 0.4.0 from 16 april10:04
*** teran has quit IRC10:04
*** yfried_ is now known as yfried|afk10:05
*** koolhead17 has joined #openstack-infra10:05
*** koolhead17 has joined #openstack-infra10:05
*** _nadya_ has joined #openstack-infra10:05
*** yfried|afk is now known as yfried_10:07
*** nosnos has quit IRC10:08
AJaegeryolanda, if a release gets tagged, we publish automatically to pypi10:09
*** jgallard has quit IRC10:09
AJaegeryolanda, 0.4.0 is the latest release - so ask jeblair to release again if there'S a need for it.10:10
*** jcoufal has quit IRC10:10
yolandai added some changes to gerritlib, to allow managing descriptions in projects, so i need that code to be in the mirror in order to use it from jeepyb10:11
*** jp_at_hp has quit IRC10:13
yolandaAJaeger, thx, i'll ping jeblair about it10:13
openstackgerritJoshua Hesketh proposed a change to openstack-infra/system-config: Update comment in logs.vhost  https://review.openstack.org/12986110:13
*** yamahata has joined #openstack-infra10:13
*** jp_at_hp has joined #openstack-infra10:14
*** teran has joined #openstack-infra10:15
*** teran_ has joined #openstack-infra10:19
openstackgerritDenis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Added description for node attribute.   https://review.openstack.org/12928710:19
*** koolhead17 has quit IRC10:20
*** teran has quit IRC10:21
*** teran_ has quit IRC10:25
*** hashar_ is now known as hashar10:28
*** gokrokve has joined #openstack-infra10:28
*** gokrokve has quit IRC10:29
*** gokrokve has joined #openstack-infra10:30
*** gokrokve has quit IRC10:35
*** koolhead17 has joined #openstack-infra10:37
*** koolhead17 has quit IRC10:37
*** koolhead17 has joined #openstack-infra10:37
*** jcoufal has joined #openstack-infra10:38
*** jcoufal has quit IRC10:39
*** teran has joined #openstack-infra10:39
*** jcoufal has joined #openstack-infra10:39
*** teran has quit IRC10:39
*** teran has joined #openstack-infra10:40
bo_shfungi: hi there.10:41
*** pc_m has joined #openstack-infra10:42
*** bradjones has quit IRC10:45
*** bradjones has joined #openstack-infra10:47
*** koolhead17 has quit IRC10:53
*** danpb has joined #openstack-infra10:53
danpbis anyone aware of problems with jenkins git merges today ?  it has marked a whole bunch of patches as having merge conflicts, when in fact no conflicts exist ?!?!10:55
*** ociuhandu has joined #openstack-infra10:56
openstackgerritA change was merged to openstack-infra/project-config: Use beaker tests on puppet storyboard  https://review.openstack.org/12609310:58
*** baoli has quit IRC11:02
AJaegerdanpb, all patches were requeued and thus we have a very long queue...11:02
hashardanpb: there was an issue yesterday, around 19:50 UTC iirc11:03
*** jamielennox_ has joined #openstack-infra11:04
hasharthough it is kind of slow processing jobs11:05
*** rcarrillocruz has quit IRC11:06
*** jamielennox_ is now known as jamielennox11:06
*** rcarrillocruz has joined #openstack-infra11:08
danpbwhy would re-queuing jobs result in them all being marked as in merge conflict though ?11:08
hasharthere was an issue with a Zuul merger  that could not connect to Gerrit11:08
hasharso it would fail the merge of the proposed patchset on tip of the target branch and report back a merge error11:09
danpbah, ok11:09
hasharsince that marked  a bunch of changes with Verified -1 , I guess that all affected patch have been manually reenqueued11:09
hasharand seems the rate at which jobs are being processed as slowed down a few hours ago11:09
hasharwhich is what the tiny url link show11:10
*** teran_ has joined #openstack-infra11:10
*** yamamoto has quit IRC11:11
*** BharatK has quit IRC11:12
*** habib has joined #openstack-infra11:13
*** teran has quit IRC11:13
*** mwagner_lap has quit IRC11:14
*** ildikov has quit IRC11:20
*** dimsum_ has joined #openstack-infra11:21
*** dmakogon_ is now known as denis_makogon11:22
*** miqui has quit IRC11:24
*** hashar has quit IRC11:25
*** BharatK has joined #openstack-infra11:26
dimsum_SergeyLukjanov: regarding docker-py, it's fine to say that we don't need it since it is used by a stack forge project. but the practical problem is that gate-nova-docker-requirements, check-tempest-dsvm-docker and check-tempest-dsvm-f20-docker fail miserably11:26
*** jistr is now known as jistr|english11:27
*** gokrokve has joined #openstack-infra11:28
*** weshay has joined #openstack-infra11:28
*** aysyd has joined #openstack-infra11:29
*** mmedvede has joined #openstack-infra11:29
*** danpb has left #openstack-infra11:30
*** isviridov_away is now known as isviridov11:31
*** gokrokve has quit IRC11:32
*** _nadya_ has quit IRC11:37
*** BharatK has quit IRC11:37
*** pelix has quit IRC11:37
*** afazekas has quit IRC11:37
*** k4n0 has quit IRC11:37
*** asettle has quit IRC11:37
*** rwsu has quit IRC11:37
*** sunil_ has quit IRC11:37
*** larsks has quit IRC11:37
*** sunil_ has joined #openstack-infra11:37
*** asettle has joined #openstack-infra11:37
*** larsks has joined #openstack-infra11:37
*** afazekas has joined #openstack-infra11:37
*** BharatK has joined #openstack-infra11:37
*** rwsu has joined #openstack-infra11:38
*** pelix has joined #openstack-infra11:38
*** k4n0 has joined #openstack-infra11:38
*** flaper87 has quit IRC11:40
*** flaper87 has joined #openstack-infra11:40
*** mfink has quit IRC11:41
*** Longgeek has quit IRC11:42
*** luqas has quit IRC11:43
*** mrmartin has joined #openstack-infra11:44
*** jp_at_hp has quit IRC11:45
*** jp_at_hp has joined #openstack-infra11:45
*** afazekas has quit IRC11:46
*** afazekas has joined #openstack-infra11:46
*** k4n0 has quit IRC11:46
*** k4n0 has joined #openstack-infra11:46
*** BharatK has quit IRC11:46
*** BharatK has joined #openstack-infra11:46
*** pelix has quit IRC11:46
*** pelix has joined #openstack-infra11:46
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/system-config: Expose the vhost_name parameter on openstack_project::nodepool  https://review.openstack.org/12987611:48
*** unicell has joined #openstack-infra11:50
*** Longgeek has joined #openstack-infra11:50
*** ZZelle has quit IRC11:51
*** yamamoto has joined #openstack-infra11:51
*** ZZelle has joined #openstack-infra11:51
*** dprince has joined #openstack-infra11:52
*** baoli has joined #openstack-infra11:53
*** HeOS_ has quit IRC11:53
*** HeOS_ has joined #openstack-infra11:54
*** HeOS_ has joined #openstack-infra11:55
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/project-config: allow for NON_STANDARD_REQS override  https://review.openstack.org/12988011:56
dimsum_SergeyLukjanov: Here's a counter proposal :) ^^^11:56
*** yamamoto has quit IRC11:57
*** maxbit has quit IRC12:01
*** hdd has joined #openstack-infra12:05
*** BharatK has quit IRC12:07
openstackgerritA change was merged to openstack/requirements: Add demjson to requirements  https://review.openstack.org/11481812:08
*** jgallard has joined #openstack-infra12:09
*** mwagner_lap has joined #openstack-infra12:11
*** mrmartin has quit IRC12:12
*** jedimike has quit IRC12:13
*** mbacchi has joined #openstack-infra12:14
*** hashar has joined #openstack-infra12:15
*** mrmartin has joined #openstack-infra12:16
*** mancdaz is now known as mancdaz_away12:18
*** mancdaz_away has quit IRC12:18
*** dkranz has joined #openstack-infra12:19
*** ildikov has joined #openstack-infra12:20
*** mancdaz_away has joined #openstack-infra12:21
*** BharatK has joined #openstack-infra12:21
*** mancdaz_away is now known as mancdaz12:21
*** rkukura has quit IRC12:22
*** koolhead17 has joined #openstack-infra12:22
*** koolhead17 has joined #openstack-infra12:22
*** mancdaz has quit IRC12:23
*** mancdaz_away has joined #openstack-infra12:23
*** mancdaz_away is now known as mancdaz12:24
*** mancdaz has quit IRC12:26
*** yamamoto has joined #openstack-infra12:28
*** habib has quit IRC12:28
*** k4n0 has quit IRC12:28
*** gokrokve has joined #openstack-infra12:28
*** mancdaz_away has joined #openstack-infra12:29
*** mancdaz_away is now known as mancdaz12:29
*** yamamoto has quit IRC12:29
*** yamamoto has joined #openstack-infra12:30
*** gokrokve has quit IRC12:33
*** yamamoto has quit IRC12:35
*** chandankumar has quit IRC12:37
*** dimsum_ has quit IRC12:38
*** dimsum_ has joined #openstack-infra12:39
*** kgiusti has joined #openstack-infra12:39
*** mfink has joined #openstack-infra12:43
*** timrc-afk is now known as timrc12:46
*** spzala has joined #openstack-infra12:47
*** kgiusti has quit IRC12:53
*** kgiusti has joined #openstack-infra12:54
*** miqui has joined #openstack-infra12:56
*** julim has joined #openstack-infra12:57
*** jistr|english is now known as jistr12:57
AJaegerlooking at the zuul queue, I see that the top jobs in check and gate queue are not moving forward. Is there a reason why those jobs are not started?13:04
*** yfried_ has quit IRC13:04
*** Sukhdev has joined #openstack-infra13:04
*** iain_king has quit IRC13:05
fungihashar: you only get a fix-proposed/in-progress update to the bug if you correctly link it in the commit message of your first patchset (this is a protection against spamming the bug on every patchset). if you want to look at improving the script which updates bugs, it's http://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/update_bug.py13:06
fungi(but as we're soon migrating to storyboard, may not be worth the headache)13:06
*** bcourt has quit IRC13:06
*** luqas has joined #openstack-infra13:06
hasharfungi: that is what I eventually figured out :_D13:07
*** dkranz has quit IRC13:07
hashari.e. only on first patchset13:07
fungiAJaeger: taking a look now that i'm caught up on scrollback13:07
hasharfungi: thanks for the follow up13:07
AJaegergood morning, fungi!13:08
AJaegerfungi, the openstack/cinder patch in check is there for ages but one was before it this morning - and the system-config in post has not moved at all.13:08
hasharfungi: also Zuul seems to no more be triggering jobs since 6am UTC   , at least the queue of jobs barely progress http://tinyurl.com/k9nfovw13:08
*** salv-orlando has quit IRC13:09
*** sdake_ has quit IRC13:09
AJaegerhashar, there are jobs triggered, I got some out of check and gate...13:09
*** sdake_ has joined #openstack-infra13:09
*** sdake_ has quit IRC13:09
*** sdake_ has joined #openstack-infra13:09
hasharyeah for some new ones13:09
AJaegerthe post queue might be due to the low priority...13:09
hasharI have the issue on my setup as well13:10
hasharfrom time to time jobs refuse to be retriggered13:10
hasharpossibly because Zuul consider them as already being running13:10
hasharI unleash it by promoting a change in the pipeline, but that causes every changes to be retriggered :/13:10
hasharor maybe some of those jobs are no more registered in Zuul gearman daemon :/13:11
*** yamahata has quit IRC13:11
*** amotoki has quit IRC13:11
*** yamahata has joined #openstack-infra13:12
AJaegerthe check queue total length is osciallating around 600 - this morning (7 hours ago) 590 then up to 620, now at 604 again.13:12
*** koolhead17 has quit IRC13:13
fungiokay, so for starters... we're maxxed out at quota in our providers, thus we're only assigning new nodes as they get recycled. we currently seem to be able to run between 200 and 300 jobs concurrently13:13
fungiany activity in gate, release, pre-release or tag pipelines will take resources first, then check pipeline gets whatever's available after that13:13
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Add a --version command line flag to show JJB version  https://review.openstack.org/12784413:14
*** mancdaz is now known as mancdaz_away13:14
jrollfungi: have y'all asked rackspace to bump quota?13:14
fungijroll: they reduce it on us regularly13:14
fungiseems the churn we create by booting and deleting nodes is significant compared to most tenants13:15
jrollgod forbid you use the cloud like a cloud13:15
jrollis that churn actually a problem?13:15
AJaegerglad that I screwed up the translation periodic jobs - that's 10 jobs less for the check pipeline ;( https://review.openstack.org/12982713:15
jaypipesBharatK: pong...13:15
fungiit has been described to us as creating negative performance impact within the regions we're using, yes13:15
jaypipesBharatK: unfortunately, the Puppet modules for setting up CI systems has changed a bit, and I have not had the time yet to adjust my blog post and the os-ext-testing repo...13:16
AJaegerSo, post will not get anything. Should we stop approving patches?13:16
jrollfungi: interesting.13:16
fungii think a lot of that may be that because we're booting from snapshots and snapshot storage is somewhere other than normal image storage, that plays into the performance issues somewhat13:16
jrollthis saddens me greatly13:16
joajaypipes: I think someone cloned it and made an updated version13:16
jrollso the solution is to throttle infra13:16
*** koolhead17 has joined #openstack-infra13:16
*** dane_leblanc has joined #openstack-infra13:16
*** jedimike has joined #openstack-infra13:17
*** radez_g0n3 is now known as radez13:17
fungijroll: well, anyway, the capacity is a fixed amount... for the most part i'm looking right now to see what we're able to do within those confines. there are potentially other issues unrelated to capacity13:17
*** ZZelle has quit IRC13:17
joajaypipes: rasselin, that was his github handle :)13:17
jrollfungi: but I don't see why the quota can't be bumped13:18
jrollbut that's just me13:18
fungijroll: the other issue described to us arising from instance churn is fixed ip assignment... apparently those can't be turned over instantaneously, so sometimes we have boot requests wait an hour for an ip address before nova reports them as complete13:18
*** cpowell has joined #openstack-infra13:19
fungiwhere "sometimes" is up to 50% of our boot requests at peak load in some regions13:19
*** freyes has quit IRC13:19
jrollfungi: yeah, letting arp clear13:19
anteayajoa: which I believe he is frustrated at maintaining, which is why he is working on helping infra implement some of our specs, like the puppet module split spec13:19
*** dmsimard_away has quit IRC13:19
anteayasince he realizes his time is better spent helping infra13:19
jrollfungi: thanks for the info :)13:19
erlon_guys, we have requested our CI account to be enable but we got not response so far13:20
fungijroll: you're welcome--and thanks for your support!13:20
*** jistr has quit IRC13:20
joaanteaya: I think so, I'm not surprised ^^'13:21
*** freyes has joined #openstack-infra13:21
erlon_Im looking at the topics in the meeting wiki but I dont know if a topic is needed for this13:21
fungiAJaeger: hashar: so i think the changes which have been waiting longest in the check pipeline are likely to be from one or more jobs aborting prematurely (something going wrong with the jenkins slave agent connection for example) which causes that job to get requeued and wait for a new worker to be assigned13:21
joaanteaya: which is why I made my propositions of having something more tied with infra for newcomers :)13:21
jrollfungi: fwiw, I'd put in a ticket to bump the quota and ask that it doesn't get lowered again, not sure if that will work but worth a try13:21
fungijroll: oh, thanks!13:22
*** erlon_ is now known as erlon13:22
*** bcourt has joined #openstack-infra13:22
anteayajroll: thank you13:22
jrollanteaya: of course :)13:22
anteayaerlon: we address requests for new accounts about once a week13:22
*** jistr has joined #openstack-infra13:23
anteayasorry tab complete fail, I was addressing joa13:23
anteayaand thank you as well jroll, :D13:23
anteayaerlon: no topic is needed for third party account requests13:23
*** nelsnels_ has quit IRC13:23
jrolloh lol13:23
anteayajroll: :D13:24
AJaegerjroll, thanks a lot!13:24
*** nelsnelson has joined #openstack-infra13:24
*** bswartz has quit IRC13:24
anteayaerlon: now if you are requesting a ci account to test stackforge and the only information in the request is the name of the stackforge project, that is a problem13:24
anteayaand mordred is taking care of that problem13:24
jedimikequick question: is CI running on trusty, or is partly migrated to trusty, or is it all still running precise? I'm looking at how we can migrate gozer to trusty13:25
erlonanteaya: hmm, ok13:25
anteayajedimike: the slaves are on trusty13:25
anteayathe infrastructure nodes are on precise13:25
*** dustins has joined #openstack-infra13:26
*** jp_at_hp has quit IRC13:26
jedimikeanteaya, is there any problem going to trusty? (I mean apart from finding the time to migrate!)13:26
fungiwell, slaves are on a combination of trusty, precise, centos 6, centos 7, fedora 20...13:26
*** dmsimard_away has joined #openstack-infra13:27
*** dmsimard_away is now known as dmsimard13:27
erlonanteaya: its been a week so I assume you guys will enable it in this next days right?13:27
erlonanteaya: another thing, is there a way to disable the CI and keep the access to the sandbox?13:27
fungijedimike: and *some* infrastructure nodes are running ubuntu trusty (our puppet 3 master in particular)13:28
*** _nadya_ has joined #openstack-infra13:28
*** yamamoto has joined #openstack-infra13:28
*** bknudson has joined #openstack-infra13:29
openstackgerritDenis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Review issues fixed.  https://review.openstack.org/12928713:29
AJaegeranteaya, could you review https://review.openstack.org/129827 today, please?13:31
anteayaerlon: you posted a paste of questions to me13:33
*** yamamoto has quit IRC13:33
anteayaAJaeger: sure13:33
*** 1JTAAQZNO has quit IRC13:33
*** hdd has quit IRC13:34
*** r-daneel has joined #openstack-infra13:34
erlonanteaya: yep, disregard the paste, irccloud tricked me, I post the questions right after13:34
AJaegeranteaya, since the load is high, consider not approving it, just +2 if you agree - without it all translation proposal jobs to fail. We can wait another day with those if needed ;)13:35
*** ZZelle has joined #openstack-infra13:35
AJaegeranteaya, and I can approve once the queue is short enough...13:35
anteayaAJaeger: +#13:35
*** pblaho has quit IRC13:35
AJaegerOk, fine as well - thanks!13:35
*** ZZelle has quit IRC13:35
anteayasorry missed your message as I was reviewing13:35
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Print context when key fails to be formatted  https://review.openstack.org/12949213:35
anteayaI only have one screen13:36
AJaeger;) np, anteaya13:36
anteayaerlon: well we will enable it depending on the request13:36
anteayabut we will look at the requests again today13:36
*** amotoki has joined #openstack-infra13:37
anteayaerlon: and you set what repos you watch13:37
erlonanteaya: what do you mean?13:37
*** pblaho has joined #openstack-infra13:37
anteayaI mean we read the email to create the account13:37
*** mriedem has joined #openstack-infra13:37
* AJaeger does some shopping now, will join the IRC meeting tonight...13:37
anteayawe review the data, if we have enough and it matches what we need, we can create the account13:37
anteayaif not, we can't, if it isn't the name of a stackforge project, we email you with more infor13:38
anteayaif it is a stackforge project, mordred will take those13:38
erlonanteaya: hmm, sorry, let me explain, me and marcusvrn_ are in charge of HDS account, we where disabled because our CI wasn't posting logs, despite it wasn't voting13:40
*** dkliban_biab is now known as dkliban13:40
anteayaoh yeah okay13:40
anteayawell I talked to thingee about this13:40
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/system-config: Use absolute path in nodepool_template default value  https://review.openstack.org/12990713:40
anteayasince i am tired of being point person for all this stuff13:40
*** masayukig has quit IRC13:40
anteayaand the projects could take up some of the responsibility here13:41
anteayabasically you use your personal gerrit account to read the stream and trigger a few tests, configure jenkins to *not* report back13:41
*** radez is now known as radez_g0n313:41
*** BharatK has quit IRC13:41
anteayapost the urls for the tests triggered by the stream you read from your personal account and have some cinder folks look at it13:42
*** rkukura has joined #openstack-infra13:42
anteayaif they tell me it looks stable, I will request you be reenabled13:42
anteayaI'm tired of reanableing systems to have to turn around and disable them again13:42
anteayaso we need some information that can be confirmed13:43
anteayaand I would rather deal with a person from each project that 80+ operators13:43
anteayaall insisting that their system works _fine_ when it doesn't13:43
erlonanteaya: hmmm ok, but I still don't the reason why the sandbox shouldn't allowed for disable CIs13:43
anteayawell I can explain13:44
erlonanteaya: but is ok, we will test this and contact thingee13:44
anteayabecause third party gerrit accounts have no gui13:44
anteayano, talk to the folks in the cinder channel13:44
anteayasince i have no doubt that thingee will want someone else to do this13:44
*** obondarev_ has joined #openstack-infra13:45
anteayaand because third party gerrit accounts have no gui, they can only communicate via ssh13:45
anteayaincluding connecting to read a stream of events13:45
fungiwell, and also we can control whether those accounts can leave verify votes on specific projects, but we can't limit their ability to comment to specific projects (we can only disable the account if it's leaving unhelpful comments on changes)13:45
anteayaso when they are disabled, the ssh connection can't be made13:45
openstackgerritA change was merged to openstack-infra/project-config: Unbreak translation proposals  https://review.openstack.org/12982713:46
*** masayukig has joined #openstack-infra13:46
fungii wonder if we should request that each project which has instituted third-party testing as a requirement for specific in-tree drivers/backends/features designate a liaison who interacts with the infra team to get accounts switched to voting, reenabled, whatever needs coordinating13:46
fungimight reduce a lot of the random finger-pointing which goes on in those situations13:47
*** katyafervent_awa is now known as katyafervent13:47
*** emagana has joined #openstack-infra13:48
fungiso either the ptl or a designated delegate the third-party testers for that project should be in touch with, who then relays the wishes of the project to us13:48
erlonfungi:anteaya: ok13:48
anteayawell right now we have mestery and emagana for neutron13:49
emaganaanteaya: Hi There!13:49
fungiright, i think in practice it's happening that way in some places13:49
*** sabeen1 has joined #openstack-infra13:49
fungiand it seems like a successful enough model we might ask other projects who have similar needs to follow13:49
anteayaI have talked to thingee and suspect that some combination of him DuncanT- and jungle_boy_j will do it13:49
anteayaemagana: hello13:50
*** freemanbrandon has joined #openstack-infra13:50
*** talluri has quit IRC13:50
anteayawas just saying that working with you to confirm neutron ci's is working well for me13:50
emaganaanteaya: Just letting you know that I am around..  well just now!13:50
anteayaand that having the same for cinder and nova would be good13:50
anteayaemagana: thank you!13:50
anteayaso cinder is in play13:50
anteayaand frankly nova accounts are few and responsible13:51
fungibecause as it stands right now, a lot of the third-party testing operators keep getting the impression they're supposed to get permission from the infra team for this sort of stuff when in actuality they should be talking to those projects13:51
anteayaI had asked in the past for a contact and had no offers, but really I haven't had any problems from nova accounts13:51
*** nelsnelson has quit IRC13:51
fungijust trying to figure out how to better pre-empt that and get them directed to the right contacts up front13:51
*** marun has joined #openstack-infra13:51
anteayaand salv and bobball and jhesketh are all awesome to talk to13:51
anteayafungi: I'm for that13:52
*** nelsnelson has joined #openstack-infra13:52
* jhesketh is happy to help out where he can13:52
anteayabut first I was working on getting the structure in place before advertising13:52
fungiright, i'm not saying we lack de-facto liaisons, but rather than those liaisons are not obvious to many of the third-party testers for those corresponding projects13:52
anteayasince all the projects move at their individual pace13:52
anteayaand infra's attempts to get them to move at teh same pace usually don't go well13:53
anteayajhesketh: and you do a great job of it13:53
anteayafungi: right13:53
anteayaand part of it has been working with them in a defacto way13:53
anteayaso I didn't scare them away from doing the work13:53
anteayasince pointing the hoards at them prematurely tends to do that13:54
fungiheh, yep. can totally see that13:54
anteayaso yeah, it is an awkward dance13:55
anteayaand I fall down a lot13:55
anteayabut I am still trying to dance as fast as I can13:55
anteayato feed the constantly hungry13:55
anteayaif I move too fast, I end up alone again13:55
anteayaI have to move at the pace of the structure13:56
anteayawhich does not suit the hoards13:56
*** chandankumar has joined #openstack-infra13:56
*** stevemar has joined #openstack-infra13:57
*** wenlock has joined #openstack-infra13:57
anteayaI guess I just have to tolerate the screaming a while longer13:57
*** freemanbrandon has quit IRC13:57
anteayasince some of the bigger pieces, slowly and of their own volition, are feeling like they are moving into a mutually beneifical place13:58
fungiwell, the infrastructure team can, as a whole, start to be more insistent about it is what i'm saying. to try and avoid burning ourselves out. the projects are the ones who have asked their specialty developers to set up external testing and reporting systems. we maintain the means to make that possible, but really shouldn't be responsible for hand-holding them13:58
*** bcourt has quit IRC13:58
anteayalet's use the word supportive rather than insistent13:58
anteayaand I'm onboard13:59
*** ayoung has joined #openstack-infra13:59
fungiand it can be up to those projects whether they decide dealing with third-party systems is in general a success or too much effort. but just heaping the necessary effort onto the infrastructure team is counter-productive13:59
*** freemanbrandon has joined #openstack-infra13:59
anteayabut keep in mind that we have just now turned a corner when all programs with this issue have had a release dealing with it first hand14:00
anteayarather than in theory14:00
fungiyep, so it's an ideal time to start iterating toward getting them to take on more of the responsibility14:00
anteayaso they now have some experience to use to bring to decison making14:00
anteayaI agree14:00
*** gokrokve has joined #openstack-infra14:01
fungiand allowing us to do what we do best (developing and maintaining good project infrastructure)14:01
anteayaoh utopia14:01
*** signed8bit has joined #openstack-infra14:01
anteayasunshine and wheat fields14:01
*** markmcclain has joined #openstack-infra14:02
*** yfried_ has joined #openstack-infra14:02
*** cody-somerville has quit IRC14:03
*** freemanbrandon has quit IRC14:04
*** todd_dsm has joined #openstack-infra14:04
*** freemanbrandon has joined #openstack-infra14:04
*** esker has joined #openstack-infra14:04
*** hdd has joined #openstack-infra14:04
*** mfer has joined #openstack-infra14:05
*** esker has quit IRC14:06
*** esker has joined #openstack-infra14:06
*** zz_dimtruck is now known as dimtruck14:06
anteayafungi: so how is zuul doing?14:08
anteayathe test node graph looks great to my eyes14:08
anteayathe zuul job queue has decreased its rapid rate of descent14:08
anteayathe fact that the test graph doesn't show the deleting and building nodes slowly coming together like teeth is awesome14:09
fungianteaya: it's running at full steam. i'm pruning some dead wood from the providers (cleaning up stale nodes which haven't hit the 8-hour mark yet, getting rid of disassociated floating ips, deleting orphaned "alien" nodes, et cetera)14:10
*** jerryz has quit IRC14:11
fungijust general de-crufting and seeing what i can do to get a little more throughput14:11
anteayathroughput looks fairly stable though for this long and full capacity14:11
*** salv-orlando has joined #openstack-infra14:11
anteayaso yay14:11
fungiso far no signs there's anything unusual going on, aside from the obvious that we heaped 0.5k rechecks onto it yesterday14:11
anteayafor this long at full capacity14:12
anteayathis is awesome14:12
anteayafor y'know stabilty stress testing14:12
fungiand just now i saw four changes merge at the head of the gate pipeline, so it's not being hindered by the pending check volume14:13
anteaya88 in post14:13
*** chandankumar has quit IRC14:13
anteayatop job has been there 19.5 hours14:14
fungiright, post pipeline is at low priority so right now it's not going to get workers until existing demand in the check pipeline has been satisfied14:14
anteayadocs and specs nodes shortage14:14
anteayawell rather than add, I think I will take my walk now, get it out of the way14:15
*** pcrews has joined #openstack-infra14:15
fungiyesterday we talked about switching post to normal priority same as check14:15
anteayaI have a few things I could do but I see a lull in the rain so going to take it14:15
anteayafungi: what were the pros and cons?14:16
*** cody-somerville has joined #openstack-infra14:16
*** dangers_away is now known as dangers14:16
fungiwe only have three priority levels, as defined in the gearman protocol specification, so any adjustments we make to move a pipeline to a higher priority potentially starves work running at lower priorities14:16
*** bradjones has quit IRC14:16
anteayaI think right now priority for post below check is fine14:18
anteayathe stuff has merged14:18
*** dkranz has joined #openstack-infra14:18
anteayajust might be the weekend before we see the docs14:18
anteayaI wonder if the only post job is docs14:18
*** bcourt has joined #openstack-infra14:18
anteayaif only the last patch to that repo needs to be in post14:19
anteayaif we have 88 in post and more to come, the docs jobs just trigger a run of the docs on the code that is there14:19
anteayawe only need one trigger per repo14:19
anteayacalculate on patches there for more than say 3 hours14:20
anteayaso when the post queue is short or rapidly cleared that algorithim isn't triggered14:21
anteayajust evaluated in situations like this14:21
*** wenlock has quit IRC14:21
*** pblaho has quit IRC14:21
fungii think right now we don't have enough information on what might run in post to say for sure that we can expire long-waiting post jobs across the board. might make sense to have a separate sort of pipeline though for jobs which can cancel their predecessors in the pipeline or something14:21
fungiperhaps they're jobs tied to the branch tip state rather than a specific sha14:23
anteayasomething to chew on14:23
*** pblaho has joined #openstack-infra14:23
fungiso rather than cancelling their predecessors in the pipeline, they just don't get new instances added if there's already one pending14:23
anteayayes and as long as "latest" is in the pipeline we get what we need, once the pipeline clears14:23
*** ZZelle has joined #openstack-infra14:24
*** marun has quit IRC14:24
anteayathat could work, yes14:24
fungifor example, the branch-tarball jobs might benefit from that. no point in rapidly replacing that tarball over and over when the most recent version will do fine14:24
*** msabramo has quit IRC14:24
*** yaguang has joined #openstack-infra14:24
anteayaas long as it clears sometime and we don't end up in a perpetual loop state14:24
*** hdd has quit IRC14:24
anteayathen once it hits a certain length, say 1514:24
anteayait can toggle a different priority, clear that queue and then go back to accumulateing more14:25
anteayacan we toggle priority based on queue length of a pipeline?14:25
fungiyeah, that's why tying teh changeish identity in the pipeline to a branch name instead of a sha could be ideal there... no lost resources from repeatedly cancelling running jobs or resetting the timer for it in the fifo14:25
anteayais that possible?14:25
jheskethright, blog posted: http://josh.people.rcbops.com/2014/10/openstack-infrastructure-swift-logs-and-performance/14:26
*** reed has joined #openstack-infra14:26
jheskethnow I'm going to disappear for sleep before the meeting14:26
fungihmm... we could in theory have zuul dynamically prioritize new work requests based on demand in a given pipeline, but i don't think we'd actually want that14:26
*** pblaho has quit IRC14:26
*** dangers is now known as dangers_away14:26
*** armax has joined #openstack-infra14:26
*** pblaho has joined #openstack-infra14:27
anteayajhesketh: enjoy sleep14:27
anteayajhesketh: thanks for the blog post14:27
anteayafungi: oh okay14:27
fungianyway, i'll whip up a feature request for the branch pipeline idea so we don't forget the discussion14:27
anteayafungi: cool thanks14:27
anteayaI will disappear for a walk14:27
anteayaback later14:27
*** mancdaz_away is now known as mancdaz14:28
*** yamamoto has joined #openstack-infra14:28
clarkbdtroyer: jogo Juno to Master is not happy http://logs.openstack.org/74/128974/1/check/check-grenade-dsvm/6b7d6f3/logs/grenade.sh.txt.gz#_2014-10-21_07_14_28_80514:28
*** mpaolino has joined #openstack-infra14:28
clarkbjhesketh: thank you for blog I am reading now14:28
*** gokrokve has quit IRC14:28
*** bradjones has joined #openstack-infra14:29
*** gokrokve has joined #openstack-infra14:29
*** chandankumar has joined #openstack-infra14:29
*** atiwari has joined #openstack-infra14:29
*** jp_at_hp has joined #openstack-infra14:30
*** yaguang has quit IRC14:32
*** yamamoto has quit IRC14:32
*** yaguang has joined #openstack-infra14:33
krotscheckQuestion: Is releasing javascript libraries to NPM (js equiv to pipi) something I need to talk to the TC about?14:33
clarkbkrotscheck: I don't think so14:33
*** davidhadas_ has joined #openstack-infra14:33
*** andreaf has quit IRC14:33
clarkbwe haven't asked their permission to release java in the past14:33
fungikrotscheck: i would say just figure out what the release upload job would look like, and get one of the infra cores info on how to set up the necessary credentials/namespace/whatever is needed14:34
krotscheckOh good.14:34
krotscheckSo, a release involves tagging the repository in a specific way.14:34
clarkbis it different than the current specific way we tag releases?14:35
krotscheckI think so.14:35
fungioh, specially-formatted tags?14:35
clarkbok that will probably require a npm-release pipeline14:35
clarkbalso npm-- for making that a thng14:35
krotscheckBut ultimately I think it’s just a string.14:35
*** msabramo has joined #openstack-infra14:36
clarkbmtreinish: ^ see above grenade log14:36
*** mmaglana has quit IRC14:36
fungiclarkb: well, we *do* have a "tag" pipeline which triggers on arbitrary tags, only used by the openstack-infra/publications project to do its conference links14:36
clarkbmtreinish: we seem to have broken juno to master14:36
krotscheckI _think_ they want a v as a prefix.14:36
krotscheckI’m going to dig a little more.14:36
clarkbfungi: oh ya we can probably overload that14:36
jeblairfungi, anteaya: ++ third-party ci liasons14:37
*** bradjones has quit IRC14:37
krotscheckBut once that tag’s there, all we have to do is register the repo with npm and they’ll lazy-scrape the tags.14:37
jeblairfungi: also, ++ branch-tip pipeline14:37
fungiso in theory if the projects publishing to npm only use tags for that purpose, they could leverage that pipeline (as opposed to release and pre-release where we need to be able to differentiate tags on a pattern)14:37
*** lttrl has quit IRC14:37
*** eharney has joined #openstack-infra14:38
*** salv-orlando has quit IRC14:38
*** ChuckC has joined #openstack-infra14:38
jogoclarkb: looking14:39
mtreinishclarkb: oh, fun14:39
mtreinishclarkb: http://logs.openstack.org/74/128974/1/check/check-grenade-dsvm/6b7d6f3/logs/new/screen-n-api.txt.gz#_2014-10-21_07_12_30_74114:39
mtreinishjogo: ^^^14:39
jogoclarkb: guuuu14:39
*** derekh has joined #openstack-infra14:39
jogojust saw that14:39
*** teran_ has quit IRC14:40
mtreinishdid we land a past change already?14:40
*** dangers_away has quit IRC14:40
*** teran has joined #openstack-infra14:40
jogoI didn't think so14:40
jogoclarkb: why are you up so early?14:40
krotscheckfungi: Awesome, I’ll peek into that.14:41
funginow that he works for the foundation, we're teaching him what morning looks like ;)14:41
*** marun has joined #openstack-infra14:41
*** ZZelle has quit IRC14:41
* krotscheck will probably do that when he’s ready to peel storyboard into libraries.14:42
jogomtreinish: I approved the patch :(14:42
jogonot sure why grenade is using the new api.paste file though14:42
ashpI have a horrible feeling all our jenkins jobs at work are useless because zuul has changed so much :/14:42
*** ZZelle has joined #openstack-infra14:42
*** mmaglana has joined #openstack-infra14:43
*** reed has quit IRC14:43
jeblairhuh, why do we run bandersnatch as root?14:43
mtreinishjogo: I don't think it is, it looks like it was an incubator thing that got deleted from the nova tree14:43
mtreinishI think...14:43
*** reed has joined #openstack-infra14:43
clarkbjogo taking larissa to work14:44
*** lttrl has joined #openstack-infra14:44
*** zz_jgrimm is now known as jgrimm14:44
fungijeblair: good question. though since it's the only thing of relevance on the servers where it's running i guess that's not an actual security risk14:44
clarkbashp the jobs shouldnt have changed any14:44
fungijeblair: however, now afs gives us a good reason to change that14:44
ashpclarkb: are they still meant to send json back to the zuul server on port 8001?14:45
ashpclarkb: because that's what all of ours are doing.  I've been experimenting with our replacement stuff and I can make a review, I see zuul pick it up, try to do something with it then jenkins never seems to get it14:45
jogomtreinish: you are right 641de5666f38532252e91735a1cd2611d96c489a14:45
fungiashp: how old is your zuul?14:46
* jogo blames me14:46
ashpfungi: our existing setup seems to be a couple of years old, I can find out exactly (maybe)14:46
ashplets just say our prod jenkins wasn't using any gearman plugins14:46
jogoclarkb: do you have a bug open for this?14:47
clarkbjogo not yet and I need to drive yet14:47
fungiashp: yeah, that's what i was wondering. so you're pre-gearman14:47
jogoclarkb: kk filing a bug and a revert patch14:47
ashpfeb 2013 for zuul-server binary, so I'm guessing then14:47
ashpI'm trying to figure out how to take our gate jobs and convert them to gearman14:47
jogoand sorry for approving the bad patch :(14:47
ashpwhich I assume is what I need to do14:47
clarkbjogo boo :P is it on master?14:48
fungiashp: well, as stated, the jobs themselves should be fine. i'm trying to remember what the pre-gearman api communication between jenkins and zuul looked like. i think there was an event stream zuul listened on?14:48
fungijeblair: clarkb: ^ ?14:48
mtreinishjogo: sigh, yeah we'll need to revert that14:48
clarkbif so how is icehouse master working?14:48
jogoclarkb: very good question14:48
ashpI'm brand new to all these bits, from what I see there's a callback from the job when done to zuul:8001 but nothing seems to listen there14:48
*** oramraz has quit IRC14:49
clarkbashp the callback is no longer used14:49
mtreinishjogo: it also should have been -1'd for not cleaning up openstack-common.conf, but that's something else :)14:49
clarkbreplaced by gearman14:49
mtreinishclarkb: it doesn't use the request_id middleware14:49
mtreinishso there is no issue trying to load something that doesn't exist14:49
clarkboh but juno does14:49
jeblairfungi, ashp: yeah, you should be able to drop the callback from all the jobs14:49
mtreinishclarkb: yep14:49
fungioh, right, job completion triggered a callback in jenkins to the zuul http api14:49
ashpis there a place on github you keep your current jenkins job builder yaml files so I can look at the diffs to see what it "should" look like now14:49
jeblairfungi, ashp: install the gearman-plugin from jenkins, and configure it to talk to zuul's gearman server14:50
ashphttps://github.com/openstack-infra/project-config/tree/master/jenkins/jobs i guess14:50
ashpjeblair: i did that, and I can test the connection works fine14:50
fungiashp: yep14:50
ashpbut it still seems like somehow the jobs don't get triggered14:50
fungiashp: check the gearman status in zuul (connect to the socket and issue the "status" command) to see what jobs your jenkins master has registered workers for14:51
ashpnot sure I understand the columns14:52
ashpbut it seems like those are the jobs in jenkins14:52
fungiashp: you may also need to update your layout.yaml if you haven't already14:52
*** hashar has quit IRC14:53
ashpi am mostly using the original stuff, so I'm sure layout.yaml is wrong at this point14:53
*** wenlock has joined #openstack-infra14:53
fungiashp: also current zuul has very detailed debug logging14:53
ashpI bet this would have been easier to do from scratch :)14:53
fungiashp: possibly14:53
*** mrmartin has quit IRC14:53
ashphttp://pastebin.com/96E1i2SZ the weird thing is it looks like zuul is doing the right stuff14:54
fungiashp: at least re-setting up zuul from scratch would likely have been easier. rebuilding jenkins and redefining all your jobs wouldn't buy you much14:54
ashpYeah, I kept the layout.yaml as I didn't want to rebuild their "workflow" at the same time as migrating to modern versions, I was hoping to get to modern versions and -then- go back and redesign things14:54
*** david-lyle_ has joined #openstack-infra14:54
*** stevemar has quit IRC14:55
*** stevemar has joined #openstack-infra14:56
jogoclarkb mtreinish: this got through for a funny reason14:56
jogorequest-id was turned into a middleware in Icehouse14:57
jogoso icehouse -> master didn't have the middleware at all14:57
*** chianingwang has quit IRC14:57
*** koolhead17 has quit IRC14:58
*** otherwiseguy has joined #openstack-infra14:58
fungiashp: hrm... looks like you may not have any mergers running14:58
jogothe good news is this was only for V314:58
fungiashp: newer versions of zuul have a worker daemon to process merge calculations14:59
fungiashp: it's a separate initscript14:59
ashpAhhhh, interesting, I definitely don't have any merger stuff running14:59
ashpWould the lack of that stop the jenkins gate checks from running?14:59
*** eharney has quit IRC14:59
fungiashp: for details of the gearman admin status fields see http://gearman.org/protocol/ (administrative protocol section)14:59
*** david-lyle_ has quit IRC15:00
fungiashp: you have a merger:merge with 2 pending requests, no workers available or running15:00
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: [WIP] Left Join bug when filtering for StorySummary Status  https://review.openstack.org/12992915:00
*** stevemar has quit IRC15:01
*** obondarev_ has quit IRC15:01
*** stevemar has joined #openstack-infra15:01
*** eharney has joined #openstack-infra15:02
ashpfungi: I think what I need to do is go back to basics, read about how it's "meant to work" in modern zuul/jenkins/gerrit, and start from there15:02
ashpas I'm just stumbling around in the dark trying to "patch things up" and it's not working especially great15:03
*** dangers_away has joined #openstack-infra15:03
fungiashp: not a bad idea15:03
ashpIs there a single great resource I shoudl be reading or just the ci.openstack.org linked docs?15:03
fungiashp: those, yep15:03
fungiashp: there's a "running your own" guide there which is probably mostly up to date, though likely covers more stuff than you need15:04
jogoclarkb: is juno -> master running everywhere?15:04
*** Sukhdev has quit IRC15:04
ashpi'll need all those things eventually :)15:04
jogoand if not can I get it to run on my fix patch15:04
*** marun has quit IRC15:05
*** dangers_away is now known as dangers15:05
jeblairashp: you should also reed the zuul docs15:05
jeblairashp: http://ci.openstack.org/zuul/  explains mergers, etc15:05
*** marun has joined #openstack-infra15:05
jogofungi: ^15:05
reedwhen did I become a verb? :)15:05
*** david-lyle has joined #openstack-infra15:06
jeblairreed: you've been verbed! ;)15:06
*** Sukhdev has joined #openstack-infra15:08
jeblairclarkb, fungi, mordred: i'm running bandersnatch on afstest to get some data in there15:09
*** thedodd has joined #openstack-infra15:09
*** lttrl has quit IRC15:09
jeblairi have to run now, but we can do some more with that later15:10
ashp__requires__ = 'zuul==' - this'll explain why so much has changed15:12
*** msabramo1 has joined #openstack-infra15:12
*** jistr has quit IRC15:14
*** _buzz has quit IRC15:14
*** msabramo has quit IRC15:16
*** jcoufal has quit IRC15:16
*** msabramo has joined #openstack-infra15:16
mordredclarkb: I disagree with your npm-- ... we made releasing via formatted tags a thing, and I honestly wish more people would too15:18
*** jedimike has quit IRC15:18
*** jedimike has joined #openstack-infra15:20
*** msabramo1 has quit IRC15:20
bo_shfungi: i get connection refused when running log-gearman-worker.py15:22
*** radez_g0n3 is now known as radez15:22
bo_shclarkb:  i get connection refused when running log-gearman-worker.py15:23
fungibo_sh: connection refused trying to retrieve logs from your log publication webserver?15:23
fungibo_sh: or connection refused truing to connect to the gearman server/.15:23
*** yaguang has quit IRC15:24
bo_shsocket.connect(self.host, self.port)) at worker.py _connect_socket method15:24
*** chandankumar has quit IRC15:24
*** jlibosva has quit IRC15:25
bo_shi guess it is during the gearman server15:25
mordrednibalizer: why does puppet crap out if "Universität" is in the file I'm trying to change?15:25
mordrednibalizer: Error: /Stage[main]/Openafs::Client/File[/etc/openafs/CellServDB]/content: change from {md5}456847e74ecd2a58120535bc52b49023 to {md5}0e828e6f71da39bf3a10f1f813eed4ba failed: invalid byte sequence in UTF-815:25
jeblairthat is _seriously_ uncool15:26
jeblairit's not like we can ask the rest of the world to change for puppet15:26
*** cody-somerville has quit IRC15:26
mordredjeblair: I'm investigating options that don't involve just editing the file by hand first15:26
*** dprince has quit IRC15:27
*** teran has quit IRC15:27
*** dprince has joined #openstack-infra15:27
*** denis_makogon has quit IRC15:28
bo_shfungi: socket.connect(self.host, self.port)) at worker.py _connect_socket method... i  guess it is during the gearman server15:28
*** yamamoto has joined #openstack-infra15:28
jeblairmordred: that _file_ predates utf8.  :)15:28
mordredjeblair: the best part is - it seems to be related to templates ... but there is not an invalid utf-8 byte _in_ the template - I removed it in the puppet source and replaced it with ae15:28
mordredjeblair: so, for some reason, puppet cares about the content of the file it's _replacing_ being valid utf-815:29
*** teran has joined #openstack-infra15:29
fungibo_sh: and you have a gearman service listening on the port you've configured to connect into?15:29
mordred*mind boggles*15:29
jeblairmordred: !!15:29
jeblairok, gotta run, biab15:29
mordredjeblair: have fun15:29
bo_shi have gearman client.15:30
bo_shfungi: ^^^15:30
bo_sh* a gearman15:30
fungibo_sh: have you checked the server to see if the service is listening on that port?15:31
*** chianingwang has joined #openstack-infra15:31
bo_shfungi: wait, i am pretty confused. i thought that the worker waits for the client to submit jobs.15:32
*** mancdaz is now known as mancdaz_away15:32
*** yamamoto has quit IRC15:33
*** _buzz has joined #openstack-infra15:33
bo_shfungi: and it listens on port 473015:33
mordredbo_sh: the worker and the client both connect to the server15:34
bo_shfungi: is there additional serivce/daemon that i have to run in addition to log-gearman-worker/client?15:34
bo_shmordred: yes, i know.15:34
bo_shmordred: hoo wait, sorry. what server?15:35
mordredthe gearman-server15:35
mordredthe worker connects to the server and tells it what jobs it can run15:35
*** tonytan4ever has joined #openstack-infra15:35
mordredthe client asks the server to runa  job, and one of the workers connected to the server picks it up15:35
*** berendt has quit IRC15:36
*** mancdaz_away is now known as mancdaz15:36
*** yamamoto has joined #openstack-infra15:36
bo_shmordred: according to http://ci.openstack.org/logstash.html (system arch part) there is nothing about a gearman-server15:36
bo_shit looks like gearman-client sends jobs to gearman-worker. so i don't follow what you're saying.15:37
mordredbo_sh: right - that doc is skipping gearman impl details and is talking about logically what happens15:37
mordredI can see how that would be confusing - sorry about that15:37
bo_shmordred: eh!? do you have any doc about the impl details of the gearman.15:38
*** alexpilotti has quit IRC15:38
*** koolhead17 has joined #openstack-infra15:38
fungithe client appears to embed the "gear" gearman server http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/log_processor/files/log-gearman-client.py#n12615:38
mordredah! ok15:39
bo_shfungi: ho ok. no problem.15:39
mordredcool. bo_sh - don't listen to me :)15:39
fungiso basically the client daemon provides a gearman server. the workers connect to that server, register themselves, and wait for work requests to appear15:39
*** gyee has joined #openstack-infra15:39
ashpafter dumb question but.. with the zuul merger, where does it get the git repos from in the first place?  I don't have a key or anything configured for it to pull right from github15:40
ashpdoes it fetch it from gerrit itself?15:40
*** hashar has joined #openstack-infra15:40
fungi(clarkb will correct me if i'm wrong about the log worker stuff)15:40
*** chianingwang has quit IRC15:40
*** yamamoto has quit IRC15:41
ashphmm, it is meant to be from gerrit, I wonder what I've done wrong, it's not fetching anything15:41
fungiashp: yes it will fetch refs from gerrit15:41
*** primeministerp has joined #openstack-infra15:41
*** primemin1sterp has joined #openstack-infra15:41
ashpI wonder if it needs additional rights in gerrit it doesn't have other than stream-events to do that15:41
ashpas all zuul-merger has done is make an empty dir so far :)15:41
*** primemin2sterp has joined #openstack-infra15:41
*** groknix has joined #openstack-infra15:41
*** primemin2sterp has quit IRC15:42
mtreinishfungi: well it's actually called a gear client, it still needs a gearman server running15:42
*** primemin1sterp has quit IRC15:42
*** primeministerp has quit IRC15:42
*** jistr has joined #openstack-infra15:43
*** 14WAADG9C has joined #openstack-infra15:43
*** 7GHAAEFQK has joined #openstack-infra15:43
*** primeministerp has joined #openstack-infra15:43
mtreinishfungi: the clients push the jobs on to the work queue and the workers pull them off. (or at least that's how it worked when I was experiementing with it for subunit2sql)15:43
*** changbl has joined #openstack-infra15:43
ashp2014-10-21 15:36:52,901 INFO zuul.IndependentPipelineManager: Unable to merge change <Change 0x7fa82c14c810 3853,1>15:46
ashpDamn. :/15:46
fungimtreinish: in this case the log-gearman-client.py script instantiates the gear server class and provides the server15:46
fungimtreinish: and then stuffs the work requests into it15:46
*** otherwiseguy has quit IRC15:47
*** pradk has joined #openstack-infra15:47
*** IvanBerezovskiy has left #openstack-infra15:47
fungimtreinish: hence lsof shows that script is listening on 4730/tcp15:48
*** Sukhdev has quit IRC15:48
*** otherwiseguy has joined #openstack-infra15:48
openstackgerritMonty Taylor proposed a change to openstack-infra/system-config: Run puppet even if git servers fail  https://review.openstack.org/12972215:51
openstackgerritMonty Taylor proposed a change to openstack-infra/system-config: Put playbooks in a directory  https://review.openstack.org/12995615:51
mtreinishfungi: hmm, I don't remember that when I was reading the script, just that it initialized a client and connected to the server (which I think was hardcoded as localhost)15:52
mtreinishbut whatever, I'm more inclined to believe you15:52
*** gokrokve has quit IRC15:52
clarkbfungi that sounds right to me15:52
mtreinishfungi: oh, nm me I see that now I just missed it before :)15:55
bo_shclarkb: fungi: looking at the logs it seems it says "Waiting for change in available connections to poll" and then it exits. any suggestions?15:55
*** pblaho has quit IRC15:55
*** emagana has quit IRC15:55
*** emagana has joined #openstack-infra15:56
*** ilyashakhat has quit IRC15:57
fungibo_sh: is it successfully connecting to the zeromq publisher plugin on your jenkins master?15:58
marcusvrn_Hi all! how can I configure jenkins to not report back the results?15:58
clarkbI am not familiar with that error bo_sh15:59
jesusaurusclarkb: i finally got my test to pass on 74943, you mind adding that to your queue?16:00
*** emagana has quit IRC16:00
bo_shfungi: the client logs doesn't mention anything about successfully connecting to the zmq publisher on my jenkins.16:01
fungimarcusvrn_: it depends on what you're using to connect to gerrit... gerrit-trigger plugin for jenkins? zuul server?16:01
*** alexpilotti has joined #openstack-infra16:01
bo_shfungi i can try and run the job that sends zmq, though16:01
marcusvrn_fungi: zuul server16:01
*** ildikov has quit IRC16:02
*** zaphod42 has joined #openstack-infra16:02
fungimarcusvrn_: look at how we configure our "silent" pipeline: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n11016:02
*** yamahata has quit IRC16:03
*** markmcclain has quit IRC16:03
bo_shfungi: i executed the job, but the logs don't say anything about receiving an event...16:03
fungibo_sh: well, first try manually connecting from the machine where you're running the log-gearman-client daemon to the zeromq port on the jenkins master16:03
*** emagana has joined #openstack-infra16:03
fungibo_sh: just to rule out connectivity issues between them, or the possibility that the zeromq event publisher isn't active or something16:04
*** alexpilotti_ has joined #openstack-infra16:04
mordredclarkb: also, 129956 and 129722 would be nice if you're reviewing things :)16:06
*** alexpilotti has quit IRC16:07
*** alexpilotti_ is now known as alexpilotti16:07
*** MarkAtwood has joined #openstack-infra16:07
bo_shfungi: okay. it was my bad. i didn't give the correct publisher. now it works (the client gets the zmq)16:08
bo_shfungi: however, the worker still fails "Errno 111 connection refused"16:08
fungibo_sh: does it say in the log what it's trying to do when it raises that exception? trying to connect to the gearman server?16:09
*** MarkAtwood has quit IRC16:10
bo_shyes, it happens when trying to connect to the gearman-host:gearman-port16:10
fungibo_sh: if so, confirm the machine where the worker daemon is running can reach the gearman server port on the machine where the client is running16:10
bo_shho wait16:10
*** cody-somerville has joined #openstack-infra16:10
*** MarkAtwood has joined #openstack-infra16:10
*** VijayTripathi has joined #openstack-infra16:10
ashpSomeone must know - does the zuul-merger require an initial checked out copy of the repo to operate on?16:10
ashpBecause it seems like it only gets some json over ssh, and doesn't know how to check out the repo at all from gerrit16:11
bo_shfungi: both the client and the worker are on the same host. in the worker.yaml i have configured gearman-host: localhost and gearman-port: 473016:11
*** jlibosva has joined #openstack-infra16:11
marcusvrn_fungi: looking at the link that you shared, the difference between silent and check pipeline is that the first one doesn't have success and failure sections, does it?16:11
fungiashp: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/merger/merger.py#n4216:12
ashpI wonder why that fails then :/16:12
ashpI have no match for "Cloning from" either in the logs16:12
ashpso it's not even trying16:13
fungibo_sh: and you see :::4730 or listening according to netstat?16:13
*** afazekas has quit IRC16:13
fungimarcusvrn_: correct, if there is no success or failure action then zuul will not report to the change for those states16:13
zaroclarkb: anytime you are ready to do the GP upgrade.16:14
marcusvrn_fungi: ok! thanks for your help!! :)16:14
*** vdo has quit IRC16:14
fungibo_sh: or maybe16:14
bo_shfungi: netstat -ntlp gives
bo_shin the local address16:15
fungibo_sh: try 'echo status|nc -q 3 localhost 4730'16:15
bo_shpush-log        43      0       016:16
bo_shfungi: ^^^^16:16
*** msabramo has quit IRC16:17
clarkbzaro I did 06 and 07 already16:17
fungibo_sh: okay, so that confirms that the gearman service is active there, and has pending work requests, but no registered workers16:17
clarkbwill do more when check queue clears16:17
bo_shfungi: i will try to figure this out. i will let you know if i need any further help. thanks16:17
*** bo_sh has quit IRC16:18
fungibo_sh: it's possible that for some reason the worker is having trouble resolving the name "localhost"16:18
fungioh poo, he left16:18
*** ihrachyshka has quit IRC16:18
ashpoh for fu... it looks like zuul-merger fails as it sits around trying to authenticate an ssh key16:19
*** isviridov is now known as isviridov_away16:19
fungiashp: you may need to manually install the gerrit service's public host key into the known hosts for your zuul merger16:20
*** signed8bit is now known as signed8bit_ZZZzz16:20
fungiashp: or manually connect to gerrit's ssh api from that account and accept the server's public key16:20
ashpyeah, going to add that now16:20
fungiashp: we use puppet to install an entry in ~zuul/.ssh/known_hosts16:20
nibalizerjeblair: mordred ohstupid puppet16:22
*** msabramo has joined #openstack-infra16:22
ashpthat definitely seemed happier16:22
ashpnibalizer: "stupid puppet", words I live by ;)16:22
mordrednibalizer: YOU FIX NOW!!!!16:23
*** mancdaz is now known as mancdaz_away16:23
mordrednibalizer: but seriously - why is it even reading the target to-be-replaced file? is it because I was running puppet agent --test and it needed to show me diffs?16:24
zaroclarkb: ohh cool, did you install thru the jenkins plugin download page (from jenkins repo)?16:24
*** rmcall has joined #openstack-infra16:24
yolandajedimike, i'm running the tests in my environment to check if works16:25
*** mpaolino has quit IRC16:25
jedimikeyolanda, come join us in #storyboard :)16:25
*** gokrokve has joined #openstack-infra16:25
*** teran has quit IRC16:28
*** jp_at_hp has quit IRC16:28
*** yamamoto has joined #openstack-infra16:28
openstackgerritKhai Do proposed a change to openstack-infra/jenkins-job-builder: add documentation for the delete command  https://review.openstack.org/12810516:29
*** cdent has quit IRC16:29
*** AJaeger_ has joined #openstack-infra16:31
clarkbzaro no I did it on disk16:31
*** thedodd has quit IRC16:31
openstackgerritKhai Do proposed a change to openstack-infra/jenkins-job-builder: add documentation for globbed parameter feature  https://review.openstack.org/12847616:32
*** thedodd has joined #openstack-infra16:32
*** yamamoto has quit IRC16:33
openstackgerritKhai Do proposed a change to openstack-infra/jenkins-job-builder: add a JJB command reference  https://review.openstack.org/12810616:33
*** packet has joined #openstack-infra16:34
*** signed8bit_ZZZzz has quit IRC16:34
*** packet has quit IRC16:34
*** packet has joined #openstack-infra16:34
nibalizermordred: well so16:34
nibalizeri mean that ticket is ancient16:34
nibalizerbut they claimed to fix it16:34
nibalizerand poking it on my local system it looks like its fixed16:34
nibalizerbut im running 3.416:34
nibalizerill go to 3.7 and see whats up16:34
nibalizerif they regressed that im gonna lol16:35
mordrednibalizer: I can send you the file that it doesn't want to overwrite if you like16:35
*** mpaolino has joined #openstack-infra16:35
nibalizerwould you mind terribly finding the file thats being replaced so I don't have to install the openafs packages ?16:35
nibalizermordred: thats exactly what i want16:35
mordrednibalizer: sent to your hp email16:36
*** teran has joined #openstack-infra16:37
nibalizermordred: woot16:38
nibalizeralso from the 'pointless and cool' desk comes this http://4.bp.blogspot.com/-GxaD0K8bvTg/VEX4t6MrgII/AAAAAAAAATQ/tjxuf83us10/s1600/diagram_grhapite.png16:38
jogomordred: how do I tell pbr to not build man pages16:38
mordredjogo: uhm.16:38
jogothe governance repo doesn't need a man page16:38
mordredjogo: oh - you mean the build_sphinx?16:38
jeblairjogo: we TOTALLY need to be able to do "man openstack-tc"16:39
zaroclarkb: ohh opps, i forgot to create a tag. so it's not on the jenkins repo anyways.  will do it now.16:39
jeblairfrankly, i would like to read that man page ;)16:39
mordredjogo: looking - honestly not sure that's an option - is it causing a problem?16:40
jogomordred: not a problem per se, but I want to enable error on warning16:40
*** otherwiseguy has quit IRC16:41
mordredjogo:             if self.builder == 'man' and len(sphinx_config.man_pages) == 0:16:41
mordred                return16:41
nibalizerim actually with jeblair, more man pages16:42
jogo mordred, ARNING: no "man_pages" config value found; no manual pages will be written16:42
*** teran has quit IRC16:42
mordredjogo: so ... pbr should not be actually running sphinx if there are no manpages to build16:43
krotscheckclarkb: So there’s this -> https://review.openstack.org/#/c/128736/16:43
mordredjogo: the bug is in the repo16:43
mordredjogo: the doc/source/conf.py in governance defines a manpage16:43
krotscheckWhich explicitly has a “Hey maybe clark should weigh back in on this” comment16:43
funginibalizer: where did you get the diagram tool?16:43
*** arxcruz has quit IRC16:44
clarkbkrotscheck Im good with it16:44
* jogo blames the crumby HP trackpad16:44
mordredjogo: it's a terrible object16:44
krotscheckclarkb: Cool, can you +3 it?16:44
clarkbkrotscheck ya let me check the one tyop thing before I do16:45
jogomordred: un wedge the governance repo: https://review.openstack.org/12997316:45
*** dimsum_ is now known as dims16:45
mordredjogo: I can't do anything ... ttx ^^^ please land16:46
nibalizerfungi: so that led to much rage, pip install diagram is on the readme, but it wasn't working last night16:46
zaropelix: +2 from you on these doc changes and i'm gonna approve.  https://review.openstack.org/#/c/128106 and https://review.openstack.org/#/c/12847616:47
*** jgallard has quit IRC16:47
jogomordred: also governance doesn't have pbr as a dep16:47
*** mpaolino has quit IRC16:47
* AJaeger_ ha sput the python 2.6 deprecation on today's infra meeting agenda - I'm confused on how to move forward with the followup patches..16:47
*** ihrachyshka has joined #openstack-infra16:47
funginibalizer: interesting16:47
*** derekh has quit IRC16:47
nibalizerfungi: okay you can pip install it now16:47
nibalizermaybe the mirror my machine was hitting hadn't synced16:47
jeblairjogo: fyi https://review.openstack.org/#/c/125782/116:47
funginibalizer: i can't wait. https://pypi.python.org/pypi/diagram looks neat16:47
nibalizerhere is teh sauce https://github.com/tehmaze/diagram16:48
jogojeblair: cool, just ran a recheck we will see16:48
*** dteselkin_ has joined #openstack-infra16:49
*** sdake_ has quit IRC16:50
*** otherwiseguy has joined #openstack-infra16:50
ttxjogo: unwedged16:50
mordredjogo: it doesn't need one16:53
pelixzaro: believe https://review.openstack.org/#/c/128106 needs to be rebased16:54
mordredjogo: pbr in requirements is only needed if there are runtime deps caused by importing pbr in the code for thigns like versions16:54
jogomordred: and the oslosphinx stuff already has it in there I guess16:54
jogottx: now to wait until this get through the check queue16:54
jogoof 60016:55
*** talluri has joined #openstack-infra16:56
clarkbjogo: where is that revert for the grenade thing?16:57
*** davidhadas_ has quit IRC16:57
openstackgerritJames E. Blair proposed a change to openstack-infra/system-config: Add kerberos and afs servers to cacti  https://review.openstack.org/12997716:57
clarkbjogo: also how did you figure out it was a paste thing?16:57
nibalizermordred: so.... http://paste.openstack.org/show/122779/ is my attempt to repro16:57
*** sarob has joined #openstack-infra16:57
mtreinishclarkb: https://review.openstack.org/12992616:57
*** mwagner_lap has quit IRC16:58
mordrednibalizer: FASCINATING16:58
*** tonytan4ever has quit IRC16:58
*** mrmartin has joined #openstack-infra16:59
*** ivar-lazzaro has joined #openstack-infra16:59
nibalizermordred: is that functionaly different than what you've been doing? it doesn't look different to me16:59
clarkbmtreinish: thank you16:59
nibalizercrinkle: ashp ^^ thoughts?16:59
mordrednibalizer: nope. it looks the same - the only difference I can see is apply vs. agent?16:59
*** dizquierdo has quit IRC17:00
*** dizquierdo has joined #openstack-infra17:01
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Remove openstack-common.conf  https://review.openstack.org/12958517:01
*** lttrl has joined #openstack-infra17:01
mordrednibalizer: WOW. I cannot reproduce it now - I just tried putting the content back and now it's happily replacing it17:01
*** dane_leblanc has quit IRC17:01
*** packet has quit IRC17:01
openstackgerritKhai Do proposed a change to openstack-infra/jenkins-job-builder: add a JJB command reference  https://review.openstack.org/12810617:02
*** _nadya_ has quit IRC17:02
mordrednibalizer: oh. ok.17:02
mordrednibalizer: so - if I directly copy the file from my box into place, it breaks17:02
mordrednibalizer: if I use vim to cut and paste the line, it does not17:02
zaropelix: ok, it's rebased.17:02
mordrednibalizer: (like, if I scp the file)17:02
jeblairmordred: so maybe you're not replacing exactly the same data with vim17:03
jogoclarkb:  knowing the code base17:03
fungimordred: yes, vim is detecting the encoding and then you're cutting to your buffer from your terminal encoding rather than the file encoding17:03
jeblairmordred: you might be encoding it in utf8 by doing that17:03
nibalizermaybe vim is helping with some cleanup or something17:03
jogoand the request_id thing17:03
jeblairfungi: what fungi said :)17:03
mordredjeblair: yah17:03
jogoclarkb: https://review.openstack.org/#/c/129926/17:03
jeblairmordred: what fungi said17:03
jogoclarkb: stuck in the check queue of doom17:03
nibalizerya fungi used words that sound smart17:03
mordredso I'm wondering if emailing the file to nibalizer had the same results17:03
jeblairmordred: possibly.  may want to tar it.17:03
* fungi is good at sounding smart while having no clue what he's actually saying17:03
anteayaah I keep trying to figure out what it means17:04
nibalizermordred: i mean you introduced outlook into the equation when you emailed it to me17:04
fungiand yes, entirely possible that the file is getting converted if it's a text/plain attachment17:04
nibalizerso all bets are off17:04
anteayalike a philosophical puzzle17:04
mordrednibalizer: sent tarred17:05
mordrednibalizer: it's the diff17:06
*** Ryan_Lane has joined #openstack-infra17:06
mordredif I run puppet agent --onetime --no-daemonize --verbose --no-usecacheonfailure --ignorecache --no-splay17:06
mordredit does not complain17:06
jogoclarkb: so it will be a day  before my patch can be approved17:07
nibalizerso the act of generating the diff casuses puppet to think harder about the situation and crap itself17:07
mordredSO - it will not be a problem for us in our normal runs because we don't use test17:07
jogoclarkb: both governance and nova17:07
clarkbjogo: well it just got approved17:07
mordrednibalizer: yes17:07
clarkbjogo: I am tempted to promote it17:07
jogoclarkb: no objection from me17:07
clarkbmordred: fungi jeblair any thoughts on that? its not gate breaking in that the gate doesn't work17:07
jeblairclarkb: what patch?17:07
clarkbbut it is gate breaking in that juno to master grenade does not work17:07
*** zigo has quit IRC17:08
clarkbjeblair: https://review.openstack.org/12992617:08
nibalizermordred: well here is the bug if you want to rage at them https://tickets.puppetlabs.com/browse/PUP-103817:08
clarkbI am fine if you want ot hold off on promoting that (I can go either way)17:08
*** zigo has joined #openstack-infra17:08
*** teran has joined #openstack-infra17:08
*** dane_leblanc has joined #openstack-infra17:08
*** hashar has quit IRC17:08
jeblairclarkb: if juno->master grenade does not work, why is the gate not broken?17:09
fungiclarkb: so that was blocking us switching master to upgrade from juno instead of icehouse?17:09
*** teran_ has joined #openstack-infra17:09
clarkbjeblair: because icehouse -> master is what we are testing now until my d-g changes get in17:09
clarkband icehouse -> master does work17:09
clarkbfungi: yup17:09
clarkbjeblair: there is apparently a corner case where previous branch to master works but current stable to master cannot17:09
*** koolhead_ has joined #openstack-infra17:09
jeblairclarkb: seems sufficiently time-sensitive and gate-breaking-related to promote to me.17:09
fungii can see an argument that delaying correcting the grenade test to be testing what we want leaves us exposed17:10
*** luqas has quit IRC17:10
clarkbfungi: ya17:10
fungiso wfm17:10
clarkbthat was my angle to nova devs reviewing :)17:10
jeblairthen let's be consistent :)17:10
clarkbok I will enqueue to gate and promote when I get a chance17:10
jogoclarkb: yeah getting grenade to juno would be nice17:10
jogoclarkb: thanks17:10
*** hdd has joined #openstack-infra17:10
clarkbstill catching up on things post breakfast17:10
fungianybody have good suggestions for where we want to host the devstack.org -> docs.o.o/developer/devstack redirect?17:11
annegentlefungi: you can do it in the openstack-manuals/www/static/.htaccess17:11
annegentleI think I'd prefer it17:12
*** teran_ has quit IRC17:12
fungiannegentle: okay, great!17:12
*** jpich has quit IRC17:12
fungiannegentle: i'll offer a patch in moments17:12
annegentlefungi: cool17:12
*** teran has quit IRC17:13
*** koolhead17 has quit IRC17:13
openstackgerritMonty Taylor proposed a change to openstack-infra/system-config: Add detailed-exitcodes to launch_node  https://review.openstack.org/12998317:13
*** signed8bit has joined #openstack-infra17:13
fungiwhere moments is defined as slightly longer than it takes me to clone openstack-manuals at ~100mb/sec17:13
fungigo hamsters, go!17:14
mordredjeblair: ^^ that should fix the thing you had a problem with when launching that node17:14
jeblairmordred: 'vos release' requires superuser access17:14
jeblairmordred: thank you17:14
mordredjeblair: oh. hrm17:14
*** signed8bit has quit IRC17:14
mordredjeblair: well that's disappointing17:14
*** rmcall has quit IRC17:14
jeblairmordred: so we may need some extra machinery around that, assuming we don't want to give bandersnatch superuser17:15
*** signed8bit has joined #openstack-infra17:15
fungisome sort of signaling event17:15
*** harlowja_away is now known as harlowja17:15
mordreda releaser worker or something that takes vos release requests17:15
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: add documentation for the delete command  https://review.openstack.org/12810517:15
*** mrmartin has quit IRC17:15
fungiand then a listener which just does that as a superuser, yeah17:15
harlowjais the gate on fire?17:16
harlowjaor zuul on fire :-P17:16
fungiharlowja: i think it's not ablaze so much as steadily smoldering17:16
harlowjaits getting barbecued17:16
jeblairshould be out by tomorrow17:16
mordredmmm. barbecue17:16
*** mfink_ has joined #openstack-infra17:17
*** HeOS has quit IRC17:17
*** rmcall has joined #openstack-infra17:17
fungislow-roasted check jobs17:17
fungiyou can taste the anticipation baked right in17:17
*** markmcclain has joined #openstack-infra17:18
*** dteselkin__ has joined #openstack-infra17:18
jogodocs look nice: http://docs-draft.openstack.org/73/129973/1/check/gate-governance-docs/5410d4c/doc/build/html/17:18
mordredalso, I'm in memphis today, so I think that's my cue to go find some bbq17:18
fungimordred: seriously, what are you doing talking to us. go get bbq asap17:19
jeblairand bring some to the infra meeting17:19
*** mfink has quit IRC17:19
fungiin memphis, there's always enough bbq to share17:19
clarkbactually gate looks mostly healthy I think an enqueue is all that I need not a promote17:19
clarkbso I will start wit that17:19
jeblairclarkb: yeah, probably won't be too long17:20
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: add documentation for globbed parameter feature  https://review.openstack.org/12847617:20
*** dteselkin_ has quit IRC17:21
*** thedodd has quit IRC17:21
*** katyafervent is now known as katyafervent_awa17:21
*** chandankumar has joined #openstack-infra17:21
fungiannegentle: looking over the .htaccess idea... i'm not sure this is a use case we can address there. isn't cloudsites going to have a problem with the request being for a name-based virtual host it doesn't know about? also what ip address would i put in dns for the devstack.org address records in that event?17:22
clarkbjeblair: for the --trigger arg to zuul enqueue I want 'gerrit' ?17:23
*** radez is now known as radez_g0n317:24
*** dizquierdo has quit IRC17:24
jogoclarkb: what happened to multi-node gate jobs?17:24
fungijogo: afazekas's change to implement them is still wip i think17:24
clarkbya I think there was some waffling on whether or not we need to use private addrs17:24
jeblairclarkb: i think?  check root's history for confirmation :)17:25
clarkband things went quiet17:25
fungiclarkb: sudo zuul enqueue --trigger gerrit --project openstack/tempest --pipeline gate --change 123510,117:25
*** patrickeast has joined #openstack-infra17:25
fungi(or whatever parameters you need for your change)17:25
jogofungi: too bad, this is something that would be nice to ahve17:26
fungijogo: no argument there17:26
clarkbfungi: jeblair ty17:26
clarkbit is enqueued now17:26
*** radez_g0n3 is now known as radez17:26
clarkbjogo: if you want to pick it up go for it17:27
clarkbI honestly thinks its 99% there and we don't need the additionaly features17:27
*** ryanpetrello has quit IRC17:27
anteayajogo: and we do need the capacity to do multi-node testing17:27
*** amcrn has joined #openstack-infra17:27
*** cpowell has quit IRC17:27
jogoanteaya: hrm yeah17:28
*** yamamoto has joined #openstack-infra17:28
*** ZZelle_ has joined #openstack-infra17:28
bknudsonhave you ever seen this oddity before? https://review.openstack.org/#/c/125923/ -- if you expand patch set 4 the author section is after the files?17:28
jogoclarkb: at this point we should pick this up at the summit I think17:29
*** tonytan4ever has joined #openstack-infra17:29
anteayabknudson: when I expand patchset 4 the author section for patchset 4 is before the patchset 4 files17:30
fungisame here17:30
*** whoops has joined #openstack-infra17:30
jeblairjogo: why?  afaik it needs doing, not talking :)17:30
anteayabknudson: and the author sections for patchset 5 is below that17:30
bknudsonanteaya: ok, weird... I'll try restarting my browser.17:30
anteayafollowed by the files for patchset 517:30
*** markmcclain1 has joined #openstack-infra17:31
bknudsonprobably the poodle attack.17:31
*** markmcclain has quit IRC17:31
anteayajeblair jogo more doing, less meetings17:31
*** ryanpetrello has joined #openstack-infra17:31
*** chandankumar has quit IRC17:31
jogojeblair: so we could move forward even if we don't get more capacity?17:32
*** yamamoto has quit IRC17:33
*** markmcclain1 has quit IRC17:33
*** markmcclain has joined #openstack-infra17:33
jeblairjogo: i believe anteaya was suggesting that as a project we need the *ability* to do multi node testing and therefore continuing work is important.  not that we require more test node quota before we can implement it.17:33
jeblairanteaya: correct me if i'm wrong :)17:33
*** tonytan4ever has quit IRC17:34
anteayacontinuing work is important yes17:34
*** markmcclain1 has joined #openstack-infra17:34
*** markmcclain has quit IRC17:34
*** thedodd has joined #openstack-infra17:35
anteayaheck if we have the abilty to do mulit-node tests serially that would work, meaning one patch at a time, with each patch in the multi-node test pipeline waiting for the patch preceding to finish17:35
fungii definitely don't see our worker capacity as a blocker to implementing a multi-node devstack job. it might be something we don't run a lot of initially due to node capacity concerns, but deciding that before we even try it seems premature17:35
*** dane_leblanc has quit IRC17:35
anteayaand by multinode we could start with two nodes17:35
anteayathat is all dvr needs to begin with17:35
jeblairyep.  i'm pretty sure that was just a misunderstanding.  if that's the case, then we are all in agreement and we do not need a summit session to determine that.  :)17:36
anteayajogo: point me to a patch to review17:37
anteayajogo: and I'll round up some friends to join me17:37
jogoahh works for me17:39
fungiso, assuming the limitations for .htaccess-based redirects will prevent us from implementing it on docs.openstack.org, is static.o.o an okay place for us to stick a hollow devstack.org vhost to do the redirecting to the docs site, or is there a better place (or do we need to make a new place for that sort of thing)?17:39
jogoanteaya: not sure where the multi node patch17:39
*** MaxV has quit IRC17:39
anteayajogo: yay17:39
anteayajogo: me either17:39
anteayalet's find it17:39
jeblairfungi: i like static for that17:39
clarkbfungi: I haven't followed everything. Why can't docs.o.o htaccess do it?17:40
*** davidhadas_ has joined #openstack-infra17:40
fungiclarkb: because the initial request is coming to a different http/1.1 name-based virtual host17:40
clarkboh right17:40
fungiclarkb: i _believe_ .htaccess-based redirects don't handle that use case at all17:40
jogoanteaya: https://review.openstack.org/#/q/status:open++branch:master+topic:mnode-poc,n,z17:40
clarkbfungi: I am slightly worried it will get lost on static.o.o But it should be fine there17:41
fungiat least i can't find an explanation of how to do it17:41
clarkbfungi: ya I think you are correct17:41
*** shashankhegde has joined #openstack-infra17:41
anteayajogo: nice17:41
clarkbso we seem to have jobs restarting in zuul17:41
fungiand also, i think cloudsites will just barf if a client connects to its http socket and passes a hostname header for a site it knows nothing about17:41
clarkbin theory because jenkins is unhappy with the number of concurrent jobs?17:41
dhellmannanteaya: as an election official, would you like to comment on the governance change to set the new TC membership? https://review.openstack.org/#/c/129274/17:41
*** zigo has quit IRC17:41
clarkbI am going to try and track one down17:41
*** amotoki has quit IRC17:41
*** zigo has joined #openstack-infra17:41
* anteaya as an election official, puts down her soup and reviews17:42
fungior else we end up setting up a devstack.org cloudsites site to host the redirect to docs.o.o17:42
clarkb(I just noticed the gate-nova-pep8 job for the nova revert change went back to queued)17:42
*** mwagner_lap has joined #openstack-infra17:42
clarkbtrying to find where it ran now17:42
fungiat least i get the impression cloudsites hosts sites for multiple vhosts on a given server and therefore its default vhost is probably a bitbucket or error page17:43
fungitesting that now17:43
*** jistr has quit IRC17:43
fungiyep, it's a page which just says "Please come back later."17:44
clarkbhrm no record of it after a quick look across the jenkinses17:44
clarkboh wait there it is17:44
clarkbzaro: jeblair https://jenkins06.openstack.org/job/gate-nova-pep8/7517/console ran and succeeded17:44
clarkbbut zuul is rerunning that job17:44
clarkbnow going to look at zuul logs17:45
clarkbbut the gearman-plugin may be broken...17:45
anteayadhellmann: thank you17:45
jeblairclarkb: do you suspect this is happening at a greater frequency on the hosts with the upgraded gearman plugin?17:45
dhellmannanteaya: next cycle we should remember to ask you to submit the patch :-)17:45
clarkbjeblair: not yet17:45
anteayadhellmann: :D17:45
anteayadhellmann: ttx is just so efficient17:46
clarkbjeblair: I only have one data point so trying to figure out why that happened17:46
dhellmannanteaya: indeed17:46
jeblairclarkb: i have noticed a lot of jobs that look like they have been restarted.17:46
clarkbjeblair: 2014-10-21 17:34:56,130 INFO zuul.Gearman: Build <gear.Job 0x7f8f3caa3450 handle: H: name: build:gate-nova-pep8:bare-trusty unique: 891401e56d2d49a793d8412b1a37b3bd> complete, result None17:47
clarkbjeblair: I think result: None is what is breaking us17:47
clarkbbut I don't think I can easily go from that back to which jenkins server ran the job17:47
*** spzala has quit IRC17:48
jeblairclarkb: grep 891401e56d2d49a793d8412b1a37b3bd17:48
*** emagana has quit IRC17:48
clarkb~3k occurences of result none17:48
*** jlibosva has quit IRC17:48
*** emagana has joined #openstack-infra17:49
*** david-lyle has quit IRC17:49
anteayathat is a whole lot of none17:49
jeblairclarkb: let's pull out 20 of them at random and see what hosts they are on?17:49
*** VijayTripathi has quit IRC17:49
*** mfer has quit IRC17:49
clarkbjeblair: ok17:49
*** david-lyle has joined #openstack-infra17:49
jeblairclarkb: you want to do that or shall i?17:49
*** dane_leblanc has joined #openstack-infra17:50
*** jedimike has quit IRC17:51
*** mrmartin has joined #openstack-infra17:51
clarkbjeblair: I am working on it17:51
clarkbI just did grep 'result None' | tail17:52
*** berendt has joined #openstack-infra17:52
clarkband am semi manually getting uuids from that and grepping17:52
clarkbdone ~6 and they are all on 07 or 06 so far17:52
*** mfer has joined #openstack-infra17:52
*** ivar-lazzaro has quit IRC17:52
fungithat doesn't bode well for the plugin upgrade :/17:53
*** teran has joined #openstack-infra17:53
*** emagana has quit IRC17:53
*** ivar-lazzaro has joined #openstack-infra17:53
clarkbyup did 1017:53
clarkbI think we need to revert17:53
clarkbzaro: ^17:53
fungiwell, at least that should help our throughput a bit17:54
marcusvrn_asselin: anteaya: Hi! I have changed the CI to use my SSH key and account...to run jobs in "silent" mode in order to generate a log to re-enable the CI, but it's not triggering the jobs....zuul-merger returns "Permission denied (publickey)."17:54
*** emagana has joined #openstack-infra17:54
clarkbfungi: jeblair: I can do that if you agree that is the next step we should take17:54
fungiclarkb: i think this is probably not a great time to try to debug it in production17:54
jeblairclarkb: yes17:55
clarkbok starting that now17:55
jeblair(just offlining 06 and 07 will start to help our throughput17:55
asselinmarcusvrn_, something probably didn't update correctly since you changed your key.17:55
*** ivar-lazzaro has quit IRC17:56
*** jistr has joined #openstack-infra17:56
*** ivar-lazzaro has joined #openstack-infra17:56
anteayamarcusvrn_: you can't change your ci account to use your ssh key17:57
clarkbzaro: jeblair: I think it is likely that gearman worker is shutdown before reporting the success back17:57
clarkbzaro: jeblair but after the job finishes17:57
clarkbalso it doesn't seem to be related to job runtime so maybe a network packet race?17:58
asselinmarcusvrn_, double-check /var/lib/zuul/ssh/id_rsa17:58
*** bdpayne has joined #openstack-infra17:58
clarkbit doesn't seem to occur on all jobs either best I can tell17:58
marcusvrn_anteaya: I change my ci account to my personal account17:58
anteayamarcusvrn_: what happens when you open a terminal and run `ssh -p 29418 yourgerritusername@review.openstack.org gerrit stream-events`17:58
clarkbboth 06 and 07 are in shutdown mode17:58
*** spzala has joined #openstack-infra17:58
openstackgerritA change was merged to openstack-infra/elastic-recheck: Don't show ER patches under open reviews  https://review.openstack.org/12511717:59
asselinmarcusvrn_, you may need to completly stop and restart zuul as well....17:59
fungimarcusvrn_: you probably need to make sure you've reconfigured zuul to use the username of your normal user, as well as the ssh key of your normal user17:59
zaroclarkb: ok.  i'm gonna test the response from the server.17:59
*** johnthetubaguy is now known as zz_johnthetubagu18:00
*** sarob has quit IRC18:00
*** sarob has joined #openstack-infra18:01
*** sarob has quit IRC18:01
asselinmarcusvrn_, yes, as fungi said: https://github.com/rasselin/os-ext-testing-data/blob/master/vars.sh.sample#L718:02
*** dane_leblanc has quit IRC18:02
marcusvrn_asselin: fungi anteaya yes, I did it...maybe I forgot to change the user/ssh-key in somewhere else...I'll check again!18:04
anteayaI did it18:04
*** VijayTripathi has joined #openstack-infra18:04
anteayadoes that mean you have achieved your goal?18:04
*** _nadya_ has joined #openstack-infra18:04
anteayaand your zuul can hear the stream?18:05
fungithe "i'll check again" makes me think it did not help18:05
anteayatwo small pieces of information that feel contradictory18:06
*** arnaud__ has joined #openstack-infra18:07
*** freemanb_ has joined #openstack-infra18:07
marcusvrn_anteaya: fungi lol, sry! let me explain..if I run `ssh -p 29418 yourgerritusername@review.openstack.org gerrit stream-events` in my personal machine I can hear the stream18:07
*** e0ne has quit IRC18:07
*** _buzz has quit IRC18:08
marcusvrn_but when I configure my personal-gerrit-user and personal-ssh-key in the machine that zuul is running...It doesn't work18:08
*** _buzz has joined #openstack-infra18:09
openstackgerritMichael Tupitsyn proposed a change to openstack-infra/jenkins-job-builder: Support additional events for HipChat plugin  https://review.openstack.org/12976018:09
fungimarcusvrn_: in this case adjusting your gerrit user in /etc/zuul/zuul.conf and replacing the content of /var/lib/zuul/ssh/id_rsa (and then possibly restarting zuul) is all you should need to do18:09
*** hashar has joined #openstack-infra18:09
*** mfink__ has joined #openstack-infra18:09
fungimarcusvrn_: also make sure you don't change the ownership/permissions on /var/lib/zuul/ssh/id_rsa when replacing it18:10
*** sdake_ has joined #openstack-infra18:10
*** teran has quit IRC18:10
*** talluri has quit IRC18:10
*** freemanbrandon has quit IRC18:11
marcusvrn_fungi: yes, let me do it and see what it returns :)18:11
fungimarcusvrn_: and backing up /var/lib/zuul/ssh/id_rsa so you can restore it to normal later when your account is reactivated is a good idea too18:13
*** tonytan4ever has joined #openstack-infra18:13
asselinmarcusvrn_, btw, this is the name of the private key that get's put in  /var/lib/zuul/ssh/id_rsa https://github.com/rasselin/os-ext-testing-data/blob/master/vars.sh.sample#L918:13
*** mfink_ has quit IRC18:13
*** radez is now known as radez_g0n318:14
*** radez_g0n3 is now known as radez18:15
*** rmcall has quit IRC18:16
*** dane_leblanc has joined #openstack-infra18:16
zaroclarkb: i'm getting valid responses from gearman server.18:16
krotscheckclarkb: Ever get around to checking that typo? https://review.openstack.org/#/c/128736/18:16
clarkbzaro: the responses are valid which is why zuul doesn't say LOST18:17
clarkbzaro: but the responses have no result. the result is None which means go rerun the job18:17
clarkbzaro: also I do not think it happens 100% of the time. I believe this is a race18:17
*** mfink_ has joined #openstack-infra18:18
zaroclarkb: the response i get looks like this: ['{"manager":"10-0-0-8.openstacklocal","name":"guava","worker":"10-0-0-8.openstacklocal_exec-0","number":24,"url":""}', '{"result":"SUCCESS","manager":"10-0-0-8.openstacklocal","name":"guava","worker":"10-0-0-8.openstacklocal_exec-0","number":24,"url":""}']18:18
zaroclarkb: isn't the result in that response valid?18:19
clarkbzaro: 2014-10-21 18:18:02,682 INFO zuul.Gearman: Build <gear.Job 0x7f8ef5d09910 handle: H: name: build:check-tempest-dsvm-neutron-full:devstack-trusty unique: 6ac4fcf9a70f49eaa0981db15cce05bc> complete, result None18:19
clarkbthat is what I get18:19
*** mfer has quit IRC18:19
zaroclarkb: i can't seem to repro that in my env.18:19
*** mfink__ has quit IRC18:20
clarkbzaro: there is no json blob like the above18:20
hashargood evening18:20
clarkbyes I think it is a race...18:20
zaroclarkb: ohh i didn't set node to offline though.  let me give that a try18:20
*** mfink has joined #openstack-infra18:20
*** rmcall has joined #openstack-infra18:20
anteayahashar: good evening18:21
hasharSeems you guys Zuul  instance has some trouble triggering jobs ..  Turns out I have more or less the same issue on my instance from time to time18:22
hasharbut have yet to figure it out :/18:22
clarkbhashar: the gearman-plugin is broken I think18:22
clarkbit doesn't handle offlined nodes properly18:23
hasharoh you upgraded it ?18:23
clarkbyes upgraded on two jenkins18:23
*** thedodd has quit IRC18:23
*** mfink_ has quit IRC18:23
asselinfungi, marcusvrn_ I just ran into this issue.  I need to double-check still, but need to step out. So just FYI for now in case you have it too: http://paste.openstack.org/show/122799/18:23
clarkband am working on downgrading them now18:23
fungiasselin: i believe there is a fix proposed for that already18:23
hasharthough I am pretty sure I manually tested that whenever a node is removed the associated functions are unregrestired18:23
clarkbhashar: that isn't the issue18:24
clarkbhashar: the problem is that jenkins doesn't report a result back to zuul18:24
clarkbso zuul reruns the job18:24
zaroclarkb: ok. i am able to repro.  the result is missing when you pass in "OFFLINE_NODE_WHEN_COMPLETE":"false"18:25
zaroclarkb: sorry i meant "OFFLINE_NODE_WHEN_COMPLETE":"true"18:25
hasharon my setup the slaves are permanent and I don't use the magic var to set a node down18:25
zaroclarkb: event must be firing on that offline thing before job completes.18:26
*** Sukhdev has joined #openstack-infra18:26
*** baoli has quit IRC18:26
clarkbzaro: I thought we tested this...18:26
zaroclarkb: i must have only tested the false case.  will fix up the code.18:28
*** yamamoto has joined #openstack-infra18:28
*** pelix has quit IRC18:28
*** Hal has joined #openstack-infra18:30
mtreinishclarkb: any chance that I can cajole you into reviewing https://review.openstack.org/108003 this week?18:30
openstackgerritMichael Tupitsyn proposed a change to openstack-infra/jenkins-job-builder: Support additional events for HipChat plugin  https://review.openstack.org/12976018:30
*** Hal is now known as Guest5319818:30
*** rmcall has quit IRC18:30
clarkbmaybe? currently fighting jenkins and grenade18:30
*** jistr has quit IRC18:31
mtreinishclarkb: cool, I can probably help with that (if there's something that needs help)18:31
clarkbmtreinish: mostly just wrangling software at this poitn thanks though18:31
clarkbI think I have it under control18:32
*** shashankhegde has quit IRC18:32
*** rmcall has joined #openstack-infra18:32
clarkb(jenkins should be happier soonish then grenade stuff and the check queue can clear)18:32
openstackgerritKhai Do proposed a change to openstack-infra/jenkins-job-builder: add a JJB command reference  https://review.openstack.org/12810618:32
*** yamamoto has quit IRC18:33
dhellmannfungi: I'm seeing those markupsafe unicode errors in some more doc builds. Before I spend a bunch of time debugging, is that something you're aware of anyone actively working on?18:35
fungidhellmann: nope--i'd not seen it before you pointed it out, and haven't looked into it18:35
dhellmannfungi: ok, thanks18:35
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Plugins may now register their own user preferences  https://review.openstack.org/12906118:37
*** jamielennox has quit IRC18:38
marcusvrn_asselin: fungi anteaya It's working now!! The problem was that sandbox repository has already been cloned...so the remote url of the repository was configured to the ci account...Then, when zuul was trying to update the repo already cloned, it returned "Permission Denied (public key)"  because it was using ci account instead of my personal account ¬¬'18:38
*** cpowell has joined #openstack-infra18:38
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Plugins may now register cron workers.  https://review.openstack.org/12960918:38
fungimarcusvrn_: oh, good point... i didn't think to suggest checking for usernames in existing git remotes18:39
*** teran has joined #openstack-infra18:39
*** Guest53198 has quit IRC18:42
*** radez is now known as radez_g0n318:43
* JayF asks for some kind core on devstack-gate to help us get https://review.openstack.org/#/c/129653/ the agent_ssh nv job for Ironic working18:43
anteayamarcusvrn_: good work18:44
jeblairclarkb or fungi if you have a sec? ^18:44
anteayafungi: I almost wonder if the process that marcusvrn_ has gone through would be useful to demonstrating that potential ci op is worthy of an account18:45
openstackgerritJeremy Stanley proposed a change to openstack-infra/system-config: Host a devstack.org redirect on static  https://review.openstack.org/13000118:45
fungiJayF: looking18:45
mtreinishJayF: why don't you just do that in devstack?18:45
anteayaso they get the ci working first and the account comes last18:45
clarkbjeblair storyboard changes?18:46
*** freemanbrandon has joined #openstack-infra18:46
jeblairclarkb: devstack-gate ironic18:46
clarkboh looking18:47
JayFmtreinish: jeblair initially asked the same question; I didn't know or have a good answer then. My only real comment is that the only thing in Devstack at all ATM that needs SWIFT_TEMPURL working is the Ironic agent driver18:47
clarkbwe decided to not devstack that?18:48
JayFjroll: ^18:48
JayFI just know we're running a check now, that fails everywhere because of that one variable, and I wanted to get it working18:48
fungiJayF: however you can switch around the same "$DEVSTACK_GATE_IRONIC_DRIVER" == "agent_ssh" test in devstack to set those things... but i agree it loks like much of this is already being set in d-g instead18:48
JayFthat's my statement and my goal :(18:48
jeblairyeah, someday i'm going to get a bee in a bonnet about that whole section, but so far, the ironic folks have said that devstack-gate is doing something really weird that no one would ever want to do outside of the gate.18:48
mtreinishJayF: ok, it just feels like something that devstack should just default and allow to be overwritten if people need it.18:48
jeblairfrankly, i don't believe them.18:48
jeblairand i agree with mtreinish and clarkb and fungi.  :)18:48
JayFjeblair: I believe us18:49
JayFjeblair: that ssh driver we use in the gate is *awful*18:49
jrollJayF: jeblair and I decided that we should move the agent configs to devstack at some point18:49
*** freemanb_ has quit IRC18:49
JayFjeblair: and nobody should ever use it at all ever outside of the gate18:49
jeblairJayF: then we should not use it in the gate18:49
JayFjeblair: it's being worked on; someone is making an ipmi proxy so we can use "real" drivers in the gate18:49
jrolljeblair: if we don't use it in the gate, we need to put hardware in the gate18:49
jrollbut yeah being worked on18:49
fungianyway, for now it's not adding any additional burden to stick yet one more line in that section, so i'm +2 on it as a stop-gap18:50
jeblairhowever, we apparently missed the opportunity to reject that section a while ago, so i don't want to hold up this fix18:50
fungiunless anyone else wants to object, i'll approve18:50
jeblairjroll: i think developers can and should be able to work on and test this code without real hardware18:50
clarkbfungi no its fine I just didnt see an answer to the earlier concerns18:50
jeblairdevstack should help them set up that kind of environment18:50
clarkband jeblairs middle road compromise wfm18:51
*** teran has quit IRC18:51
jrolljeblair: right, so by "never use this outside of the gate" we actually mean "never use this outside of dev/test"18:51
*** freemanbrandon has quit IRC18:51
jeblairjroll: yep.  and that's what the "dev" in "devstack" stand for :)18:51
jeblairjroll: i'm absolutely not arguing that it's production ready18:51
fungiit's for gettin devs all up in your stack18:51
jeblairjroll: i'm just saying that the development and test environment should be managed by devstack18:51
mtreinishfungi: I'm fine with it. I'm just not sure why the key needs to be set in d-g. Why doesn't devstack just default it to something?18:51
JayFjeblair: clarkb: fungi: So your concern is basically that as a dev, I should be able to flip a single switch and get a devstack that "works" for IPA?18:51
dhellmannfungi: it looks like the problem has to do with the encoding of the templates; I'm testing to make sure they are coming from the right package18:51
jeblairJayF: yep, like everything else in devstack18:52
fungiJayF: more or less, yes18:52
JayFjeblair: clarkb fungi: and with needing to set ~10 vars in ds-g, it means the devs have to set most of those vars too (in theory)18:52
jeblairJayF: exactly.  way too much work18:52
jeblairtoo much magic in the wrong place18:52
jrolljeblair: to be clear, some of those things in that block are tempest settings, so we'll probably still need that block long-term. it should just be smaller.18:52
fungidhellmann: encoding as in the text encoding, or as in the way the templates are coded?18:52
*** oramraz has joined #openstack-infra18:52
*** jedimike has joined #openstack-infra18:53
jrolljeblair: actually, exactly half of them are gate-related18:53
jeblairjroll: yep.  we need to tell devstack how to set it self up, but we shouldn't need to tell the things that devstack sets up how to set themselves up :)18:53
dhellmannfungi: as in the text encoding of the template file -- the spot the error comes from looks like it is a code path used when the encoding can't be determined18:53
jrolljeblair: yeah. I'd like to fix this. priorities are hard.18:53
fungidhellmann: oh, eww. utf-8 everywhere! ;)18:54
lifelessfungi: seen fidoalliance.org ?18:54
jeblairjhesketh: good morning, how was your nap? :)18:54
dhellmannfungi: yeah, these files are ascii so they should be ok, but I wonder if something being injected into them isn't18:54
anteayaasselin: where is your jenkins module split patch?18:55
fungilifeless: now i have. some interesting options there18:56
jheskethHeh, short ;-)18:56
*** thedodd has joined #openstack-infra18:57
jeblairi assume that's how you log into fidonet18:57
fungilifeless: i'll bring this to the attention of the devs who have been working on openstackid18:57
fungilifeless: could provide some useful possibilities for moving away from passwords18:57
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: multi-node dg setup  https://review.openstack.org/10604318:58
lifelessfungi: thanks ;)18:58
fungilifeless: thanks to you too18:58
*** mfink_ has joined #openstack-infra18:59
fungimeeting time in #openstack-meeting !18:59
*** mfink has quit IRC19:01
*** Guest53198 has joined #openstack-infra19:02
*** emagana has quit IRC19:02
*** emagana has joined #openstack-infra19:02
*** guibuild has joined #openstack-infra19:03
*** mfer has joined #openstack-infra19:03
*** Longgeek has quit IRC19:04
*** mfink_ has quit IRC19:04
*** sarob has joined #openstack-infra19:04
*** derekh has joined #openstack-infra19:06
*** emagana has quit IRC19:07
*** andreykurilin_ has joined #openstack-infra19:08
*** jistr has joined #openstack-infra19:10
*** tonytan4ever has quit IRC19:10
*** sarob has quit IRC19:10
*** sarob has joined #openstack-infra19:11
*** jamielennox has joined #openstack-infra19:12
*** cody-somerville has quit IRC19:13
*** radez_g0n3 is now known as radez19:14
*** ayoung has quit IRC19:15
*** otherwiseguy has quit IRC19:15
*** hdd has quit IRC19:16
openstackgerritMatthew Treinish proposed a change to openstack-infra/project-config: Add a new qa-specs-core group and use it  https://review.openstack.org/13001219:17
ashpurgh, this is a really dumb apache question but it's that kind of day - when I add the AliasMatch stuff for /p/ for zuul so jenkins can see repos19:17
ashpi just get "request not supported" from apache, am I missing something obvious like.. do I need a <Directory> on /var/zuul/git as well?19:17
*** emagana has joined #openstack-infra19:18
*** hashar has quit IRC19:18
*** emagana has quit IRC19:18
fungiashp: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/zuul/templates/zuul.vhost.erb19:19
ashpahhh that's where it is, I must be blind19:19
ashpI was looking in the zuul module too but didn't see anything in manifests to drop it in place19:19
ashpshit, you don't have anything I don't, why does apache hate me. :/19:19
fungiwhat distro/release?19:20
fungiwe're using apache 2.2 on ubuntu precise there19:20
ashp12.04, yeah19:20
*** sdake_ has quit IRC19:20
ashpRequest not supported: '/var/zuul/git/metacloud/horizon refs/zuul/mc/2013.1.7/Zfb69f1305040430c95dc5c1db65f999e'19:20
ashpweirdly not a lot of useful stuff about this online either :/19:20
ashpMaybe git fetch does something a regular http request wouldn't, so I can't debug it as easily, that kind of makes sense19:22
ashpbecause the actual job just gives me: 19:10:15 fatal: Couldn't find remote ref refs/zuul/mc/2013.1.7/Zd10acef0f2704b03a133c70e752b056d19:22
ashpI can confirm the actual ref is there, it's just the git fetch seems to fail :/19:23
*** freemanbrandon has joined #openstack-infra19:24
ashpit's annoying, it gets the gerrit stuff, then just fails when it hits up zuul :/19:25
ashpok, so no apache failures when git fetch is ran, it just.. doesn't work19:26
*** thedodd has quit IRC19:27
openstackgerritKen Giusti proposed a change to openstack-infra/project-config: Add tests to verify oslo.messaging's AMQP 1.0 protocol support  https://review.openstack.org/12443319:28
*** yamamoto has joined #openstack-infra19:28
*** MaxV has joined #openstack-infra19:28
*** dprince has quit IRC19:28
fungidoes that ref exist?19:28
ashp-rw-r--r-- 1 zuul zuul 40 Oct 21 19:29 Z49f440ba6f5b42f895a08bb82eda5b9b19:30
ashp(different ref, but I had to trigger it again)19:30
*** thedodd has joined #openstack-infra19:30
*** yamamoto has quit IRC19:33
asselinanteaya, nibalizer puppet module split. what I've done so far: https://review.openstack.org/#/c/129768/  https://github.com/rasselin/puppet-jenkins19:33
*** packet has joined #openstack-infra19:33
anteayaasselin: I've added an agenda item to the infra meeting agenda19:34
anteayado join us in -meeting19:34
asselinok thanks19:34
*** ildikov has joined #openstack-infra19:34
*** packet has quit IRC19:35
asselinanteaya, fyi, what I was doing was just poc so I understand the process e2e19:35
openstackgerritMike Heald proposed a change to openstack-infra/storyboard: Paginate after filtering query  https://review.openstack.org/13001619:36
anteayaasselin: perfect19:38
*** jamielennox has quit IRC19:38
anteayaasselin: that was what I was hoping19:38
*** tonytan4ever has joined #openstack-infra19:39
*** gyee has quit IRC19:42
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/devstack-gate: Ability to flag projects to use non-standard libraries  https://review.openstack.org/13001819:42
*** shashankhegde has joined #openstack-infra19:42
*** otherwiseguy has joined #openstack-infra19:42
*** yolanda has quit IRC19:43
*** sarob has quit IRC19:47
*** hashar has joined #openstack-infra19:47
*** sarob has joined #openstack-infra19:47
*** e0ne has joined #openstack-infra19:51
*** terryw has joined #openstack-infra19:51
pleia2off the plane! (connection in CLT)19:52
*** sarob has quit IRC19:52
derekhWeird gierrit issue : I'm logged into gerrit and hit this link https://review.openstack.org/#/c/126513/19:53
derekhI am getting told "The page you requested was not found, or you do not have permission to view this page."19:53
*** dane_leblanc has quit IRC19:53
*** otherwiseguy has quit IRC19:53
derekhif I log out, I can see the patch as expected19:53
*** amcrn has quit IRC19:54
*** changbl has quit IRC19:55
*** kirscht has quit IRC19:56
*** changbl has joined #openstack-infra19:58
*** Guest53198 has quit IRC19:58
anteayapleia2: yay19:58
*** boris-42 has quit IRC19:59
*** dane_leblanc has joined #openstack-infra19:59
*** markmcclain1 has quit IRC19:59
*** markmcclain has joined #openstack-infra19:59
jeblairfungi, clarkb, mordred, SergeyLukjanov (if you happen to be around): anyone interested in an interactive afs walkthrough after the tc meeting?20:00
fungijeblair: absolutely20:01
* zaro is20:01
*** jistr has quit IRC20:01
fungijeblair: i played around a little on afstest20:01
*** unicell has quit IRC20:01
fungijeblair: should i use that system as a testbed for scripting the uid/gid remapping?20:01
*** derekh has quit IRC20:01
*** baoli has joined #openstack-infra20:02
jeblairfungi: i think that would be fine -- though i have a bandersnatch running right now20:02
ianwfungi / jeblair / clarkb : didn't get to it in the meeting, but can we merge https://review.openstack.org/#/c/128498/ (Add HP Cloud Centos7 images).  we agreed last week to use these images until we're in a position to have d-i-b working20:02
fungioh, no worries. just trying to figure out somewhere good where we can demo that the uids/gids are showing up correctly after changing20:02
fungi(in relation to afs-created files)20:03
nibalizerjeblair: HI I AM TEACH ME20:03
nibalizeralthough if it requires root/shell access probably hard for me to actually participate20:03
clarkbjeblair maybe? will I need to be somewhere quiet?20:03
fungiyeah, i get the impression this is intended to be "how to manage the afs cell and kdcs"20:04
*** dteselkin__ has quit IRC20:04
clarkbI am lunching in not quiet location20:04
jeblairclarkb: i was thinking phone conf + screen session might be the best way?  but won't be for another hour anyway20:04
*** david-lyle has quit IRC20:04
jeblairfungi: a little of afs 101 too20:04
openstackgerritKhai Do proposed a change to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client  https://review.openstack.org/13002520:05
zaroclarkb: ^20:05
sdakewhen I push a signed key for a release, where does gerrit get my gpg public key from?20:05
*** ociuhandu has quit IRC20:05
fungisdake: it doesn't know anything about your key20:06
openstackgerritA change was merged to openstack-infra/storyboard: Paginate after filtering query  https://review.openstack.org/13001620:06
fungisdake: it's just authenticating you based on your ssh credentials20:06
sdakefungi so I can use any key I like and gerrit wont auth it?20:06
sdakecool but I still must use a signed key?20:06
sdakerather a signed tag?20:06
fungisdake: yes, you need to sign a tag to be able to get gerrit to accept it, but can sign it with any key you want20:07
* anteaya is interested in afs after meetings20:07
sdakefungi++ thanks bro20:07
fungiideally it would be a key you use for these sorts of things, and getting it into the web of trust where possible is encouraged20:07
AJaeger_dhellmann: So, for https://review.openstack.org/#/c/129433/1/zuul/layout.yaml - we will keep python 2.6 on both master and branches for all oslo projects for now?20:08
pleia2anteaya: I'm assuming as nibalizer is, hoping the 101 will help update docs for the rest of us though :) /me still has initial docs patch to review20:08
fungisdake: down the road we might start setting up keyrings in the release automation and allow ptls to decide on a list of key fingerprints which are authorized to sign release tags for their projects or something, as an additional safety20:08
sdakeyup makes sense20:08
clarkbjeblair I can mute my mic I guess20:08
AJaeger_SergeyLukjanov: please review https://review.openstack.org/#/c/129433/1/zuul/layout.yaml - see fungi's comment on sahara-dashboard20:08
clarkbor move location20:08
jeblairclarkb: or we can wait a bit :)  when would be a good time for you?20:09
*** cody-somerville has joined #openstack-infra20:09
anteayapleia2: we certainly will try to ensure the broader interest is served20:09
*** mrmartin has quit IRC20:10
hasharzaro: poor gearman plugin :D20:10
lifelesssetuptools on python3.2 is bust :<20:11
lifelessit writes an entrypoint using execfile, which doth not exist20:11
*** aysyd has quit IRC20:11
fungilifeless: i thought the only releases more broken than python 3.2 were 3.1 and 3.020:12
dhellmannAJaeger_: that's probably the safest thing for now, yes20:12
openstackgerritKhai Do proposed a change to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client  https://review.openstack.org/13002520:12
lifelessfungi: yeah but 3.2 has security fixes through 201620:12
lifelessfungi: the other two are buried and dead dead dead20:12
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Limit python26 jobs to icehouse/juno for server projects  https://review.openstack.org/12943320:13
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Removed python26 from python-jobs  https://review.openstack.org/12873620:13
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Remove python26 jobs from all stackforge projects  https://review.openstack.org/12943420:13
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Remove python26 jobs from various projects  https://review.openstack.org/12943520:13
clarkbjeblair all day tomorrow should be good20:13
AJaeger_ok, rebased the python 2.6 pachtes and updated commit messages, there was indeed a merge conflict. No other changes done to them.20:13
clarkbor thurs or friday20:13
SergeyLukjanovAJaeger_, okay, looking20:14
jeblairclarkb: i pretty much only have this afternoon available20:14
*** david-lyle has joined #openstack-infra20:15
SergeyLukjanovjeblair, I am (if I will not fall asleep)20:15
*** ayoung has joined #openstack-infra20:16
* AJaeger_ waves good bye!20:16
anteayaoh I was going to say something in the meeting, in the in case anyone cares category20:16
anteayaI'm away starting next Tuesday20:16
anteayaat the summit20:16
*** AJaeger_ has quit IRC20:16
anteayaand then off the week after20:16
anteayaI don't think i have anything big outstanding20:16
pleia2I have much travel too, All Things Open this week, travel days next Monday & Friday20:17
clarkbjeblair hrm I can probably swing something later in afternoon. maybe 3ish?20:17
pleia2then heading out Thursday after summit for Proper Vacation for a week20:17
pleia2anteaya: maybe we need a team travel calendar :)20:18
fungiyeah, i expect a lot of people are travelling next week. i fly out wednesday20:18
*** emagana has joined #openstack-infra20:18
jeblairclarkb: ok, so 2200?20:18
clarkbya lets do that20:18
jeblairfungi: work for you?20:18
jeblairSergeyLukjanov: ?20:18
anteaya2200 is good for me20:19
anteayapleia2: no20:19
anteayapleia2: I don't want to be responsible to a calendar20:19
SergeyLukjanovjeblair, it's in 40 mins?20:20
*** r1chardj0n3s_afk is now known as r1chardj0n3s20:20
jeblairSergeyLukjanov: 1:4020:20
*** jamielennox has joined #openstack-infra20:20
SergeyLukjanovjeblair, oh, not sure, it'll be 2 am for me20:20
openstackgerritA change was merged to openstack-infra/project-config: Add a new qa-specs-core group and use it  https://review.openstack.org/13001220:21
*** Guest53198 has joined #openstack-infra20:24
*** e0ne has quit IRC20:25
jeblairclarkb: any chance 2pm would work for you?  otherwise SergeyLukjanov, maybe we can schedule another (earlier) time some other day  (maybe next week?)20:25
*** Sukhdev has quit IRC20:26
*** sarob has joined #openstack-infra20:26
clarkb2 will be hard20:27
anteayafungi: following the tc meeting?20:27
anteayasince you are often the add me to the gerrit group person20:28
fungianteaya: trying20:28
*** yamamoto has joined #openstack-infra20:28
clarkbmaybe you all do 2 and I catch up later?20:28
openstackgerritA change was merged to openstack-infra/system-config: Add HP Cloud Centos7 images  https://review.openstack.org/12849820:28
anteayafungi: so for tc repos ttx says add the original commiter to the first group for the repo20:29
anteayafungi: unless the commit message says differently20:29
fungisounds fine20:29
*** miqui has quit IRC20:30
*** ayoung has quit IRC20:30
SergeyLukjanovjeblair, IMO don't change time slot for me, I hope we could find some time later (before summit) to teach me a bit about afs20:30
SergeyLukjanovjeblair, and I will read logs (as always :) )20:31
*** boris-42 has joined #openstack-infra20:32
*** yamamoto has quit IRC20:32
*** changbl has quit IRC20:32
*** oramraz has quit IRC20:33
*** julim has quit IRC20:33
fungiand stackforge/heat-translator finds its way into the rename pile20:33
*** teran has joined #openstack-infra20:33
*** _nadya_ has quit IRC20:33
fungialso also its lack of working pn python 3.4 ceases to be a low priority i guess20:34
*** teran_ has joined #openstack-infra20:34
*** Guest53198 has quit IRC20:34
clarkbalso next week works too20:35
anteayafungi: and dhellmann says tc group is also part of any tc repo gerrit group20:35
fungiyep, got that20:35
*** teran has quit IRC20:38
*** jedimike has quit IRC20:38
*** oramraz has joined #openstack-infra20:38
*** teran_ has quit IRC20:38
*** dmsimard is now known as dmsimard_away20:40
*** jamielennox has quit IRC20:41
openstackgerritA change was merged to openstack-infra/project-config: Add API Working Group project  https://review.openstack.org/12846620:41
* fungi still never saw an explanation for why the content for that shouldn't have just gone into openstack/openstack-specs instead of creating yet another repo20:43
*** lttrl has quit IRC20:43
*** sabeen1 has quit IRC20:45
*** ayoung has joined #openstack-infra20:45
anteayafungi: I don't know that anyone ever asked that question20:46
anteayaI didn't think of it20:46
*** radez is now known as radez_g0n320:46
openstackgerritKen Giusti proposed a change to openstack-infra/project-config: Add tests to verify oslo.messaging's AMQP 1.0 protocol support  https://review.openstack.org/12443320:46
fungioh, i broughht it up at least once in irc, but should have mentioned it in that change as well as the governance one, and prior to the tc meeting, so mea culpa20:47
*** ayoung has quit IRC20:47
anteayaguess I missed it in the backscroll20:47
anteayabut in hindsight I agree it should have been considered20:47
*** ayoung has joined #openstack-infra20:47
fungii did add that question to the governance change, but not until after it was on the agenda20:47
anteayabecause if things that should be specs just make their own repo, why have specs?20:47
fungiat which point the horse is already out of the barn20:48
*** whoops has quit IRC20:48
* anteaya watches the horse run around in the fields20:48
*** carl_baldwin has joined #openstack-infra20:49
*** whoops has joined #openstack-infra20:50
*** katyafervent_awa has quit IRC20:53
*** katyafervent_awa has joined #openstack-infra20:53
*** gyee has joined #openstack-infra20:55
*** freemanbrandon has quit IRC20:55
*** teran has joined #openstack-infra20:56
*** HeOS has joined #openstack-infra20:56
*** yjiang5 has left #openstack-infra20:58
ianwjeblair / clarkb : so the nodepool logs are starting to get really quite large.  turning on gzip turned out to be a whole afternoon adventure, after i realised that the current puppet-apache doesn't allow override in its template.  so i've proposed https://review.openstack.org/#/c/129096/ - a new template for nodepool.o.o and also https://review.openstack.org/#/c/129496/ to fork puppet-apache, because we are so divergent from upstrea21:00
ianwm that upgrading in-place is, in my estimation, no feasible21:00
*** cpowell has quit IRC21:01
*** wenlock has quit IRC21:02
*** hdd has joined #openstack-infra21:02
*** gary-smith has joined #openstack-infra21:02
*** mfer has quit IRC21:02
fungianteaya: there are lemmings around hudson bay... do they not range farther west?21:02
anteayafungi: not sure21:03
anteayanot as far as vancouver21:03
fungioh, too bad21:03
anteayaand as far as I know, no cities named after them21:03
fungibecause i totally want an openstack lemming shirt21:03
funginow that it's been suggested21:03
anteayaI shall not stand in your way21:03
jeblairSergeyLukjanov, clarkb: either of you around now?21:03
anteayaI'm sure mikal wants one too21:03
SergeyLukjanovjeblair, I'm on a project meeting, but mostly available ;)21:04
anteayafungi: hudson bay to vancouver would be about 3000+ km21:04
jeblairSergeyLukjanov: was wondering what we decided about the afs show and tell21:04
jogojeblair: was there headway on getting governance repo published in the TC meeting?21:04
anteayaa bit far to travel on such short legs21:04
jeblairjogo: i don't think it was on the agenda21:05
fungijogo: it was discussed in the foundation staff meeting21:05
jogojeblair: guess it will stay on the ML then, thanks21:05
SergeyLukjanovjeblair, if everyone else are ok21:05
jogofungi: oh exciting21:05
*** ChuckC has quit IRC21:05
fungii think it was decided that we can do either or both options proposed21:05
anteayattx flag would be considered federal21:05
anteayafederal is national level of government21:05
anteayaor national flag21:05
anteayacanada's next level of government is provincial21:06
openstackgerritA change was merged to openstack-infra/devstack-gate: Set SWIFT_TEMPURL_KEY for Ironic agent_ssh checks  https://review.openstack.org/12965321:06
anteayathen municipal21:06
fungijbryce suggested an scp/sftp account could be created to dump files to the www site in "a day or two" so as soon as someone's ready to do that we can show them to the jenkins ssh public key. in the meantime we can also publish to static.o.o and use whatever vhost name we like for that21:06
fungijogo: ^21:06
jeblairfungi: scp?21:07
fungijeblair: so that we could use it with the jenkins scp publisher plugin?21:07
*** emagana has quit IRC21:07
jeblairfungi: yeah, that's my question; did not know cloud sites supported scp.21:07
fungijeblair: it's no longer on cloud sites21:07
jeblairfungi: where is it?21:08
fungia rackspace-managed server21:08
*** emagana has joined #openstack-infra21:08
fungitipit migrated it from cloudsites to a server plus trove21:08
* anteaya wonders about inviting rick mercer to the vancouver openstack summit21:09
fungithey also just gave me and fifieldt access to the private git repo of the source code and a sanitized dump of the database usable for recreating the bits under the silverstripe cms for testing purposes, and assuming we see no serious red flags they want to open it up public before paris21:09
*** dizquierdo has joined #openstack-infra21:10
jeblairfungi: so that we can start working on putting it on an openstack-managed server?21:10
fungijeblair: that's what i see as the next step, yes21:10
clarkbjeblair: ya now is still bad. I can catch up later if you want ot go ahead21:11
*** ChuckC has joined #openstack-infra21:11
clarkbalso I should probably fix teh jenkinses21:11
fungithe migration off cloud-sites was apparently more prompted by security concerns over cloudsites databases (being even worse than trove security in rackspace) and by deployment needs for openstackid21:11
clarkbor next week is fine with me21:11
jeblairclarkb: i'm trying to get you up to speed since you asked for a tldr :)21:12
anteayafungi: can you add cyeoh to https://review.openstack.org/#/admin/groups/469,members21:12
anteayaand is release automatically a member of core?21:12
clarkbI know its just hard since I ended up outside of home to carpool with larissa21:12
fungianteaya: nobody is automatically a member of anything, no21:12
*** emagana has quit IRC21:13
anteayaah okay21:13
anteayaso perhaps release as a member of core then21:13
*** oramraz has quit IRC21:13
clarkbzaro: so the fix is to just stop triggering on those events?21:13
anteayaso he can add members to core21:13
clarkbzaro: why did we want to trigger on those events in the first plcae? I want to make sure that we don't break somethign else in the process21:13
gary-smithHorizon patchsets (e.g. https://review.openstack.org/#/c/112741/12) are being verified by Sahara Hadoop Cluster CI in a questonably short amount of time. Any idea why a Sahara CI is building horizon changes?21:14
anteayaSergeyLukjanov: ^^21:14
fungijeblair: i'm personally okay reviewing http://docs-draft.openstack.org/48/129748/1/check/gate-ci-docs/86e5548/doc/build/html/ and experimenting myself in non-impacting ways if we want to reschedule a tol;dr at a time when more root admins are around than just you and me21:14
*** kgiusti has left #openstack-infra21:14
clarkbzaro: looks like we expect the saveablelisteners onChange to catch these cases?21:15
anteayaSergeyLukjanov: plus the ci should not be commenting "starting check jobs"21:15
*** terryw has quit IRC21:15
anteayagary-smith: how long as it been doing this?21:15
gary-smithat least since yesterday21:15
*** markmcclain1 has joined #openstack-infra21:15
*** packet has joined #openstack-infra21:15
*** packet has quit IRC21:15
*** packet has joined #openstack-infra21:15
*** markmcclain1 has quit IRC21:15
*** markmcclain has quit IRC21:15
*** markmcclain1 has joined #openstack-infra21:15
*** emagana has joined #openstack-infra21:16
gary-smithI just came back from a week of vacation, so it could have been longer21:16
jeblairclarkb, SergeyLukjanov, fungi: what about friday at 9am pst / 1600 utc?21:16
*** packet has quit IRC21:16
anteayagary-smith: thanks21:16
fungijeblair: that works for me21:16
clarkbjeblair: I can do that21:16
*** packet has joined #openstack-infra21:16
anteayaSergeyLukjanov: looks like I will have to disable your system unless you show up to tell me you have beaten me to it21:16
SergeyLukjanovgary-smith, hm, yeah, I see, we'll disable voting tomorrow to make it stable first21:16
anteayaSergeyLukjanov: can we disable teh system now please?21:16
fungiSergeyLukjanov: can we disable voting for you now?21:16
SergeyLukjanovgary-smith, the reason is that there are too many changes in horizon done that are breaks sahara part21:17
fungiheh. anteaya beats me to the question ;)21:17
SergeyLukjanovanteaya, fungi, okay, I'll disable the job in 10 mins21:17
anteayaSergeyLukjanov: thank you21:17
gary-smithSergeyLukjanov: thanks21:17
fungithanks SergeyLukjanov!21:17
jeblairSergeyLukjanov: does 1600utc friday work for talking about afs?21:17
SergeyLukjanovjeblair, let me check21:18
SergeyLukjanovjeblair, yeah, it works for me21:18
*** otherwiseguy has joined #openstack-infra21:18
SergeyLukjanovgary-smith, I think we'll raise this question on the next horizon meeting, because we need to do something with changes that are breaking sahara pages21:19
gary-smithSergeyLukjanov: sounds good, so long as it actually builds it :-)21:20
clarkbzaro: I have 06 and 07 in shutdown mode. If maybe we can get that fix merged I can upgrade to that new stuff21:21
clarkbotherwise I will just be reverting21:21
SergeyLukjanovgary-smith, the job you linked runs sahara integration tests using horizon21:21
jeblairclarkb: link?21:21
clarkblet me know what you think but hoping you can comment on my questions21:21
clarkbjeblair: https://review.openstack.org/#/c/130025/221:21
gary-smithSergeyLukjanov: In less than a minute? Really?21:21
clarkbjeblair: I am concerned that we are just removing the code that I asked to remove in the past21:21
SergeyLukjanovgary-smith, "it should" *21:21
clarkbjeblair: want to make sure we don't break something else since keeping that code was intentional21:22
anteayaSergeyLukjanov: also let's make sure your "starting check jobs" comment is removed21:23
anteayaSergeyLukjanov: that isn't allowed in ci accounts21:23
SergeyLukjanovanteaya, we decided to keep it for sahara jobs to make the vote cleared21:23
jeblairclarkb, zaro: whoah21:23
jeblairclarkb, zaro: why would de-registering a job cause a disconnect.  that makes no sense.21:23
*** markmcclain1 has quit IRC21:23
clarkbjeblair: it was my initial concern21:24
SergeyLukjanovanteaya, if we'll build the jobs for other project it'll be removed21:24
clarkbjeblair: because we offline when the job starts21:24
jeblairclarkb: not sure what you are saying.21:24
clarkbjeblair: so gearman sends h edisconnect before it sends the result or something21:24
*** dkranz has quit IRC21:24
jeblairclarkb: right, that's totally wrong.21:24
clarkbjeblair: that event fires in jenkins as soon as the job starts running21:24
clarkbso its ahead of the other gearman worker event stuff? (I don't actually understand the entire mechanism yet)21:24
*** emagana has quit IRC21:25
*** terryw has joined #openstack-infra21:25
*** todd_dsm has quit IRC21:25
anteayaSergeyLukjanov: who is we?21:25
anteayaof the we decided21:25
anteayaand where is the log of this decison21:26
*** emagana has joined #openstack-infra21:26
anteayasince making exceptions for third party tends not to go over well21:26
anteayawith the 90+ others21:26
anteayain that they have no ability to comprehend21:26
*** hashar has quit IRC21:27
anteayaand will use it as an example to be not compliant themselves21:27
SergeyLukjanovanteaya, it was discussed by sahara team and the main reason is that the vote from sahara-ci is a mandatory vote for approving changes21:27
*** yjiang5 has joined #openstack-infra21:27
anteayaso you can get the vote21:27
anteayaI don't understand why you post starting check jobs21:28
anteayaand this sint' a sahara patch21:28
clarkbjeblair: sounds like maybe I should just revert 06 and 07 to old plugin? or would you like me to wait?21:28
anteayathis is horizon21:28
SergeyLukjanovanteaya, to reset the CI vote21:28
*** yamamoto has joined #openstack-infra21:28
SergeyLukjanovanteaya, it's not related to horizon21:28
jeblairclarkb: i think that letting offline nodes have functions registered is asking for it21:28
anteayaso when the new vote arrives, it will replace the old vote21:28
*** otherwiseguy has quit IRC21:28
fungiSergeyLukjanov: we don't even do that with openstack "jenkins" account votes in the check pipeline any longer21:28
ianwanteaya: if you could re-evaluate https://review.openstack.org/#/c/128793/, that would be cool21:29
SergeyLukjanovanteaya, as I said it was a test job for horizon that I was unaware about and I've just disable it21:29
* anteaya reevaluates21:29
anteayaSergeyLukjanov: and let's remove "starting check jobs" comments please21:29
clarkbjeblair: I think the problem is that we are stopping the worker21:30
SergeyLukjanovanteaya, okay, I'll assign someone to do update the layout21:30
clarkbjeblair: so maybe we need to decouple stop from unregister function?21:30
anteayaSergeyLukjanov: thank you21:30
clarkbjeblair: then run stop when the node is actually removed21:30
*** berendt has quit IRC21:30
*** emagana has quit IRC21:30
jeblairclarkb: i do not believe we previously stopped the worker when a node when temporarily offline21:31
anteayajeblair mtreinish this changes a devstack and devstack-gate job from experiemental to check: https://review.openstack.org/#/c/128793/121:31
clarkbjeblair: correct21:31
anteayajeblair mtreinish was about to approve unless you disagree21:32
jeblairclarkb: why was that change made?21:32
*** sarob has quit IRC21:32
clarkbjeblair: we discussed it in the previous change and zaro indicated he order of ops was correct. And the move was made because we are changing the listenable type we use21:32
clarkbjeblair: jenkins has made internal api changes so you can't rely on the old listener things that we relied on before21:32
mtreinishanteaya: I'll defer to dtroyer there, whether he considers keeping devstack on centos a first class support thing21:32
anteayamtreinish: great, when is he next around?21:33
*** yamamoto has quit IRC21:33
jeblairclarkb: understood, but this seems like a behavior change.  why would we start doing this action on temporarily on/offline when we did not before?21:33
clarkbjeblair: that was my question... My understanding was it should work (but doesn't) and its so that we dregister functions when the node is offlined21:34
mtreinishanteaya: dunno, he was online ealier today21:34
*** esker has quit IRC21:34
*** emagana has joined #openstack-infra21:34
anteayamtreinish: kk thanks, I made a note in the comment21:35
anteayaI'll add them to the review21:35
anteayahim and jeblair21:35
cyeohanteaya, btw the email address listed here: https://review.openstack.org/#/admin/groups/469,members is mine, but its not the email address I use for gerrit. I'm not sure if it came from launchpad where I used to have two accounts or something silly I did when I first started working on openstack21:35
anteayaoh they already are21:35
dtroyerTuesdays are fun…I'm around mostly21:35
*** alexpilotti has quit IRC21:35
anteayadtroyer: cool21:36
anteayadtroyer: your thoughts: https://review.openstack.org/#/c/128793/21:36
jeblairanteaya: i don't find reviews because i have been added to them21:36
anteayajeblair: I didn't think so21:36
*** dizquierdo has quit IRC21:36
jeblairclarkb: those methods should probably call "registerJobs()" then :)21:36
anteayajust trying to do my best to leave ianw with something he can follow up on21:36
*** sabeen1 has joined #openstack-infra21:37
clarkbjeblair: that won't work because the node is still up21:37
anteayafungi: hmmm, how do we straighten up things for cyeoh?21:37
jeblairclarkb: registerJobs knows to deregister them when the node is offline.  that's why it worked before.21:37
*** sabeen1 has quit IRC21:37
clarkboh ok so that I completely missed21:37
clarkbzaro: ^21:37
*** sabeen1 has joined #openstack-infra21:37
jeblair        if (!computer.isOffline()) {21:37
jeblairthe whole job calculation is within that conditional21:38
fungianteaya: cyeoh: sounds like another case of duplicate launchpad openids21:38
jeblairand if it is false, then we register the empty set21:38
fungianteaya: cyeoh: i'll see what i can do to patch that up21:38
anteayafungi: thanks21:38
clarkbjeblair: ok that was not communicated in the reviews21:38
cyeohanteaya, fungi: thank! I also have submitted this bug report re: api-wg-core https://bugs.launchpad.net/openstack-ci/+bug/138126521:38
uvirtbotLaunchpad bug 1381265 in openstack-ci "Request for Initial Gerrit Core Group Member for API WG" [Undecided,New]21:38
*** alexpilotti has joined #openstack-infra21:38
clarkb(which is how I got confused)21:39
openstackgerritSpencer Krum proposed a change to openstack-infra/system-config: Test failure of apply test  https://review.openstack.org/12983721:39
*** rlandy has quit IRC21:39
fungicyeoh: in the meantime i've added the other christopher yeoh instead21:39
jeblairclarkb: yep, i barely understood what was happening in that review.  i just took zaro at his word that he tested it and it continued to work.21:39
clarkbjeblair: me too :)21:39
cyeohfungi: heh, we both thank you :-)21:39
ekarlsowould it be negative if I used "falcon" for a github trigger in zuul ?21:40
fungicyeoh: i've added you to api-wg-core now as well21:40
cyeohfungi: thanks!21:40
*** alexpilotti has quit IRC21:41
jeblairekarlso: use what storyboard is using, or convince them to change.  i believe that's pecan.21:41
*** koolhead_ has quit IRC21:43
*** alexpilotti has joined #openstack-infra21:43
clarkbjeblair: so I should probably put 06 and 07 back in service using the old plugin ya?21:43
nibalizerjeblair: so i re: the 'apply test is lies' i did make the apply test fail with a fail statement21:43
clarkb(prior to first patch to make it handle new jenkins better)21:44
jeblairclarkb: i think so21:44
clarkbwill start that now21:44
funginibalizer: i've got puppet-apply-precise failures i'm debugging on my devstack.org redirect change too, so i think it is actually breaking for some things anyway21:44
jeblairnibalizer: okay, that's mildly reassuring21:45
fungipresumably breaking because i was bad, and not just because $random_stuff21:45
jeblairso it sounds like we should try to track down why mordred's spefific failure wasn't caught21:45
nibalizerfungi: jeblair im trying the opposite, which is a noop change that should pass21:46
clarkbkilling 07 now21:46
ekarlsojeblair: needs to be the same ? :p21:46
*** mjturek has quit IRC21:46
jeblairekarlso: if you want my +2, yes :)21:46
*** mjturek has joined #openstack-infra21:46
clarkb07 is starting21:47
ekarlsoi think pecan is meh vs falcon21:47
fungiconsistency means fewer things in whose caveats need to be well versed21:47
fungier, we need to be well versed21:47
clarkbits up stuff is happening21:48
clarkbdoing 06 now21:48
fungiekarlso: sounds like you opt for #2... "convince them to change"21:48
ekarlsofungi: meh, that's too much work :P21:48
*** signed8bit has quit IRC21:49
*** signed8bit has joined #openstack-infra21:49
clarkb06 is coming back up now21:50
*** tonytan4ever has quit IRC21:51
*** andreykurilin_ has quit IRC21:51
*** pc_m has quit IRC21:51
jeblairfungi: so back to the governance repo publishing...21:52
fungioh, right, that21:53
clarkb06 looks up21:53
jeblairfungi: someone creates an ssh account21:53
clarkbhopefully this helps chew through the backlog21:53
jeblairfungi: with the jenkins ssh public key21:53
jeblairfungi: which is here: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/init.pp#n521:53
openstackgerritSteve Baker proposed a change to openstack-infra/project-config: Define experimental check-heat-dsvm-functional-postgres  https://review.openstack.org/12950921:53
jeblairfungi: then we do an scp publisher for that?21:54
*** amcrn has joined #openstack-infra21:54
fungijeblair: right. and spit the rendered file tree onto www.openstack.org in /some/path/to/foundation/tech-committee/21:54
jeblairfungi: i'm moderately uncomfortable creating a publishing workflow with an endpoint that we don't control (and is generally unresponsive) using technologies we are going to migrate away from21:55
fungijeblair: the people who will create the account will also be able to tell us what /some/path/to should really be21:55
jeblairfungi: because we could easily end up with this being the last thing on this particular tech stack21:55
*** MaxV has quit IRC21:55
fungijeblair: i am personally in favor of just putting it on static.o.o but it sounded like maybe ttx had a preference for it being directly on the www site? (ttx can you comment?)21:56
fungiseemed related to public discoverability of governance-related policies21:57
jeblairfungi: i think this could be mitigated by universal understanding that we expect www.o.o to be running in infra by the time that becomes an issue, or the likewise understanding that we would simply drop governance publication if it is the last holdout (an action which would hopefully prompt some kind of swift positive resolution).21:57
jeblairoops, i said swift.  sorry john.  :)21:58
*** terryw has quit IRC21:58
notmynamejeblair: SWIFT!!!21:58
*** mmedvede has quit IRC21:59
*** spzala has quit IRC21:59
jeblairfungi: mind you, i do think this is the best way to proceed now.  i just don't like backing us into a corner.  :)21:59
*** camunoz has joined #openstack-infra22:00
fungijeblair: understood. i think the "big hurdle" here is going to be convincing the people who currently control that system to entrust its management to not-a-24x7-staffed-company-with-an-sla-and-pagers22:00
fungialso, convincing the people who currently hack on the source code underlying it to go through a code review system they've never seen before22:01
*** dustins has quit IRC22:01
ttxfungi: I think there is value in pushign that info under www.o.o/foundation/tech-committee22:02
jeblairfungi: if you think that's a hurdle unlikely to be overcome, then i don't think we should do it.22:02
ttxand with the sftp creds that should be quite easy to do22:02
jeblairit == publish governance there.22:02
zaroclarkb, jeblair : relooking at the GP code again.  jeblair, i believe you are saying that temporary online/offline should only register jobs not stop the executors.  I think that's the real problem.22:02
fungiat some point we need to decide whether we're a free software project or a money-making corporate enterprise, because some of us are operating under one assumption and some possibly under the other22:02
ttxI'd keep governance.o.o as a plan B if for some reason we can't get the promised creds this week22:03
jeblairttx: what about my concerns above?22:03
clarkbzaro: ya, can you update the code to just register rather than stop then test that?22:03
ttxjeblair: I may have missed them22:03
zaroclarkb, jeblair will try to put those events back in and retest.  i think that's better than re-enabling the saveablelistener22:03
clarkbzaro: yes22:03
jeblairttx: i'm moderately uncomfortable creating a publishing workflow with an endpoint that we don't control (and is generally unresponsive) using technologies we are going to migrate away from22:03
jeblairttx: because we could easily end up with this being the last thing on this particular tech stack22:04
ttxjeblair: I think risk is limited. We could further mitigate risk by publishing in both places and internally linking to the governance.o.o page ?22:05
fungistemming from questions around whether we'll be able to convince the foundation and tipit, once they have the code published, to work toward moving it into gerrit and onto infra-managed servers22:05
*** gokrokve has quit IRC22:05
*** wenlock has joined #openstack-infra22:05
ttxI kind of like the idea that the page with the TC roster would not get stale22:06
ttxso anything we can update directly from governance repo sounds like a good thing to do22:06
fungii mean getting the source public has taken, what, 2 years at least? so wondering how fast we'll be able to get the rest done if ever22:07
anteayattx you said there is value to pushing to o.o/foundation/tech-committee22:07
jeblairfungi: the rest can happen in about 15 minutes.  :)22:07
anteayattx what value do you feel tehre is?22:07
fungijeblair: hah. depends on the ttl!22:07
ttxanteaya: discoverability for one22:07
* anteaya is leaning goverance.o.o22:07
ttxanteaya: and correctness.22:07
anteayaokay, fair22:08
anteayaI get discoverabiltiy22:08
ttxI don't want stale pages on www.o.o22:08
anteayaI'm missing the correctness point22:08
ttxsince that's what 99% of people will read22:08
anteayaI was feeling that part of jeblair's concern publishing to www.o.o is staleness22:09
ttxanteaya: the page at https://www.openstack.org/foundation/tech-committee/ should be built from governance repo22:09
anteayasince we do't control the endpoint22:09
ttxif we publish elsewhere and not on www.o.o, then this page won't be geenrated from governance repo22:09
anteayattx oh, because there is already material at that endpoint22:09
anteayaI was missing that22:09
anteayaI see22:10
ttxyes, you drill down from https://www.openstack.org/foundation22:10
anteayabut how fresh will it stay if we do't control the endpoint?22:10
fungianteaya: i think the concern about lack of responsiveness was more about how to deal with it when it breaks22:10
*** cody-somerville has quit IRC22:10
funginot as much about it being stale22:10
ttxanteaya: I don't get the "control the endpoint"22:10
jeblairand when i say unresponsive, i don't just mean about the www move, but also the fact that the plan was to set up the credentials we're talking about 10 months ago22:10
ttxwe control what will end up there22:10
*** pradk has quit IRC22:11
ttxjeblair: I only recently took office :)22:11
jeblairttx: we're locked into a tech stack that we can't change.  we will soon get to a point where none of our publishing uses scp, but this will.22:11
jeblairttx: you are empowered to change this?22:11
anteayattx I think jeblair is explaining the "control the endpoint" concerns22:12
ttxjeblair: I'm empowered to question it, for sure22:12
fungiso far my questioning it hasn't seemed to hasten anything22:12
*** bknudson has quit IRC22:12
fungibut i suppose i could have questioned it more... persistently?22:13
ttxsorry really need to jump in bed22:13
funginight ttx!22:13
jeblairttx: good night.  thanks :)22:13
anteayattx night22:13
ttxfungi: hopefully will all be clearer after our first internal team meeting22:13
ttxand after some sleep22:13
fungittx: looking forward22:13
*** david-lyle has quit IRC22:13
* jeblair would love it if the direction of the openstack project's web site didn't need an *internal* team meeting to be made clear22:14
*** cody-somerville has joined #openstack-infra22:15
fungimy first action in the foundation engineering meeting will be to move for a change of venue to freenode22:15
*** david-lyle has joined #openstack-infra22:15
anteayafungi: thank you22:15
anteayagood luck with that motion being adopted22:15
jeblairfungi: well, that's fine, but i won't be joining it22:15
jeblairfungi: because i don't think the foundation should be unilaterally engineering things22:16
fungijeblair: nor i22:16
*** atiwari has quit IRC22:16
jeblairwe have processes and venues in place within the project for that22:16
*** bswartz has joined #openstack-infra22:17
fungii've already asserted to those involved that the infra teams priorities are to do what is best for the project, so if the priorities of the foundation aren't in sync with those then there's a miscommunication somewhere22:17
anteayafound out why recent third-party-requests are not giving us the requested information22:19
anteayathey are following http://www.joinfu.com/2014/02/setting-up-an-external-openstack-testing-system/#Getting_an_Upstream_Service_Account22:19
anteayanot ci.openstack.org22:19
*** erlon has quit IRC22:19
*** emagana has quit IRC22:19
*** groknix has quit IRC22:20
anteayajaypipes: are you about?22:20
fungiof course they are! isn't www.joinfu.com where we maintain all our important documentation?22:20
clarkbok the restarts of 06 and 07 have killed the green on the node graphs22:20
*** emagana has joined #openstack-infra22:20
jeblairjaypipes: your blog gets out of date very quickly22:20
openstackgerritKhai Do proposed a change to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client  https://review.openstack.org/13002522:20
anteayajaypipes: we have a problem, current requests for infra accounts are not starting off with required information: http://ci.openstack.org/third_party.html#requesting-a-service-account22:20
jeblairjaypipes: perhaps you could reconstruct it to do little more than point people to the right place22:20
clarkband we seem to be killing the backlog so that is good22:20
clarkband I missed the fun discussion22:21
reedthe effects of google juice :/22:21
jeblairjaypipes: and if our docs are too confusing, maybe you could improve them? :)22:21
fungiclarkb: there's always more fun discussion22:22
fungiclarkb: we can rehash it in said secret foundation cabal conspiracy meeting tomorrow22:22
clarkbnow waiting on my d-g stack to get through the queue of doom so that I can recheck it with the nova revert in place22:22
clarkbbut that should be the last of the grenade stuff I hope22:23
zaroclarkb, jeblair: ok.  new PS for GP uses temp offline/online events again for node state changes.  better than saveablelistner.  sorry about that, i was rushing.22:23
clarkbzaro: is that tested too?22:23
clarkband works under the offline node case?22:24
*** mmedvede has joined #openstack-infra22:24
clarkbawesome thanks22:24
zaroyes, ran the same tests documented in gerrit comments.22:24
clarkbcool will take a look22:24
*** eharney has quit IRC22:24
*** emagana has quit IRC22:25
mtreinishclarkb, fungi, jogo: do we have a bug open for a fedora heat image download going so slowly that it hits the jenkins job timeout?22:26
clarkbnot that I am aware of22:26
fungisounds new to me22:26
mtreinishclarkb, fungi: http://logs.openstack.org/45/129045/1/gate/gate-tempest-dsvm-neutron-heat-slow/7b888df/logs/devstacklog.txt.gz#_2014-10-21_21_20_56_43622:26
mtreinishok I'll open one then22:26
clarkbmtreinish: the fix is likely to make it so that devstack lists that image when we cache all the things22:28
clarkbthen it iwll be local and don't have to wait?22:28
*** yamamoto has joined #openstack-infra22:28
mtreinishhmm, I thought it did that already22:28
anteayathere was one request and I replied to the email instructing the poster to follow the instructions for requesting an account22:28
anteayaso nothing this week, clarkb22:29
mtreinishclearly not based on that log though22:29
clarkbanteaya: ok good to know. let me know if anything else pops up though I have been trying to get better at tracking it myself22:29
anteayaclarkb: thank you for that22:29
anteayanext monday will be my last day before I am off22:30
anteayaso I will look again then22:30
clarkbanteaya: sounds good22:30
clarkbare you vacationing prior to summit?22:31
anteayasome time on the Brittany coast22:31
clarkbcool have fun22:31
anteayahoping the b and b doens't have a constant fan/generator/pump noise22:31
anteayathen a week to myself after summit22:31
* anteaya hopes for quiet22:32
anteayaand I'm off for a walk22:32
anteayaback in a bit22:32
*** yamamoto has quit IRC22:32
clarkbkrotscheck: into the stackforge projects... trying to do this review in chunks but I am still doing it :)22:34
mtreinishclarkb: oh, I think it might be because it was on stable/icehouse22:34
krotscheckclarkb: Thanks!22:34
*** dims_ has joined #openstack-infra22:35
*** dims_ has quit IRC22:36
*** dims_ has joined #openstack-infra22:36
*** dims has quit IRC22:38
clarkbkrotscheck: and approved22:42
krotscheckclarkb: Thanks22:42
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Write cache to file on process exit  https://review.openstack.org/11783822:42
*** nelsnelson has quit IRC22:44
jogomtreinish: hmm I *think* so22:45
jogosounds familier22:45
jogomtreinish: 131478622:45
jogomtreinish: I64f371d2f2f30d5ea692b10c4d0b3c6cdaf9cc3422:45
mtreinishclarkb, fungi: do you know off your top of the head where in (nodepool or config I guess) the images from devstack get cached22:45
*** thedodd has quit IRC22:46
fungimtreinish: devstack has a list of files it needs and the nodepool prepare_devstack.sh script pre-caches them22:46
fungimtreinish: in openstack-infra/project-config:nodepool/scripts22:46
clarkbit runs a devstack script that says this is the list of things you should care baout22:47
mtreinishfungi: ah, ok thanks22:47
fungimtreinish: but ideally the change should be addition of a url in some file in devstack22:47
* fungi doesn't recall which one just now22:47
*** dimtruck is now known as zz_dimtruck22:47
*** hdd has quit IRC22:47
mtreinishfungi: I'm thinking it's a image version mismatch between icehouse and master in devstack22:48
*** hdd has joined #openstack-infra22:48
*** gokrokve has joined #openstack-infra22:48
mtreinishso I was going to look how hard it would be to cache the image from the stable branches too22:48
*** jgrimm is now known as zz_jgrimm22:49
fungimtreinish: oh, good point. i bet right now we assume devstack master gives us a list of all the things we should cache22:49
*** marun has quit IRC22:50
clarkbwe should do stable branches too already22:50
fungimtreinish: is it just stable jobs which are timing out?22:50
clarkbit iterates over all the branches it can find22:50
fungiahh, so it does22:50
mtreinishyeah just found that too22:51
mtreinishfungi: I only hit it on a stable job22:52
openstackgerritSteve Martinelli proposed a change to openstack/requirements: Bump oslotest minimum requirements to 1.2.0  https://review.openstack.org/13005822:52
*** wenlock has quit IRC22:52
*** alexpilotti has quit IRC22:52
*** carl_baldwin has quit IRC22:53
*** mmedvede has quit IRC22:56
*** msabramo has quit IRC22:57
*** whoops has quit IRC22:58
mtreinishbut one sample really isn't useful for saying it's limited to stable22:59
*** emagana has joined #openstack-infra22:59
*** baoli has quit IRC23:01
openstackgerritA change was merged to openstack-infra/project-config: Removed python26 from python-jobs  https://review.openstack.org/12873623:02
*** dane_leblanc has quit IRC23:03
*** derekh has joined #openstack-infra23:05
*** MaxV has joined #openstack-infra23:06
clarkbok I have rechecked d-g grenade stack. it should work now23:06
mtreinishclarkb: awesome23:07
openstackgerritSpencer Krum proposed a change to openstack-infra/project-config: Use sudo with apt-get in puppet jobs  https://review.openstack.org/13005923:07
*** MarkAtwood has quit IRC23:08
*** MaxV has quit IRC23:10
*** msabramo has joined #openstack-infra23:10
*** wenlock has joined #openstack-infra23:11
*** frank_zhou has joined #openstack-infra23:12
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Write cache to file on process exit  https://review.openstack.org/11783823:13
*** doug-fish has left #openstack-infra23:13
*** dmsimard_away is now known as dmsimard23:13
clarkbjeblair: fungi mordred any reason to not move centos7 test to check from experimental as a non voting test on devstack and d-g?23:15
clarkbdtroyer jhesketh and anteaya are all +X23:15
clarkband we merged the change to add hpcloud centos7 nodes so are multiclouding that now23:15
jeblairclarkb: if it's multicloud, i'm okay.  link?23:16
clarkbit has a link to the multicloud nodepool change in the commit message23:16
*** yamahata has joined #openstack-infra23:16
jeblairclarkb: +223:17
clarkbcool I can approve23:17
*** VijayTripathi has quit IRC23:17
*** _buzz has quit IRC23:18
ianw^ thanks ... i'll keep on it.  glusterfs people are very interested, might be some of the first consumers23:18
*** signed8bit has quit IRC23:19
*** marun has joined #openstack-infra23:19
*** frank_zhou has quit IRC23:19
*** frank_zhou has joined #openstack-infra23:19
*** erlon has joined #openstack-infra23:20
openstackgerritSteve Baker proposed a change to openstack-infra/project-config: Define experimental check-heat-dsvm-functional-postgres  https://review.openstack.org/12950923:21
clarkbianw: can't you gluster on ubuntu?23:22
*** dangers is now known as dangers_away23:22
*** sarob has joined #openstack-infra23:23
clarkbjeblair: https://review.openstack.org/#/c/130025/ appears to be what you indicated would be the proper way to solve this gearman thing23:24
clarkbit reregisters jobs whenever temporary* fires23:24
clarkbthis should unregister on temp offlines and register when going back online (if say you manually do that)23:24
ianwclarkb: i'm no gluster expert, it is not entirely clear how straight-forward it is.  i'm sure you can somehow.  anyway i got a ping from some people about it, i am trying to encourage them to work upstream if possible23:24
clarkbianw: ++ to upstream23:25
jaypipesjeblair, anteaya: ok, loud and clear. will do that.23:25
*** yamamoto has joined #openstack-infra23:28
*** wenlock has quit IRC23:30
*** david-lyle has quit IRC23:30
*** bdpayne_ has joined #openstack-infra23:31
openstackgerritA change was merged to openstack-infra/project-config: Promote devstack centos7 to non-voting check job  https://review.openstack.org/12879323:32
*** ihrachyshka has quit IRC23:32
*** bdpayne has quit IRC23:33
*** yamamoto has quit IRC23:33
*** dmsimard is now known as dmsimard_away23:35
*** mmedvede has joined #openstack-infra23:36
openstackgerritIan Wienand proposed a change to openstack-infra/system-config: Rotate out nodepool logs every 8-hours  https://review.openstack.org/13006423:36
clarkbhttps://etherpad.openstack.org/p/py26-deprecation is my draft of the py26 depreaction warning email23:37
clarkbalso a russellb twitter link is likely not best link23:39
clarkbbut its still drafty23:39
openstackgerritA change was merged to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client  https://review.openstack.org/13002523:39
ianwclarkb: do you just want suggestions made with direct edits?23:41
clarkbianw: sure23:41
ianwclarkb: that would be all i would suggest.  seems clear to me23:44
clarkbianw: thanks those are good edits23:44
*** zz_jgrimm is now known as jgrimm23:46
jeblairclarkb: https://etherpad.openstack.org/p/juno-cross-project-future-of-python for link [0] ?23:47
jeblairclarkb: rest lgtm23:47
*** frank_zhou has quit IRC23:48
clarkbjeblair: much better link23:48
clarkbI will update23:49
*** frank_zhou has joined #openstack-infra23:50
*** frank_zhou has quit IRC23:50
*** jamielenz has joined #openstack-infra23:52
*** jamielenz is now known as jamielennox23:52
*** sarob has quit IRC23:53
*** VijayTripathi has joined #openstack-infra23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!