Tuesday, 2014-09-09

clarkbfungi: for 118859 is that the right place to put the param? should it be in jenkins_params.pp instead?00:00
*** changbl has joined #openstack-infra00:00
*** bdpayne_ has joined #openstack-infra00:01
fungiclarkb: good question. i waffled over the options, but that's where we source the packages list for slaves00:02
clarkbfungi: ya I think so00:02
*** _buzz has quit IRC00:02
clarkbfungi: but we source that package list for all nodes00:02
clarkbfungi: in modules/openstack_project/manifests/base.pp00:02
clarkbfungi: so I think a more appropriate location is jenkins_params then ensure both are instaleld?00:03
clarkbthat way test packages don't leak into non test slaves00:03
openstackgerritMarga Millet proposed a change to openstack-infra/config: puppet-n1k-vsm project should set acl-config  https://review.openstack.org/11791500:03
*** zns has quit IRC00:03
*** paulrad has quit IRC00:04
fungiclarkb: hrm... that only seems to get used for thick slaves, which maybe is still sufficient00:04
*** bdpayne has quit IRC00:04
clarkbfungi: it should be at least for now00:04
fungiokay, i'll give that a whirl00:04
*** Sukhdev has quit IRC00:05
*** amcrn has quit IRC00:05
clarkbfungi: I only care so much because I just cleaned up jenkins::params.pp and that was :(00:05
clarkbfungi: we were double accounting stuff in weird ways00:05
*** rmcall has quit IRC00:06
*** sarob_ has quit IRC00:07
openstackgerritMarga Millet proposed a change to openstack-infra/config: puppet-n1k-vsm project should set acl-config  https://review.openstack.org/11791500:07
*** asettle has quit IRC00:09
*** rmcall has joined #openstack-infra00:10
*** bhuvan has joined #openstack-infra00:11
*** SumitNaiksatam has quit IRC00:13
*** bdpayne_ has quit IRC00:14
*** baoli has joined #openstack-infra00:14
*** tomoe__ has quit IRC00:15
*** tomoe_ has joined #openstack-infra00:15
*** hdd has quit IRC00:17
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Update StackForge doc for Zuul project templates  https://review.openstack.org/11886100:17
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add Python 3.4 and PyPy to Ubuntu Trusty slaves  https://review.openstack.org/11885900:18
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add non-voting gate-git-review-python34 job  https://review.openstack.org/11886000:18
clarkbfungi: thanks. one last thing and this doesn't need a -1. there is another lsbdistcodename test in thick_slave.pp should we combine the checks together or keep them separate as they are logically dealing with different things?00:19
fungioh, good eye. i'll combine them. i also just noticed i typo'd the variable name for the python3_package in thick_slave.pp00:20
clarkbbut between that change and my change to claenup jenkins::params I think we have a couple good slave image update chagnes to get in this week00:21
*** bknudson has joined #openstack-infra00:22
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add Python 3.4 and PyPy to Ubuntu Trusty slaves  https://review.openstack.org/11885900:23
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add non-voting gate-git-review-python34 job  https://review.openstack.org/11886000:23
clarkbfungi: is https://review.openstack.org/#/c/118860/4/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml a bad rebase?00:25
clarkbthat says docbookrestapi not git review00:25
fungigit rebase you bastard00:26
*** sabeen has quit IRC00:26
*** _nadya_ has quit IRC00:26
fungiclarkb: oh, i see00:27
fungiclarkb: no, that's me prepping sphinxcontrib-docbookrestapi as it was the only project in that file to directly instantiate the gate-{name}-python33 job-template00:28
*** radez is now known as radez_g0n300:28
fungiclarkb: per the commit message, i'm making sure projects which create gate-{name}-python33 jobs also create gate-{name}-python34 jobs00:29
*** SumitNaiksatam has joined #openstack-infra00:29
fungimost are covered by adding it to the python-jobs group00:29
fungithat was the one outlier00:29
clarkboh so git review has it by way of py33? I assume that was another migration chagne that I must've missed00:29
harlowjadid u guys see the lwn.net article about openstack last week?00:29
clarkbharlowja: no?00:29
fungiideally, we should probably just switch it to use the python-jobs group as well00:29
harlowjaah, let me get u a link00:29
JayFharlowja: I read the mailing list before the article existed :P00:29
fungiclarkb: that's a good suggestion. fixing00:30
harlowja    http://lwn.net/SubscriberLink/610769/7f9939039559b630/00:30
fungiclarkb: i'll also break that into multiple changes for better clarity00:30
harlowjasubscriber only, for some reason00:30
harlowjabut i think u guys should be able to access through that00:31
*** dmsimard is now known as dmsimard_away00:31
harlowja*nothing that special honestly, just was interesting that it got mentioned00:31
*** dims has quit IRC00:32
*** chuckC has joined #openstack-infra00:35
*** yamamoto has quit IRC00:35
*** yamahata has joined #openstack-infra00:36
*** timrc is now known as timrc-afk00:37
*** dims has joined #openstack-infra00:37
*** rmcall has quit IRC00:41
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add a Python 3.4 job template  https://review.openstack.org/11886000:41
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add non-voting gate-git-review-python34 job  https://review.openstack.org/11996400:41
fungiclarkb: better?00:41
*** zns has joined #openstack-infra00:42
clarkboh I think I understand why I was confused. the commit message in 118860 says it is adding it to the job group buit I don't see that happening00:42
fungihuh... where am i actually adding that to python-jobs though? i must have lost that edit00:42
fungihold on00:42
clarkbya that :)00:42
fungier, it's already there00:43
clarkbright which is why jenkins didn'00:43
clarkbt -100:43
clarkbbut the commit message is not quite right00:43
clarkbI grok now :)00:43
fungiaha. zul added it in https://review.openstack.org/10284700:44
*** gokrokve has joined #openstack-infra00:44
*** yongli has joined #openstack-infra00:45
*** dmsimard_away is now known as dmsimard00:46
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add non-voting gate-git-review-python34 job  https://review.openstack.org/11996400:46
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Switch sphinxcontrib-docbookrestapi to python-jobs  https://review.openstack.org/11886000:46
fungiokay, i think those represent reality now00:47
*** dmsimard is now known as dmsimard_away00:47
clarkbyup ty00:47
*** Sukhdev has joined #openstack-infra00:48
*** otter768 has joined #openstack-infra00:48
fungiinterestingly, the bare-trusty node i held a few minutes ago already has python3.4 installed, so that'll make manual testing easier but the change explicitly installing it is still good from a correctness standpoint (and adds pypy which is not installed yet)00:48
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Update for new home for puppetboard module  https://review.openstack.org/11996600:48
clarkbfungi: I think py3.4 is default on trusty00:49
clarkbso you should mostly get it everywhere but being explicit is good if we end up with minimal images (eg docker)00:50
*** weshay has quit IRC00:50
fungiwow, we peaked at 900 jobs per hour earlier today00:52
*** _nadya_ has joined #openstack-infra00:52
clarkbby the way I had to start some logstash worker processes today00:52
clarkbthey seemed to be victims of the file descriptor issue and died due to being unable to resolve names00:52
*** zz_dimtruck is now known as dimtruck00:55
*** manybubbles has quit IRC00:55
*** nelsnelson has joined #openstack-infra00:57
*** hdd has joined #openstack-infra00:59
jogohow often do you guys do the equivalent of 'nova show server-name' or 'nova delete server-name' versus using UUIDs?00:59
fungijogo: for me, roughly equally often01:00
jogowhat about in nodepool or other scripts?01:00
clarkbjogo: I almost always do nova list then use uuids for actions01:00
clarkbjogo: because you can have the same name > once01:00
jogoclarkb: yup, I am working on a bug to make using names suck a little less https://bugs.launchpad.net/python-novaclient/+bug/120217901:01
uvirtbotLaunchpad bug 1202179 in python-novaclient "findall in novaclient/base.py is inefficient" [Undecided,In progress]01:01
fungiyeah, i use the instance name as a convenience when i know there is one and only one with that name, to avoid having to look up its uuid01:01
fungibut if in doubt, i use the instance uuid instead01:01
jogofungi: and what about nodepool?01:02
*** asettle has joined #openstack-infra01:02
jogothat uses pythonclient so it should use UUIDs for things01:02
clarkbjogo: it uses uuids01:02
fungiyeah, stores them in a db and uses them with the library to instantiate nova server objects01:03
*** dims_ has joined #openstack-infra01:03
fungiand then calls the various methods novaclient provides01:04
*** stevemar has quit IRC01:04
jogofungi: cool, so I am trying to make 'nova {show|delete..} instance-name faster01:04
*** dims has quit IRC01:05
fungihrm... based on zuul's parsing of the current layout.yaml, we have 63 unique voting python34 jobs... i'm not sure it's worth my time to manually run tests for all of them01:05
*** marcoemorais1 has quit IRC01:05
fungier, unique voting python33 jobs01:05
*** zns has quit IRC01:05
*** winston-d has joined #openstack-infra01:06
fungimaybe i can just script it01:06
*** dims_ has quit IRC01:06
fungibut that goes onto tomorrow's docket i think01:06
*** Alexandra_ has joined #openstack-infra01:06
*** dims has joined #openstack-infra01:06
winston-dnot sure this is the right place for gertty, but anyone successfully run gertty on Mac OS X?01:06
fungiwinston-d: i think zaro has01:07
*** manybubbles has joined #openstack-infra01:07
*** asettle has quit IRC01:08
fungiwinston-d: also mordred mentioned making a shell account on a mac mini available remotely to jeblair for reproducing some obscure error conditions01:08
*** Alexandra_ is now known as asettle01:09
jeblairwinston-d: yes, notmyname has as well01:10
jeblairwinston-d: make sure you start with version 1.0.1 (i just released it this morning)01:10
notmynamewait not that one01:10
notmynamethat one01:10
notmynamewinston-d: yes, I've run it a little on os x01:10
winston-djeblair: unfortunately I have 1.0.1, but still not working.01:12
winston-djeblair: http://paste.openstack.org/show/108575/01:12
winston-dnotmyname: so gertty is working on your Mac?01:12
*** jyuso has joined #openstack-infra01:13
notmynamewinston-d: yup. just upgraded to 1.0.1 and synced. looks fine01:13
jeblairwinston-d: this is going to be a weird suggestion, but actually, can you "rm ~/.gertty.db" (or mv it to a safe place), and then install 1.0.0?01:13
jeblairwinston-d: "pip install gertty==1.0.0" in a new virtualenv should do it i think01:14
jeblairwinston-d: i'd be interested in knowing if that fixes it, and then, separately, if you "rm ~/.gertty.db" and then try again with 1.0.1 if that works01:14
winston-djeblair: let me try01:14
jeblairwinston-d: my two theories are (a) a "harmless" change to the migrations that we merged into 1.0.1 wasn't actually harmless01:15
jeblairwinston-d: or (b) something random happened during the db setup and maybe it would work if you just removed it and started over01:15
mordredjeblair: (a) dad-blast it01:15
jeblairmordred: yeah, i tested a full migration from scratch before i merged that, so it'd have to be something weird like sqlite-on-osx caused it to derp.01:16
winston-djeblair: remove db and 1.0.0 doesn't work: http://paste.openstack.org/show/108576/01:16
*** dims has quit IRC01:16
mordredjeblair: same here01:17
jeblairwinston-d: that's a completely unexpected error01:17
openstackgerritBoris Pavlovic proposed a change to openstack-infra/config: Remove large-ops from places where it is useless to safe resources  https://review.openstack.org/11996901:17
winston-djeblair: but you know what, remove the db again and install 1.0.1 is working this time!01:17
jeblairwinston-d: any chance you kept a copy of the db?01:18
jeblairi realize i didn't ask you too, i'm still learning how to remote-debug gertty myself :)01:18
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Configure triggers dynamically  https://review.openstack.org/11953401:19
openstackgerritJoshua Hesketh proposed a change to openstack-infra/zuul: Call driver methods more dynamically  https://review.openstack.org/11953301:19
mordredjeblair: oh! actually01:19
mordredjeblair: so, that might have been fixed by 1.0.101:19
*** dims has joined #openstack-infra01:19
mordredjeblair: but when I was hacking on the migration changes - I did notice that once a migration breaks, it goes to hell until the end of time01:19
winston-djeblair: nope. :( but since it never worked, I guess the db should be just as-in for a new installation.01:20
mordredjeblair: and maybe sqlite on osx throws keyword errors similar to mysql01:20
jeblairbut notmyname was able to get it to work, however, i think maybe he also ran into a migration error?01:20
jeblair(before the 1.0.1 migrations change)01:21
winston-djeblair: the only changes need for gertty.yaml is username and pwd, right?01:21
jeblairwinston-d: yeah, if you start with openstack-gertty.yaml (assuming you're using it with openstack ;)01:21
winston-di have it running, but doesn't seem syncing anything01:21
jeblairwinston-d: you need to subscribe to one or more projects first;01:21
*** r-daneel has quit IRC01:22
jeblairwinston-d: hit 'L' to show the full list, 's' to subscribe, 'L' again to shorten the list01:22
*** gokrokve_ has joined #openstack-infra01:23
*** hdd has quit IRC01:24
*** dims has quit IRC01:25
*** dims has joined #openstack-infra01:26
*** gokrokve has quit IRC01:26
*** dims_ has joined #openstack-infra01:26
winston-djeblair: ok, that helps. :)01:26
winston-djeblair: btw, very nice tool. thx!01:27
*** gokrokve_ has quit IRC01:27
*** tgohad has quit IRC01:28
jeblairwinston-d: you're welcome, thanks for trying it out, and sorry it didn't work right away01:28
*** homeless has quit IRC01:28
jeblairgotta run now for mid-autumn festivities :)01:29
*** skelpter has joined #openstack-infra01:29
winston-djeblair: have a good one, enjoy the moon-cakes01:29
anteaya0 degrees, 5 minutes to the full moon01:30
*** dims has quit IRC01:30
anteayadhellmann: so the patch to report changes to the -dev channel01:30
anteayadhellmann: right now we don't report any changes to the -dev channel01:31
anteayawould it be worth at least having a chin wag on the ml before we go ahead with this?01:31
anteayajust so folks can be informed and offer their thoughts on it?01:31
anteayaalso the commit message says resolutions, but the code will post any patch to the governance repo01:32
fungii fail at evenings. i've now got a loop of py33 jobs running on a held precise-py3k node for all projects where it's voting, and a similar loop on bare-prusty with py34, generating lists of what passes. hopefully i can compare the results to find out where we're going to run into pain01:32
fungier, bare-trusty01:32
anteayaincluding stuff like dependencies and such01:32
anteayaha ha ha01:32
anteayaevening fail01:32
anteayagood thing your wife knew that before she married you01:33
fungiwell, she's got classes monday/wednesday/friday evenings01:33
anteayathat helps01:33
fungiso easier to get away with it tonight01:33
anteayawise woman she is01:33
*** yamamoto_ has joined #openstack-infra01:36
*** dimtruck is now known as zz_dimtruck01:37
StevenKfungi: Are you going to share the results on the list? I'm rather curious about the results.01:38
clarkbfungi: nice! fwiw I don't expect much failure but testing is good01:38
anteayafull moon01:39
*** yamamoto_ has quit IRC01:41
fungiStevenK: hopefully the results will be "it all works, lets switch asap" but if not then definitely01:41
fungiStevenK: i'm mainly just trying to rule out false negatives where 3.4 is failing because 3.3 is also failing and the project in question just doesn't have any patches merging to it01:42
*** dims_ has quit IRC01:42
fungithere seem to be quite a few of those mostly very, very recently-added projects01:43
*** bknudson has quit IRC01:43
*** hdd has joined #openstack-infra01:44
*** Sukhdev has quit IRC01:44
StevenKfungi: Hmm, I added a bunch of py33 jobs very recently ...01:45
*** dims has joined #openstack-infra01:47
anteayajyuso: hello01:49
anteayajyuso: let's talk here01:49
anteayaso what is your question01:49
*** dims has quit IRC01:50
*** dims has joined #openstack-infra01:50
jyusoI think I just found a email..it said that our China government blocked 29418 port..01:51
*** wenlock has joined #openstack-infra01:51
*** nosnos has joined #openstack-infra01:51
anteayajyuso: yes01:52
*** dims_ has joined #openstack-infra01:53
jyusoanteaya: Sorry,can't access that page:(01:54
*** dims has quit IRC01:54
anteayajyuso: well that doesn't help01:55
anteayagood thing irc works01:55
anteayawinston-d: you posted to that mailing list thread01:55
anteayaany support or advice for jyuso?01:55
anteayajyuso is having some difficulty with the firewall out of china01:56
*** mestery has quit IRC01:57
*** mestery_ has joined #openstack-infra01:57
*** yamamoto_ has joined #openstack-infra01:59
*** rlandy has quit IRC01:59
*** tchaypo has quit IRC02:02
*** tchaypo has joined #openstack-infra02:02
*** dims_ has quit IRC02:04
*** koolhead17 has joined #openstack-infra02:04
*** dims has joined #openstack-infra02:04
*** pcrews has quit IRC02:06
jyusoOK,thx anteaya.I'll contact more people about this issue,there should be a solution I think:)02:06
*** yamamoto_ has quit IRC02:06
mordredjyuso: it is possible to use https02:07
*** tomoe_ has quit IRC02:07
*** tomoe__ has joined #openstack-infra02:07
anteayamordred: he has a ci system he is asking about02:07
anteayasorry I was hoping he would re-ask the question in channel02:08
anteayajyuso: what ci system is yours?02:08
anteayathe ci systems have unique problems with the firewall02:08
anteayasince they can't https02:08
anteayaonly ssh02:08
anteayathe gerrit stream is only available via ssh02:09
jyusoanteaya: It's Intel-PCI-CI.Our CI is still in testing.02:09
wenlockjyuso , if you worked for a corporation with corporate proxies in US, you could use there proxies by directing your ssh connection to the corporate proxy with an application like corscrew02:09
*** dims has quit IRC02:09
*** vponomaryov has quit IRC02:09
*** bogdando has quit IRC02:10
*** rpodolyaka_pto has quit IRC02:10
anteaya11103 intelotccloud Intel PCI CI otc_cloud@163.com02:10
*** andreykurilin has quit IRC02:10
*** denis_makogon has quit IRC02:10
*** denis_makogon has joined #openstack-infra02:10
*** andreykurilin has joined #openstack-infra02:10
*** bogdando has joined #openstack-infra02:11
*** vponomaryov has joined #openstack-infra02:11
*** rpodolyaka has joined #openstack-infra02:11
jyusowenlock: Yes.We have a proxy in US.But for some reason I can't use that proxy. :(02:11
*** tomoe_ has joined #openstack-infra02:11
*** tomoe__ has quit IRC02:11
anteayajyuso: you have been in testing since at least june: https://review.openstack.org/#/c/99623/02:12
clarkbjyuso: hrm you should be able to. at least you were when I was at Intel02:12
clarkbjyuso: you have to specify the folsom or oregon proxies directly though probably rather than using the geo load balancer names02:12
wenlockjyuso good article here, i think you can adapt that for review.openstack.org http://stackoverflow.com/questions/1040089/how-do-i-use-github-through-harsh-proxies02:16
*** mmaglana has quit IRC02:16
wenlockjyuso just adjust host and ports02:16
*** bhuvan has quit IRC02:19
anteayajyuso: can you access stackoverflow?02:19
jyusoanteaya: Yes.I can access it.I think I should try to use https first.02:22
jyusowenlock: OK,thx.02:23
*** pcrews has joined #openstack-infra02:23
wenlockjyuso in case stackoverflow access is limited http://paste.openstack.org/show/108600/02:23
*** Ryan_Lane has quit IRC02:24
wenlockjyuso gl02:24
anteayajyuso: your progress is slow if you are still testing and have been since june02:28
anteayabe sure to stay available in this channel and ask questions if you get stuck02:28
anteayahopefully someone might be able to offer you suggestions so that you can continue to make progress02:28
*** mbacchi has quit IRC02:31
*** mbacchi has joined #openstack-infra02:31
jyusoanteaya: Yes,It is still not stable enough:( Because I don't use zuul,but gerrit trigger.Our testing is based on physical machine.so lot of problem caused by environment.I can't just simple delete a instance:(02:32
*** openstackgerrit has quit IRC02:33
*** bdpayne has joined #openstack-infra02:33
*** mbacchi has quit IRC02:33
*** asettle has quit IRC02:34
*** manybubbles has quit IRC02:34
*** yamamoto_ has joined #openstack-infra02:37
anteayajyuso: wow, many obstacles for you02:39
anteayajyuso: okay stay active in this channel02:39
anteayalet us know your progress02:39
anteayado you know how to use paste?02:39
jyusoanteaya: OK,thx.02:39
jyusoanteaya: yes.02:39
anteayawell paste your output from logs or stdout or err02:40
anteayaand ask for help when you need it02:40
*** hdd has quit IRC02:40
jyusoanteaya: OK,I'll do that when I need help.Thx a lot:)02:43
*** sabeen has joined #openstack-infra02:45
*** baoli has quit IRC02:47
anteayajyuso: welcome02:47
anteayathis is the right time to ask for help, feature freeze is past and we have time to answer questions right now02:47
anteayatry to make as much progress as you can in the next 4 weeks02:48
*** adalbas has quit IRC02:51
*** arnaud has joined #openstack-infra02:53
*** KanagarajM has joined #openstack-infra02:57
*** sabeen has quit IRC02:57
*** sabeen has joined #openstack-infra02:58
*** yamahata has quit IRC02:58
*** MaxV has joined #openstack-infra02:58
*** Azher has joined #openstack-infra03:00
*** tsg has joined #openstack-infra03:00
*** arnaud__ has joined #openstack-infra03:02
*** timrc-afk is now known as timrc03:10
*** amcrn has joined #openstack-infra03:11
*** bdpayne has quit IRC03:12
*** unicell has joined #openstack-infra03:12
*** bdpayne has joined #openstack-infra03:12
*** zz_jgrimm has quit IRC03:13
*** zz_jgrimm has joined #openstack-infra03:14
*** tsg has quit IRC03:22
*** stevemar has joined #openstack-infra03:24
*** openstack has joined #openstack-infra03:42
*** fungi has joined #openstack-infra03:42
*** craigbr has joined #openstack-infra03:42
*** ZZelle has joined #openstack-infra03:44
*** hdd has joined #openstack-infra03:46
*** gothicmindfood has joined #openstack-infra03:46
*** dosaboy has joined #openstack-infra03:46
*** jlk has joined #openstack-infra03:46
*** juice has joined #openstack-infra03:46
*** andreaf_ has joined #openstack-infra03:46
*** yamahata_ has joined #openstack-infra03:46
*** bookwar has joined #openstack-infra03:46
*** camunoz has joined #openstack-infra03:46
*** torandu has joined #openstack-infra03:46
*** masayukig has joined #openstack-infra03:46
*** spiffxp_ has joined #openstack-infra03:46
*** iax7 has joined #openstack-infra03:46
*** mriedem1 has joined #openstack-infra03:46
*** luqas has joined #openstack-infra03:46
*** wenlock has joined #openstack-infra03:46
*** koolhead17 has joined #openstack-infra03:46
*** denis_makogon has joined #openstack-infra03:46
*** pcrews has joined #openstack-infra03:46
*** dmellado has joined #openstack-infra03:46
*** asadoughi has joined #openstack-infra03:46
*** jmeridth has joined #openstack-infra03:46
*** coolsvap has joined #openstack-infra03:46
*** dutsmoc has joined #openstack-infra03:46
*** dolphm has joined #openstack-infra03:46
*** yaguang has joined #openstack-infra03:46
*** dtroyer has joined #openstack-infra03:46
*** odyi has joined #openstack-infra03:46
*** ttx` has joined #openstack-infra03:46
*** mrodden has joined #openstack-infra03:46
*** stevebak` has joined #openstack-infra03:46
*** pinajo has joined #openstack-infra03:46
*** pabelanger_ has joined #openstack-infra03:46
*** melwitt1 has joined #openstack-infra03:46
*** flaper87 has joined #openstack-infra03:46
*** Guest56441 has joined #openstack-infra03:46
*** StevenK has joined #openstack-infra03:46
*** bodepd_ has joined #openstack-infra03:46
*** QuinnyPi1 has joined #openstack-infra03:46
*** NobodyCa1 has joined #openstack-infra03:46
*** Hefeweizen has joined #openstack-infra03:46
*** davidlenwell_ has joined #openstack-infra03:46
*** sandywalsh has joined #openstack-infra03:46
*** KanagarajM has joined #openstack-infra03:46
*** ildikov has joined #openstack-infra03:46
*** enikanorov__ has joined #openstack-infra03:46
*** stevemar has joined #openstack-infra03:46
*** zz_jgrimm has joined #openstack-infra03:46
*** unicell has joined #openstack-infra03:46
*** amcrn has joined #openstack-infra03:46
*** arnaud__ has joined #openstack-infra03:46
*** Azher has joined #openstack-infra03:46
*** sabeen has joined #openstack-infra03:46
*** arnaud has joined #openstack-infra03:46
*** yamamoto_ has joined #openstack-infra03:46
*** tomoe_ has joined #openstack-infra03:46
*** rpodolyaka has joined #openstack-infra03:46
*** vponomaryov has joined #openstack-infra03:46
*** bogdando has joined #openstack-infra03:46
*** andreykurilin has joined #openstack-infra03:46
*** tchaypo has joined #openstack-infra03:46
*** mestery_ has joined #openstack-infra03:46
*** nosnos has joined #openstack-infra03:46
*** skelpter has joined #openstack-infra03:46
*** jyuso has joined #openstack-infra03:46
*** winston-d has joined #openstack-infra03:46
*** nelsnelson has joined #openstack-infra03:46
*** _nadya_ has joined #openstack-infra03:46
*** yongli has joined #openstack-infra03:46
*** chuckC has joined #openstack-infra03:46
*** SumitNaiksatam has joined #openstack-infra03:46
*** changbl has joined #openstack-infra03:46
*** _buzz_ has joined #openstack-infra03:46
*** itzikb has joined #openstack-infra03:46
*** bswartz has joined #openstack-infra03:46
*** mwagner_lap has joined #openstack-infra03:46
*** ociuhandu has joined #openstack-infra03:46
*** miqui has joined #openstack-infra03:46
*** jpich has joined #openstack-infra03:46
*** gpocentek has joined #openstack-infra03:46
*** sbadia has joined #openstack-infra03:46
*** erlon has joined #openstack-infra03:46
*** vladan has joined #openstack-infra03:46
*** Shrews has joined #openstack-infra03:46
*** gyee has joined #openstack-infra03:46
*** jerryz has joined #openstack-infra03:46
*** [HeOS] has joined #openstack-infra03:46
*** Morgan_ has joined #openstack-infra03:46
*** reed has joined #openstack-infra03:46
*** pballand has joined #openstack-infra03:46
*** pfallenop has joined #openstack-infra03:46
*** julim has joined #openstack-infra03:46
*** Daviey has joined #openstack-infra03:46
*** ilyashakhat has joined #openstack-infra03:46
*** gargola has joined #openstack-infra03:46
*** Mithrandir has joined #openstack-infra03:46
*** zul has joined #openstack-infra03:46
*** Steap has joined #openstack-infra03:46
*** JadnG has joined #openstack-infra03:46
*** amotoki has joined #openstack-infra03:46
*** imcsk8 has joined #openstack-infra03:46
*** annegentle has joined #openstack-infra03:46
*** toabctl has joined #openstack-infra03:46
*** boris-42 has joined #openstack-infra03:46
*** bmwiedemann has joined #openstack-infra03:46
*** mika has joined #openstack-infra03:46
*** laxity has joined #openstack-infra03:46
*** dkehn has joined #openstack-infra03:46
*** schang has joined #openstack-infra03:46
*** GheRivero_pto has joined #openstack-infra03:46
*** jab has joined #openstack-infra03:46
*** liusheng has joined #openstack-infra03:46
*** enikanorov has joined #openstack-infra03:46
*** cody-somerville has joined #openstack-infra03:46
*** greghaynes has joined #openstack-infra03:46
*** Svedrin has joined #openstack-infra03:46
*** fifieldt_ has joined #openstack-infra03:46
*** mattymo has joined #openstack-infra03:46
*** pbelanyi has joined #openstack-infra03:46
*** katyafervent has joined #openstack-infra03:46
*** krtaylor has joined #openstack-infra03:46
*** dtantsur|afk has joined #openstack-infra03:46
*** shardy has joined #openstack-infra03:46
*** ryanpetrello has joined #openstack-infra03:46
*** jamielennox has joined #openstack-infra03:46
*** psedlak has joined #openstack-infra03:46
*** leifmadsen has joined #openstack-infra03:46
*** primeministerp has joined #openstack-infra03:46
*** jpeeler has joined #openstack-infra03:46
*** bnemec has joined #openstack-infra03:46
*** jhesketh has joined #openstack-infra03:46
*** paul-- has joined #openstack-infra03:46
*** morganfainberg has joined #openstack-infra03:46
*** crinkle has joined #openstack-infra03:46
*** dangers_away has joined #openstack-infra03:46
*** SpamapS has joined #openstack-infra03:46
*** blkperl has joined #openstack-infra03:46
*** Clabbe has joined #openstack-infra03:46
*** YorikSar has joined #openstack-infra03:46
*** skraynev has joined #openstack-infra03:46
*** medieval1 has joined #openstack-infra03:46
*** dkehnx1 has joined #openstack-infra03:46
*** afazekas has joined #openstack-infra03:46
*** ianw has joined #openstack-infra03:46
*** kevinbenton has joined #openstack-infra03:46
*** jamespage has joined #openstack-infra03:46
*** gnuoy has joined #openstack-infra03:46
*** jgriffith has joined #openstack-infra03:46
*** sdake has joined #openstack-infra03:46
*** sweston has joined #openstack-infra03:46
*** rkukura has joined #openstack-infra03:46
*** harlowja has joined #openstack-infra03:46
*** markvan has joined #openstack-infra03:46
*** kmartin has joined #openstack-infra03:46
*** aviau has joined #openstack-infra03:46
*** bcrochet has joined #openstack-infra03:46
*** grantbow has joined #openstack-infra03:46
*** portante has joined #openstack-infra03:46
*** ekarlso- has joined #openstack-infra03:46
*** bookwar_ has joined #openstack-infra03:46
*** mrda has joined #openstack-infra03:46
*** mmitchell has joined #openstack-infra03:46
*** Ng has joined #openstack-infra03:46
*** m1dev has joined #openstack-infra03:46
*** adam_g has joined #openstack-infra03:46
*** roaet_ has joined #openstack-infra03:46
*** wendar has joined #openstack-infra03:46
*** mflobo has joined #openstack-infra03:46
*** dougwig has joined #openstack-infra03:46
*** esmute has joined #openstack-infra03:46
*** dmitryme has joined #openstack-infra03:46
*** jesusaurus has joined #openstack-infra03:46
*** dhellmann has joined #openstack-infra03:46
*** jcooley has joined #openstack-infra03:46
*** devananda has joined #openstack-infra03:46
*** slagle has joined #openstack-infra03:46
*** therve has joined #openstack-infra03:46
*** nikhil_k has joined #openstack-infra03:46
*** zz_gondoi has joined #openstack-infra03:46
*** nimrodsun_ has joined #openstack-infra03:46
*** vishy has joined #openstack-infra03:46
*** zz_johnthetubagu has joined #openstack-infra03:46
*** arosen has joined #openstack-infra03:46
*** zeus has joined #openstack-infra03:46
*** gema has joined #openstack-infra03:46
*** sileht has joined #openstack-infra03:46
*** freyes has joined #openstack-infra03:46
*** rushiagr_away has joined #openstack-infra03:46
*** erw_ has joined #openstack-infra03:46
*** maurosr has joined #openstack-infra03:46
*** rossella_s has joined #openstack-infra03:46
*** msuriar has joined #openstack-infra03:46
*** mancdaz has joined #openstack-infra03:46
*** ctracey has joined #openstack-infra03:46
*** thingee has joined #openstack-infra03:46
*** tristanC has joined #openstack-infra03:46
*** simonmcc___ has joined #openstack-infra03:46
*** willroberts has joined #openstack-infra03:46
*** eglynn-office has joined #openstack-infra03:46
*** cyeoh has joined #openstack-infra03:46
*** nijaba has joined #openstack-infra03:46
*** jraim__ has joined #openstack-infra03:46
*** saper has joined #openstack-infra03:46
*** lyxus has joined #openstack-infra03:46
*** dteselkin has joined #openstack-infra03:46
*** charz has joined #openstack-infra03:46
*** lxsli has joined #openstack-infra03:46
*** Alex_Gaynor has joined #openstack-infra03:46
*** Mmike has joined #openstack-infra03:46
*** dkliban has joined #openstack-infra03:46
*** fabo has joined #openstack-infra03:46
*** niska has joined #openstack-infra03:46
*** Hunner has joined #openstack-infra03:46
*** redrobot has joined #openstack-infra03:46
*** sc68cal has joined #openstack-infra03:46
*** EmilienM has joined #openstack-infra03:46
*** abramley has joined #openstack-infra03:46
*** niedbalski_ has joined #openstack-infra03:46
*** nibalizer has joined #openstack-infra03:46
*** pleia2 has joined #openstack-infra03:46
*** jamespd_ has joined #openstack-infra03:46
*** zhiyan has joined #openstack-infra03:46
*** joa has joined #openstack-infra03:46
*** dstufft has joined #openstack-infra03:46
*** isviridov_away has joined #openstack-infra03:46
*** timrc has joined #openstack-infra03:46
*** zz_dimtruck has joined #openstack-infra03:46
*** russellb has joined #openstack-infra03:46
*** Vek_ has joined #openstack-infra03:46
*** radez_g0n3 has joined #openstack-infra03:46
*** gus has joined #openstack-infra03:46
*** jogo has joined #openstack-infra03:46
*** DuncanT has joined #openstack-infra03:46
*** mattoliverau has joined #openstack-infra03:46
*** serverascode has joined #openstack-infra03:46
*** michchap_ has joined #openstack-infra03:46
*** persia has joined #openstack-infra03:46
*** mnaser has joined #openstack-infra03:46
*** phschwartz has joined #openstack-infra03:46
*** Vivek has joined #openstack-infra03:46
*** chmouel has joined #openstack-infra03:46
*** notmyname has joined #openstack-infra03:46
*** funzo has joined #openstack-infra03:46
*** soren has joined #openstack-infra03:46
*** bradm has joined #openstack-infra03:46
*** JoshNang has joined #openstack-infra03:46
*** kashyap has joined #openstack-infra03:46
*** timfreund has joined #openstack-infra03:46
*** gaelL has joined #openstack-infra03:46
*** SergeyLukjanov has joined #openstack-infra03:46
*** zz_naotok has joined #openstack-infra03:46
*** jvrbanac has joined #openstack-infra03:46
*** asselin__ has joined #openstack-infra03:46
*** spiffxp has joined #openstack-infra03:46
*** sdague has joined #openstack-infra03:46
*** rakhmerov has joined #openstack-infra03:46
*** plars has joined #openstack-infra03:46
*** dvorak has joined #openstack-infra03:46
*** mikal has joined #openstack-infra03:46
*** dmsimard_away has joined #openstack-infra03:46
*** mugsie has joined #openstack-infra03:46
*** ubuntu3 has joined #openstack-infra03:46
*** vipul has joined #openstack-infra03:46
*** marzif__ has joined #openstack-infra03:46
*** mkoderer has joined #openstack-infra03:46
*** uvirtbot has joined #openstack-infra03:46
*** wolsen has joined #openstack-infra03:46
*** DinaBelova has joined #openstack-infra03:46
*** gmurphy has joined #openstack-infra03:46
*** tteggel has joined #openstack-infra03:46
*** proffalk1n has joined #openstack-infra03:46
*** TheJulia has joined #openstack-infra03:46
*** jd__ has joined #openstack-infra03:46
*** NikitaKonovalov has joined #openstack-infra03:46
*** dansmith has joined #openstack-infra03:46
*** s0nea has joined #openstack-infra03:46
*** ruhe has joined #openstack-infra03:46
*** zaro has joined #openstack-infra03:46
*** dguerri has joined #openstack-infra03:46
*** froek has joined #openstack-infra03:46
*** clarkb has joined #openstack-infra03:46
*** mtreinish has joined #openstack-infra03:46
*** BobBall_AWOL has joined #openstack-infra03:46
*** clayg has joined #openstack-infra03:46
*** Kiall has joined #openstack-infra03:46
*** lifeless has joined #openstack-infra03:46
*** tcammann_ has joined #openstack-infra03:46
*** _shaps_ has joined #openstack-infra03:46
*** jeblair has joined #openstack-infra03:46
*** anteaya has joined #openstack-infra03:46
*** jbjohnso__ has joined #openstack-infra03:46
*** a2hill has joined #openstack-infra03:46
*** tsufiev has joined #openstack-infra03:46
*** loquacities has joined #openstack-infra03:46
*** zigo has joined #openstack-infra03:46
*** gilliard_ has joined #openstack-infra03:46
*** BadCub has joined #openstack-infra03:46
*** JayF has joined #openstack-infra03:46
*** cinerama has joined #openstack-infra03:46
*** mgagne has joined #openstack-infra03:46
*** hemna_ has joined #openstack-infra03:46
*** krotscheck has joined #openstack-infra03:46
*** mordred has joined #openstack-infra03:46
*** SlickNik has joined #openstack-infra03:46
*** jdsn has joined #openstack-infra03:46
*** morgabra has joined #openstack-infra03:46
*** d0ugal has joined #openstack-infra03:46
*** ChanServ has joined #openstack-infra03:46
*** dstanek has joined #openstack-infra03:46
*** sendak.freenode.net sets mode: +o ChanServ03:46
*** arnaud__ has quit IRC03:46
*** jroll has quit IRC03:46
*** jroll has joined #openstack-infra03:46
*** antonym is now known as Guest3927003:46
*** fungi is now known as Guest3720903:46
*** ZZelle is now known as Guest8194103:46
*** melwitt1 has quit IRC03:47
*** bdpayne has joined #openstack-infra03:48
*** SumitNaiksatam has quit IRC03:48
*** koolhead17 has quit IRC03:48
*** SumitNaiksatam has joined #openstack-infra03:50
*** bdpayne has quit IRC03:50
*** mmaglana has joined #openstack-infra03:55
*** daya_k has joined #openstack-infra03:57
*** zul has quit IRC04:02
*** unicell has quit IRC04:02
*** pcrews has quit IRC04:06
*** HenryG has joined #openstack-infra04:07
*** Ryan_Lane has joined #openstack-infra04:07
*** unicell has joined #openstack-infra04:08
*** praneshp has joined #openstack-infra04:12
*** asettle has joined #openstack-infra04:12
*** ildikov has quit IRC04:12
*** MaxV has joined #openstack-infra04:13
*** Alexandra_ has joined #openstack-infra04:15
*** asettle has quit IRC04:17
*** praneshp_ has joined #openstack-infra04:17
*** Sukhdev has joined #openstack-infra04:19
*** Alexandra_ is now known as asettle04:20
*** praneshp has quit IRC04:21
*** praneshp_ is now known as praneshp04:21
*** pcrews has joined #openstack-infra04:21
*** mriedem1 has quit IRC04:25
*** craigbr has quit IRC04:28
*** arnaud has quit IRC04:29
*** rushiagr_away is now known as rushiagr04:29
kashyapI couldn't recall the Ubuntu One password, and when I reset it, I get an "Invalid OpenID transaction" - Arrgh, any hints?04:29
kashyapOkay, despite the above, I see it is reset. Probably just poor error reporting.04:31
*** stevebak` is now known as stevebaker04:32
*** tomoe__ has joined #openstack-infra04:33
*** dutsmoc is now known as comstud04:33
*** _nadya_ has quit IRC04:34
*** _nadya_ has joined #openstack-infra04:34
*** pinajo has quit IRC04:36
*** asselin__ has quit IRC04:36
*** IAX has joined #openstack-infra04:36
*** hemna_ has quit IRC04:36
*** tomoe_ has quit IRC04:36
*** hemna_ has joined #openstack-infra04:37
*** QuinnyPi1 is now known as QuinnyPig04:37
*** pcrews has quit IRC04:38
*** arnaud has joined #openstack-infra04:43
*** gokrokve has joined #openstack-infra04:46
*** kmartin has quit IRC04:47
*** ildikov has joined #openstack-infra04:48
*** gokrokve has quit IRC04:51
*** reed has quit IRC04:52
*** stevemar has quit IRC04:52
*** TravT has joined #openstack-infra04:53
*** MaxV has quit IRC04:58
*** tomoe__ has quit IRC05:02
*** tomoe_ has joined #openstack-infra05:02
*** bdpayne has joined #openstack-infra05:05
*** ppai has joined #openstack-infra05:06
*** rushiagr is now known as rushiagr_away05:07
*** ociuhandu has quit IRC05:09
*** julim has quit IRC05:10
*** ociuhandu has joined #openstack-infra05:11
*** mrmartin has joined #openstack-infra05:16
*** MaxV has joined #openstack-infra05:20
*** rushiagr_away is now known as rushiagr05:21
*** gyee has quit IRC05:22
*** mrmartin has quit IRC05:23
*** jlibosva has joined #openstack-infra05:23
*** bdpayne has quit IRC05:23
*** sabeen has quit IRC05:26
*** cipcosma has joined #openstack-infra05:35
*** hdd has quit IRC05:36
*** yfried has joined #openstack-infra05:45
*** gokrokve has joined #openstack-infra05:46
*** harlowja is now known as harlowja_away05:47
*** gokrokve has quit IRC05:50
*** afazekas_ has joined #openstack-infra05:55
*** MaxV has quit IRC05:58
*** MaxV has joined #openstack-infra06:02
*** koolhead17 has joined #openstack-infra06:03
*** andreykurilin has quit IRC06:08
*** loki184 has joined #openstack-infra06:08
*** camunoz is now known as camunoz_gone06:11
*** jcoufal has joined #openstack-infra06:12
*** MaxV has quit IRC06:14
*** ociuhandu has quit IRC06:16
*** Sukhdev has quit IRC06:19
*** nelsnelson has quit IRC06:21
*** flaper87 has quit IRC06:22
*** flaper87 has joined #openstack-infra06:22
*** Guest56441 is now known as Adri200006:27
*** Adri2000 has quit IRC06:27
*** Adri2000 has joined #openstack-infra06:27
*** asettle has quit IRC06:27
*** nelsnelson has joined #openstack-infra06:31
*** MaxV has joined #openstack-infra06:33
*** jamespage_ has joined #openstack-infra06:36
*** itzikb has quit IRC06:39
*** jamespage_ has quit IRC06:39
*** itzikb has joined #openstack-infra06:40
*** andreykurilin has joined #openstack-infra06:40
*** nelsnelson has quit IRC06:40
*** arnaud has quit IRC06:42
*** [HeOS] has quit IRC06:43
*** wenlock has quit IRC06:45
*** ubuntu3 has quit IRC06:46
*** gokrokve has joined #openstack-infra06:46
*** andreykurilin has quit IRC06:47
*** andreykurilin has joined #openstack-infra06:48
*** MaxV has quit IRC06:50
*** gokrokve has quit IRC06:51
*** manybubbles has joined #openstack-infra06:54
*** nelsnelson has joined #openstack-infra06:54
*** e0ne has joined #openstack-infra06:57
*** jpich has quit IRC06:57
*** e0ne has quit IRC06:59
*** AJaeger has joined #openstack-infra07:03
*** doude has joined #openstack-infra07:04
*** _nadya_ has quit IRC07:05
*** baohua has joined #openstack-infra07:05
*** arxcruz has joined #openstack-infra07:10
*** mpaolino has joined #openstack-infra07:13
*** jgallard has joined #openstack-infra07:13
*** pkoniszewski has joined #openstack-infra07:16
*** dtantsur|afk is now known as dtantsur07:18
*** dtantsur is now known as Morning07:18
*** Morning is now known as dtantsur07:18
*** MaxV has joined #openstack-infra07:27
*** MaxV has quit IRC07:27
*** MaxV_ has joined #openstack-infra07:28
*** ociuhandu has joined #openstack-infra07:29
yfrieddo you guys know what's up with the citrix CI? it's voting down patches and failing without even running tests07:29
clarkbyfried I think anteaya had it disabled. it was breaking on a devstack clone check iirc07:30
*** arxcruz has quit IRC07:30
*** ubuntu3 has joined #openstack-infra07:31
clarkbthere should be a link to the wiki in the comments with contact info07:31
*** camunoz has joined #openstack-infra07:31
yfriedclarkb: contact info is BobBall_AWOL07:31
yfriedclarkb: ^ as you can see, he's not around07:32
clarkbno email address?07:32
yfriedclarkb: and it seems like it's not disabled. any chance you could do that?07:32
clarkbhrm not right now have a link to an example change?07:32
clarkbsergeylukjanov may be able to if he is around07:33
clarkbcurious to see recent comments07:33
*** jistr has joined #openstack-infra07:33
yfriedclarkb: https://review.openstack.org/#/c/119611/07:34
*** arxcruz has joined #openstack-infra07:34
yfriedclarkb: https://review.openstack.org/#/c/112199/07:35
yfriedclarkb: https://review.openstack.org/#/c/102700/07:35
clarkbon 119611 that comment would have been made before it was disabled07:35
*** _nadya_ has joined #openstack-infra07:35
*** dkehn has quit IRC07:35
yfriedclarkb: when was it disabled?07:35
yfriedclarkb: what can be done in that case?07:36
*** dkehnx1 has quit IRC07:36
clarkbyfried late yesterday UTC probably 1800ish07:36
clarkbits disabled not much else to do07:36
yfriedclarkb: so how can we remove the -1? simply push a new patch?07:37
*** dkehnx1 has joined #openstack-infra07:37
clarkbor just ignore it07:37
*** manybubbles has quit IRC07:37
*** dkehn has joined #openstack-infra07:37
AJaegeryfried, that -1 is not blocking you to accept the patch...07:38
mattoliverauI'm calling it a night, at my parents so probably should be social!07:38
yfriedclarkb: AJaeger: personally, I usually don't bother reviewing patches with -107:39
*** jcoufal has quit IRC07:39
yfriedclarkb: AJaeger: review queue is so large that I wouldn't click on it to see which CI gave it a -1.07:39
yfriedclarkb: if 3rd party CI is not a blocker, it shouldn't register on the same level as Jenkins07:40
AJaegeryfried, that's a completely different issue. I was talking about gerrit...07:40
AJaegeryfried, so, if you want reviews, just push a new patch...07:41
clarkbyfried it doesnt you can filter by user in column and they cannot -2 or +207:41
*** isviridov_away is now known as isviridov07:41
clarkbthis is no different than a human giving a -1 you disagree with07:41
*** ociuhandu has quit IRC07:43
*** andreykurilin has quit IRC07:45
yfriedclarkb: that's a personal issue, but I think people don't bother to look at random patches that have -1 on Verification column since they assume it's not ready for review.07:45
*** Azher has quit IRC07:46
*** Guest15849 has joined #openstack-infra07:46
*** gokrokve has joined #openstack-infra07:46
yfriedclarkb: IMO, if we don't consider these CI systems to be credible, we shouldn't publish their results by default, and only expose them by request. (same thing as the "Toggle CI" button)07:47
yfriedclarkb: or have them report to their owners without bothering us regular upstream contributors.07:47
AJaegerclarkb, once you're awake again, could you give https://review.openstack.org/119547 a review, please?07:47
*** rpodolyaka has quit IRC07:48
*** rpodolyaka has joined #openstack-infra07:48
clarkbyfried we do do that. they cannot vote at all until they show there is reliability07:48
yfriedclarkb: I've spent a large amount of time yesterday trying to understand what Citrix is, and what it has against me as a person...07:48
AJaegeryfried, this is a phase of change in OpenSTack where the projects require these votes and I see people like anteaya working hard to make this work smoothly.07:49
clarkbyfried unfortunately they wont always be perfect after getting those additional voting abilities07:49
AJaegerAs you've noticed, it's not always easy.07:49
yfriedAJaeger: clarkb: I know. and I appreciate your hard work, guys07:49
AJaegeryfried, the external CI systems now follow a naming pattern so that it's easy to recognize them.07:50
AJaegeryfried, rule of thumb: Jenkins is OpenStack, everything else not ;)07:50
yfriedAJaeger: yeah, but my problem was that when I look on a bunch of patches I can't tell which is -1ed by Jenkins and which by a faulty 3rd party CI, so I just won't review any of them07:51
*** gokrokve has quit IRC07:51
AJaegeryfried, did you use the "Toggle CI" button to see the message there from Citrix? If you have suggestions on how to make it better, please discuss with anteaya once she's awake (US timezone)07:51
yfriedAJaeger: I did07:52
yfriedAJaeger: and I will07:52
*** koolhead17 has quit IRC07:52
AJaegeryfried, yes, this is unfortunate...07:52
clarkbyou can also search via vote and voter. something like verified:>0,jenkins07:52
yfriedclarkb: didn't know that. tnx07:52
clarkbI forget the exact sytax but it should auto complete and is documented07:53
*** yamamoto_ has quit IRC07:53
*** derekh has joined #openstack-infra07:53
yfriedAJaeger: i'm a little angry that Citrix has only one guy in charge of it. when he's is on a much deserved vacation, I'm left to wonder what's up with this unknown system or have to bother you guys07:53
*** dosaboy has quit IRC07:54
*** dosaboy has joined #openstack-infra07:54
*** itzikb1 has joined #openstack-infra07:54
*** itzikb has quit IRC07:55
yfriedAJaeger: vmware-gate has their all team available to handle problems with their CI on a dedicated channel published in the wiki. I assumed citrix gate would have the same support. I was wrong07:55
clarkbapparently >0 is not the same as >=1 in this integer world07:55
*** praneshp has quit IRC07:58
*** rkukura_ has joined #openstack-infra08:01
*** jcoufal has joined #openstack-infra08:03
*** rkukura has quit IRC08:04
*** rkukura_ is now known as rkukura08:04
AJaegerclarkb, SergeyLukjanov we lost openstack bot in #openstack-doc ;( Could you reenable it, please?08:05
*** daya_k has quit IRC08:05
*** daya_k has joined #openstack-infra08:06
*** zz_johnthetubagu is now known as johnthetubaguy08:06
*** [HeOS] has joined #openstack-infra08:10
*** _nadya_ has quit IRC08:14
*** koolhead17 has joined #openstack-infra08:15
*** _nadya_ has joined #openstack-infra08:15
*** [HeOS] is now known as HeOS08:17
*** yamamoto_ has joined #openstack-infra08:20
*** Ryan_Lane has quit IRC08:23
*** andreaf_ is now known as andreaf08:25
*** Guest81941 has quit IRC08:26
*** ZZelle has joined #openstack-infra08:26
*** dizquierdo has joined #openstack-infra08:28
*** koolhead17 has quit IRC08:30
*** daya_k has quit IRC08:36
*** KanagarajM2 has joined #openstack-infra08:38
*** KanagarajM has quit IRC08:38
*** bookwar has quit IRC08:38
*** daya_k_ has joined #openstack-infra08:39
*** daya_k_ is now known as daya_k08:39
*** flaviof_zzz has joined #openstack-infra08:40
*** e0ne has joined #openstack-infra08:43
*** yamamoto_ has quit IRC08:44
*** yamamoto_ has joined #openstack-infra08:44
*** IvanBerezovskiy has joined #openstack-infra08:44
*** amuller has joined #openstack-infra08:45
*** xianghuihui has joined #openstack-infra08:45
*** gokrokve has joined #openstack-infra08:46
*** jp_at_hp has joined #openstack-infra08:47
*** ihrachyshka has joined #openstack-infra08:47
*** gokrokve has quit IRC08:47
*** koolhead17 has joined #openstack-infra08:48
*** koolhead17 has joined #openstack-infra08:48
*** gokrokve has joined #openstack-infra08:48
*** zaro has quit IRC08:49
*** mika has quit IRC08:49
*** mika has joined #openstack-infra08:49
*** mika has joined #openstack-infra08:49
*** xianghuihuihui has joined #openstack-infra08:49
*** yamamoto_ has quit IRC08:49
*** xianghuihui has quit IRC08:49
*** gokrokve has quit IRC08:52
*** e0ne has quit IRC08:54
*** amcrn has quit IRC08:54
*** Dafna has joined #openstack-infra08:55
*** ihrachyshka has quit IRC08:56
*** trinaths has joined #openstack-infra08:56
*** yamamoto_ has joined #openstack-infra08:58
*** loki184 has quit IRC08:58
*** andreykurilin has joined #openstack-infra08:58
*** loki184 has joined #openstack-infra08:59
*** e0ne has joined #openstack-infra09:03
*** _nadya_ has quit IRC09:05
*** cdent has joined #openstack-infra09:06
*** yamamoto_ has quit IRC09:06
*** yamamoto_ has joined #openstack-infra09:07
*** sarob has joined #openstack-infra09:07
*** dtantsur is now known as dtantsur|lunch09:11
*** jgallard has quit IRC09:14
*** jgallard has joined #openstack-infra09:15
*** jgallard has quit IRC09:15
*** jgallard has joined #openstack-infra09:15
*** jgallard has quit IRC09:16
*** jgallard has joined #openstack-infra09:16
*** openstackgerrit has joined #openstack-infra09:20
*** daya_k has quit IRC09:20
*** rushiagr is now known as rushiagr_away09:21
*** trinaths has left #openstack-infra09:25
*** che-arne has joined #openstack-infra09:26
*** yfried has quit IRC09:28
*** _nadya_ has joined #openstack-infra09:29
*** yfried has joined #openstack-infra09:29
*** rushiagr_away is now known as rushiagr09:34
*** yfried_ has joined #openstack-infra09:35
*** e0ne has quit IRC09:38
*** yfried has quit IRC09:38
*** KanagarajM2 has quit IRC09:39
*** e0ne has joined #openstack-infra09:44
*** KanagarajM has joined #openstack-infra09:44
*** gokrokve has joined #openstack-infra09:46
*** tomoe__ has joined #openstack-infra09:50
*** gokrokve has quit IRC09:51
*** tomoe_ has quit IRC09:54
*** rlandy has joined #openstack-infra09:54
*** berendt has joined #openstack-infra09:55
*** koolhead17 has quit IRC09:57
*** unicell has quit IRC09:58
*** pelix has joined #openstack-infra09:59
*** zul has joined #openstack-infra10:02
*** _nadya_ has quit IRC10:05
*** jgallard has quit IRC10:05
*** KanagarajM has quit IRC10:08
*** mmaglana has quit IRC10:09
*** jedimike has joined #openstack-infra10:09
*** baohua has quit IRC10:11
*** baohua has joined #openstack-infra10:11
*** KanagarajM has joined #openstack-infra10:11
*** tomoe__ has quit IRC10:13
*** mkerrin has joined #openstack-infra10:14
*** jyuso has quit IRC10:14
openstackgerritMartin Mágr proposed a change to openstack-infra/config: Packstack changes  https://review.openstack.org/11833310:15
*** pcm_ has joined #openstack-infra10:17
*** pcm_ has quit IRC10:19
*** pcm_ has joined #openstack-infra10:19
*** koolhead17 has joined #openstack-infra10:22
*** koolhead17 has joined #openstack-infra10:22
*** yfried__ has joined #openstack-infra10:24
*** alexpilotti has joined #openstack-infra10:25
*** yfried_ has quit IRC10:28
*** alexpilotti has quit IRC10:30
*** tomoe_ has joined #openstack-infra10:32
*** Longgeek has joined #openstack-infra10:33
*** tomoe_ has quit IRC10:34
AJaegerflaper87, ping10:35
*** john-davidge has joined #openstack-infra10:35
*** doude has quit IRC10:37
*** doude has joined #openstack-infra10:38
*** mmaglana has joined #openstack-infra10:40
*** xianghuihuihui has quit IRC10:40
john-davidgeJust got a -1 from vAmour CI due to the test aborting: : ABORTED10:41
john-davidgelink doesn't lead to any logs10:41
*** mmaglana_ has joined #openstack-infra10:41
*** gilliard_ is now known as gilliard10:42
*** mmaglana has quit IRC10:44
*** gokrokve has joined #openstack-infra10:46
*** mmaglana_ has quit IRC10:46
*** ttx` is now known as ttx10:47
*** mpaolino has quit IRC10:47
*** flaviof_zzz is now known as flaviof10:48
*** ttx has quit IRC10:48
*** ttx has joined #openstack-infra10:48
*** e0ne has quit IRC10:51
*** gokrokve has quit IRC10:51
*** dtantsur|lunch is now known as dtantsur10:51
*** isviridov is now known as isviridov_away10:59
lifelessmordred: re dox: bindep may be relevant too11:01
*** ildikov_ has joined #openstack-infra11:02
*** amuller_ has joined #openstack-infra11:03
*** amuller has quit IRC11:03
*** yaguang has quit IRC11:03
*** ildikov has quit IRC11:05
*** johnthetubaguy is now known as zz_johnthetubagu11:06
*** e0ne has joined #openstack-infra11:07
*** gilliard is now known as gillilunch11:08
*** _nadya_ has joined #openstack-infra11:09
*** MaxV_ has quit IRC11:10
*** sdake has quit IRC11:11
*** MaxV has joined #openstack-infra11:11
*** dims has joined #openstack-infra11:11
*** mmaglana has joined #openstack-infra11:12
*** sdake has joined #openstack-infra11:12
lifelessmordred: also with dox, being able to treat it as a backend for crcache might fit well into my plans for testr11:12
*** loki184 has quit IRC11:13
*** MaxV has quit IRC11:15
*** mmaglana has quit IRC11:17
*** marun has joined #openstack-infra11:21
flaper87AJaeger: pong11:22
AJaegerflaper87, can you join us on #openstack-doc, please? WE had some glance_store discussion11:23
flaper87lifeless: mmh, dox as backend sounds interesting. It's actually already possible to do that. The Runner can be imported and used standalone: https://github.com/stackforge/dox/blob/master/dox/runner.py#L3211:23
flaper87AJaeger: sure11:23
openstackgerritSean Dague proposed a change to openstack-infra/os-loganalyze: Fix lost content escaping for html safety  https://review.openstack.org/12006411:27
sdagueyfried__: there is a setting in gerrit that lets you see what's leaving votes11:28
*** mbacchi has joined #openstack-infra11:28
*** isviridov_away is now known as isviridov11:29
*** MaxV has joined #openstack-infra11:31
*** amuller_ has quit IRC11:31
*** dmsimard_away is now known as dmsimard11:33
sdagueSergeyLukjanov: https://review.openstack.org/120064 is probably going to be pretty helpful to people looking at gate logs, it's an issue jogo found last night11:34
*** pblaho has joined #openstack-infra11:35
john-davidgeanteaya: IBM SDN-VE CI is now exhibiting the same error as Citrix Xenserver was yesterday11:39
john-davidgeanteaya: http://sng01.objectstorage.softlayer.net/v1/AUTH_bbc5e0c7-0d10-46a0-a57a-29607ab57956/sdnve-openstack-neutron-plugin/1819/11:39
john-davidgeanteaya: See devstacklog.txt11:39
*** skelpter has left #openstack-infra11:39
*** che-arne has quit IRC11:40
*** amuller_ has joined #openstack-infra11:40
*** amuller_ is now known as amuller11:40
*** camunoz has quit IRC11:40
*** mmaglana has joined #openstack-infra11:42
*** mmedvede has joined #openstack-infra11:46
*** gokrokve has joined #openstack-infra11:46
*** mmaglana has quit IRC11:46
ekarlso-phschwartz: up ? ;)11:48
*** baohua_ has joined #openstack-infra11:50
*** gokrokve has quit IRC11:51
*** baohua has quit IRC11:51
*** ZZelle has quit IRC11:52
*** nosnos has quit IRC11:52
*** nosnos has joined #openstack-infra11:52
*** eharney has joined #openstack-infra11:54
*** dmellado has quit IRC11:54
*** andreaf_ has joined #openstack-infra11:54
*** dmellado has joined #openstack-infra11:54
*** pblaho is now known as pblaho|afk11:55
*** andreaf has quit IRC11:56
*** juice has quit IRC11:56
*** nosnos has quit IRC11:57
*** amotoki has quit IRC11:57
*** juice has joined #openstack-infra11:57
*** juice has quit IRC11:57
*** pblaho|afk is now known as pblaho11:57
*** juice has joined #openstack-infra11:57
*** juice has quit IRC11:57
*** juice has joined #openstack-infra11:58
*** juice has quit IRC11:58
*** masayukig has quit IRC11:58
*** masayukig_ has joined #openstack-infra11:58
*** juice has joined #openstack-infra11:59
*** denis_makogon has quit IRC11:59
*** spiffxp_ has quit IRC11:59
*** denis_makogon has joined #openstack-infra11:59
*** juice has joined #openstack-infra11:59
*** spiffxp_ has joined #openstack-infra11:59
*** gillilunch is now known as gilliard11:59
*** alexpilotti has joined #openstack-infra11:59
*** juice has joined #openstack-infra12:00
*** juice has joined #openstack-infra12:00
*** juice has quit IRC12:00
Kiallrussellb / mordred / anteaya - Incubated projects + Merging code to Horizon/DevStack/Tempest etc thread started http://lists.openstack.org/pipermail/openstack-dev/2014-September/045441.html12:00
*** juice has joined #openstack-infra12:01
*** juice has quit IRC12:01
*** luqas has quit IRC12:01
*** eharney has quit IRC12:01
*** pcm_ has quit IRC12:01
*** andreykurilin has quit IRC12:01
*** nelsnelson has quit IRC12:01
*** IAX has quit IRC12:01
*** asadoughi has quit IRC12:01
*** iax7 has quit IRC12:01
*** torandu has quit IRC12:01
*** camunoz_gone has quit IRC12:01
*** yamahata_ has quit IRC12:01
*** jlk has quit IRC12:01
*** gothicmindfood has quit IRC12:01
*** juice has joined #openstack-infra12:01
*** rfolco has joined #openstack-infra12:02
*** juice has joined #openstack-infra12:02
*** juice has joined #openstack-infra12:02
*** juice has quit IRC12:02
*** unicell has joined #openstack-infra12:03
*** juice has joined #openstack-infra12:03
*** juice has quit IRC12:03
*** torandu has joined #openstack-infra12:03
*** koolhead17 has quit IRC12:04
*** baoli has joined #openstack-infra12:04
*** koolhead17 has joined #openstack-infra12:04
*** koolhead17 has quit IRC12:04
*** koolhead17 has joined #openstack-infra12:04
*** juice has joined #openstack-infra12:05
*** iax7 has joined #openstack-infra12:05
*** eharney has joined #openstack-infra12:05
*** pcm_ has joined #openstack-infra12:05
*** andreykurilin has joined #openstack-infra12:05
*** nelsnelson has joined #openstack-infra12:05
*** IAX has joined #openstack-infra12:05
*** asadoughi has joined #openstack-infra12:05
*** camunoz_gone has joined #openstack-infra12:05
*** yamahata_ has joined #openstack-infra12:05
*** jlk has joined #openstack-infra12:05
*** gothicmindfood has joined #openstack-infra12:05
*** iax7 has quit IRC12:06
*** iax7 has joined #openstack-infra12:06
*** eharney is now known as Guest5720712:06
*** luqas has joined #openstack-infra12:07
SergeyLukjanovsdague, looking12:08
*** koolhead17 has quit IRC12:09
*** manybubbles has joined #openstack-infra12:10
kashyapAny Gertty users, what's wrong with this invocation? - http://paste.openstack.org/show/108860/  (It's from the latest release from pip)12:10
*** dims has quit IRC12:11
*** dims has joined #openstack-infra12:12
*** dims_ has joined #openstack-infra12:12
*** dims_ has quit IRC12:14
*** dims_ has joined #openstack-infra12:15
*** mpaolino has joined #openstack-infra12:15
*** dims has quit IRC12:16
*** dkranz has joined #openstack-infra12:16
dhellmannanteaya: if we have a policy of not posting there, that's fine. I can start a ML thread, or we can just drop it.12:16
*** yamamoto_ has quit IRC12:17
*** mwagner_lap has quit IRC12:17
*** yamahata has joined #openstack-infra12:18
kashyapAnswering my own question: Removing the SQLite db and reinvoking `gertty` does the trick (thanks flaper87!)12:19
*** koolhead17 has joined #openstack-infra12:21
*** weshay has joined #openstack-infra12:21
*** KanagarajM has quit IRC12:21
*** Clabbe has quit IRC12:23
*** kgiusti has joined #openstack-infra12:24
*** ildikov_ has quit IRC12:27
*** ildikov has joined #openstack-infra12:27
*** amuller has quit IRC12:27
*** cipcosma has quit IRC12:27
*** _buzz_ has quit IRC12:27
*** gargola has quit IRC12:27
*** amuller has joined #openstack-infra12:28
*** gargola has joined #openstack-infra12:28
*** _buzz has joined #openstack-infra12:28
*** ociuhandu has joined #openstack-infra12:28
*** cipcosma has joined #openstack-infra12:28
*** Guest37209 is now known as fungi12:32
*** dprince has joined #openstack-infra12:33
*** weshay is now known as weshay_mtg12:35
*** adalbas has joined #openstack-infra12:36
*** mestery_ is now known as mestery12:37
*** hdd has joined #openstack-infra12:39
*** aysyd has joined #openstack-infra12:39
openstackgerritDmitry Teselkin proposed a change to openstack-infra/config: Pin pupppetdb-terminus package  https://review.openstack.org/11866012:41
*** mmaglana has joined #openstack-infra12:42
*** gokrokve has joined #openstack-infra12:46
*** mmaglana has quit IRC12:46
*** jgallard has joined #openstack-infra12:47
*** bswartz has quit IRC12:49
*** marun has quit IRC12:50
*** gokrokve has quit IRC12:50
sdaguejhesketh: you still up?12:51
sdaguejogo: found a pretty substantial bug in os-loganalyze that we've missed for a while - https://review.openstack.org/12006412:51
*** che-arne|2 has joined #openstack-infra12:52
jheskethsdague: yep12:52
*** cdnchris has joined #openstack-infra12:55
jheskethsdague : lgtm, nice catch by jogo. I'll merge it12:55
*** yolanda has joined #openstack-infra12:58
*** julim has joined #openstack-infra12:59
*** tomoe_ has joined #openstack-infra13:01
sdaguejhesketh: thanks!13:01
jheskethno worries13:01
*** Guest57207 has quit IRC13:01
openstackgerritA change was merged to openstack-infra/os-loganalyze: Fix lost content escaping for html safety  https://review.openstack.org/12006413:03
jheskethsdague: did you have an example log to test it on now it's merged?13:03
sdagueany nova-api log, the Calling method string will get you to an escape point13:04
sdaguefungi: how often is the os-loganalyze code CDed in?13:05
jheskethsdague: I believe 15min13:05
fungisdague: puppet pulse every 15 minutes, can take a few minutes to complete. probably no longer than half an hour from merging13:05
sdaguejhesketh: http://logs.openstack.org/61/118661/1/check/check-tempest-dsvm-full/00a36b7/logs/screen-n-api.txt.gz#_2014-09-03_18_00_17_929 is a line that is currently wrong in the old/current code13:06
sdagueso that line should change to show angle brackets once this lands13:06
fungisdague: jhesketh: i just double-checked the cron job on our puppet master and it is indeed still every 15 minutes13:07
sdaguefungi: cool13:07
*** koolhead17 has quit IRC13:07
*** zz_johnthetubagu is now known as johnthetubaguy13:08
*** radez_g0n3 is now known as radez13:10
*** joa has left #openstack-infra13:10
*** dkranz has quit IRC13:11
*** Guest39270 is now known as antonym13:13
*** antonym is now known as Guest5417313:14
*** julim has quit IRC13:15
*** vhoward has joined #openstack-infra13:16
*** ppai has quit IRC13:16
*** mestery has quit IRC13:17
*** mjturek has joined #openstack-infra13:17
*** mriedem has joined #openstack-infra13:17
*** mestery has joined #openstack-infra13:17
*** pblaho is now known as pblaho|afk13:17
jrollanyone mind taking a quick look at https://review.openstack.org/#/c/119473/ ? :)13:18
*** yamamoto has joined #openstack-infra13:18
*** Longgeek has quit IRC13:19
*** dane_leblanc has joined #openstack-infra13:20
*** cdnchris has quit IRC13:21
*** julim has joined #openstack-infra13:22
*** yamamoto has quit IRC13:23
*** hdd has quit IRC13:23
*** cdnchris has joined #openstack-infra13:24
andreykurilinsdague, hi!13:26
*** maxbit has joined #openstack-infra13:26
*** Guest15849 has quit IRC13:27
sdagueandreykurilin: hello, what's up?13:31
sdaguejhesketh: looks like the fix is live now - http://logs.openstack.org/61/118661/1/check/check-tempest-dsvm-full/00a36b7/logs/screen-n-api.txt.gz#_2014-09-03_18_00_17_92913:31
*** ildikov has quit IRC13:31
*** stevemar has joined #openstack-infra13:31
*** ildikov has joined #openstack-infra13:32
*** dustins has joined #openstack-infra13:32
*** bswartz has joined #openstack-infra13:33
andreykurilinsdague, you already made a review to my patch in novaclient ( https://review.openstack.org/#/c/117277/ ). Based on your comment, I send two patches to global-requirements. Can you review it?13:34
*** mestery has quit IRC13:34
jheskethsdague: awesome13:34
*** mestery has joined #openstack-infra13:34
sdagueandreykurilin: +213:35
*** dmellado has quit IRC13:35
*** bknudson has joined #openstack-infra13:35
andreykurilinsdague, thanks!13:35
*** ildikov_ has joined #openstack-infra13:36
*** ildikov has quit IRC13:36
*** sabeen has joined #openstack-infra13:36
*** pblaho|afk is now known as pblaho13:37
*** markmcclain has joined #openstack-infra13:38
*** jaypipes has joined #openstack-infra13:40
*** baohua_ has quit IRC13:41
*** baohua has joined #openstack-infra13:41
*** yamamoto has joined #openstack-infra13:41
*** Longgeek has joined #openstack-infra13:41
*** emagana has joined #openstack-infra13:42
*** mwagner_lap has joined #openstack-infra13:42
*** mmaglana has joined #openstack-infra13:42
*** r-daneel has joined #openstack-infra13:43
*** manybubbles has quit IRC13:44
*** gokrokve has joined #openstack-infra13:46
*** mfer has joined #openstack-infra13:46
*** yamamoto has quit IRC13:46
*** mmaglana has quit IRC13:47
*** cdnchris has quit IRC13:49
*** yamamoto has joined #openstack-infra13:49
*** manybubbles has joined #openstack-infra13:49
*** gokrokve has quit IRC13:51
*** hdd has joined #openstack-infra13:51
bookwar_where can I find definitions for jenkins-job-builder macros/scripts/etc, like devstack-logs publisher for example?13:56
*** ildikov_ has quit IRC13:56
fungibookwar_: openstack-infra/config:modules/openstack_project/jenkins_jobs/config/macros.yaml13:56
bookwar_fungi: thanks13:57
fungibookwar_: er, that was from memory and slightly mistyped... http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/macros.yaml is an actual url13:57
*** changbl has quit IRC13:59
bookwar_yes, got it, i've been there, just searched for some separate directory for some reason, as i didn't realize it is all there13:59
*** amotoki has joined #openstack-infra13:59
john-davidgeVMware NSX CI seems to have a bug which is causing tests to fail14:04
*** marun has joined #openstack-infra14:04
john-davidgeis this known?14:04
john-davidgejust had another failure on this patch14:04
*** homeless has joined #openstack-infra14:04
john-davidgelooking at the logs for that and other patches it seems that error has come up in every test for the past couple of days14:05
*** nelsnelson has quit IRC14:05
*** nelsnelson has joined #openstack-infra14:06
openstackgerritJordan OMara proposed a change to openstack-infra/config: Add xstatic-angular-fileupload support  https://review.openstack.org/12010114:06
*** alkari has joined #openstack-infra14:08
*** wenlock has joined #openstack-infra14:08
*** pcrews has joined #openstack-infra14:11
*** craigbr has joined #openstack-infra14:14
*** ildikov has joined #openstack-infra14:14
*** annegentle_ has joined #openstack-infra14:14
*** annegentle has quit IRC14:14
*** alkari has quit IRC14:17
*** NobodyCa1 is now known as NobodyCam14:17
john-davidgeCan somebody please disable VMware NSX CI?14:19
fungijohn-davidge: sure, i'll shut down their account and send a notice to the third-party-announce ml14:19
john-davidgesame two failures appear on every test since yesterday14:19
*** e0ne has quit IRC14:19
john-davidgefungi: tahnks14:19
john-davidgefungi: *thanks!14:19
*** craigbr has quit IRC14:22
*** esker has joined #openstack-infra14:22
fungilooks like they're https://wiki.openstack.org/wiki/ThirdPartySystems/VMware_CI14:24
*** zz_dimtruck is now known as dimtruck14:24
fungii don't see salv-orlando or rhsu in irc at the moment14:24
*** mfer has quit IRC14:24
*** david-lyle has joined #openstack-infra14:25
john-davidgeyeah, i had a look but didnt see them either, but I've sent an email to sorlando alerting them to the bug14:25
*** yfried__ has quit IRC14:25
*** zaro has joined #openstack-infra14:25
fungiokay, i've switched their account to inactive and am notifying the third-party-announce mlml14:26
john-davidgefungi: is there an easy way to have the -1 taken off my patch or do i need to wait until their issues are fixed?14:27
*** gokrokve has joined #openstack-infra14:27
*** homeless has quit IRC14:28
fungijohn-davidge: it can be safely ignored, or they can update it with a correct result once their system is back in operation, or it will go away when you update your current change with a new patchset14:28
*** jistr has quit IRC14:28
john-davidgefungi: ok, thanks14:29
*** craigbr has joined #openstack-infra14:29
*** paulrad has joined #openstack-infra14:29
dansmithI've seen the python26 nova job time out twice now.. is that just load or is there something up?14:31
mordreddansmith: I think it's because python 2.6 is old and doddery and should be put out to pasture14:34
*** e0ne has joined #openstack-infra14:34
sdagueholy crap, is there someone at canonical we can light on fire about launchpad timeouts14:34
dansmithmordred: heh14:34
*** atiwari has joined #openstack-infra14:34
fungisdague: nah, mordred hired them all14:34
*** homeless has joined #openstack-infra14:35
sdagueI'm very close to rage deleting all openstack bugs, because actually triaging them requires approximately 10 clicks to change a nova state14:36
mordredfungi: I did not hire _all_ of them - wgrant still works there14:36
*** pabelanger_ is now known as pabelanger14:37
*** pabelanger has joined #openstack-infra14:37
mordredsdague: we're getting closer with storyboard ...14:37
*** mfer has joined #openstack-infra14:37
* krotscheck agreees14:38
*** dkranz has joined #openstack-infra14:38
mordredkrotscheck: I wrote and then ignored an import script - did anybody ever do anything with that?14:39
sdaguewell, if there is a chance to get some other projects over there, it might be nice. I think nova is too big to make such a transition.14:40
*** eharney has joined #openstack-infra14:41
jeblairsdague: we're not close to that point.  the transition plan is to get it working enough for infra to use, bash at it a bit, then get it working well enough for openstack to use.14:41
sdagueok, so we're still then talking about a year for something like nova?14:41
krotscheckmordred: No yet.14:41
sdaguewhich leads me back to the initial question, because it seems like just increating the timeout value on launchpad might really freaking help, if we could get someone to do that14:42
*** mmaglana has joined #openstack-infra14:42
mordredsdague: I'm pretty sure they actually lowered it and did that on purpose14:43
mordredsdague: when lifeless is awake, he can tell us more14:43
jeblairmordred: i got a gertty bug report via email and understand the alembic issues14:44
mordredjeblair: ooh! excellent. what are they?14:44
jeblairmordred: we require alembic >0.6.4 (i think; double checking)14:45
sdaguemordred: ok, well I'm am having somewhere around a 30% success rate in changing the triage state on bugs right now14:45
mordredahhhh. that would be a thing that would make since14:45
mordredsdague: jeez14:45
jeblairsdague: chat with ttx, i think he has alternate ways of doing that14:46
sdaguemordred: this is why I want to light all canonical people on fire14:46
sdaguejeblair: he does14:46
jeblairsdague: i want to say it's using email api?14:46
sdaguejeblair: I can't triage 1000 nova bugs via email14:46
sdaguethat's completely unreasonable interface14:46
sdagueat that point we should just delete all bugs in launchpad14:46
sdaguethis isn't on giant cross linked bugs14:47
*** mmaglana has quit IRC14:47
sdaguethis is on nova only bugs14:47
ttxsdague: yeah.. juno-3 was pretty painful14:47
jeblairsdague: i'm not suggesting it's reasonable, i'm suggesting it's functional.  i believe you can bulk edit, so you can do it all in one (or several) emails14:47
ttxTook me like about 40 tries to get the API not to timeout14:48
*** tonytan4ever has joined #openstack-infra14:48
ttxthe email interface trick is not working so well these days. Maybe they closed the loophole14:48
*** tomoe_ has quit IRC14:48
ttxSO I use awesome tricks14:48
ttxlike removing the milestone, setting FixReleased, putting the milestone back14:48
ttxtakes about 20 tries each, but in general it works14:49
ttxBY FAR the most painful part of releasing, those days14:49
ttxI was tempted more than once to just remove the task.14:49
sdaguemordred: yeh, this is basically beyond the pale. Honestly, we need a communication channel here.14:49
*** lttrl has joined #openstack-infra14:49
* ttx could use a /v2/i_m_ok_to_wait_forever LP API endpoint14:50
ttxi still don't get why aggressive timeouts are present on the API side.14:50
*** alkari has joined #openstack-infra14:51
ttxI think that comes with zope territory14:51
dkranzjeblair: I like seeing the jenkins results up top but I no longer see any jenkins results except for the last run. Is there a way to see results from previous patchsets now?14:51
jeblairdkranz: hit the 'toggle ci' button14:52
*** jistr has joined #openstack-infra14:52
dkranzjeblair: Ah, cool. Thanks14:52
*** datsun180b has joined #openstack-infra14:52
*** signed8bit has joined #openstack-infra14:53
*** paulrad has quit IRC14:53
ttxjeblair: still checking out options for the Design Summit. I think Infra could make use of a meetup space on Friday. I was thinking we could share the same room with QA and Release management14:54
openstackgerritKurt Taylor proposed a change to openstack-infra/config: An alternate recheck syntax for third-party CI systems  https://review.openstack.org/11862314:54
ttxjeblair: with that in mind, would you rather have it last for half a day or the full day ?14:54
jeblairttx: full day i think14:54
ttxmore time, but less potential to join other rooms (and probably smaller space)14:55
ttxjeblair: ok14:55
openstackgerritMark Vanderwiel proposed a change to openstack-infra/config: Fix ruby-191-prep to uninstall 1.8 and instal bundler  https://review.openstack.org/12011614:56
ttxjeblair: maybe we'll have less "scheduled sessions" since the informal time is generally better for us14:56
*** emagana has quit IRC14:56
*** jistr has quit IRC14:56
*** jistr has joined #openstack-infra14:57
jeblairttx: yeah, we will probably establish some structure too, like making sure that qa/infra/release all get to talk about things that are important to us; time-boxing some things to make sure we cover what we need to14:58
jrolljeblair: fwiw, I'd love to switch to storyboard asap for ironic :)14:58
krtayloranteaya, thanks for the comment on https://review.openstack.org/#/c/118623/  good catch14:59
jeblairttx: i think that informal things take a bit of time to get going, so i think a full day is better; half would work if we need to -- we'll just need to be efficient about it :)14:59
jeblairjroll: know any angularjs? ;)14:59
ttxjeblair: ack14:59
jrolljeblair: a bit...15:00
openstackgerritJamie Finnigan proposed a change to openstack-infra/config: Adding Bandit project to StackForge  https://review.openstack.org/11986515:00
* jroll runs15:00
jeblairjroll: in all honesty, i'm as eager as anyone, but switching openstack early could be a really big mess, and i think we only get one shot at it.15:01
jrolljeblair: it's hard to get motivated to work on things I don't use... but I'm happy to help if there's work that needs done15:01
jrolljeblair: yeah... I mean, I'd like ironic to beta test for you :)15:01
jeblairjroll: the other thing is that we do have to switch all of openstack at once; the key feature of both of these systems is cross-project bug tracking15:02
jrollgah, true15:02
jeblairjroll: but krotscheck would _love_ someone else looking at the angular stuff15:02
krotscheckjroll: yes15:02
krotscheckWHat he said15:03
jeblairjroll: and with mordred back <nudge, nudge> we should have infra projects ported over soon15:03
jeblairjroll: which should give us a good chunk of real data to work with.  i'm sure we can find ways for infra to mess up ironic.  ;)15:03
jeblairand then you could use it for real15:03
*** zz_jgrimm is now known as jgrimm15:03
jeblairjroll: also, storyboard's bugs are in storyboard, so once you're hacking on it, you'll be set :)15:04
krotscheckFair wrning, I’m on post-op meds15:04
*** reed has joined #openstack-infra15:04
fungii read that as top-post meds15:04
mordredkrotscheck: and this is different from normal days how?15:04
*** virmitio has joined #openstack-infra15:04
*** markmcclain has quit IRC15:05
krotscheckmordred: No attention span…..cooikies!15:06
mordredkrotscheck: mmm15:06
* jroll "borrows" some of krotscheck's cookies15:06
mordredjeblair: is "migrate infra in" essentially blocked on me saying the migration is ready?15:06
*** cdnchris has joined #openstack-infra15:06
*** amuller_ has joined #openstack-infra15:07
jeblairmordred: and probably you running it ;)15:07
mordredyah - I'm guessing you want  me to check with you before running it though15:07
*** mfer has quit IRC15:07
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Update alembic requirements  https://review.openstack.org/12012015:08
jeblairmordred: probably so; we might want to talk about what to do with openstack-ci15:08
*** skelpter1 has joined #openstack-infra15:09
jeblairmordred: i lean toward import all the bugs into 'infra/config', then close all of them in lp, but leave the project there to receive elastic-recheck bugs15:09
*** amuller has quit IRC15:09
*** afazekas_ has quit IRC15:11
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add support for disk-image-builder in nodepool  https://review.openstack.org/8847915:16
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add pre-requisites needed for diskimage-builder  https://review.openstack.org/10258315:16
*** mfer has joined #openstack-infra15:16
*** david-lyle has quit IRC15:19
jrollfungi: https://review.openstack.org/#/c/119473/2/devstack-vm-gate.sh15:19
jrollfungi: should I have the dsg variable default to 'True' rather than '1', and set to 'False' to disblae (rather than set to 0)15:19
bookwar_i see that you install jenkins-job-builder locally on jenkins master, are there any reasons for that? It seems i can just store everything in the repository, check it out every time i need to update jobs and run it from virtualenv (the same way one runs ansible playbooks)15:21
*** yamamoto has quit IRC15:22
fungijroll: on the phone, but left a comment15:22
*** e0ne has quit IRC15:22
fungijroll: you could do that too if you want15:22
jrollfungi: ok, thanks, wasn't sure what conventions were :)15:23
*** Longgeek has quit IRC15:23
fungijroll: conventions in d-g are "terse but clear/readable"15:24
openstackgerritJim Rollenhagen proposed a change to openstack-infra/devstack-gate: Add option for building Ironic deploy ramdisks  https://review.openstack.org/11947315:24
*** e0ne has joined #openstack-infra15:25
mordredbookwar_: that seems more complicated15:25
jrollfungi: cool, fixed :)15:25
mordredbookwar_: we already have a fairly large set of servers on which stuff happens in general, we don't need to optimize in our environment for not installing a python package on a server15:26
*** mikedillion has joined #openstack-infra15:27
fungianteaya: pleia2: my post to the tpa ml about disabling the vmware nsx ci account is held for moderation15:30
*** dkehn has quit IRC15:30
*** dkehn has joined #openstack-infra15:30
*** mmaglana has joined #openstack-infra15:31
*** kgiusti has quit IRC15:31
*** dkehn has quit IRC15:32
*** andreykurilin has quit IRC15:32
*** dkehn has joined #openstack-infra15:32
fungiseems to be especially bad today15:33
*** ndipanov has joined #openstack-infra15:33
ndipanovhey guys - anyone can tell me if I can go ahead and rebase 112977 based on zuul15:34
ndipanovthe ui seems to suggest it failed a merge, but when I try to rebase it locally there seem to be no conflicts15:34
ndipanovand I'd rather not rebae needlesly if it still stands a chance of making it since it's a 13h+ penalty atm15:35
bookwar_mordred: but i am thinking about updates: jjb's development is fast, and frequent updates on a server-side give more troubles, while keeping local env up-to-date seems to be easy - just fix the requirements.txt in repository. On the other hand - easy updates => easy errors.15:35
*** dkehn has quit IRC15:36
*** dimtruck is now known as zz_dimtruck15:38
*** asselin has joined #openstack-infra15:38
*** _buzz has quit IRC15:38
*** dhellmann is now known as dhellmann_15:38
*** dtantsur is now known as dtantsur|afk15:38
fungibookwar_: i think distros package jjb, if you're looking for a stable version to put on your servers... or only install from the releases to pypi rather than every commit in a cd fashion, or...15:39
*** dkehn has joined #openstack-infra15:40
*** garyh has joined #openstack-infra15:40
*** gokrokve_ has joined #openstack-infra15:41
anteayafungi: cleared and hopefully you won't be moderated henceforth15:42
fungianteaya: okay, cool15:42
fungianteaya: thanks!15:42
*** pblaho is now known as pblaho|afk15:42
*** ericpeterson has joined #openstack-infra15:43
*** jlibosva has quit IRC15:44
*** jlibosva has joined #openstack-infra15:44
anteayakrtaylor: sure15:44
*** gokrokve has quit IRC15:44
*** Longgeek has joined #openstack-infra15:45
*** Longgeek has quit IRC15:46
*** cdnchris has quit IRC15:48
*** david-lyle has joined #openstack-infra15:48
*** zz_dimtruck is now known as dimtruck15:48
*** cdent has quit IRC15:48
*** e0ne has quit IRC15:49
*** cdent has joined #openstack-infra15:49
*** maxbit has quit IRC15:50
*** cdent has quit IRC15:50
*** mpaolino has quit IRC15:50
*** cdent has joined #openstack-infra15:51
*** yamamoto has joined #openstack-infra15:52
*** ZZelle has joined #openstack-infra15:53
clarkbndipanov: zuul/gerrit should only report that you need to rebase if that is the case15:53
*** ericpeterson has left #openstack-infra15:53
clarkbndipanov: not gerrit uses jgit not the C git implementation and sometimes it has merge conflicys that C git handles for you.15:53
*** changbl has joined #openstack-infra15:54
ndipanovclarkb, when I hover over the gray circlrcle next to my change it says something about merge conflicts15:54
*** pblaho|afk is now known as pblaho15:54
clarkboh is it still in the gate?15:54
ndipanovclarkb, hmmm maybe not anymore let me check15:54
ndipanovclarkb, yes it is15:55
ndipanovand now it's green15:55
ndipanovI mean queued but green15:55
fungindipanov: probably there was a change ahead of it in the gate which was in conflict with your change, but then failed out15:56
*** MaxV has quit IRC15:56
clarkbI dont see it but I may be blind15:56
ndipanovfungi, I assumed so but there were really no other nova changes15:56
ndipanovoh well15:57
*** yamamoto has quit IRC15:57
ndipanovclarkb, 11095715:57
*** Longgeek has joined #openstack-infra15:57
clarkbya that change should not need a rebase. grey circles means you are outside the window and shouldnt even be checked for merge conflicts hrm15:58
*** lcheng_ has joined #openstack-infra15:59
*** maxbit has joined #openstack-infra15:59
fungilooks like nodepool may be having trouble building new nodes, just based on the ratio of yellow to magenta on the graph now15:59
fungichecking logs to confirm/diagnose16:00
ndipanovclarkb, fungi anyway - thanks guys!16:01
*** arxcruz has quit IRC16:02
fungindipanov: yeah, if the circle was gray then it wasn't far enough up the pipeline to begin testing. red or black generally indicate a problem with a test, dependency or merge conflict16:03
ndipanovfungi, ok cool good to knwo16:04
fungiwe're seeing a fair number of node launching exceptions, most in rax-dfw16:04
fungithough only on the order of about 1 per minute16:04
*** pkoniszewski has quit IRC16:05
*** cdent has quit IRC16:05
*** marcoemorais has joined #openstack-infra16:05
fungi112 nodes in a building state in dfw right now... that's the majority of its quota16:05
fungisome are building for as much as an hour16:06
fungi46 in a building state in dfw for between .5 and 1 hour16:08
*** jgallard has quit IRC16:08
*** lcheng__ has joined #openstack-infra16:09
fungilooks like the build exceptions are for timeouts in dfw (hitting the one hour mark and then giving up)16:09
*** dangers_away is now known as dangers16:09
fungiphschwartz: if you're around, we're seeing significant performance issues in dfw, taking more than an hour (maybe indefinitely? that's the point at which we abort) to boot from snapshots16:10
*** lcheng_ has quit IRC16:12
jeblairsdague: are you using emacs24?  i just upgraded to trusty, and i can't seem to set the mark (except using M-x set-mark-command); did you run into anything like that?16:12
*** yfried__ has joined #openstack-infra16:13
uvirtbotLaunchpad bug 1289570 in emacs24 "Can't set mark (dup-of: 1278569)" [Undecided,Confirmed]16:13
jeblairhuh, i wonder if that's it16:13
uvirtbotLaunchpad bug 1278569 in ibus "ibus breaks emacs and eclipse control-space keybinding" [High,Triaged]16:13
homelesshi guys, I need your help, Im configuring nodepool but im having problems with the private-key, I dont really undestand which key and user should I use16:13
*** Longgeek has quit IRC16:14
*** Longgeek has joined #openstack-infra16:14
homelessbecause my nodes they just created, and then they just got destroyed because they are not able to ssh connect16:14
*** julim has quit IRC16:14
clarkbhomeless: it should be a key and user created in the images as part of the image build process.16:14
clarkbhomeless: in our case we create a jenkins user with authorized_keys file that matches the private key there16:14
*** Longgeek has quit IRC16:14
homelessgotcha, thansk clarkb16:15
fungihomeless: for our servers, we "create" it on our images by installing it with a puppet manifest16:15
homelessthanks fungi16:15
fungiso it's not something nodepool magically installs for you. you'll probably need some sort of automation/config management to put it on there16:16
fungias part of the image creation16:16
*** prad has joined #openstack-infra16:16
homelessright, thanks16:16
*** masayukig_ has quit IRC16:18
*** mpaolino has joined #openstack-infra16:18
*** pblaho_ has joined #openstack-infra16:19
JayFdhellmann_: I run my unit tests outside of devstack as well... I can run linux in a local VM for unit tests, but devstack is too heavy/slow16:19
openstackgerritKurt Taylor proposed a change to openstack-infra/config: An alternate recheck syntax for third-party CI systems  https://review.openstack.org/11862316:20
*** zeus has quit IRC16:21
*** pblaho has quit IRC16:21
*** julim has joined #openstack-infra16:22
*** zeus has joined #openstack-infra16:22
*** zeus is now known as Guest3995616:22
*** yamamoto has joined #openstack-infra16:23
*** masayukig has joined #openstack-infra16:24
mordredjeblair: so - I'm working on the nodepool dib sutff16:25
mordredjeblair: and there are several places where teh non-dib codepath does things with hostname but we're not really doing that in the dib path right now16:26
mordredjeblair: actually, I think I grok what I should do16:26
mordredjeblair: thanks for the help!16:26
clarkbmordred: is that for the dns resolution of localhost?16:26
clarkbmordred: iirc nodepool doesn't actually deal with it right because the hostname is not passed as $1 to the scripts16:27
clarkbso it may be somewhat noopy right now16:27
*** derekh has quit IRC16:27
anteayafungi: do you want to test if you can post unmoderated to tpa? I have two more ci's that need to be disabled16:27
anteayafungi: if yes, I'll give you details and if no, I'll just give you names and take care of it myself16:28
openstackgerritA change was merged to stackforge/gertty: Rename doc environment to docs  https://review.openstack.org/11991416:28
openstackgerritA change was merged to stackforge/gertty: Update alembic requirements  https://review.openstack.org/12012016:28
*** yamamoto has quit IRC16:28
winston-djeblair: morning. wondering how i can change the keybinding in gertty. don't really know how 'gertty -k' works16:28
*** rmcall has joined #openstack-infra16:28
fungianteaya: i can disable them for you--that's fast. if you want to write the ml posts though, i'd be grateful. trying to get back to comparing python 3.3->3.4 transition test results16:29
jeblairwinston-d: good morning.  i think we found and fixed the db problem you were having yesterday; it turns out alembic 0.6.4 is required.16:29
*** mpaolino has quit IRC16:29
jeblairwinston-d: see git.openstack.org/cgit/stackforge/gertty/tree/examples/reference-gertty.yaml#n8616:29
winston-djeblair: that's awesome!16:30
jeblairwinston-d: there's an example there16:30
anteayadhellmann_: I don't think we have a policy around not posting in -dev, we just never have before so we don't have a precedent16:30
*** mpaolino has joined #openstack-infra16:30
jeblairwinston-d: and gertty -k will give you the list of commands that go on the left hand side (like 'diff' or 'review' in the example)16:30
anteayadhellmann_: I think others may want to see the changes too, so I think it is worth an ml thread16:30
*** markmcclain has joined #openstack-infra16:30
anteayaKiall: thanks16:30
jeblairwinston-d: oh wait, that's wrong, 1 second16:30
*** bhuvan has joined #openstack-infra16:31
jeblairwinston-d: "gertty --print-keymap" will do that16:31
mordredclarkb: right. I think we need to to pass the self.dib_image.image_name in at the very least16:31
*** markmcclain has quit IRC16:32
anteayajohn-davidge: can I get a url of a patch with a IBM SDN-VE CI failure that you have noticed?16:32
winston-djeblair: thx, that's enough for me to get started.16:32
*** gokrokve_ has quit IRC16:32
john-davidgeanteaya: https://review.openstack.org/#/c/112836/16:33
*** markmcclain has joined #openstack-infra16:33
*** koolhead17 has joined #openstack-infra16:34
*** koolhead17 has quit IRC16:34
*** koolhead17 has joined #openstack-infra16:34
funginow this is downright odd... stackforge/wsme unit tests fail on precise-py3k under py33 right now, but pass on bare-trusty under py3416:35
*** david-lyle has quit IRC16:35
clarkbfungi: huh. perhaps a test order thing and py34 got lucky?16:35
clarkbis py33 wsme job voting?16:36
*** david-lyle has joined #openstack-infra16:36
anteayayfried__ clarkb diabled notices for ci's are posted to third-party-announce, archives available at lists.openstack.org http://lists.openstack.org/pipermail/third-party-announce/2014-September/thread.html#116:36
fungiclarkb: yeah, i only analyzed jobs which were voting according to zuul's parsing of the current production layout file16:36
*** kgiusti has joined #openstack-infra16:36
clarkbfungi: yup looks like its voting16:36
clarkbanteaya: hrm I may not be subscribed like I thought I was16:37
anteayajohn-davidge: thanks16:37
fungithe bad news is that my initial findings (repeated the loop twice just to rule out intermittent errors) has 36 jobs which will fail under 3.4 but are working and voting under 3.316:37
fungionly 10 of those are stackforge, the rest are official projects16:37
clarkbfungi: wow, that is unexpected. Let me know if you want eyes on debugging16:38
anteayafungi: kk, I'll get you the names, looks like this might be a ci round up day for me *sigh*16:38
fungiclarkb: will do. i need to manually rerun a few to see if i can spot something common like missing dependencies16:38
fungiclarkb: but if it gets esoteric, i'll bring you up to speed16:38
clarkbfungi: or possibly unset PYTHONHASHSEED?16:38
openstackgerritMonty Taylor proposed a change to openstack-infra/nodepool: Export image name and filename to DIB  https://review.openstack.org/12014416:39
mordredclarkb, jeblair ^^ I think we want that so taht I can duplicate the current functionality of hostname16:39
anteayajohn-davidge: well IBM SDN-VE CI gets disable for prompting me to download the logs, rather than being able to read them16:39
anteayajohn-davidge: so I can't even see the error you are seeing, since I don't download logs from ci's16:40
john-davidgeanteaya: Great, I hate when they do that16:40
anteayajohn-davidge: they get disabled if they do16:40
anteayajohn-davidge: so if you spot any, bring me a url16:40
john-davidgeanteaya: will do!16:41
jeblairmordred: where will that be used?16:41
jeblairoh, like for debugging and stuff16:42
jeblairso we know "this instance was build from this dib filename/image name" ?16:42
openstackgerritMonty Taylor proposed a change to openstack-infra/nodepool: Export image name and filename to DIB  https://review.openstack.org/12014416:42
*** rmcall has quit IRC16:42
mordredand actually, probably more like that for consistency16:42
*** mpaolino has quit IRC16:43
*** rmcall has joined #openstack-infra16:43
mordredalthough /etc/image-hostname.txt is about to be poorly named ...16:43
jeblairmordred: are you about to push another one, or is that your final patch? ;)16:44
mordredjeblair: that's it16:44
anteayafungi: 3 to disable: http://paste.openstack.org/show/108963/ ids and reasons in paste16:45
*** lcheng__ has quit IRC16:45
*** lcheng_ has joined #openstack-infra16:45
*** HeOS has quit IRC16:45
fungianteaya: all three accounts are now set to inactive in gerrit16:46
anteayathank you16:46
* anteaya composed emails16:46
openstackgerritJames E. Blair proposed a change to openstack-infra/infra-specs: Add config repo split spec  https://review.openstack.org/11073016:47
jeblairclarkb, fungi: that's with layout-dev too ^  sorry about that16:47
*** yidclare has joined #openstack-infra16:48
clarkbfungi: any chance you can give me a project that failed py34 but not py33? curious to see if I can reproduce locally16:48
*** andreykurilin has joined #openstack-infra16:49
clarkbjeblair: no problem. should've caught that16:49
*** arnaud__ has joined #openstack-infra16:49
clarkbjeblair: just as much on the reviewers :)16:49
*** rushiagr is now known as rushiagr_away16:49
clarkblooks like fungi did catch it16:49
jeblairclarkb: yep, and my brain function apparently returned two different results on two different invocations :)16:50
jeblairnondeterminism in humans is so ick.16:50
clarkbI am looking at python3.4 change log. there is a lot of stuff but nothing too backward incompatible16:51
mordredjeblair: hrm. did we give any thought as to how ready_script might work with dib images?16:51
mordredjeblair: asking before I read the code in case you have the answer off the top of your head16:51
jeblairmordred: ready_script is used on instances, not images16:51
fungiclarkb: yeah, http://paste.openstack.org/show/10897916:51
mordredjeblair: yes - but the scripts have to get onto the image16:51
fungiclarkb: i'm starting at the top with pbr, and am getting a wonderfully opaque failure16:51
*** lcheng__ has joined #openstack-infra16:51
fungiclarkb: nevermind--not at all opaque. we need python3.4-dev16:52
mordredjeblair: maybe same as before, just have dib copy the entire contents of the nodepool scripts dir into the image?16:52
jeblairmordred: ah, i assume we can stick them in as elements or something?  if not, we could scp them onto the instance like we do the image scripts (for the current snapshot image build process)16:52
fungiclarkb: fixing and seeing how many of these drop off the failing list now16:52
*** lcheng_ has quit IRC16:52
jeblairmordred: yeah, if you can have dib copy them in place, that would be the smallest delta16:52
clarkbfungi: oh right because pip16:52
mordredjeblair: k. I'll do that16:52
*** andreaf has joined #openstack-infra16:53
clarkbfungi: python3-all-dev is probably what we want for all the things16:53
*** Daviey has quit IRC16:53
clarkbfungi: which has the nice benefit of being future compat with packages in future releases16:53
*** amuller__ has joined #openstack-infra16:54
fungiclarkb: yep, very good idea16:54
*** sarob_ has joined #openstack-infra16:55
anteayaBobBall_AWOL: you back?16:55
clarkbfungi: though the package description says it may eventually depend on jython and ironpython at some point :)16:56
pleia2oof, multihomed and both internet connections are having trouble this morning16:56
fungiclarkb: fun... happybase installation fails with "NameError: name 'execfile' is not defined"16:56
clarkbpleia2: you need to tether off of your phone for maximum redundancy16:56
pleia2clarkb: haha, perhaps!16:56
fungiclarkb: so ceilometer is breaking on that16:57
openstackgerritMonty Taylor proposed a change to openstack-infra/nodepool: Pass the nodepool scriptdir to disk-image-create  https://review.openstack.org/12015116:57
mordredjeblair: I believe we'll need that too ^^16:57
jeblairtagged gertty 1.0.216:57
clarkbfungi: execfile was removed in 3.0. so ceilometer must not have been passing py3k testing?16:58
*** amuller_ has quit IRC16:58
*** bdpayne has joined #openstack-infra16:58
clarkbhrm the job is voting16:59
clarkbthat is an odd one16:59
*** lcheng_ has joined #openstack-infra16:59
fungiclarkb: more likely happybase is doing python version detection very poorly and falling back to a 2.x function on 3.416:59
*** lcheng__ has quit IRC16:59
clarkboh right its in the dependency16:59
*** mestery has quit IRC16:59
jeblairmordred: lgtm.  we should swing back and normalize those variable names at somepoint.16:59
fungiclarkb: ceilo's py33 jobs seem to be working judging from https://jenkins01.openstack.org/job/gate-ceilometer-python33/16:59
*** mestery has joined #openstack-infra17:00
clarkbfungi: ya17:00
clarkbhappybase doesn't seem to claim any py3k support. maybe ceilometer is doing conditional import?17:01
fungiclarkb: oh... could be17:01
clarkbno happybase in the log for py33 jobs linked above17:02
clarkbhrm and they have a py3k requireemtns without happybase in it17:03
*** lcheng_ has quit IRC17:03
clarkbso maybe a pbr bug?17:03
pleia2anteaya: how are we handling messages about outages to -announce? let them through?17:03
pleia2(one about Midokura CI in the queue)17:04
*** dizquierdo has quit IRC17:04
anteayapleia2: I'm reading them and if the content is a direct reply to something posted I let it through17:04
*** johnthetubaguy is now known as zz_johnthetubagu17:05
pleia2ok, this one is not17:05
clarkbfungi: I think pbr is correct so probably a ceilometer thing17:05
*** tonytan4ever has quit IRC17:05
anteayapleia2: I don't think Midokura CI is currently down, if it is announcing it will be down hmmmm, that is what this wikipage is designed for: https://wiki.openstack.org/wiki/ThirdPartySystems17:05
jeblairpleia2: i don't think -announce is the right place for outage notifications, they should go to -dev.17:05
jeblairand yeah, the wikiu17:05
fungiclarkb: agreed, i'm digging deeper inside ceilometer now17:05
*** Daviey has joined #openstack-infra17:06
pleia2they did update their wiki page17:06
pleia2so I guess they're just some confusion about where these notifications should go17:06
fungiclarkb: oho, they're one of the projects overriding their testenv:py33 so they need to merge the patch i proposed last week17:06
anteayapleia2: but they haven't changed the row colour to show their status is down: https://wiki.openstack.org/wiki/ThirdPartySystems17:06
anteayapleia2: instructions for which are bullet point four at the bottom of the page17:07
anteayapleia2: yes, some confusion17:07
pleia2anteaya: yeah, they only updated https://wiki.openstack.org/wiki/ThirdPartySystems17:07
fungiclarkb: https://review.openstack.org/#/c/118770/17:07
pleia2Current Status: Down for maintenance17:07
anteayaI'm for having one place for things, not multiple17:07
*** luqas has quit IRC17:07
anteayapleia2: great, then that will show in their history, or they can keep a list of status updates like others do17:08
*** Longgeek has joined #openstack-infra17:08
clarkbfungi: honestly the utility of that seems minimal... but ok17:08
pleia2so they updated their specific wiki page, and emailed the announce list; they should have updated the ThirdPartySystems page and not emailed anyone?17:08
clarkbfungi: we are running a very restricted set of tests. I imagine the intention was to expand that over time. I wonder how much work is happening on that17:08
anteayapleia2: update their specific wikipage and ThirdPartySystems and not email anyone, correct17:08
fungiclarkb: well, the utility of that seems to be so as to support their py3k-specific reqs fwr17:08
pleia2anteaya: ok, so still updating 2 spots, just no email17:08
clarkbfungi: right but it is running a small subset of the tests17:09
anteayapleia2: the wikipage keeps a history if they need an archive17:09
anteayapleia2: correct17:09
ruhehi infra core folks! it seems that meetbot passed away. at least it doesn't change topics in #openstack-meeting-alt17:09
pleia2anteaya: got it :)17:09
anteayapleia2: thanks17:09
fungiruhe: checking17:09
clarkbfungi: one file out of all ceilometer/tests17:09
jeblairfungi: looks like it's running, maybe it just lost perms17:09
fungijeblair: yep, it has17:10
*** davidlenwell_ is now known as davidlenwell17:10
*** coolsvap has quit IRC17:10
*** SumitNaiksatam has quit IRC17:10
*** cdnchris has joined #openstack-infra17:11
*** Ajaeger1 has joined #openstack-infra17:12
fungijeblair: is it supposed to be a chanop or just have topic perms?17:12
*** pblaho_ is now known as pblaho|afk17:13
fungimeetbots seem to get +O in http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/accessbot/channels.yaml#n2517:13
*** dhellmann_ is now known as dhellmann17:13
ruhefungi, jeblair: hooray! you fixed it. thank you17:13
*** rushiagr_away is now known as rushiagr17:14
jeblairfungi: yep.  and i think it's missing in all 3 channels.17:14
*** TravT has quit IRC17:14
fungijeblair: it is... but the chanserv access list says it has +O granted17:14
jeblairfungi: i'm assuming services got confused after a netsplit and forgot to +o it17:15
*** zaro has quit IRC17:15
*** jp_at_hp has quit IRC17:15
fungijeblair: that seems most likely17:16
fungijeblair: anyway, i've restored its mode in all three channels now17:16
jeblairfungi: thanks17:17
clarkbfungi: I poked #openstack-ceilometer about that change17:17
anteayapleia2: you have all you need then?17:17
anteayapleia2: I was about to go have a walk17:17
pleia2anteaya: yes, I rejected the mail with a friendly message about the proper procedure :) thanks17:18
fungiclarkb: yeah, once i have a list of projects with outstanding support changes blocking the transition i'll start hounding them doggedly17:18
*** coolsvap has joined #openstack-infra17:18
pleia2anteaya: enjoy your walk!17:18
*** Ryan_Lane has joined #openstack-infra17:18
anteayapleia2: thanks for doing that, and I hope to17:18
*** eglynn-office is now known as eglynn-officeafk17:19
clarkbharlowja_away: how is that simplification of taskflow tox.ini files going?17:19
*** pblaho|afk is now known as pblaho_17:19
*** garyh has quit IRC17:19
*** rushiagr is now known as rushiagr_away17:20
clarkbmordred: I really wonder if having all the supporting code (setup.py, tox.ini, run_tests.sh, LICENSE et al) in a git submodule that is shared across projects would make this more sane17:20
*** gokrokve has joined #openstack-infra17:21
*** rushiagr_away is now known as rushiagr17:21
mordredclarkb: doubtful. almost nobody knows how to use submodules17:21
JayFclarkb: what's to prevent projects from never bumping their submodule version just like they don't merge patches sometimes :(17:21
clarkbJayF: we acn have gerrit update them17:21
*** sarob_ has quit IRC17:22
mordredJayF: in gerrit, submodules can be set up to be managed by gerrit17:22
*** isviridov is now known as isviridov_away17:22
clarkbmordred: ya :(17:22
mordredJayF: so taht a commit to the submodule auto-updates ... this is how openstack/openstack works17:22
clarkbmordred: I just we have bascially reimplemented autoconf for tox apparently17:22
JayFThen I really don't like the idea. I would not want my tests/gate broken by gerrit without my interaction17:22
mordredclarkb: really?17:22
clarkbmordred: see taskflow17:22
*** sarob_ has joined #openstack-infra17:22
clarkbI mean its not super terrible bad but they are using 'toxgen' to explode a giant tox.ini17:22
nibalizerprobably easier to write a tool to keep all those files in sync17:22
*** ZZelle_ has joined #openstack-infra17:22
nibalizercrinkle: recently wrote that tool for the puppet modules17:23
*** rushiagr is now known as rushiagr_away17:23
mordredclarkb: uhm17:23
clarkbnibalizer: yeah we have some tools too. the issue is more on the enforcement side. I can propose a tox.ini file to you, but it is very hard to enforce you use a particular one without overwriting it each time like we do with requirements17:23
*** yamamoto has joined #openstack-infra17:23
*** craigbr has quit IRC17:23
clarkbmordred: I think harlowja_away has a change to stop or reduce the use of it17:23
jeblairyes he does17:24
nibalizerya isn't that PBR17:24
clarkbnibalizer: no, pbr only touches setup.py stuff and makes it declarative17:24
jeblairmordred: speaking of which, why is customization of the tox ini needed to include a changelog in a docs build?17:24
jeblairmordred: isn't the changelog generated by the tox sdist step, and therefore available to a docs build even in the current 'venv' system?17:25
clarkbnibalizer: so similar problem but with a lot of freedom still17:25
*** cdnchris has quit IRC17:25
*** sarob_ has quit IRC17:25
clarkbjeblair: no because we do not sdist anymore17:25
clarkbjeblair: I am slowly thinking we should go back to sdisting with tox17:25
clarkbit is slower but ensures we can build sdists and provides all build artifacts to the tox targets17:25
*** harlowja_away is now known as harlowja17:25
fungiclarkb: one of the projects i patched the tox.ini for (i forget which... maybe pecan?) carries a fork of toxgen in its tree so you can't regenerate the tox.ini with a normal toxgen either17:26
*** pblaho_ has quit IRC17:26
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add support for disk-image-builder in nodepool  https://review.openstack.org/8847917:26
jeblairclarkb: oh.  well, can we call pbr.make_me_a_changelog in the docs conf.py?17:26
jeblairmordred: ^?17:26
*** SumitNaiksatam has joined #openstack-infra17:26
mordredclarkb: no17:26
clarkbjeblair: that is probably doable17:26
harlowjaclarkb mordred https://review.openstack.org/#/c/118941/17:26
mordredthere is NO REASON to use sdist17:26
mordredwe can make the other thing work just fine without doing the sdist thing17:26
jeblairmostly i'm thinking that putting docs build steps in tox.ini feels like the wrong place17:26
clarkbharlowja: thanks17:26
mordredI'll get to fixing pbr so that it does that ... it's about 3 down on the list right now17:27
mordredit's a very easy fix17:27
jeblair(i'm still happy to make the docs PTI change; i just also want building docs to be easy and consistent)17:27
clarkbmordred: what is wrong with sdists? aiui it is still the only install method that is expected to work on all supported platforms17:27
mordredclarkb: they are not needed17:27
mordredand no, pip install . shoudl also work on all supported platforms17:27
mordredthe sdist step is not needed for any reason17:27
*** praneshp has joined #openstack-infra17:28
*** yamamoto has quit IRC17:28
clarkbmordred: doesn't pip install . make an sdist?17:28
*** lcheng_ has joined #openstack-infra17:28
jeblairmordred: and when you 'pip install .', pbr does not make a changelog?17:28
fungiclarkb: not that i'm inclined to care, but stackforge/gnocchi seems to try to use ConfigParser under 3.4 but configparser (correctly) under 3.3. wonder what craziness they've implemented there17:28
clarkbfungi: interesting17:28
*** amcrn has joined #openstack-infra17:28
clarkbfungi: they probably check the minor version when they should check only major version?17:28
mordredjeblair: no, it does not. we could make it if people think that's something they want17:29
jeblairmordred: probably not, just asking :)17:29
fungiclarkb: oh, wait! i do care... it's mysql-python doing it on install, not gnocchi17:29
*** otter768 has joined #openstack-infra17:29
*** amotoki has quit IRC17:29
mordredjeblair: I think the easier thing would be to have build_Sphinx just trigger the "create changelog" step17:29
fungitup_posix.py", line 2, in <module>17:30
fungigah, stray newlines17:30
jeblairmordred: explicitly (like in conf.py?) or implicitly?17:30
*** amotoki has joined #openstack-infra17:30
harlowjamordred like https://review.openstack.org/#/c/119688/  ??17:30
jeblairmordred: i kind of think that saying "i need a changelog to build docs" in conf.py is reasonable17:30
jeblair(since not everyone needs a changelog to build docs)17:30
clarkbharlowja: so in that tox.ini cleanup. why does [testenv] explicitly list requireemnts that are not in files?17:30
mordredwhat is conf.py ?17:31
fungiclarkb: oh, right... it's contagious https://review.openstack.org/#/c/118821/1/tox.ini17:31
mordredharlowja: yes. exactly like that17:31
clarkbfungi: that change fixes it?17:31
mordredharlowja: except it might be nice for us to include a boolean flag in setup.cfg17:32
harlowjaclarkb mainly those things are optional, not strictly required17:32
clarkbharlowja: except you are making them required to do tests17:32
clarkbharlowja: so why not list them in test requireemnts?17:32
harlowjathink thats fair, can probably move them17:32
mordredjeblair: not everyone needs that - however, the cost for building docs is so much higher than the cost for writing the changelog - I wonder if we don't just make that the behavior17:32
fungiclarkb: apparently they only import mysql-python in test-requirements.txt but not in test-requirements-py3.txt17:32
clarkbfungi: oh I see17:32
mordredjeblair: (ultimately, every time we have an option flag for pbr I die inside a little bit)17:32
jeblairmordred: fair enough :)17:32
harlowjaclarkb let me move those out of there17:33
dhellmannfungi: I think that fork of toxgen predates toxgen being released standalone (and I think it was WSME first, but maybe pecan has one, too)17:33
jeblairharlowja: you are way ahead of me :)17:33
fungidhellmann: oho--that 'splains it17:33
mordredharlowja: so, that said - I am +2 on that patch, and I have changed my mind, I do not think we need a config option17:34
*** amotoki has quit IRC17:34
harlowjamordred i tried putting in a [build_sphinx] option, that didn't go so well17:34
harlowjaso i just went back to that easier solution17:34
*** jcoufal has quit IRC17:34
jeblairmordred: is there ever a case where pbr would delete uncommitted files from my working tree?17:34
*** jistr has quit IRC17:35
*** e0ne has joined #openstack-infra17:35
*** dtantsur|afk is now known as dtantsur17:35
clarkbfungi: find any other major headaches beyond those projects needing tox.ini changes?17:37
*** emagana has joined #openstack-infra17:38
fungiclarkb: keystonemiddleware hung and started complaining about urllib3 timeouts, signing errors and system time being way off (which i confirmed in another shell on the same system was not the case)17:38
fungios-apply-config fails inside a test (possibly a nondeterminism?)17:39
*** andreykurilin has quit IRC17:40
clarkbI can give os-apply-config a run to see if it is consistent17:40
fungiyeah, i'm waiting for the loop to complete and then i'll have a short-list we can iterate more tightly over17:40
fungibut feel free to get a head start on one if you like17:41
clarkbthe urllib3 thing is odd. I wonder if they are mocking/patching time.time17:41
fungiseems possible17:42
fungiit was like the tests for keystonemiddleware had hung indefinitely, which i don't think i saw on earlier passes17:42
fungiso am eager to try to reproduce that17:42
clarkbFAIL: os_apply_config.tests.test_apply_config.OSConfigApplierTestCase.test_load_list_from_json17:43
clarkbwith no JSON object could be decoded messages17:43
fungiyep. that's the one i was hitting17:43
*** pelix has quit IRC17:43
clarkbERROR:os-apply-config:cannot interpret value 'foo' as type int may be the underlying cause17:44
clarkboh I see17:44
clarkbthey are asserting an error message17:44
fungihah, which possibly changed17:44
clarkbbut the error message seems to have changed. that seems... gfragile17:44
clarkbshould just assert it failed imo17:44
* clarkb hops over to #tripleo17:45
*** e0ne has quit IRC17:46
*** gyee has joined #openstack-infra17:46
dteselkinHi, I need an advice about my commit to openstack-infra17:47
dteselkinIt's here https://review.openstack.org/11866017:47
fungiclarkb: whilst you're bringing that to their attention, os-collect-config is also breaking17:47
clarkbfungi: kk, will look at that one too17:48
dteselkinThe problem is that there is another commit with similar functionality, here https://review.openstack.org/11760417:48
dteselkinSo I would like to know, what should I with mine - abandon it or let the reviewers decide?17:49
fungidteselkin: you should probably discuss it with nibalizer (he's the author of the other patch)17:49
*** jlibosva has quit IRC17:49
*** skelpter1 has left #openstack-infra17:49
*** otter768 has quit IRC17:49
*** yfried__ has quit IRC17:49
*** dtantsur is now known as dtantsur|afk17:50
*** david-lyle has quit IRC17:50
dteselkinfungi, ok, thanks!17:50
clarkbdteselkin: fungi nibalizer it is probably reasonable for nibalizer to rebase his change on your change since 118660 is a bit more specific17:50
*** lcheng_ has quit IRC17:50
clarkbfungi: wow the keystonemiddleware system time error is fun :)17:51
*** lcheng_ has joined #openstack-infra17:51
*** lcheng_ has quit IRC17:51
*** todd_dsm has joined #openstack-infra17:52
*** homeless_ has joined #openstack-infra17:52
*** amuller__ is now known as amuller17:53
ryanpetrellodhellmann: fungi: pecan doesn't use toxgen,17:53
ryanpetrelloWSME does17:53
ryanpetrello(unless somebody snuck it in when I wasn't looking)17:54
openstackgerritA change was merged to openstack-infra/infra-specs: Add config repo split spec  https://review.openstack.org/11073017:54
dhellmannryanpetrello: ok, I couldn't remember if you'd used it for the extra configurations you were testing or not17:54
bknudsonhas anyone seen this before? during gate glance-manage isn't installed17:54
fungiryanpetrello: that was probably the one i was remembering then17:54
*** homeless has quit IRC17:55
SlickNikCan I get some eyeballs on https://review.openstack.org/#/c/105692/ when folks get a chance? Trying to get trove CI off of rdjenkins. Thanks much!17:55
jeblairSlickNik: you said all the right things to get my attention :)17:57
jeblairSlickNik: lgtm17:57
fungiSlickNik: spotted an error. see inline17:58
clarkbbknudson: it looks like the install_glance step failed without failing hard http://logs.openstack.org/29/119629/4/check/check-tempest-dsvm-postgres-full/f6e1480/logs/devstacklog.txt.gz#_2014-09-09_17_34_26_91817:58
SlickNikfungi: just saw it. Nicely spotted. Will follow up with an update right now.17:59
clarkbfungi: with the eagle eyes17:59
bknudsonclarkb: ah, that's tricky!17:59
fungiSlickNik: other than that, looks ready to approve17:59
*** emagana has quit IRC18:00
*** emagana has joined #openstack-infra18:00
*** emagana_ has joined #openstack-infra18:01
openstackgerritNikhil Manchanda proposed a change to openstack-infra/config: Fixed gate-trove-functional-dsvm jobs  https://review.openstack.org/10569218:01
fungioslo.rootwrap looks like it also has a test issue18:02
*** emagana has quit IRC18:02
fungiFAIL: tests.test_rootwrap.RootwrapTestCase.test_RootwrapConfig18:02
fungiValueError: Unexpected syslog_log_level: INFO18:02
*** rmcall has quit IRC18:03
clarkbfungi: os-apply-config fix pushed18:05
*** rmcall has joined #openstack-infra18:06
clarkbrunning collect-config tests now18:06
fungiclarkb: awesome. hopefully i'll have a more accurate list shortly, and then i'll etherpad it with links to pending tox.ini changes and other fixes18:06
*** praneshp has quit IRC18:07
*** praneshp has joined #openstack-infra18:07
*** _buzz has joined #openstack-infra18:09
*** praneshp has quit IRC18:10
*** praneshp has joined #openstack-infra18:10
*** dims has joined #openstack-infra18:11
*** dims_ has quit IRC18:11
*** changbl has quit IRC18:11
*** mwagner_lap has quit IRC18:11
*** dims has quit IRC18:12
*** marun has quit IRC18:12
*** mrmartin has joined #openstack-infra18:12
clarkbfungi: I am not convinced collect-config should pass under py33 either. it does locale.getlocale(locale.LC_ALL) which the docs say is a no no18:13
fungiclarkb: good point... let me see if it actually did18:14
*** tonytan4ever has joined #openstack-infra18:14
fungiclarkb: you're right... the paste i linked earlier didn't have it in the list18:15
fungiclarkb: which means it's failing on 3.3 as well. presumably they turned the job to voting without confirming it worked18:15
*** dims_ has joined #openstack-infra18:16
*** changbl has joined #openstack-infra18:16
*** koolhead17 has quit IRC18:17
*** _nadya_ has quit IRC18:19
clarkbfungi: though the py2.7 docs say the same thing18:19
clarkbso I am not sure what is going on there but I think we can ignore it ;)18:19
dhellmanndoc build question: Now that we do a full pypi mirror, if I have a package I only need for my documentation build, can I just add it to tox.ini instead of test-requirements.txt? Or do I still need that to go through the global requirements list and test-requirements.txt?18:19
dhellmannI should say that's for a sphinx job in something like a specs repo or project repo18:19
*** e0ne has joined #openstack-infra18:20
clarkbdhellmann: technically you can avoid global requirements. However, the docs builds are potentially important to packagers (for man pages in particular) and should probably go through global reqs18:20
dhellmannclarkb: ok, that's a good point I hadn't thought of18:20
dhellmannI'm looking into how to add RSS feeds to the specs repos, and didn't want to go through the global requirements just for that, but it sounds like I need to.18:21
clarkbdhellmann: you can probably do the two things in parallel18:22
dhellmannclarkb: yeah, I'll get it working and then submit all the parts at once I think18:22
*** yamamoto has joined #openstack-infra18:23
*** pcrews has quit IRC18:24
*** pcrews has joined #openstack-infra18:25
nibalizerdteselkin: i dont really care, i kinda prefer my way of doing it but it doesn't really matter18:26
clarkbnibalizer: I think I prefer your way too. I don't like that case statement bceause it splits the handling of the pin values into two locations18:28
*** yamamoto has quit IRC18:28
nibalizeralso its basically just an if statement18:28
*** che-arne|2 has quit IRC18:29
*** ndipanov is now known as ndipanov_gone18:31
*** dprince has quit IRC18:31
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Enable puppet3 master bootstrap  https://review.openstack.org/11760418:31
clarkbnibalizer: that bothers me less. I just don't want to have to remember to update two locations when fixing the next dep18:31
* clarkb is selfish18:32
*** mestery has quit IRC18:32
*** mestery has joined #openstack-infra18:32
anteayaclarkb: glad I am not the only selfish one around here18:32
jeblairnibalizer: you have merge conflict detritus in that patchset18:33
*** jistr has joined #openstack-infra18:33
*** jistr has quit IRC18:34
anteayaSlickNik: jenkins is unhappy with https://review.openstack.org/#/c/105692/ which is a shame since it was +A'd18:34
*** jistr has joined #openstack-infra18:34
SlickNikanteaya: The "fatal: read error: Connection reset by peer' strikes again. :(18:34
jeblairlooks like hpcloud-b5 this time18:35
*** kmartin has joined #openstack-infra18:35
fungiSlickNik: just in case you needed proof that neutron isn't just broken when being tested... ;)18:35
clarkbfungi: the rootwrap failure is subtle. I think it has to do with how getlevelname works across versions? I am still digging18:35
nibalizerjeblair: oops!18:35
SlickNikfungi: lol18:35
fungiclarkb: cool18:35
jeblairfungi, clarkb, mordred: i got word our routers have been moved; i'm going to double check the output i was sent, then re-enable b1 and b218:36
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Enable puppet3 master bootstrap  https://review.openstack.org/11760418:36
clarkbjeblair: sounds good18:36
fungijeblair: w00t!18:36
nibalizerjeblair: good catch, fixed now18:36
fungiclarkb: thankfully i'm seeing a lot more successes now that the dev headers are installed18:37
jeblairclarkb: while the iron is hot, do you want to create routers 6-9?18:37
jeblairclarkb: then we can go aheand and ask that they be placed?18:37
SlickNikfungi / anteaya: What's the recommended course of action in this case? Is there a bug to recheck such network issues against?18:37
clarkbjeblair: is that something we want to go ahead and do? I can create them if so18:37
clarkbjeblair: wasn't sure if the image question was settled18:38
jeblairclarkb: i don't think we want to use them, but there's no harm in creating them, right?18:38
clarkboh good point18:38
jeblairclarkb: im mostly just thinking while we've got rick on the line and thinking about this :)18:38
fungiSlickNik: top one at http://status.openstack.org/elastic-recheck/ right now, unfortunately18:38
clarkbjeblair: yup18:38
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Revert "Reduce hpcloud-b1 and b2 usage to 0"  https://review.openstack.org/12017818:39
jeblairclarkb, fungi, mordred: ^18:39
*** amcrn has quit IRC18:40
*** Longgeek has quit IRC18:40
SlickNikfungi: thanks18:40
*** amuller has quit IRC18:41
*** Longgeek has joined #openstack-infra18:43
clarkbjeblair: 6 is done18:47
*** todd_dsm has quit IRC18:47
*** _nadya_ has joined #openstack-infra18:49
*** baoli has quit IRC18:49
*** e0ne has quit IRC18:50
clarkb7 is done18:51
clarkb8 is done18:52
*** garyh has joined #openstack-infra18:52
clarkb9 is done.18:52
ekarlso-phschwartz: around ? :)18:53
clarkbthe process is create new subnet, create new network on that subnet, create new router, attach router to external gateway, give router interface on new subnet created18:53
clarkbbrb should be back before meeting18:53
*** rmcall has quit IRC18:53
jeblairclarkb: ack; requested moves for 6-918:56
sc68calclarkb: I think you need to make the network first18:56
sc68calthen the subnet18:56
jeblairi reckon it's about meeting time19:00
fungii reckon so19:00
jeblairwhen keystone finishes19:00
*** craigbr has joined #openstack-infra19:00
lifelesssdague: all openstack bugs or just nova? multi-task bugs or nova alone?19:01
*** sdake_ has joined #openstack-infra19:01
lifelesssdague: and what change is it making?19:02
*** lcheng_ has joined #openstack-infra19:02
*** openstackgerrit has quit IRC19:02
*** rmcall has joined #openstack-infra19:03
*** sabeen has quit IRC19:03
*** emagana_ has quit IRC19:03
*** sabeen has joined #openstack-infra19:04
*** john-davidge has quit IRC19:04
mordredlifeless: just nova alone19:05
clarkbsc68cal: oh ok19:05
mordredlifeless: trying to set state to triaged19:05
clarkbsc68cal: I used horizon because this is one of thsoe things I will never get right via the client19:05
lifelessso the reason the timeout is there is to protect the system from long locks19:06
*** emagana_ has joined #openstack-infra19:06
mordredyah. I thought I remembered that as being the reason19:06
lifelessa 5s query is 5s of locks held across all involved objects (read locks on referenced tables, write locks on mutated tables)19:06
lifeless5s API call I mean19:06
lifelessif the timeout is raised it quickly increases the number of interactions of such locks, which cascades to make other operations slow etc19:07
lifelesswhen I was working on driving it down we drove it down slowly at the 99th or higher percentile so that we didn't destroy the site for everyone19:08
*** yidclare has quit IRC19:08
lifelessif you're seeing 1 in 3 writes timeout thats exceptional; the known bad spots in bugs are19:08
lifelessor were19:08
lifelesslarge subscriber lists19:09
lifeless(notifications are partially processed in-line [eek I know - I had a service planned])19:09
lifelessand multi-task bugs19:09
lifelesswgrant will be up in a few hours and I'll ask him to poke at the stats a little19:09
*** maxbit has quit IRC19:09
*** emagana_ has quit IRC19:11
fungilifeless: i've discovered more and more things trhat influence subscriber lists. recently i had a cinder-only bug i couldn't update the privacy on without first removing its only tag19:11
jamielennoxkeystoneclient is looking to spawn out some of it's more difficult auth plugins into seperate repos. should these look to live in the openstack/ namespace or go to stackforge?19:11
*** maxbit has joined #openstack-infra19:12
*** baoli has joined #openstack-infra19:13
*** emagana_ has joined #openstack-infra19:15
mordredjamielennox: are they still intended to be owned by the keystone team?19:17
jamielennoxmordred: yes19:17
mordredjamielennox: then openstack19:17
jamielennoxmordred: cheers19:17
morganfainbergmordred, cool jamielennox beat me to the question19:17
morganfainbergmordred, thanks!19:17
mordredjamielennox: also, does this mean that keystoneclient won't need lxml anymore???? (/me waits in anticipation)19:17
*** maxbit has quit IRC19:18
jamielennoxmordred: different repo to the one we are discussing but yes, same idea19:18
morganfainbergmordred, well............19:18
morganfainbergmordred, hopefully... but it needs some documentation so we can not break people "oh if you're using this, install this new package"19:18
jamielennoxnot sure what sort of issue we'll have with compatibility there19:18
mordredI thnk keystoneclient only needs lxml to support saml - which is a feature that not everyone using keystoneclient needs ...19:18
morganfainbergjamielennox, i *think* we can get away with it.19:19
morganfainbergmordred, ++19:19
jamielennoxmorganfainberg: afaik it's mostly cern anyway19:19
morganfainbergjamielennox, right.19:19
mordredand tim bell is smart19:19
jamielennoxso long as we don't accidently crash the collider or anything19:19
jamielennoxi'd feel really bad19:19
mordredmorganfainberg: oh, btw - I'm sure you know this, but it's not possible to auth with keystoneclient and then pass a token in to cinderclient19:19
mordredmorganfainberg: I think mainly because bunny rabbits hate me19:20
morganfainbergjamielennox, ^?19:20
morganfainbergmordred, that seems... weird.19:20
jamielennoxmorganfainberg: i've got it working via sessions, i don't know about a token directly19:20
morganfainbergjamielennox, ah something we probably need to look at.19:20
jamielennoxi can't remember what OSC does19:20
mordredjamielennox: oh - neat. is there code I can look at to see how I shoudl do that? OSC?19:21
jamielennoxmordred: the cinderclient hasn't had a release with sessions afaik19:21
jamielennoxosc = openstackclient19:21
mordredjamielennox: because I'm doinga  thing in ansible and I would love for all auth paths to go direct to keystoneclient19:21
mordredjamielennox: nod19:21
*** rlandy has quit IRC19:22
*** tonytan_brb has joined #openstack-infra19:22
jamielennoxmordred: I kind of use it here: https://github.com/jamielennox/nova/blob/session-test/nova/volume/cinder.py19:23
jamielennoxrelies on things in keystoneclient master and cinderclient master19:23
*** yamamoto has joined #openstack-infra19:23
*** itziikb has joined #openstack-infra19:23
*** dimtruck is now known as zz_dimtruck19:23
jamielennoxalso involves hacking around a lot of old assumptions19:24
itziikbmordred: ping19:24
*** tonytan4ever has quit IRC19:25
mordredjamielennox: looking in session.py - I don't see anyway to pass in username/password - am I missing something stupid?19:26
mordreditziikb: sup?19:26
*** dizquierdo has joined #openstack-infra19:27
jamielennoxmordred: session takes an auth plugin as a parameter, auth plugins are in keystoneclient/auth19:27
jamielennoxyou probably want auth.identity.v2.Password(username, password)19:27
*** _nadya_ has quit IRC19:27
*** _nadya_ has joined #openstack-infra19:28
*** yamamoto has quit IRC19:28
mordredjamielennox: you're going to love me over the next few minutes ...19:28
mordredjamielennox: ok - so Password makes sense ... now where do things like auth_url and region_name go?19:29
itziikbmordred: Just wondered if I  was mistaken19:29
mordredjamielennox: ok - so Password makes sense ... now where do things like tenant_id and region_name go?19:29
*** mrmartin has quit IRC19:29
jamielennoxtenant_id is passed to password as well19:30
*** dprince has joined #openstack-infra19:30
jamielennoxregion_name is still a client parameter, we don't use that in auth at all19:30
*** amotoki has joined #openstack-infra19:30
jamielennoxauth_url is also a Password param19:31
jamielennoxmordred: are you loading this from ansible/oslo.conf/cli?19:31
ekarlso-fungi: u know where phschwartz is hiding ? ;)19:31
fungiekarlso-: how would i know?19:32
mordredI see. tenant_id comes from the Auth superclass19:32
ekarlso-just wondered :D19:32
funginope, no idea19:32
mordredjamielennox: https://github.com/ansible/ansible/pull/892419:32
mordredjamielennox: I'm not doing full keystoneclient for auth yet ... but I would like to get that patch there19:33
*** zz_dimtruck is now known as dimtruck19:33
mordredjamielennox: https://github.com/emonty/ansible/blob/features/all-the-nova/lib/ansible/module_utils/openstack.py#L19519:33
*** lcheng_ has quit IRC19:33
jamielennoxmordred: so ideally i'd say don't take username/password as params there at all and accept a session object19:34
mordredjamielennox: yah. I think once I understand session a little better, I can make all of that code betterer19:35
*** amotoki has quit IRC19:35
jamielennoxlets you do more funky things like auth_plugin = keystoneclient.auth.load_from_conf_options(CONF)19:35
jamielennoxsession = keystoneclient.session.Session.load_from_conf_options(CONF, auth=auth_plugin)19:35
anteayanibalizer: let's ease back on the colourful cursing19:36
jamielennoxstandardizes options19:36
*** julim has quit IRC19:37
*** otherwiseguy has joined #openstack-infra19:37
nibalizeranteaya: sure19:37
nibalizermy bad19:38
berendtcan somebody please check why the openstackgerritbot is offline?19:38
*** maxbit has joined #openstack-infra19:38
anteayanibalizer: thank you19:38
*** dane_leblanc has quit IRC19:38
berendtthe bot left #openstack-doc at 09:02:57 PM19:38
fungiberendt: 2014-09-09 19:02:57     <--     openstackgerrit (~openstack@review.openstack.org) has quit (Ping timeout: 264 seconds)19:38
*** otherwiseguy has quit IRC19:38
anteayanibalizer: you have such great phrases for things without the cursing anyway19:39
anteayanibalizer: I like it far better19:39
fungiberendt: probably freenode is struggling, but i'll restart it and try to get it to reconnect19:39
berendtfungi: thanks19:39
*** david-lyle has joined #openstack-infra19:40
*** openstackgerrit has joined #openstack-infra19:41
jogojeblair: any news on the HP public cloud networking issues?19:41
openstackgerritIan Wienand proposed a change to openstack-infra/config: Add bare-f20 nodes  https://review.openstack.org/11739719:41
openstackgerritIan Wienand proposed a change to openstack-infra/config: Get postgresql puppet from upstream master  https://review.openstack.org/11739619:41
openstackgerritDmitry Teselkin proposed a change to openstack-infra/config: Pin pupppetdb-terminus package  https://review.openstack.org/11866019:42
anteayajogo: we are having our weekly meeting19:43
jogoanteaya: oh right19:43
fungijogo: we're pinning routers to distinct neutron controllers as a next attempt, though it sounds like it could be resource starvation from random misbehaving neighbor tenants19:44
jogofungi: interesting19:45
*** armax_ has joined #openstack-infra19:45
jogoarosen: ^ ohh neutron19:45
jamielennoxmordred: http://www.jamielennox.net/blog/2014/02/24/client-session-objects/ is fairly old now but might be useful in understanding19:46
jogojamielennox: you work on the openstack SDK right?19:46
*** vhoward has left #openstack-infra19:46
jamielennoxjogo: i'm involved19:46
*** dane_leblanc has joined #openstack-infra19:47
jogojamielennox: so got a question, in nova some filtering options (such as filter servers by name) are regex and some are string matching19:47
jogojamielennox: does the SDK have a way of helping the user distinguish which is which19:48
openstackgerritDoug Hellmann proposed a change to openstack/requirements: Add yasfb for making RSS feeds in docs  https://review.openstack.org/12019919:48
*** BadCub has quit IRC19:49
jamielennoxjogo: this is the first i've heard about being able to pass regexp to nova filters honestly19:49
jamielennoxyou can actually pass an expression as a query filter?19:49
*** BadCub has joined #openstack-infra19:49
jogojamielennox: yup!19:50
jogojamielennox: https://review.openstack.org/#/c/119081/19:51
jamielennoxjogo: so i guess there are two options, first we treat it as any old string parameter and pass it through as normal19:52
*** e0ne has joined #openstack-infra19:52
*** manybubbles has left #openstack-infra19:52
*** _nadya_ has quit IRC19:52
jamielennoxsecond we look for a regexp object and convert to string (which i don't know if that's any more beneficial than just being a string)19:53
jogojamielennox: that doesn't work because regex does substring matching19:53
jogoif  I have servers named: {bob,bobby,john} and I regex filter based on "bob" I get {bob,bobby19:53
jamielennoxbut i don't think SDK has gotten sufficiently far to deal with anything that's not passed all the way through19:54
*** maxbit_ has joined #openstack-infra19:54
jogojamielennox: passed all the way through?19:54
jamielennoxjogo: a string that user inputs into a client object which we just print back out in the URL19:56
jogojamielennox: the regex work that way19:56
jogojamielennox: but they are regex not strings19:57
jogoso as a user I need to be able to identify what is a regex and what isn't so the substring matching doesn't get me19:57
dkranzDoes any one know how long we are keeping log files in logstash now?19:57
fungiapparently we have some leaks in some unit tests... my py34 test node eventually gummed up with a python and java (zookeeper?) processes chewing up all available memory19:57
jamielennoxjogo: if this is a server side filter - what do i need to do on the client side?19:57
*** maxbit has quit IRC19:57
*** unicell has quit IRC19:58
jogojamielennox: have a way for the user to know which is which19:58
jogojamielennox: so I can use it properly19:58
jamielennoxwouldn't the user be giving me that filter initially?20:00
*** HeOS has joined #openstack-infra20:01
*** timo has joined #openstack-infra20:01
*** timo is now known as Guest6995520:02
clarkbdkranz 10 days20:02
dkranzclarkb: thanks. Makes it tough to see very-low-probability race bugs but I know it is all we can do20:03
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Event actor now added to message payload  https://review.openstack.org/12020120:03
*** kgiusti has left #openstack-infra20:04
fungiclarkb: did you try running py34 for keystonemiddleware and did it hang indefinitely for you or eventually complete?20:04
jogojamielennox: yes but as a user how do I know which fitler is regex and which is simple string?20:04
jogojamielennox: shouldn't there be a way to discover that via the client20:05
fungiclarkb: actually, watching top, i predict it's in a memory-gobbling loop and will eventually get oom'd20:06
jamielennoxjogo: oh - you want a way to know which parameters on the server will accept a regex filter?20:06
jogojamielennox: yup20:06
*** zehicle_at_dell has joined #openstack-infra20:07
clarkbfungi ya I ^C'd it20:07
jamielennoxok - no we don't have anything like that anywhere AFAIK, including existing clients20:07
jamielennoxjogo: how does nova export this information?20:07
clarkbsame with rootwrap20:07
jogojamielennox: it doesn't really20:08
clarkbfungi: but rootwrap had that weird logging thing which I am still digging into20:09
jogojamielennox: so my understanding of the SDK effort is to build a framework that can be used to write a better nova python client20:09
jamielennoxjogo: there's nothing really about our discovery that would allow this atm, is nova going to look at jsonhome?20:09
*** oanufriev has joined #openstack-infra20:09
fungiclarkb: so the list of failing jobs is whittled down to 17, but a few of those were dying because the machine was basically out of memory so i'm rerunning those now to trim the list further20:09
jogojamielennox: in which case the person writing the python-novalcient based on the SDK can code it in20:09
*** masayukig has quit IRC20:09
jogojamielennox: but I may be missing something about the SDK20:09
jamielennoxjogo: not really, SDK is a user facing project it's a way to make a full openstack client library rather than have to deal with all the disjoin -clients20:10
*** craigbr has quit IRC20:10
jogojamielennox: ahh so how do you do discovery now?20:10
jamielennoxthere is a lot of stuf going on in the keystoneclient and session around discovery20:10
jamielennoxhowever we only really do version detection20:10
jamielennoxso there are a couple of things in keystoneclient where you can make a request and say it requires API version 3.3 and if it's not present it raises an error without making a request20:11
jamielennox(in truth the first real commands doing that are in review now)20:11
jogojamielennox: ahh anyway I don't have answer on how this should work20:11
jogojamielennox: I more wanted to raise this use case so you are aware of it20:12
jamielennoxjogo: so jsonhome allows you to specify parameters regarding the URLs you call20:12
jogojamielennox: interesting, so we can use that for service discovery?20:12
jamielennoxa common example is that this URL accepts application/json, application/xml so you can set your Accept in advance20:12
jamielennoxkeystone should have jsonhome support for juno20:13
jogojamielennox: neat do you have a link to that?20:13
jamielennoxi don't know yet exactly how we are going to consume it20:13
clarkbfungi: the rootwrap failure is a test checking that logging.getlevelname(lvl) doesn't return the default string back20:13
clarkbfungi: on py27 it works, supposedly on py33 it works, but in py34 passing in INFO returns "Level: INFO" instead of INFO the object20:13
clarkber rather instead of the string repr20:14
clarkbso I am wondering if possibly the repr changed?20:14
jamielennoxjogo: this is one of the patches: https://review.openstack.org/#/c/111576/20:14
*** jistr has quit IRC20:14
jamielennoxthere are a number to do the whole thing - but the keystone implemenation is kind of messy because we have a crappy wsgi layer20:15
*** masayukig has joined #openstack-infra20:15
jamielennoxi think looking up the jsonhome standard is a better start and figure out how that fits in with nova20:15
jogojamielennox: so we are missing a step here20:16
jogojamielennox: trying to have all OpenStack services agree to standardize around X20:16
*** jedimike has quit IRC20:16
*** dhellman_ has joined #openstack-infra20:16
jamielennoxjogo: i tried that maybe a year ago, people mostly still did there own thing20:17
jogoso we have a more unified API experience.20:17
jogojamielennox: heh yeah we don't have the forum to do that yet20:17
dprincejeblair: hi, any chance you think this could merge this week https://review.openstack.org/#/c/112331/20:17
jogojamielennox: hopefully if we can solve the forum issue in Kilo and make this one of the first things to tackle20:17
jamielennoxjogo: https://wiki.openstack.org/wiki/VersionDiscovery was just an attempt to standardize all the services - it wasn't really an improvement20:17
jogojamielennox: I mainly wanted to let you know about the existence of regex filtering params20:17
jamielennoxjogo: the advantage of jsonhome is that it's not our standard, if we say you should use jsonhome if a service doesn't implement it correctly then it's a bug20:18
*** maxbit_ has quit IRC20:18
jogojamielennox: ++20:18
*** dims_ has quit IRC20:18
jamielennoxjogo: no, we don't have that sort of options yet. We can do certain things like if the nova API version < X.X then error without making the request20:19
fungiclarkb: is oslo.rootwrap py34 hanging for you too or does it eventually return?20:19
*** dims_ has joined #openstack-infra20:19
jogojamielennox: thats a really good first step and once we get micro versioning in ...20:19
jamielennoxbut the SDK project isn't there yet20:19
*** maxbit has joined #openstack-infra20:19
jamielennoxjogo: yep, was surprised nova didn't do that20:19
*** yidclare has joined #openstack-infra20:19
jogojamielennox: hehe20:19
*** otter768 has joined #openstack-infra20:19
*** yidclare has left #openstack-infra20:19
jamielennoxkeystone, glance and cinder do20:20
jamielennoxhaven't looked further than tha20:20
clarkbfungi: it hangs too20:20
fungiok, thx20:20
clarkbfungi: on the second thing which is test eventlet which won't work on python320:20
clarkbfungi: so not completely sure how py33 is/was working20:20
*** sabeen1 has joined #openstack-infra20:20
clarkboh wit there is a py33 target20:20
clarkbfungi: did you write a py34 target change for it?20:20
* clarkb reruns with hacked target20:21
*** sabeen has quit IRC20:21
clarkbstill fails on loglevel thing but no hanging20:21
fungiclarkb: ahh, https://review.openstack.org/#/c/118792/1/tox.ini20:22
clarkblogging.getLevelName(logging.INFO) works properly under python3.4 so this must be an actualy rootwrap thing20:22
*** homeless_ has quit IRC20:22
*** dims_ has quit IRC20:23
clarkboh I see20:23
*** weshay_mtg has quit IRC20:23
*** yamamoto has joined #openstack-infra20:23
*** homeless has joined #openstack-infra20:24
clarkblogging.getLevelName('INFO') returns 20 on py2.7 and 'Level: INFO' under py34. I don't have py33 handy without getting on a test slave20:24
*** yamamoto has quit IRC20:24
*** mmaglana has quit IRC20:25
*** mikedillion has quit IRC20:25
*** yamamoto has joined #openstack-infra20:25
clarkbfungi: woo and that is an undocumented py3.4 or I can find where it is documented if it was documented20:26
*** emagana_ has quit IRC20:26
fungiactually, when i test that on 3.4 manually i also get 'INFO' (not 'Level: INFO')20:27
*** emagana has joined #openstack-infra20:27
clarkbfungi: huh20:27
clarkbhttp://paste.openstack.org/show/109079/ is what I get20:28
fungiaha, i was testing the wrong thing20:28
clarkbfungi: what does getLevelName('INFO') do on py33?20:28
fungiyeah, on 3.3 it's 20 and on 3.4 it's 'Level INFO'20:29
*** yamamoto has quit IRC20:29
fungiso that does indeed appear to be a legit 3.3->3.4 stdlib behavior change20:30
*** emagana_ has joined #openstack-infra20:30
fungigoes from <class 'int'> to <class 'str'> for the return type20:31
clarkbthe logging docs don't seem to say anything different so this must just be a code change20:31
* clarkb tries to find python logging change log20:31
openstackgerritDavid Stanek proposed a change to openstack-dev/pbr: Adds option for excluding files from autodoc trees  https://review.openstack.org/12021620:31
*** emagana has quit IRC20:31
clarkbwow it is incredibly hard to understand where the actual python code lives20:32
*** dprince has quit IRC20:33
*** itziikb has quit IRC20:33
clarkbbut I assume hg.python.org is what I want20:33
openstackgerritDavid Shrewsbury proposed a change to stackforge/dox: Add option to keep test image  https://review.openstack.org/12021720:35
*** dims_ has joined #openstack-infra20:35
*** maxbit has quit IRC20:36
*** e0ne has quit IRC20:37
*** radez is now known as radez_g0n320:37
*** fifieldt_ has quit IRC20:38
*** tonytan_brb has quit IRC20:39
*** craigbr has joined #openstack-infra20:41
*** Guest69955 has quit IRC20:42
*** emagana_ has quit IRC20:42
*** mmaglana has joined #openstack-infra20:43
*** emagana has joined #openstack-infra20:43
clarkbAlex_Gaynor: you broke it20:43
clarkbhttp://hg.python.org/cpython/rev/5629bf4c6bba?revcount=60 can I demand a fix?20:44
fungiclarkb: demand a refund20:44
*** ociuhandu has quit IRC20:45
Alex_Gaynorfungi, clarkb: Yeah, you can have every penny you paid back :-)20:46
fungiclarkb: so far what we know is recorded at https://etherpad.openstack.org/p/py34-transition though lines 21-23 are still not done rerunning yet so could have been from my earlier oom situation20:46
clarkbAlex_Gaynor: so how do you suppose we deal with this? has ship sailed on changing ti again in python and we should deal with it in oslo.rootwrap?20:46
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add non-voting gate-git-review-python34 job  https://review.openstack.org/11996420:47
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Add Python 3.4 and PyPy to Ubuntu Trusty slaves  https://review.openstack.org/11885920:47
clarkbalso we should at least document that change imo20:47
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Switch sphinxcontrib-docbookrestapi to python-jobs  https://review.openstack.org/11886020:47
Alex_Gaynorclarkb: it's been a while since I did this, but yeah, this change was really supposed to just be cleaning up some internals, the fact that it effected public things is really a bug in them IMO, relying on implementation details20:47
Alex_Gaynorit's definitely unlikely to be reverted20:47
*** emagana has quit IRC20:47
fungiAlex_Gaynor: clarkb: it's the overheating spacebar20:47
clarkbAlex_Gaynor: I dunno the docs seem pretty clear. I think the public thing broke and is broken right now20:48
clarkband either we fix internals or change the doc20:48
*** andreaf has quit IRC20:48
dims_fungi: great one!20:49
*** dolphm has left #openstack-infra20:49
fungii'm supposed to go find some food, then i'll come back and bang my head against the list a while longer20:49
anteayaha ha ha20:50
*** fifieldt_ has joined #openstack-infra20:50
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Remove fingerprint for bug 1291605  https://review.openstack.org/12022020:52
uvirtbotLaunchpad bug 1291605 in nova "unit test test_create_instance_with_networks_disabled race fail" [Medium,Invalid] https://launchpad.net/bugs/129160520:52
mroddenhttp://pypi.openstack.org/simple/Flask-RESTful/ works but http://pypi.openstack.org/simple/flask-restful/ doesn't20:52
openstackgerritDavid Shrewsbury proposed a change to stackforge/dox: Add option to keep test container  https://review.openstack.org/12021720:52
mroddenpypi.python.org apparently is case-insensitive20:53
mroddene.g. https://pypi.python.org/simple/flask-restful/20:53
clarkbmrodden: I think that is correct. pip is supposed to normalize for you or some such20:54
mroddenso the url canonicalization (word?) is done by pip?20:54
clarkbmrodden: yes, though this is new iirc20:54
mroddeni suppose that makes sense20:54
clarkbmrodden: the problem with doing it server side is that it means anyone running a mirror would have to do it too20:55
mroddenjust weird that the behavior of bandersnatch mirrors is different than pypi upstream20:55
clarkbmrodden: or the client can just do it20:55
clarkbmrodden: bandersnatch does no normalization20:55
mroddenwas super confused and thought bandersnatch was broken20:55
*** mestery has quit IRC20:57
*** mestery has joined #openstack-infra20:57
*** praneshp has quit IRC20:59
openstackgerritDavid Shrewsbury proposed a change to stackforge/dox: Add option to keep test container  https://review.openstack.org/12021720:59
* clarkb works on a rootwrap patch to ask for more forgiveness rather than doing error checking20:59
*** praneshp has joined #openstack-infra20:59
*** arnaud__ has quit IRC20:59
*** lcheng_ has joined #openstack-infra20:59
*** praneshp has quit IRC21:00
*** praneshp has joined #openstack-infra21:00
*** marcoemorais has quit IRC21:01
*** marcoemorais has joined #openstack-infra21:01
*** dustins has quit IRC21:02
*** changbl has quit IRC21:02
*** lcheng_ has quit IRC21:02
*** pcm_ has quit IRC21:04
*** julim has joined #openstack-infra21:05
mordredmrodden: patches are coming to both pip and bandersnatch to help this21:05
*** maxbit has joined #openstack-infra21:07
*** weshay has joined #openstack-infra21:07
dougwighi infra, has anyone ever considered modifying zuul to retry jobs on failure, before commenting?  or is there a known way to do that today?21:08
*** bswartz has quit IRC21:09
clarkbdougwig: I don't think that is a use case that we have considered. we expect tests to pass every time. passing half the time is still a failure21:10
clarkbfungi: https://review.openstack.org/120228 so thats a major hack to deal with the log level thing21:13
clarkbfungi: not sure I am happy with it21:13
mroddenmordred: what are they patching exactly?21:14
*** maxbit has quit IRC21:14
*** maxbit has joined #openstack-infra21:15
openstackgerritBarnaby Court proposed a change to openstack-infra/jenkins-job-builder: Adds credentials-binding plugin support to wrappers  https://review.openstack.org/12018421:15
clarkbdougwig: are you wanting this behavior for a third party zuul or a zuul that has nothing to do with openstack?21:16
*** aysyd has quit IRC21:16
clarkbdougwig: a different way to achieve that behavior is to encode it in the test directly21:16
dougwigthird-party.  you're totally right.  i just get occasionally failures because pypi connresets, or the backend has a rare core, and i don't like putting a failure in gerrit when it has nothing to do with openstack code.21:17
dougwig /occasionally/occasional/21:17
mordredmrodden: they are going to make pip normalize case and -/_ first before it makes a request21:18
mordredmrodden: and I believe it will make bandersnatch write out two dirs - one normalized, and one as received21:18
mroddenshould just ban caps in URLs21:18
*** lcheng_ has joined #openstack-infra21:19
mroddendoesn't make for the best migration story though21:19
*** dims__ has joined #openstack-infra21:19
*** cipcosma has quit IRC21:20
*** dims__ has quit IRC21:20
anteayadougwig: good instincts21:20
dhellmannclarkb: is https://review.openstack.org/#/c/120228/1 a juno or kilo fix?21:20
clarkbdhellmann: it is a juno fix hopefully. part of the py34 push21:20
*** dims__ has joined #openstack-infra21:21
reedjust FYI, since mikal asked during the meeting, you can resign from Foundation membership https://www.openstack.org/profile/resign/21:21
*** dims_ has quit IRC21:23
*** yamamoto has joined #openstack-infra21:23
*** mwagner_lap has joined #openstack-infra21:25
*** mfer has quit IRC21:25
anteayareed great21:25
*** dims__ has quit IRC21:25
Morgan_reed: interesting. It never occurred to me someone would want to resign.21:25
jeblairreed, fungi, mordred: so actually that means that we *currently* have the same problem that we would have by decoupling cla signing and foundation membership21:26
anteayathough how do we access the list of people who have resigned for the purposes of the election roll?21:26
jeblairreed, fungi, mordred: namely, that we need to correlate the set of contributors with the set of individual members for election purposes21:26
dhellmannclarkb: it would help me if there was a bug # associated with it, are you tracking any of that work with bugs or blueprints?21:26
clarkbfungi: ^21:26
reedjeblair, it just dawned on me that we will have a different set of people soon21:26
clarkbdhellmann: its part of the modernize infra work we do every time lts release happens21:26
jeblairreed, fungi, mordred: so anyway, maybe we should get started on brainstorming that soon21:26
clarkbdhellmann: its less about bugs and blueprints and more about things just have to work21:26
reedjeblair, the 'resign' button is new, like yesterday21:26
*** e0ne has joined #openstack-infra21:27
*** mmedvede has quit IRC21:27
reedjeblair, and it comes because some people have never voted therefore will loose their rigths to election21:27
dhellmannclarkb: ok, I just don't want it to fall off the review radar, which I've finally gotten organized on launchpad21:27
jeblairreed: ya21:27
reedeverything goes back into the horrible delay of launching openstackid.org21:27
* reed sighs21:28
jeblairreed: i think that's all quite separate, actually :)21:28
dhellmannclarkb: we have so many different repos to review, we're trying to use launchpad to help identify the priority items21:28
clarkbdhellmann: gotcha21:28
reedprobably it is, would make me feel better :)21:28
dhellmannclarkb: I hate asking for paperwork...21:28
clarkbdhellmann: hoepfully fungi will fill us in when he gets back21:28
*** yamamoto has quit IRC21:28
dhellmannclarkb: ok, thanks21:29
jeblairdhellmann: unfortunately, for technical reasons, opening infra bugs against all of the projects in launchpad is a bad idea at this point21:29
jeblairdhellmann: the ones we have done in the past we are unable to change status for21:29
*** ociuhandu has joined #openstack-infra21:29
*** e0ne has quit IRC21:30
anteayaso as of right now, since there are no access points for infra to read the list of those who resign, it is possible that the electoral roll this election might be inaccurate21:30
dhellmannjeblair: yeah, I remember that21:30
clarkbdhellmann: https://etherpad.openstack.org/p/py34-transition is a summary that fungi did throw up on etherpad21:30
anteayathis is not filling me with confidence21:30
dhellmannclarkb: thanks21:31
anteayareed: are you willing to turn off the resign button at least until we have created the electoral rolls for the upcoming elections?21:31
*** amotoki has joined #openstack-infra21:31
anteayareed: and can you tell me if anyone has chosen to resign after doing so?21:31
dhellmannjeblair: it would be good if we could solve the tracking and prioritization problem some other way. Maybe storyboard is our answer, I don't know. :-/21:32
reedanteaya, I can ask, I have absolutely no control over that site21:32
anteayathen we have at least 6 more months before we have to address this21:32
anteayabecause it won't be addressed accurately in the next 2-3 weeks21:32
anteayareed: please ask21:32
jeblairdhellmann: in the long run, yes, but not yet.  hopefully by the next lts ;)21:32
jeblair(hopefully much sooner)21:32
*** TheDodd has joined #openstack-infra21:33
dhellmannjeblair: yeah, I guess that project has resource issues as much as the rest of us21:33
jeblairdhellmann: i expect infra to be able to start using it soon21:34
jeblairwhich is a big milestone21:34
dhellmannyeah, that's great21:34
reedanteaya, drafting the message I realized that it may be too late... let me check the dates but I think there is a need according to the bylaws to remove some people from the pool anyway21:34
dhellmannI would not object to oslo being a relatively early adopter, too, but maybe not "first" :-)21:34
clarkbjeblair: fungi nibalizer ok need a break fighting python. reviewing the puppet3 master change now21:35
nibalizerclarkb: woot21:35
anteayareed remove them by when?21:35
anteayareed and I can understand the need to be in compliance with the foundation bylaws21:35
jeblairdhellmann: because of the cross-project tracking, we actually need to move all of openstack together, so our migration plan is basically get basic functionality working, move infra, get enough functionality for openstack, move openstack.21:36
*** amotoki has quit IRC21:36
anteayareed but I have a problem if doing so makes the ptl and tc elections in non-compliance with the tc charter21:36
reedanteaya, trying to find out more details... give me a sec21:36
anteayareed: thank you21:36
clarkbnibalizer: in https://review.openstack.org/#/c/117659/3/install_puppet.sh what sets PUPPET_MASTER?21:37
clarkbnibalizer: is the calling env expected to set that?21:37
mordredclarkb: magic21:37
clarkbjeblair: ^ does that cause problems for launch node?21:37
*** e0ne has joined #openstack-infra21:38
*** mriedem has quit IRC21:38
openstackgerritA change was merged to openstack-infra/config: Fixed gate-trove-functional-dsvm jobs  https://review.openstack.org/10569221:38
nibalizerclarkb: yea calling env21:38
clarkb"Since tooz is part of stackforge, it's somewhat part of OpenStack." hrm21:38
clarkbnibalizer: ok, i am not sure that will work with the current way we build nodes /me double checks21:38
reedanteaya, membership to the foundation for about 2,800 people will expire on Oct 4.21:39
*** TheDodd has quit IRC21:39
*** amcrn has joined #openstack-infra21:39
nibalizerclarkb: hrm, we'll also need to put the script in puppet 3 mode21:39
*** dims_ has joined #openstack-infra21:39
anteayareed: okay, so why the need for the resign button now?21:39
*** bswartz has joined #openstack-infra21:39
clarkbnibalizer: I think what you want to do is have puppet do the symlink itself21:39
reedanteaya, as far as I know, these 2,800 people have already been notified of the expiration21:39
clarkbnibalizer: since puppet doesn't need the symlink to run right? just hiera?21:40
nibalizerclarkb: the hiera symlink?21:40
reedanteaya, I think (in general) it makes sense to provide that option to members21:40
clarkbnibalizer: https://review.openstack.org/#/c/117659/3/install_puppet.sh the thing that happens in that change21:40
*** e0ne has quit IRC21:40
anteayareed sure except that I can no longer fulfill my duties as election official21:40
reedanteaya, so what I think would be good to do is to ask get a list of expired/unrenewed members and check the electoral pool against that21:40
anteayareed: since I can no longer run the ptl and tc elections as stipulated by the tc charter21:40
clarkbya we don't copy the calling env onto the target node which is probably a good thing21:41
openstackgerritJJ Asghar proposed a change to openstack-infra/config: Fix ruby-191-prep to uninstall 1.8 and install bundler  https://review.openstack.org/12011621:41
anteayareed: I have no objections21:41
mordredclarkb: hahahaha. where did you see that? (re: tooz)21:41
anteayareed: how do you propose we do that?21:41
clarkbmordred: it is in solly's message21:41
nibalizerclarkb: okay we can pull that out and let puppet do it21:41
nibalizerworks for me21:41
clarkbmordred: in the thread about tooz21:41
anteayaand we have to do it so that it is current when we create the roll on Septeber 26th21:41
reedanteaya, I'm asking Jimmy and Todd, hopefully they have a record we can pull around Oct 4th21:41
nibalizerwill take a few minutes to make those changes then rebse everything21:41
anteayaI create the roll prior to that21:42
anteayaand does taht list take into account email drift?21:42
reedanteaya, when do you create the roll?21:42
clarkbnibalizer: well hold on. maybe jeblair has alternative ideas?21:42
clarkbbut I think we will have trouble if we try to enc this like that21:42
*** emagana has joined #openstack-infra21:42
anteayareed: fungi and I create it after September 26, 2014 05:59 UTC21:42
clarkbexternal node classifier? isn't that what this is doing21:42
reedanteaya, ok21:43
clarkbit is trying to say you are a puppet 3 master, you are not a puppet3 master, and so on21:43
clarkbexcept we have none of the framework to actually do it that way21:43
*** dolphm has joined #openstack-infra21:43
anteayareed and gerrit only knows about the gerrit preferred email for the roll21:43
*** maxbit has quit IRC21:43
nibalizerim not sure what you mean? do you mean at node launch?21:43
anteayareed: so if the foundation does not have their current gerrit preferred email we won't have a match21:43
*** bookwar has joined #openstack-infra21:43
nibalizeri had assumed a rooter would ssh in and run the script21:43
clarkbnibalizer: yes at node launch. that is the only time install_puppet.sh runs21:43
anteayareed: and I don't know if not a match means they will be removed from the electoral roll in error, or left on in error21:44
* jeblair catches up21:44
anteayareed: since I don't know how we will be matching21:44
nibalizerif thats not what happens then we need to readjust our plan21:44
reedanteaya, yep, it'll be an imprecise count21:44
openstackgerritJJ Asghar proposed a change to openstack-infra/config: Fix ruby-191-prep to uninstall 1.8 and install bundler  https://review.openstack.org/12011621:44
reedanteaya, how imprecise is hard to know21:44
anteayareed: I can't tolerate an imprecise count21:44
anteayait is my job to be precise21:44
clarkbnibalizer: it can probably be made to happen that way, but isn't how we normally run that script21:44
*** dims_ has quit IRC21:45
reedanteaya, let's keep talking and frame the issue precisely so we can find a solution21:45
anteayaI have to be precise for candidates and I have to be precise for the electorate21:45
*** dims_ has joined #openstack-infra21:45
anteayathe tc charter doesn't say anything about a fudge factor in ascertaining voting status21:45
reedanteaya, where is the TC charter so I can read it?21:45
jeblairclarkb: launch_node sets the puppet_version env var for that script, it could do the same for master...21:46
jeblairnibalizer, clarkb: however, what happens if the hiera stuff doesn't happen?21:46
clarkbjeblair: right, except I don't think we need to. the puppet for the master knows it is both puppet3 and a master21:46
openstackgerritJamie Lennox proposed a change to openstack/requirements: Add kite to projects.txt  https://review.openstack.org/12025221:46
*** masayukig has quit IRC21:46
jeblairclarkb: agreed, that's kind of where i'm heading21:47
clarkbjeblair: well the files are empty as is in that change. so I don't think it matters21:47
jesusaurusjeblair: if you dont make the symlink then running the 'hiera' command-line tool will be different from running the function in puppet21:47
jeblairnibalizer: will puppet work well enough to correct that?21:47
jeblairjesusaurus: but it will make no difference during the bootstrap process21:47
jesusaurusfixing it with puppet will work21:47
nibalizerno difference with bootstrap21:48
jeblairjesusaurus, nibalizer: so if we ignore that during bootstrap, and let puppet fix it on run #1, we should be good21:48
*** 77CAAEEHU has joined #openstack-infra21:48
nibalizercan be removed from shell script entirely21:48
*** praneshp has quit IRC21:48
jesusaurusyep, we can let puppet fix it on run #121:48
*** dhellman_ has quit IRC21:48
nibalizerokay i will remove reference to the symlink21:49
*** dizquierdo has quit IRC21:50
*** 77CAAEEHU is now known as che-arne21:50
*** masayukig has joined #openstack-infra21:51
*** praneshp has joined #openstack-infra21:51
nibalizerdo we also want the shell scrit to be  putting in an initial puppet.conf or do we want puppet to do that too?21:52
nibalizerim a bit unclear how the 'sever' configuration variable is set initially21:53
clarkbnibalizer: I think it should do the minimal necessary to do run 121:53
clarkbnibalizer: if that is possible without a puppet.conf then ++21:53
nibalizersohow do the normal nodes now know to hit cipuppetmaster as their server?21:54
* clarkb looks21:55
clarkbnibalizer: looks like launch-node.py passes it in as a command line arg21:56
clarkbvia puppet agent --server21:56
*** todd_dsm has joined #openstack-infra21:56
nibalizerdoes launch_node.py live in config?21:57
clarkbnibalizer: yes it is under the launch/ dir21:57
*** alexandra_ has joined #openstack-infra21:57
nibalizerok me reads21:57
*** alexandra_ is now known as asettle21:57
* nibalizer wondering how cert dance is done21:57
openstackgerritJJ Asghar proposed a change to openstack-infra/config: Fix ruby-191-prep to uninstall 1.8 and install bundler  https://review.openstack.org/12011621:57
nibalizerjeblair once told me that certs are generated then copied over21:58
clarkbnibalizer: its in there too, basically checks if cert exists, and copy pastas if so21:58
clarkbnibalizer: yup21:58
mordredall in launch_node.py21:59
mordredat least, it is until my ansible changes are solid ...21:59
nibalizeri see it now22:00
nibalizerokay /me goes to refactor22:00
*** stevemar has quit IRC22:01
*** rmcall has quit IRC22:01
openstackgerritJamie Lennox proposed a change to openstack-infra/config: Add keystoneclient-kerberos repository  https://review.openstack.org/12026122:02
*** datsun180b has quit IRC22:02
*** bdpayne_ has joined #openstack-infra22:02
*** rmcall has joined #openstack-infra22:03
*** flaper87 is now known as flaper87|afk22:03
*** bdpayne has quit IRC22:03
*** dkranz has quit IRC22:04
*** Ajaeger1 has quit IRC22:04
openstackgerritJamie Lennox proposed a change to openstack-infra/config: Add keystoneclient-kerberos repository  https://review.openstack.org/12026122:04
mtreinishclarkb: btw, did you figure out the zookeeper on the dsvm slave thing?22:06
clarkbmtreinish: ya, https://review.openstack.org/#/c/119936/ is the fix. basically we isntall it everywhere22:06
jeblairjhesketh: good morning22:08
*** bknudson has quit IRC22:08
jeblairjhesketh: when you have a moment, can you read the section on priority blueprints from today's meeting?22:08
lifelessmordred: any thoughts on my comments re dox above?22:09
openstackgerritJJ Asghar proposed a change to openstack-infra/config: Fix ruby-191-prep to uninstall 1.8 and install bundler  https://review.openstack.org/12011622:09
jeblairjhesketh: i want to make sure we highlight work needed to accomplish logs-in-swift in next weeks meeting22:09
jheskethjeblair: will do22:10
*** homeless has quit IRC22:10
mordredlifeless: I do not see them ... how far above?22:11
mordredlifeless: ah - I foudn them22:11
mtreinishclarkb: ok, cool22:11
mordredyes, both of those comments seem to be things I can agree with22:11
*** homeless has joined #openstack-infra22:11
fungidhellmann: clarkb: i haven't opened any bugs yet because i didn't know in advance what would turn out broken and how. i figured once i identified broken projects i'd file corresponding bugs and give them a chance to identify fixes or switch the jobs non-voting before the nodes they run on went away22:14
fungiclarkb: turns out that stackalytics and tooz were also false failures brought on by memory exhaustion, so i've striken them from the affected list22:18
clarkbgreat news22:18
jheskethjeblair: I'll figure out some next steps (as I've been meaning to) and put them in next weeks agenda22:18
clarkbjhesketh: jeblair also wondered if we should hae a meeting annex thati s more friendly to your timezone22:18
jheskethIt'll also be daylight savings next month which will make it easier for me to attend22:19
*** berendt has quit IRC22:19
openstackgerritA change was merged to openstack-infra/config: Revert "Reduce hpcloud-b1 and b2 usage to 0"  https://review.openstack.org/12017822:19
jheskethAt the moment it's 5am which means that I've been looking at the agenda and deciding if I should set an alarm. 6am from next month I can make22:20
clarkbthat is dedication22:20
jhesketh(I have been keeping an eye on minutes and logs but I do feel slightly disconnected mainly because I'm not a morning person)22:21
*** emagana has quit IRC22:21
*** emagana has joined #openstack-infra22:21
clarkbwe should all just operate on UTC and ignore what the sun does out the window22:21
*** alexpilotti has quit IRC22:22
jeblairjhesketh: cool, having something you can put in the agenda means we can highlight reviews that are important to your work during the regular meeting, regardless of whether you attend22:22
jheskethI thought the solution was to never sleep and always be working?22:22
fungijhesketh: i'm still working on the logistics for that, but yes22:22
jeblairjhesketh: and then if you want to have a meeting annex or, we can just resolve to chat informally when you wake up, we can fill in the blanks22:22
jheskethjeblair: sounds good22:22
*** jgrimm is now known as zz_jgrimm22:23
*** atiwari has quit IRC22:23
*** yamamoto has joined #openstack-infra22:23
*** lcheng_ has quit IRC22:23
jheskethSo with dls I can commit to attending. I'm also okay with the occasional one at 5am too. So the question about a second meeting perhaps should be do we need one. Are there others who might attend if there was another or perhaps more agenda items that we'd be able to cover etc22:24
clarkbfungi: the glanceclient errors look like hashseed things. but hashseed is 022:24
clarkbfungi: so I am wondering if maybe the seed of 0 means smomething new?22:25
*** emagana has quit IRC22:26
fungiclarkb: well, hash ordering is not guaranteed between interpreter versions even if the seed is kept constant22:26
clarkbargle bargle22:26
* clarkb tries a thing22:26
fungiso it could easily be a bug exposed by ordering differences in 3.3 vs 3.422:26
clarkbya I think that may be what it is, will look a bit closer though22:27
fungiwe could try random seeds and see if the test behavior changes at all22:27
fungithough ultimately, it's probably just yet-another-test-bug-in-need-of-fixing22:27
*** yamamoto has quit IRC22:28
*** camunoz_gone is now known as camunoz22:28
openstackgerritJoshua Harlow proposed a change to openstack-infra/config: Add py33 to sqlalchemy-migrate zuul jobs  https://review.openstack.org/12026922:29
clarkbharlowja: ^ can you ensure that py34 works too22:29
harlowjashould the above be targeting 3.4 and i should just leave 3.3 as dead?22:29
clarkbharlowja: ya, I think we want to do 3.4 at this point22:29
harlowjaok dokie22:29
clarkbharlowja: can do both for a bit if necessary too, but lets not forget about 3.422:29
harlowjaya, i'm just trying to get the whole oslo.db starting to move toward 3.x and thats one of those that seems like it was fixed for 3.x but never published for 3.x :-/22:30
*** prad has quit IRC22:31
*** sdake has quit IRC22:31
*** homeless has quit IRC22:31
*** homeless_ has joined #openstack-infra22:31
openstackgerritJoshua Harlow proposed a change to openstack-infra/config: Add py33/py34 to sqlalchemy-migrate zuul jobs  https://review.openstack.org/12026922:31
*** patrickeast has joined #openstack-infra22:31
*** jpich has joined #openstack-infra22:32
*** wenlock has quit IRC22:33
fungiharlowja: right, the plan is that i'll (hopefully) replace gate-{name}-python33 with gate-{name}-python34 in the python3-jobs project-template in zuul's layout.yaml once done confirming that we've fixed any outlier jobs which worked and are voting on 3.3 but presently broken under 3.422:36
harlowjafungi cool22:36
*** dimtruck is now known as zz_dimtruck22:37
*** emagana has joined #openstack-infra22:37
fungiharlowja: the list is whittled down to https://etherpad.openstack.org/p/py34-transition and i think we have fixes for most of what's there now22:37
clarkbglanceclient is not trivial22:37
clarkbonly half the tests are dict order related22:37
clarkbunsure of what is going on with the otehrs22:37
harlowjataskflow on there, not good, ha22:38
fungiharlowja: i think that one just need the tox.ini patch i posted last week, but i haven't re-tested with that to confirm yet22:38
harlowjaoh, ok, phew22:39
harlowjai'll double check22:39
fungiharlowja: basically any projects which had "special" testenv:py33 overrides needed them duplicated for testenv:py3422:39
*** dims_ has quit IRC22:42
*** dims_ has joined #openstack-infra22:43
*** alkari has quit IRC22:44
*** dane_leblanc has quit IRC22:44
*** dmsimard is now known as dmsimard_away22:45
*** dims_ has quit IRC22:47
jeblairnibalizer: we're waiting on a revised patch from you for install_puppet, right?22:48
*** dims_ has joined #openstack-infra22:49
*** _nadya_ has joined #openstack-infra22:53
*** garyh has quit IRC22:56
*** _nadya_ has quit IRC22:57
*** craigbr has quit IRC22:58
nibalizerjeblair: yes, i've been changing locations22:59
nibalizerwill update now22:59
*** dims_ has quit IRC23:00
*** dims_ has joined #openstack-infra23:00
*** emagana has quit IRC23:00
fungii'm available to re-review once it's up23:01
*** emagana has joined #openstack-infra23:01
fungistill around for the moment, poking at py34 tests to make sure the proposed changes solve the breakage i'm seeing23:01
*** emagana has quit IRC23:01
*** emagana has joined #openstack-infra23:02
fungii need to do a similar pass for voting pypy jobs, but i expect that to go a lot more smoothly23:02
jeblairThis message is to inform you that the host that your cloud server zm02.openstack.org' resides on has become unresponsive as of 23:05 UTC. We have rebooted the server and will continue to monitor for any further alerts.23:04
harlowjahmmm, i'm seeing alot of ones that look like those in https://code.google.com/p/mock/issues/detail?id=234 :(23:04
harlowjadamn mock, ha23:04
anteayait is 23:0423:05
*** hdd has quit IRC23:05
anteayamessage from the future23:05
*** dims_ has quit IRC23:05
jeblair23:05 < anteaya> it is 23:0423:05
jeblairanteaya: ^ :)23:05
anteaya23:04:30         jeblair | This message is to inform you23:06
harlowjahmmmm, do people use mock on 3.4 or something else, weird23:06
anteayaI'm slower23:06
jeblairanteaya: yeah, my clock said 23:04 when i sent that too.  i believe you are correct.23:06
clarkbharlowja: does unittest.mock work with 3.423:07
clarkbharlowja: if so that would be an odd bug23:07
harlowjalets see23:07
clarkbbut may just mean that the pypi package needs to be updated with 3.4 latest and greatest23:08
fungijeblair: so... they arbitrarily rebooted zm02? what sort of mess has/will that cause(d) us?23:08
clarkbfungi: it may  cause some changes to get reported as being unmergeable?23:09
fungihopefully only failed git clone attempts from jobs?23:09
harlowjaclarkb ya, seems like i need to do some updates to do that type of import23:10
harlowjathx for the hint23:10
clarkbharlowja: is it related specifically to the autospec introspection?23:10
harlowjaseems so23:10
*** praneshp has quit IRC23:11
jeblairclarkb, fungi: at most, one change reported as unmergeable, i'd hope.  :)23:11
harlowjacreate_autospec ...23:11
*** bdpayne_ has quit IRC23:11
jeblairclarkb, fungi: and actually, if i did it right, not even that.23:11
clarkbjeblair: oh good23:11
clarkbharlowja: so I gues worst case you spec it out yourself?23:12
arosenany one see this error before related to No local packages or download links found for oslo.config>=
arosen    http://paste.openstack.org/show/109185/23:12
arosenI wonder if it's because there is a letter in the version?23:12
harlowjaclarkb ya, worst case, although seems to be fixed if i do 'from unittest import mock'23:12
harlowjaso i guess just needs to be updated23:12
clarkbarosen: yup I think that is a result of us only publishing wheels for alpha versions and setup.py using easy_install23:12
clarkbarosen: try pip intall -e .23:12
clarkbharlowja: thats new in 3.3 iirc. so you might need some conditional importing. or maybe we can get the mock package updated23:13
anteayaI'm going to get another walk in23:13
*** bdpayne has joined #openstack-infra23:13
arosenclarkb:   sudo pip install -e  oslo.config>= ?23:13
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Formatting updates in hiera.yaml  https://review.openstack.org/11766423:13
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Enable puppet3 master bootstrap  https://review.openstack.org/11760423:13
jeblairclarkb, fungi: i don't think i was that clever though, so probably we'll lose one change.23:14
fungijeblair: what's the protection against a job which has been fed a zuul_url to zm02 from trying to clone refs while it was rebooting?23:14
clarkbarosen: no `pip install -e .` in the repo you are trying to setup.py develop. the -e flag is equivalent to develop but pip will install and use wheels23:14
asselinI'm running into a strange issue with zuul. All jobs targeted for the master get an immediate -1 "NOT_REGISTERED" comment. But still they are run, and the results post fine after all jobs finish. any ideas where to look/debug?23:14
clarkbfungi: oh good question23:14
harlowjaclarkb ya, conditional importing ftw23:15
nibalizerjeblair: clarkb fungi okay those two reviews are hot^^ a third review that didn't make sense any more has been abandoned23:15
arosenclarkb: thanks23:15
nibalizeralso a fresh rebase atop master23:15
*** david-lyle has quit IRC23:15
jeblairfungi: oh, those will fail :)23:15
clarkbjogo: https://bugs.launchpad.net/nova/+bug/1362596 can you provide more info on what you mean there? and how you found out?23:16
uvirtbotLaunchpad bug 1362596 in openstack-ci "API Test Failure: tempest.api.compute.servers.test_server_actions.[ServerActionsTestJSON,ServerActionsTestXML]" [Undecided,New]23:16
jeblairreading the code, it's possible we could actually end up with a stuck change.23:16
clarkbjogo: "wedged" isn't super helpful23:16
*** virmitio has left #openstack-infra23:17
jogoclarkb:  woops I thought I added in the link23:17
jogoclarkb: http://logs.openstack.org/99/115799/7/gate/gate-tempest-dsvm-neutron-pg/6394ec4/logs/screen-dstat.txt.gz23:17
jogoat 08:4823:17
clarkbnibalizer: so you went with the case statement anyways?23:18
jogoclarkb: 99% of CPU is in "wai"23:18
jogoclarkb: and there is 0 disk activity23:18
jogoclarkb: so thinking its a HP cloud issue23:18
winston-djeblair: after spending a day with gertty, i'm pretty happy with it. definitely an efficiency booster to me, thx!23:18
nibalizerclarkb: i did23:19
nibalizeri meant for there to be a default case there23:19
nibalizerlet me update23:19
jeblairwinston-d: yay!23:19
*** laxity has quit IRC23:20
winston-djeblair: anyway I can do a search in 'Diff' mode? It always make me want to hit '/' to get to the part i'm interested in quickly.23:20
clarkbjogo: is wait blocking on kernel?23:20
jogoclarkb: not sure23:20
clarkboh its percentage of cpu spent on iowait23:20
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Enable puppet3 master bootstrap  https://review.openstack.org/11760423:20
clarkbjogo: so ya I think you are right underlying node when uh, hold on I am trying to io23:21
fungiclarkb: yeah, that's "your cpu hopes your disk eventually responds"23:21
jogoclarkb: yeah I added openstack-ci so that you guys would be aware of this. I think its a bug with HP cloud sucking23:21
*** esker has quit IRC23:22
jogofungi: haha23:22
jheskethjeblair: btw, I've been working towards separating out connections in zuul.. I've got a bunch of changes starting here https://review.openstack.org/#/c/115187/7 if you're interested23:22
*** yamamoto has joined #openstack-infra23:23
clarkbnibalizer: https://review.openstack.org/#/c/117604/17/util/puppetmaster_bootstrap.sh so is that intended on running before or after we run our normal launch node install_puppet stuff?23:23
jeblairjhesketh: cool, i am, but i'm sort of nodepool+puppet focused right at the moment :/23:24
jheskethyep, all good :-)23:24
clarkbnibalizer: this is a similar question to the previous stuff. The way we launch nodes is spin them up and point them at puppetmaster. If we can't actually have puppet bootstrap itself here thats fine, but wanting to make sure I udnerstand how you intend on doing this stuff23:24
nibalizerclarkb: so, puppetmaster_bootstrap.sh can just die, that was how i built mine and tested the puppet code23:25
clarkbnibalizer: gotcha23:25
nibalizeryou were asking for docs on how to bootstrap, including that script seemed natural23:25
nibalizerinstead we should just write down what we actually do23:25
clarkbwell have the script is fine. just checking if a rooter was intended to log into host and run that23:26
nibalizerthat script does provide some use for people who come in here asking how to get a master set up so they can start hacking on infra23:26
nibalizerbut it suffers from 'we're probably never going to test it so its automatically broken'23:26
jeblairclarkb, fungi, mordred: we have 9 routers on 9 hosts now.  the names of the hosts have azs in them.  that sort of suggests to me that we may want to, when we get to this point, have each of our 9 pseudo providers affiliated only with the az that its router is in.23:27
jeblairclarkb, fungi, mordred: i'm going to ask more about that.23:27
nibalizerjeblair: i feel like there is a RIP joke in here somewhere23:28
*** yamamoto has quit IRC23:28
fungijeblair: that's an interesting suggestion... and probably has implications for the current load-balance-by-default az implementation in nodepool?23:28
jrollclarkb: mind a quick review when you have a moment? :) https://review.openstack.org/#/c/119473/23:28
clarkbfungi: you can "override" that behavior by specifcying a single az23:28
clarkbfungi: so we can game that a bit23:29
fungijeblair: oh, right, we can just list only one az in each "provider23:29
fungiclarkb: yep23:29
jeblairyeah, and we're currently doing that in order to favor az323:29
jeblairit may be a delicate balancing az capacity and network size :/23:29
jeblair(if we have to put 2/3 of our nodes on az3 then we have to weigh those router hosts more)23:30
jeblairi mean we will end up driving more traffic to them23:30
clarkbnibalizer: fungi jeblair ok I am +2 on that puppet stack. didn't approve the first chagne just in case you all wanted to look at it again without its old dependency23:31
harlowjaalright got taskflow running on 3.4 (i think fine)23:31
harlowjaRan 948 (+16) tests in 200.991s (+6.001s)23:31
harlowjaPASSED (id=3, skips=2)23:31
harlowjawoot :-P23:31
*** openstack__ has joined #openstack-infra23:31
*** bhuvan_ has joined #openstack-infra23:32
*** signed8bit has quit IRC23:32
*** amotoki has joined #openstack-infra23:32
*** laxity has joined #openstack-infra23:32
fungiharlowja: excellent. i'll be double-checking it next anyway once i'm done confirming nothing else is needed for oslo.rootwrap23:32
patrickeasthey everyone, I have a possibly dumb question about zuul configuration if someone has a min to help out, I’m working on the 3rd party CI for a cinder driver and am trying to figure out how to disable the gerrit comment zuul makes when starting a job so it only comments a single time as per the requirements… browsing through the zuul source it looks like there isn’t a (obvious) configuration option for this and is part of the23:33
patrickeast“normal” flow for zuul when enqueueing events, so how do I change that behavior in my builds?23:33
fungiclarkb: oslo.messaging apparently need more work. testr is segfaulting on me23:33
clarkbfungi: wow thats awesome23:33
*** JadnG has quit IRC23:33
harlowjafungi cool, i had to do https://review.openstack.org/#/c/120281/23:33
harlowjabut after that seemed to go ok23:33
clarkbpatrickeast: in your layout file under the pipelines section there should be entries with starting comments, just remove those entries23:33
jeblairpatrickeast: zuul is entirely configuration driven, so if your system is commenting on a patch, it's because the layout.yaml instructs it to...23:34
fungiharlowja: was that in addition to https://review.openstack.org/118813 ?23:34
jeblairpatrickeast: were you following some instructions to set it up in a certain way?23:34
harlowjafungi correcy23:34
*** david-lyle has joined #openstack-infra23:34
jeblairpatrickeast: if so, can you point them out to us?  it seems like they may be leading you astray.23:34
*** bhuvan has quit IRC23:34
fungiharlowja: thanks. i've added it to the etherpad so i don't forget23:34
clarkbfungi: now I am going to see if I can segfault too23:34
patrickeastjeblair: i was following these blog posts http://www.joinfu.com/2014/02/setting-up-an-openstack-external-testing-system-part-2/23:35
fungiclarkb: please do. i don't entirely trust this node any longer and may recycle it and grab another23:35
patrickeastjeblair: clarkb: so I have triggers for gerrit comments, but I don’t see anything for posting a comment, only for “verified”, unless “verified: 0” means to report starting?23:36
jeblairpatrickeast: where's the layout.yaml that gets installed?23:36
patrickeastjeblair: its under /etc/zuul/23:37
jeblairpatrickeast: sorry, i meant the source file.23:37
*** amotoki has quit IRC23:37
jeblairpatrickeast: i see instructions that say to edit it, but i don't see the original.23:37
patrickeastjeblair: here is my current one http://paste.openstack.org/show/109193/23:38
jeblairis this it?  https://github.com/jaypipes/os-ext-testing-data/blob/master/etc/zuul/layout.yaml23:38
patrickeastjeblair: yep23:38
jeblairpatrickeast: the one on github does not have a start section, which is correct.  but yours does23:39
*** ZZelle_ has quit IRC23:39
jeblairpatrickeast: anyway, get rid of the 'start' sections, and get rid of the 'gate' pipeline entirely.23:39
patrickeastjeblair: ohhh interesting, ok, so that must be it23:39
patrickeastjeblair: ok, ill give that a try23:39
jeblairjaypipes: why does your example layout.yaml have a gate section?23:39
clarkbfungi: Segmentation fault (core dumped) bam!23:39
clarkbfungi: this is awesome23:40
fungiclarkb: okay, so... yeah. probably one of the c extensions23:40
patrickeastjeblair: thanks for your help!23:40
jeblairpatrickeast: you're welcome23:41
clarkbfungi: lookes like 6/8 testr runners did it on my box23:41
clarkbso consistent but not for every test23:41
jeblairjaypipes: i think you should be able to drop the gate pipeline and the includes section23:41
*** emagana has quit IRC23:42
jaypipesjeblair: copy/paste from upstream :(23:42
nibalizerclarkb: thanks :)23:42
jeblairjaypipes: the resulting file should be delightfully small if you do that :)23:42
jaypipesjeblair: and IIRC, a vestigiality of perhaps when we might have thought external systems may vote in the gate (did we ever, not sure?)23:42
jaypipesjeblair: I'll do so right now! :)23:42
jeblairjaypipes: oh, i'd drop the 'precedence' line from the check pipeline too23:43
clarkbfungi: I am trying to narrow down the culprit23:43
clarkbfungi: runnign tests serially23:43
*** MaxV has joined #openstack-infra23:43
*** openstack__ has quit IRC23:45
jaypipesjeblair: a-ok? https://github.com/jaypipes/os-ext-testing-data/pull/523:45
fungiharlowja: confirmed that taskflow is testing clean now with those two patches cherry-picked onto the tip of master23:49
harlowjafungi woot23:49
clarkbrunning tests serially is slow23:49
fungiclarkb: i KNEW there had to be a reason we parallelized testing23:49
jaypipesjeblair: sorry for os-ext-testng-data causing confusion :( trying to remedy ASAP.23:50
*** dmsimard_away is now known as dmsimard23:51
*** daya_k has joined #openstack-infra23:51
openstackgerritEmilien Macchi proposed a change to openstack-infra/config: Introduce tripleo-puppet-elements into Stackforge  https://review.openstack.org/12028523:53
fungiclarkb: okay, i've tested the py34 changes we have posted so far and split the problem project list in the etherpad between ones which i've confirmed solved and those still under research23:55
*** dmsimard is now known as dmsimard_away23:55
clarkbadded notes about glanceclient23:56
clarkbok messaging segfaults in serial execution too23:56
* clarkb reads logs23:56
fungii think i'm going to call it a night and start back in on these and sizing up the pypy job transition as well first thing tomorrow, barring new fires23:56
clarkbfungi: sounds good. Nothing pops out screaming why it segfaulted23:57
clarkbprobably going to need to be a bit more methodolical23:58
*** camunoz has quit IRC23:59
*** oomichi has joined #openstack-infra23:59
fungiif you break off, just leave a note about the minimum breaking case you've been able to isolate and i can pick up on trying to break it down further when i get back to it23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!