Monday, 2016-08-08

*** itisha has quit IRC00:00
*** pahuang has joined #openstack-infra00:00
*** zhurong has quit IRC00:01
*** armax has quit IRC00:08
*** rbuzatu has joined #openstack-infra00:09
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: grafana: Correct pipeline of midonet periodic jobs
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: grafana: Add the rest of midonet stable jobs
*** rbuzatu has quit IRC00:13
*** sflanigan has joined #openstack-infra00:15
openstackgerritCraige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio
*** johnchalekson has quit IRC00:22
*** Teresita-Warrior has joined #openstack-infra00:27
*** markvoelker has joined #openstack-infra00:28
*** zz_dimtruck is now known as dimtruck00:34
openstackgerritGoutham Pacha Ravi proposed openstack-infra/project-config: Publish manila install guide from tree
*** dimtruck is now known as zz_dimtruck00:44
*** kzaitsev_mb has quit IRC00:46
*** amitgandhinz has joined #openstack-infra00:48
*** thorst has joined #openstack-infra00:52
*** amitgandhinz has quit IRC00:52
*** edmondsw has joined #openstack-infra00:54
openstackgerritGoutham Pacha Ravi proposed openstack-infra/project-config: Manila - Skip running devstack jobs on api-ref/install-guide changes
openstackgerritGoutham Pacha Ravi proposed openstack-infra/project-config: Publish manila install guide from tree
jheskethcraige: I left +W off as I wanted the second patch to be ready (I hadn't reviewed it at the time)01:05
*** baoli has joined #openstack-infra01:05
jheskethcraige: they probably should be reordered but it's no problem01:06
*** mriedem has quit IRC01:07
craigeAh, I read your comment and it made sense, I just didn't join the dots on why there was no +W.01:07
craigeThanks :-)01:07
craigeWhile have your eyes, do you have a second to look at a minor spec issue?01:08
craigeI've got this doc compiling with a bad link (first sentence, third link) but the code looks good to me.01:08
*** karthikp_ has quit IRC01:09
*** baoli has quit IRC01:11
openstackgerritMerged openstack-infra/system-config: Update iLO IP and rack for compute011.chocolate
*** zhurong has joined #openstack-infra01:12
*** camunoz has joined #openstack-infra01:17
*** hichihara has joined #openstack-infra01:19
*** thorst_ has joined #openstack-infra01:24
*** rhallisey has quit IRC01:25
*** thorst has quit IRC01:27
*** yanyanhu has joined #openstack-infra01:30
*** baoli has joined #openstack-infra01:41
*** kzaitsev_mb has joined #openstack-infra01:43
*** gildub has joined #openstack-infra01:46
*** sandanar has joined #openstack-infra01:46
*** amitgandhinz has joined #openstack-infra01:49
*** sandanar has quit IRC01:52
*** sandanar has joined #openstack-infra01:52
*** amitgandhinz has quit IRC01:53
*** signed8bit_Zzz is now known as signed8bit02:01
*** signed8bit has quit IRC02:02
*** baoli has quit IRC02:02
*** zz_dimtruck is now known as dimtruck02:03
*** baoli has joined #openstack-infra02:04
*** thorst_ has quit IRC02:04
*** Sukhdev has joined #openstack-infra02:05
*** baoli has quit IRC02:07
*** sandanar_ has joined #openstack-infra02:12
*** sandanar has quit IRC02:16
openstackgerritMerged openstack-infra/irc-meetings: move HA meeting an hour later, to 0900 UTC
*** kzaitsev_mb has quit IRC02:19
*** Sukhdev has quit IRC02:27
*** bswartz has quit IRC02:30
*** julim has joined #openstack-infra02:35
*** markvoelker has quit IRC02:36
*** gildub has quit IRC02:39
*** amitgandhinz has joined #openstack-infra02:49
*** Sukhdev has joined #openstack-infra02:52
*** asilenkov has quit IRC02:53
*** amitgandhinz has quit IRC02:54
*** gouthamr has quit IRC02:55
*** asilenkov has joined #openstack-infra02:57
tonybIs anyone else seeing the gerrot web UI behave poorly? slow response and frequent proxy errors?03:06
*** armax has joined #openstack-infra03:08
*** Sukhdev has quit IRC03:11
*** esberglu has joined #openstack-infra03:12
*** armax has quit IRC03:13
*** kzaitsev_mb has joined #openstack-infra03:16
*** bswartz has joined #openstack-infra03:17
craigeYes but that's why I've almost always used gertty, tonyb, so that experience is probably of no use ;-)03:18
*** esberglu has quit IRC03:19
*** kzaitsev_mb has quit IRC03:21
*** Sukhdev has joined #openstack-infra03:21
*** sandanar_ has quit IRC03:24
*** gbraad has joined #openstack-infra03:25
*** chlong has quit IRC03:25
tonybcraige: Oh the REST API is slow too but as that's scripted the slow isn't as visible03:27
*** julim has quit IRC03:28
prometheanfiretonyb: is that like a carrot?03:31
tonybprometheanfire: Umm yes, just like a carrot.03:33
prometheanfireso where's the stick?03:37
*** ayoung has quit IRC03:39
masayukig+1 for craige to use gertty :)03:41
prometheanfiregertty rocks :D03:41
*** fengchaoyang has joined #openstack-infra03:50
*** amitgandhinz has joined #openstack-infra03:50
fengchaoyanghello,everybody, now when I "git review",  fatal: ICLA contributor agreement requires current contact information.  Please review your contact information:   fatal: Could not read from remote repository.  Please make sure you have the correct access rights and the repository exists.03:51
*** gongysh has joined #openstack-infra03:52
fengchaoyangwhen I go to ,it say : Code Review - Error Server Error Cannot store contact information03:52
fengchaoyanghere  is my git config  :stack@fe:~/python-gnocchiclient$ git  config -l gitreview.username=fengchaoyang core.repositoryformatversion=0 core.filemode=true core.bare=false core.logallrefupdates=true remote.origin.url= remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* branch.master.remote=origin branch03:53
fengchaoyangand ICLA is registed StatusNameDescription VerifiedICLAOpenStack Individual Contributor License Agreement New Contributor Agreement03:54
fengchaoyangmy git remote is : stack@fe:~/python-gnocchiclient$ git remote -v gerritssh:// (fetch) gerritssh:// (push) origin (fetch) origin (push)03:55
*** amitgandhinz has quit IRC03:55
fengchaoyangwho kown where I wrong?03:55
fengchaoyangI can git review -s ,but I can't git review03:56
fengchaoyangIs this is a bug?03:57
*** sdake has joined #openstack-infra03:58
*** jesusaur has joined #openstack-infra03:58
fengchaoyangI have google much times,but can't sort it03:58
*** spiffxp has quit IRC03:58
*** vipuls has quit IRC03:59
*** spiffxp has joined #openstack-infra03:59
clarkbfengchaoyang: which directions did you follow to set up your accouny04:00
*** vipuls has joined #openstack-infra04:00
clarkbplease follow the ones in the openstack infra manual04:00
clarkbmake sure your openstack foundation profile is configured properly04:00
*** zhurong has quit IRC04:04
*** zhurong has joined #openstack-infra04:05
*** roxanaghe has joined #openstack-infra04:08
fengchaoyanghere is my openstack community member
fengchaoyangI don't know whether configured properly04:09
clarkbyou need to be a foundation member not community I think. there should be a button for you to change it04:11
clarkbthen update the contact info and you should be done04:11
craigeoh clarkb is about04:11
* craige prods clarkb for more clues on working with OpenStackID as the last leads came up empty :-(04:12
fengchaoyang You are logged in as: feng chaoyang Resign Membership Change to Community Member Current Member Level: Foundation Member04:13
*** binbincong has joined #openstack-infra04:13
*** jaosorior has joined #openstack-infra04:14
clarkbcraige: did you try with the dev server?04:14
clarkbfengchaoyang: ok foundation member is correct, make sure the email address matches in gerrit and update your contact info04:15
openstackgerritIan Wienand proposed openstack-infra/project-config: Add Fedora 24 work-around kernel
fengchaoyangthis is review Username fengchaoyang Full Namefengchaoyang Email RegisteredJun 29, 2016 7:00 PM Account ID2251404:16
fengchaoyanggit remote -v gerritssh:// (fetch) gerritssh:// (push) origin (fetch) origin (push)04:16
*** kzaitsev_mb has joined #openstack-infra04:17
*** baoli has joined #openstack-infra04:19
fengchaoyangwhen I update my contact information, It' will  Error Server Error Cannot store contact information04:20
*** kzaitsev_mb has quit IRC04:22
openstackgerritRyan Tidwell proposed openstack-infra/project-config: Make gate-neutron-dynamic-routing-dsvm-tempest job voting.
*** senk_ has joined #openstack-infra04:24
*** baoli has quit IRC04:24
clarkbfengchaoyang: yes it will do that if your email address in gerrit doesntatch thr address in your foundation profilw04:24
craigeoh, no I didn't clarkb! That was one of your suggestions I failed to act on. Address for the dev server again?04:25
craige(and is the backend data the same?)04:25
clarkbopenstackid-dev.o.o I think04:27
clarkband no thr passwords are different smarcet can tell you wjat it is your reset yours04:27
craigeI'll give it a whirl. thanks clarkb04:28
craigeIf it gets as far as asking for a passowrd, I'll hit smarcet up.04:29
*** Ravikiran_K has joined #openstack-infra04:29
openstackgerritCraige McWhirter proposed openstack-infra/infra-specs: Corrected Phabricator URL
AJaegerinfra-root, I think we need to restart gerrit, the UI is just really slow04:35
*** markvoelker has joined #openstack-infra04:37
*** tonytan4ever has quit IRC04:39
jheskethAJaeger: it feels okay to me04:41
*** markvoelker has quit IRC04:42
AJaegermmh, do we have networking issues?04:42
* AJaeger just had a 50204:42
craigeNo love with that either clarkb :-(04:43
craigeFYI: I replied ealier but didn't ping you, jhesketh, in case you missed it.04:43
jheskethcraige: oh, sorry, /me looks04:44
AJaegertonyb also had problems earlier - see backscroll, around 3:0604:44
craigenot urgent, jhesketh, I've just been thinking my lack of nick may have caused it to get lost.04:44
*** coolsvap has joined #openstack-infra04:46
AJaegercraige: your specs change fixes the problems, thanks04:46
clarkbcraige: do you have a paste of your config?04:47
clarkbI cqn dig mine uo when actually on computer with keys04:47
craigeThank *you* AJaeger. It was driving me spare and I had no idea that I had doubled up *or* that doubling up was a problem.04:47
craigeI do clarkb04:47
jheskethAJaeger: resources on gerrit seem high but not abnormal... if it continues maybe a restart but I'm not seeing a reason at hte moment04:48
jheskethcraige: I can't see a fault in your rst... maybe because _Phabricator is defined twice? have you tried building the docs locally?04:49
*** amitgandhinz has joined #openstack-infra04:51
craige and clarkb, where I've replaced with or
AJaegerjhesketh: is gerrit doing garbage collection?04:52
craigeah, that question, thanks jhesketh. It was just resolved in 35221504:52
AJaegerjhesketh: craige's RST issue is solved04:52
craigeand it was the double definition of _Phabricator.04:52
clarkbcraige: and it does work with ubuntu openid?04:53
*** kushal has joined #openstack-infra04:53
craigeYes, it does clarkb04:53
*** kushal has quit IRC04:53
*** kushal has joined #openstack-infra04:53
clarkbok I will have to compare what I have when at real conputer04:54
openstackgerritRyan Tidwell proposed openstack-infra/project-config: Make gate-neutron-dynamic-routing-dsvm-tempest job voting.
craigeCool, thanks clarkb04:55
clarkband you did try the discovery url?04:55
*** amitgandhinz has quit IRC04:56
openstackgerritAndreas Jaeger proposed openstack-infra/infra-specs: Fix warnings
craigeI was unable to determine the discovery URL, clarkb04:56
openstackgerritAndreas Jaeger proposed openstack-infra/infra-specs: Fix warnings
clarkbcraige: look at the http response headers for / to find it04:57
openstackgerritAndreas Jaeger proposed openstack-infra/infra-specs: Fix warnings
clarkbits one of them iirc04:58
AJaegerclarkb: have you found some internet on vacation or are you back and catching up? ;)05:01
*** fengchaoyang has quit IRC05:02
jheskethAJaeger: I don't think it is running05:02
*** nmagnezi_ has joined #openstack-infra05:03
*** roxanaghe has quit IRC05:03
*** nmagnezi_ has quit IRC05:03
*** pcaruana has quit IRC05:03
*** roxanaghe has joined #openstack-infra05:03
*** nmagnezi_ has joined #openstack-infra05:03
*** roxanaghe has quit IRC05:04
AJaegerjhesketh: what is not running? Sorry, you lost me...05:04
*** roxanaghe has joined #openstack-infra05:04
*** roxanaghe has joined #openstack-infra05:05
AJaegerjhesketh: got it - you mean, it's not doing heavy GC - thanks for confirming.05:05
clarkbAJaeger: I have intermittent internets on phone05:05
*** roxanaghe has quit IRC05:06
*** roxanaghe has joined #openstack-infra05:06
*** roxanaghe has quit IRC05:06
AJaegerclarkb: I see. Then continue to enjoy your vacation ;)05:07
*** dimtruck is now known as zz_dimtruck05:08
openstackgerritRyan Tidwell proposed openstack-infra/project-config: Make gate-neutron-dynamic-routing-dsvm-tempest job voting.
*** senk_ has quit IRC05:09
*** senk_ has joined #openstack-infra05:13
jheskethAJaeger: yes, sorry05:14
openstackgerritJianghua Wang proposed openstack-infra/devstack-gate: xenapi: Skip multi-host setup for nova network setup
AJaegerinteresting - gerrit is still slow for me. Normally it's garbage collecting in such cases...05:16
*** oanson has joined #openstack-infra05:18
*** kzaitsev_mb has joined #openstack-infra05:18
craigeI'll see if I can find them in the logs. I'll turn on Apache debugging clarkb, as they weren't appearing normally.05:22
clarkbcraige: just curl it05:23
*** kzaitsev_mb has quit IRC05:23
craigeUnsurprisingly that is much better, clarkb :-)05:24
* craige wades through the output05:24
*** psachin has joined #openstack-infra05:25
*** senk_ has quit IRC05:26
*** hichihara has quit IRC05:27
sshnaidmAJaeger, hi, I have a question about
sshnaidmAJaeger, btw derekh is in pto, so I think he's back next week only05:31
sshnaidmAJaeger, what does mean "Why not add the job to the same template as before?"? Which template do you mean?05:32
*** pgadiya has joined #openstack-infra05:35
*** Teresita-Warrior has quit IRC05:38
*** rbuzatu has joined #openstack-infra05:39
*** fengchaoyang has joined #openstack-infra05:48
*** PollyZ has joined #openstack-infra05:48
fengchaoyangsorry, my question already can't solve, this is my question when i git review Enter passphrase for key '/opt/stack/.ssh/id_rsa':  Enter passphrase for key '/opt/stack/.ssh/id_rsa':  fatal: ICLA contributor agreement requires current contact information.  Please review your contact information:   fatal: Could not read from remote repository.  Please make sure you have th05:50
fengchaoyangmy openstack profile email and gerrit email is same email address05:51
fengchaoyangcould you tell me what can i do?05:52
*** amitgandhinz has joined #openstack-infra05:52
*** javeriak has joined #openstack-infra05:52
*** PollyZ has quit IRC05:52
*** _nadya_ has joined #openstack-infra05:52
fengchaoyanggit remote -v gerritssh:// (fetch) gerritssh:// (push) origin (fetch) origin (push)05:53
*** PollyZ has joined #openstack-infra05:53 gitreview.username=fengchaoyang core.repositoryformatversion=0 core.filemode=true core.bare=false core.logallrefupdates=true remote.origin.url= remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* branch.master.remote=origin branch.master.merge=refs/heads/master remote.gerrit.url=ssh://fengchaoyang@revi05:53
*** yamahata has joined #openstack-infra05:53
*** amitgandhinz has quit IRC05:56
*** chlong has joined #openstack-infra05:59
*** roxanaghe has joined #openstack-infra06:02
*** sarob has joined #openstack-infra06:02
*** sarob has quit IRC06:02
*** sarob has joined #openstack-infra06:02
*** mrmartin has joined #openstack-infra06:05
*** roxanaghe has quit IRC06:06
openstackgerritMerged openstack-infra/tripleo-ci: Set --use-public option for dlrn execution instead of env var
*** dkehn has quit IRC06:08
*** gildub has joined #openstack-infra06:09
*** dkehn_ has quit IRC06:10
*** Hal has quit IRC06:10
*** dkehn has joined #openstack-infra06:15
*** kzaitsev_mb has joined #openstack-infra06:19
craigefound the discovery URL for prod and dev instances and have been unable to get a working response using the URI contained there in (for either) clarkb06:20
*** vsaienko1 has quit IRC06:23
*** _nadya_ has quit IRC06:23
*** javeriak has quit IRC06:23
*** kzaitsev_mb has quit IRC06:24
*** PollyZ has quit IRC06:25
*** senk_ has joined #openstack-infra06:28
*** javeriak has joined #openstack-infra06:28
*** dkehn_ has joined #openstack-infra06:29
*** baoli has joined #openstack-infra06:31
*** sarob has quit IRC06:35
*** baoli has quit IRC06:36
*** sdake has quit IRC06:38
*** markvoelker has joined #openstack-infra06:38
*** pcaruana has joined #openstack-infra06:38
*** rcernin has joined #openstack-infra06:41
*** markvoelker has quit IRC06:42
*** xnox has quit IRC06:44
*** matbu|pto is now known as matbu06:44
*** fengchaoyang has quit IRC06:45
*** fengchaoyang has joined #openstack-infra06:46
*** rbuzatu has quit IRC06:48
*** Hal has joined #openstack-infra06:52
*** amitgandhinz has joined #openstack-infra06:52
*** tesseract- has joined #openstack-infra06:53
yolandagood morning06:54
craigeGood morning yolanda06:54
*** slaweq has joined #openstack-infra06:55
AJaegersshnaidm: see the change I referenced that removed the job - the job was in a different template than the one you add it to now.06:56
AJaegerfengchaoyang: If you already followed the instructions at and still get that, see for additional troubleshooting tips.06:57
*** amitgandhinz has quit IRC06:57
AJaegermorning, yolanda06:57
openstackgerritShinya Kawabata proposed openstack-infra/project-config: Add gate-tempest for Cassandra implementation of monasca
sshnaidmAJaeger, not sure I understand.. I see it there in experimental-tripleo as well:
AJaegersshnaidm: You say basically "Add back as previous" - but do it differently. And I wonder why it's not a full revert to the status before. I'm fine with either but like to be convinced that what you do is really what the tripleo team intends to do.06:59
AJaegersshnaidm: I suggest you wait until either derekh is back or pabelanger is online (this European afternoon - Canada morning) and discuss with either of them.07:00
craigeping jhesketh07:01
jheskethcraige: pong07:01
craigeGot a moment for a higher bandwidth chat about 342481?07:02
sshnaidmAJaeger, I see, I don't think I can really "revert" it, because there were a huge amount of changes in job definitions related to all these movements from rh1, to rh1, etc. But what I need is to be able to run tempest gate job with "check experimental", as it was possible before (this I meant "revert"). From my side it could be considered even as new definition..07:02
*** _nadya_ has joined #openstack-infra07:02
sshnaidmAJaeger, sure, I'll talk to pabelanger about this as well07:03
*** ifarkas_afk is now known as ifarkas07:03
*** _nadya_ has quit IRC07:04
*** _nadya_ has joined #openstack-infra07:05
AJaegersshnaidm: there were quite some changes - but you can still look what was the status before those and what it is now ;) And if you don't want to revert, then explain what you do.07:06
AJaegersshnaidm: Just some simple grep why your change make a difference:07:06
AJaegergrep experimental-tripleo-jobs zuul/layout.yaml|wc -l07:06
AJaegerbut the same grep with check-tripleo-jobs gives 16. So, before the job was added to 6 repos, now you want to add it to 16.07:07
*** xnox has joined #openstack-infra07:07
AJaegerDoing it as you propose might be the right thing to do - but then please explain *why* you do it that way.07:07
*** HeOS has quit IRC07:08
*** slaweq has quit IRC07:08
*** HeOS has joined #openstack-infra07:08
sshnaidmAJaeger, if I understand you right - it was a split for tripleoci jobs to run different sets of them for different repos, the tempest job could run for everything before this split, as well as now should be07:09
*** _nadya_ has quit IRC07:10
sshnaidmAJaeger, are we talking about the same..?07:10
AJaegersshnaidm: I don't know - and don't have the time to dig deeper into this right now. I'll wait for your discussion with pabelanger ...07:11
sshnaidmAJaeger, yeah, anyway I'll talk to him, he's on top of last changes in jobs definitions there07:11
*** roxanaghe has joined #openstack-infra07:12
*** xnox has quit IRC07:13
*** dkehn has quit IRC07:13
*** yamamoto has quit IRC07:14
*** dkehn_ has quit IRC07:14
*** xnox has joined #openstack-infra07:14
*** roxanaghe has quit IRC07:17
*** _nadya_ has joined #openstack-infra07:18
openstackgerritCraige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio
*** _nadya_ has quit IRC07:18
*** kzaitsev_mb has joined #openstack-infra07:20
*** jlanoux has joined #openstack-infra07:23
fengchaoyangthanks all07:23
*** fengchaoyang has quit IRC07:23
*** slaweq has joined #openstack-infra07:24
*** permalac has joined #openstack-infra07:24
*** kzaitsev_mb has quit IRC07:25
*** _nadya_ has joined #openstack-infra07:31
*** mikelk has joined #openstack-infra07:33
*** vincentll has joined #openstack-infra07:34
*** jpich has joined #openstack-infra07:36
*** e0ne has joined #openstack-infra07:36
openstackgerrityolanda.robla proposed openstack-infra/shade: Update delete flavor test to work with betamax
*** matthewbodkin has joined #openstack-infra07:37
openstackgerritMarton Kiss proposed openstack-infra/puppet-askbot: Add explicit configuration support for Akismet
openstackgerritAleksey Zvyagintsev proposed openstack-infra/project-config: [jenkinss]Add {murano-app}-unittest job
yolandais very slow for you?07:43
yolandaAJaeger, ^07:43
*** yaume has joined #openstack-infra07:44
*** rbuzatu has joined #openstack-infra07:45
*** yaume has quit IRC07:47
*** rbuzatu has quit IRC07:49
*** e0ne has quit IRC07:51
zigoI get a very weird result on the deb-auto-backports, everything should just work, but it doesn't, I just get sbuild invoked, and then nothing happends instead of building.07:51
zigoHow can I debug this?07:52
*** amitgandhinz has joined #openstack-infra07:53
*** rossella_s has quit IRC07:53
openstackgerritMerged openstack-infra/project-config: Add net-info to openstack-infra jobs
openstackgerritMerged openstack-infra/project-config: Manila - Skip running devstack jobs on api-ref/install-guide changes
zigoAJaeger: ^07:57
openstackgerritIan Wienand proposed openstack/diskimage-builder: Add "audit"package to yum-minimal
zigoMaybe if I get a copy of the image?07:57
*** gildub has quit IRC07:57
zigoianw: ^07:57
*** amitgandhinz has quit IRC07:58
*** matrohon has joined #openstack-infra07:58
ianwzigo: ?08:00
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-infra08:00
zigoianw: How can I get to understand why sbuild is doing nothing here?
zigoianw: It's supposed to just build, but it does nothing...08:00
zigo(see the very end of the log)08:01
*** dtantsur|afk is now known as dtantsur08:02
*** ramishra_ has joined #openstack-infra08:02
ianwzigo: hmm, odd ... it does go quiet for 27 seconds, is that about the build time?08:02
ianwseems like probably the output goes somewhere not captured08:03
zigoianw: I don't know.08:03
*** Hal has quit IRC08:03
zigoianw: Maybe I can add an "ls" after the build, just to check the output.08:03
*** nherciu has joined #openstack-infra08:04
* zigo does that08:04
ianwzigo: i can probably put a host on hold and get you a login for it, if that would help08:04
zigoianw: That would help *A LOT* yes.08:04
ianwzigo: ok, give me a minute, we can hold hosts after a failed job08:05
zigoianw: I just added the ls in
*** ramishra has quit IRC08:05
*** jtomasek has quit IRC08:07
ianw| 7  | pkgdeb-build-auto-backports         | 1               |08:08
ianwzigo: ok, we should hold a node on failure of that job now.  try kicking it off for me and we'll see how it goes when it fails08:08
*** jtomasek has joined #openstack-infra08:12
*** javeriak has quit IRC08:13
*** roxanaghe has joined #openstack-infra08:13
zigoianw: As I wrote, I just did in
*** Hal has joined #openstack-infra08:13
openstackgerritMerged openstack-infra/project-config: Rename tripleo-centos-7-ovb to tripleo-centos-7
*** roxanaghe has quit IRC08:18
openstackgerritMarton Kiss proposed openstack-infra/system-config: Update solr package repository url
*** matrohon has quit IRC08:19
*** amoralej|off is now known as amoralej08:20
*** Goneri has joined #openstack-infra08:20
*** kzaitsev_mb has joined #openstack-infra08:21
*** ggnel_t has joined #openstack-infra08:24
openstackgerritMerged openstack-infra/project-config: Add gate-tripleo-quickstart-docs-ubuntu-xenial job
*** rbuzatu has joined #openstack-infra08:26
*** kzaitsev_mb has quit IRC08:26
AJaegeryolanda: yes, gerrit was/is slow, see my earlier discussion with jhesketh. But haven't touched review for the last 90 mins, so cannot speak about current status.08:28
*** dingyichen has quit IRC08:32
senk_AJaeger: still slow currently.. i guess it started yesterday..08:32
AJaegerzigo: One tip - you can also check the log files from ansible in . They contain the rest of your job setup. Not sure whether that helps with this example, just wanted to point it out. I agree with ianw's suggestion to hold a note.08:32
AJaegersenk_: let's tell yolanda, she has the powers to restart gerrit ;)08:32
yolandagoing to take a look at logs and restart if needed08:33
senk_yolanda: would be great if you can investigate the issue.. the "Proxy Error" happens more frequently08:34
openstackgerritMerged openstack-infra/system-config: Add compute023.chocolate metadata for better tracking
Jokke_AJaeger & yolanda: it still is dead slow and giving 502 proxy errors08:35
*** matrohon has joined #openstack-infra08:36
yolandagoing to restart it08:36
*** matrohon has quit IRC08:36
*** acabot has joined #openstack-infra08:36
*** rossella_s has joined #openstack-infra08:37
yolanda#status notice Gerrit is going to be restarted08:37
openstackstatusyolanda: sending notice08:37
-openstackstatus- NOTICE: Gerrit is going to be restarted08:38
*** mrmartin has quit IRC08:39
openstackstatusyolanda: finished sending notice08:40
yolandaok, it has been restarted08:41
Jokke_thanks yolanda08:42
*** dtantsur is now known as dtantsur|sick08:44
*** jistr|mtg is now known as jistr08:50
*** tosky has joined #openstack-infra08:51
*** mrmartin has joined #openstack-infra08:52
*** matrohon has joined #openstack-infra08:53
*** mhickey has joined #openstack-infra08:54
*** yaume has joined #openstack-infra08:56
*** kzaitsev_mb has joined #openstack-infra08:56
*** roxanaghe has joined #openstack-infra09:00
*** binbincong has quit IRC09:03
*** acoles_ is now known as acoles09:04
*** roxanaghe has quit IRC09:05
*** chem has joined #openstack-infra09:06
*** matrohon has quit IRC09:08
*** kushal has quit IRC09:08
*** Mmike has quit IRC09:11
*** e0ne has joined #openstack-infra09:11
*** yamamoto has joined #openstack-infra09:12
senk_thanks yolanda09:12
*** Mmike has joined #openstack-infra09:13
yolandano problem09:14
*** yamamot__ has joined #openstack-infra09:14
*** daemontool has joined #openstack-infra09:14
*** yamamoto has quit IRC09:17
*** Sukhdev has quit IRC09:18
openstackgerritMarton Kiss proposed openstack-infra/system-config: Askbot add editor_deselector for the staging settings
zigoianw: The build is done, now 1/ I get what the problem is 2/ the issue is indeed that there's no output redirection or something.09:19
zigoianw: Can you give me access so that I can attempt to find out the output issue?09:19
e0necan anybody please review
*** javeriak has joined #openstack-infra09:22
*** DaiDV has joined #openstack-infra09:24
AJaegere0ne: I'd like to see some review of that especially by OpenStack QA and cinder folks09:24
*** matrohon has joined #openstack-infra09:24
e0neAJaeger: I'm not sure that QA guys can help with it. it's related only to BDD driver:(09:25
e0neAJaeger: I'll ask somebody else from cinder team to review it09:25
* zigo pushed a new patch fixing the issue09:27
*** binbincong has joined #openstack-infra09:27
AJaegere0ne: thanks09:28
mrmartinMaybe someone else have a better eye for this, why this trusty job fails?
ianwzigo: ok, try jenkins@
mrmartinit is the related patchset:
zigoianw: I'm in, thanks !09:30
*** daemontool has quit IRC09:31
zigoWeird, sbuild is producing output when I just type "sbuild"09:31
zigoTrying with the ssh trick.09:32
zigoHum... it's not doing it with the ssh thing.09:33
*** narayrak has joined #openstack-infra09:34
*** yamamot__ has quit IRC09:35
*** zeroDivisible has joined #openstack-infra09:35
*** yamamoto has joined #openstack-infra09:35
*** kzaitsev_mb has quit IRC09:36
*** ams__ has joined #openstack-infra09:38
*** binbincong has quit IRC09:40
*** yamamoto has quit IRC09:41
*** Sudha has joined #openstack-infra09:41
ianwzigo: i bet it checks isatty() or something...09:43
*** narayrak has quit IRC09:43
ianwssh -l might help?09:43
zigoianw: It's looking like it needs $verbose = 1; in .sbuildrc09:44
zigoNot sure why...09:44
zigoI'll fix that in deb-openstack-pkg-tools.09:44
*** narayrak has joined #openstack-infra09:44
ianwmrmartin: i think09:45
ianwthose logs are always really hard to parse09:45
mrmartinianw, oh thanks, really09:46
*** Hal has quit IRC09:47
*** Hal has joined #openstack-infra09:47
zigoianw: Any idea why it's taking so long to schedule packaging jobs, when we're supposed to be alone using the Jessie image?09:48
openstackgerritAdam Coldrick proposed openstack-infra/storyboard: Fix /v1/worklists/:id/items endpoint
zigoianw: It used to take only a few minutes, now I'm waiting more than 30.09:48
zigoianw: And I guess you can now destroy the VM, I've seen what I wanted.09:49
zigoThanks a lot for your help.09:49
*** nherciu_ has joined #openstack-infra09:52
ianwzigo: ok, cool.  umm, well there might be something in nodepool that's not working too well09:52
zigoianw: Yeah, that's my feeling too. It may be worse looking into it.09:53
zigo(as it may be slowing down everyone...)09:53
ianwzigo: when we first rolled out centos, we hit starvation in a similar situation of only using a few nodes.  i did a bunch of work on the scheduler at that point, but things might have changed09:53
*** amitgandhinz has joined #openstack-infra09:55
*** nherciu has quit IRC09:56
ianwzigo: ok, removed the "hold after failure" on that job09:56
*** amitgandhinz has quit IRC09:59
openstackgerritMarton Kiss proposed openstack-infra/puppet-askbot: Add explicit configuration support for Akismet
*** matrohon has quit IRC10:01
*** zhurong has quit IRC10:03
*** binbincong has joined #openstack-infra10:04
*** yamamoto has joined #openstack-infra10:05
*** yamamoto has quit IRC10:05
*** yamamoto has joined #openstack-infra10:06
*** rbuzatu has quit IRC10:09
*** yanyanhu has quit IRC10:12
*** matrohon has joined #openstack-infra10:17
*** sshnaidm has quit IRC10:20
*** _degorenko|afk has quit IRC10:26
*** cshastri has joined #openstack-infra10:27
*** HeOS has quit IRC10:29
*** degorenko_ has joined #openstack-infra10:33
*** degorenko_ has quit IRC10:33
*** yamamoto has quit IRC10:33
openstackgerritsandhya proposed openstack/diskimage-builder: Add support for building images capable of UEFI
*** yamamoto has joined #openstack-infra10:39
*** sshnaidm has joined #openstack-infra10:40
*** sshnaidm is now known as sshnaidm|afk10:40
*** Ravikiran_K has quit IRC10:40
*** kzaitsev_mb has joined #openstack-infra10:42
bookwarhi, is there a specific irc channel for generic JJB development questions or can i ask here? I wonder what is the status of Pipeline Plugin support in jjb and if anyone working on it10:48
*** dkehn has joined #openstack-infra10:48
*** dkehn_ has joined #openstack-infra10:49
*** yamamoto has quit IRC10:51
*** yamamoto has joined #openstack-infra10:55
*** yamamoto has quit IRC10:55
openstackgerritAleksey Zvyagintsev proposed openstack-infra/project-config: [jenkinss]Add {murano-app}-unittest job
*** amitgandhinz has joined #openstack-infra10:56
*** yamamoto has joined #openstack-infra10:56
*** matrohon has quit IRC10:57
*** yamahata has quit IRC10:58
*** senk__ has joined #openstack-infra10:59
*** sdague has joined #openstack-infra11:00
*** amitgandhinz has quit IRC11:00
*** senk_ has quit IRC11:01
*** yamamoto has quit IRC11:01
*** Sudha has quit IRC11:03
*** yamamoto has joined #openstack-infra11:03
*** yamamoto has quit IRC11:03
*** rbuzatu has joined #openstack-infra11:04
*** yamamoto has joined #openstack-infra11:05
*** pcaruana has quit IRC11:06
*** pcaruana has joined #openstack-infra11:07
*** sdake has joined #openstack-infra11:08
*** sdake has quit IRC11:08
*** sdake has joined #openstack-infra11:08
*** mrmartin has quit IRC11:09
*** permalac has quit IRC11:09
*** yamamoto has quit IRC11:10
*** yamamoto has joined #openstack-infra11:10
*** DaiDV has quit IRC11:11
*** coolsvap is now known as coolsvap_11:12
*** matrohon has joined #openstack-infra11:13
*** thiagolib has joined #openstack-infra11:17
*** lucasagomes is now known as lucas-hungry11:21
*** jkilpatr has joined #openstack-infra11:23
*** e0ne has quit IRC11:23
*** furlongm_ has joined #openstack-infra11:23
*** yamamoto has quit IRC11:26
*** yamamoto has joined #openstack-infra11:26
*** jaosorior has quit IRC11:27
openstackgerritTim Irnich proposed openstack-infra/project-config: Adding project for Gluon fuel plugin.
*** jaosorior has joined #openstack-infra11:28
*** sshnaidm|afk is now known as sshnaidm11:28
*** yolanda has quit IRC11:29
*** matthewbodkin has quit IRC11:30
*** yamamoto has quit IRC11:31
*** yolanda has joined #openstack-infra11:33
*** rodrigods has quit IRC11:33
*** rodrigods has joined #openstack-infra11:33
*** yamamoto has joined #openstack-infra11:34
*** yamamoto has quit IRC11:34
*** sdake has quit IRC11:36
*** ldnunes has joined #openstack-infra11:38
*** sdake has joined #openstack-infra11:39
*** kzaitsev_mb has quit IRC11:39
*** sdake has quit IRC11:41
*** gordc has joined #openstack-infra11:43
*** sdake has joined #openstack-infra11:43
*** jimbaker has quit IRC11:44
*** furlongm_ has quit IRC11:44
*** markvoelker has joined #openstack-infra11:45
*** matrohon has quit IRC11:45
*** yamamoto has joined #openstack-infra11:46
*** yamamoto has quit IRC11:46
*** thorst has joined #openstack-infra11:47
*** thorst has quit IRC11:47
*** jimbaker has joined #openstack-infra11:47
*** jimbaker has joined #openstack-infra11:48
*** thorst has joined #openstack-infra11:48
*** furlongm_ has joined #openstack-infra11:48
*** florianf has joined #openstack-infra11:48
*** florianf has quit IRC11:48
openstackgerritJuan Antonio Osorio Robles proposed openstack/diskimage-builder: TESTING CCAMACHO'S CHANGE
strigaziHi infra, how can I delete a file from our docs that was published in the past but currently is deleted from our repo?11:51
*** cshastri has quit IRC11:51
*** furlongm_ has quit IRC11:51
strigaziis replaced by
AJaegerstrigazi: you cannot ;) I can do it for you...11:52
AJaegerstrigazi: it's a manual step ;(11:52
openstackgerritAleksey Zvyagintsev proposed openstack-infra/project-config: Add {murano-app}-unittest job
strigaziAJaeger: Thanks11:54
*** furlongm_ has joined #openstack-infra11:54
AJaegerstrigazi: there are a few more old files in that directory, can those be removed as well? dev-heat-mesos.html, dev-mesos.html, dev-functional-test.html, tls.html?11:54
*** baoli has joined #openstack-infra11:54
*** binbincong has quit IRC11:55
*** tpsilva has joined #openstack-infra11:56
*** kushal has joined #openstack-infra11:56
*** amitgandhinz has joined #openstack-infra11:56
*** raildo has joined #openstack-infra11:56
strigaziWhatever is not in the tree of our repo should be removed11:57
*** larainema has quit IRC11:57
strigaziall the files you mentioned can be deleted11:57
strigaziAJaeger: ^^11:58
*** larainema has joined #openstack-infra11:59
*** furlongm_ has quit IRC11:59
*** baoli has quit IRC11:59
*** javeriak has quit IRC11:59
*** dprince has joined #openstack-infra12:00
AJaegerstrigazi: deleted the old content12:00
strigaziAJaeger: Thanks a lot12:01
*** amitgandhinz has quit IRC12:01
*** amoralej is now known as amoralej|off12:02
*** matrohon has joined #openstack-infra12:02
*** baoli has joined #openstack-infra12:02
*** rossella_s has quit IRC12:03
*** amoralej|off is now known as amoralej|lunch12:03
*** cshastri has joined #openstack-infra12:04
*** baoli_ has joined #openstack-infra12:04
*** senk has quit IRC12:04
*** senk__ is now known as senk12:04
*** gongysh has quit IRC12:04
*** senk has quit IRC12:05
*** jimbaker has quit IRC12:07
*** baoli has quit IRC12:08
*** senk_ has joined #openstack-infra12:08
zigoianw: Are you still around? Can I bother you with this one?
*** furlongm_ has joined #openstack-infra12:10
*** jimbaker has joined #openstack-infra12:11
*** jimbaker has joined #openstack-infra12:11
*** yamamoto has joined #openstack-infra12:12
*** yamamoto has quit IRC12:12
*** pradk has joined #openstack-infra12:12
*** matthewbodkin has joined #openstack-infra12:13
*** yamamoto has joined #openstack-infra12:13
*** chmouel has joined #openstack-infra12:13
*** yamamoto has quit IRC12:14
*** yamamoto has joined #openstack-infra12:14
*** yamamoto has quit IRC12:15
*** HeOS has joined #openstack-infra12:15
*** rhallisey has joined #openstack-infra12:15
*** gouthamr has joined #openstack-infra12:17
*** lucas-hungry is now known as lucasagomes12:18
*** moravec has quit IRC12:18
*** amitgandhinz has joined #openstack-infra12:19
*** yamamoto has joined #openstack-infra12:20
*** e0ne has joined #openstack-infra12:20
*** furlongm_ has quit IRC12:21
openstackgerritCsaba Henk proposed openstack-infra/project-config: Add jobs for Manila glusterfs, glusterfs-native drivers with Heketi layout
openstackgerritCsaba Henk proposed openstack-infra/project-config: templatize Manila GlusterFS jobs
*** matrohon has quit IRC12:23
*** yamamoto has quit IRC12:25
*** weshay has joined #openstack-infra12:26
openstackgerritMerged openstack-infra/system-config: Add compute043.chocolate metadata for better tracking
openstackgerritMerged openstack-infra/system-config: Update iLO IP and rack for compute010.chocolate
openstackgerritMerged openstack-infra/system-config: Update iLO IP and rack for compute089.chocolate
*** igorn has quit IRC12:30
*** javeriak has joined #openstack-infra12:30
*** amotoki has joined #openstack-infra12:31
e0neAJaeger: thanks for the review, Andreas12:31
*** kzaitsev_mb has joined #openstack-infra12:31
openstackgerritMerged openstack-infra/system-config: Update iLO IP and rack for compute084.chocolate
*** swat30 has joined #openstack-infra12:32
strigaziWe have a document in etherpad that I can't open. It contains the summary of our midcycle. Can someone check, I hope the content is not lost.
openstackgerritMerged openstack-infra/system-config: Update iLO IP and rack for compute013.chocolate
openstackgerritMerged openstack-infra/system-config: Add compute061.chocolate metadata for better tracking
openstackgerritMerged openstack-infra/system-config: Add compute065.chocolate metadata for better tracking
*** psilvad has joined #openstack-infra12:33
openstackgerritMerged openstack-infra/system-config: Add compute067.chocolate metadata for better tracking
openstackgerritMerged openstack-infra/system-config: Add compute063.chocolate metadata for better tracking
*** hichihara has joined #openstack-infra12:34
openstackgerritMerged openstack-infra/system-config: Update iLO IP and rack for compute086.chocolate
*** chem has quit IRC12:34
openstackgerritMerged openstack-infra/system-config: Add compute070.chocolate metadata for better tracking
*** igorn has joined #openstack-infra12:35
openstackgerritMerged openstack-infra/system-config: Add compute028.chocolate metadata for better tracking
openstackgerritMerged openstack-infra/system-config: Update iLO IP and rack for compute012.chocolate
openstackgerritMerged openstack-infra/system-config: Add compute004.chocolate metadata for better tracking
*** amitgandhinz has quit IRC12:37
openstackgerritMerged openstack-infra/system-config: Add compute048.chocolate metadata for better tracking
*** cshastri has quit IRC12:38
*** amitgandhinz has joined #openstack-infra12:38
*** Julien-zte has joined #openstack-infra12:38
*** Ravikiran_K has joined #openstack-infra12:38
*** chem has joined #openstack-infra12:39
*** matrohon has joined #openstack-infra12:39
openstackgerritMerged openstack/os-client-config: Update Internap information
*** javeriak has quit IRC12:45
openstackgerritMerged openstack-infra/system-config: Add compute018.chocolate metadata for better tracking
*** ihrachys has joined #openstack-infra12:46
openstackgerritMerged openstack-infra/system-config: Add documentation on howto consume events from firehose.o.o
strigaziAJaeger: We have a document in etherpad that I can't open. It contains the summary of our midcycle. I hope the content is not lost.
*** rlandy has joined #openstack-infra12:48
openstackgerritMerged openstack-infra/system-config: Handle non-ASCII characters in body and subject
mordredstrigazi: I also cannot open it - it's entirely likely that someone added an evil unicode character by mistake (has happened before) ... I'm looking right now12:49
*** oanson has quit IRC12:52
*** PollyZ has joined #openstack-infra12:52
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: if we fail to send mail, the job should fail
zigomordred: Hi there! The backport repo is looking good, building and all, I'm now waiting for the first POST job to publish stuff. However, I've seen waiting time which makes me think that there's a serious problem in scheduling jobs. I am the only one using the Jessie image, so my jobs should be scheduled pretty fast, though it's taking 30mins + to get check jobs to run.12:53
zigoSomething not right is going on.12:53
zigomordred: ianw wrote here in the channel that probably there's something wrong that was merged into nodepool.12:53
zigomordred: Your thoughts?12:54
AJaegerzigo, check - we'Re using all our images right now.12:54
openstackgerritMerged openstack-infra/system-config: Remove puppet for jeepyb's trivial rebase script
AJaegermorning, pabelanger !12:54
*** rfolco has joined #openstack-infra12:54
zigoAJaeger: Oh, so it's just infra being too busy?12:54
pabelangerzigo: post jobs have lowest priority, so the queue is starved for resources.  It has often taken 24 hours the last few weeks to clear the queue.12:55
zigopabelanger: I'm not even talking about POST jobs, but also check jobs.12:55
pabelangerzigo: right, same applies, it is just slow12:56
AJaegerzigo, right now we'Re at our limit12:56
*** yolanda has quit IRC12:56
zigopabelanger: AJaeger: Is there something planned to fix the situation?12:56
*** amotoki has quit IRC12:56
pabelangerinfracloud will help add more resources12:56
pabelangerbluebox should be coming back online12:56
pabelangerand osic is switching to IPv6, which should be more nodes online12:57
*** furlongm_ has joined #openstack-infra12:57
AJaegerpabelanger: could you check and together, please? sshnaidm was asking for tripleo job reviews and I'm a bit confused on the way forward.12:57
pabelangerzigo: I'd suggest updating your deb-auto-backports jobs to git clone deb-openstack-tools, over using packaging.  That way you don't have to wait for zuul to merge / build it12:58
openstackgerritMerged openstack-infra/project-config: grafana: Correct pipeline of midonet periodic jobs
*** yolanda has joined #openstack-infra12:58
pabelangerAJaeger: sure, I can do it shortly12:58
zigopabelanger: I don't really need to change openstack-pkg-tools so often, so that's fine.12:58
zigopabelanger: FYI, the first POST job of deb-auto-backports will run soon.12:59
zigoLet's hope we get there! :)12:59
openstackgerritMathieu Rohon proposed openstack-infra/project-config: net-bgpvpn : move install and tempest tests to the gate queue
AJaegerpabelanger: not that urgent - just wanted to connect them together for you since looking only at 351120 looks wrong - and note that the two changes beside the title do not restore the status before the RH1->RH2 change AFAIU. So, get some coffee and take your time to review. Thanks!13:00
pabelangerAJaeger: once we land 352086 and 352087, it will need to be updated.13:01
pabelangeryolanda: do you mind helping with 352086?13:01
*** matrohon has quit IRC13:01
*** PollyZ has quit IRC13:02
mordredzigo: I agree with pabelanger ... we have capacity issues for sure13:02
*** mrmartin has joined #openstack-infra13:03
yolandasorry on a meeting, i can take a look in a while13:03
mordredzigo: but super exciting about the first POST job!13:04
zigoSure thing.13:04
sdaguewe're seeing odd network behavior on ovh tempest tests (it reflecting host packets back some times), any idea how we get to the bottom of this?13:04
*** _nadya_ has quit IRC13:05
*** rossella_s has joined #openstack-infra13:06
pabelangerAJaeger: looking at the 2 tripleo reviews, they appear correct. But lets update the image name first, then we can see about reviewing that code again13:06
pabelangeryolanda: np, thanks13:07
pabelangermordred: do you have a moment to help land 352086? Cleaning up image names in nodepool13:07
*** _nadya_ has joined #openstack-infra13:09
AJaegerpabelanger: syntax wise yes. But the tempest job that 351120 readds was before in the experimental-tripleo-jobs template which is used by 6 repos and now gets added to check-tripleo-jobs template which is used by 16 repos. So, that can be the riht decision. But looking at the commit message, I see no reason why it should be the right approach ;) I miss the explanation.13:09
*** mdrabe has joined #openstack-infra13:09
*** amitgandhinz has quit IRC13:09
*** amitgandhinz has joined #openstack-infra13:10
AJaegerpabelanger: if it would have "restored the old behaviour" - adding to experimental template - I would have given +2 but seeing the discussion and the lack of comments, I doubt this change is done intentional this way.13:10
openstackgerritMerged openstack-infra/system-config: Rename attended_upgrades playbook to unattended_upgrades
*** rbuzatu has quit IRC13:11
*** dkehn has quit IRC13:11
*** tonytan4ever has joined #openstack-infra13:12
AJaegermordred: could you +2A 352086 for pabelanger, please?13:12
*** dkehn_ has quit IRC13:12
*** eharney has quit IRC13:13
*** Na3iL has joined #openstack-infra13:13
*** rbuzatu has joined #openstack-infra13:13
*** amotoki has joined #openstack-infra13:13
* mordred looking13:14
AJaegerthanks. pabelanger ^13:15
*** furlongm_ has quit IRC13:15
*** zhoragh has joined #openstack-infra13:16
zhoraghhave trouble with diskimage-builder inside lxc-container13:16
zhoragh2016-08-08 15:12:08,454 INFO + /usr/local/share/diskimage-builder/lib/img-functions:run_d_in_target:85          :   sudo umount -f /opt/dib_t13:16
zhoragh2016-08-08 15:12:08,460 INFO umount2: Permission denied13:16
zhoragh2016-08-08 15:12:08,460 INFO umount: /opt/dib_tmp/dib_build.2sv2ZIqd/mnt/tmp/in_target.d: block devices not permitted on fs13:16
zhoragh2016-08-08 15:12:08,460 INFO umount2: Permission denied13:16
zhoraghany idea?13:16
*** _ari_ has joined #openstack-infra13:16
*** xyang1 has joined #openstack-infra13:16
mordredpabelanger: how's your etherpad content recovery skills?13:17
*** matrohon has joined #openstack-infra13:17
*** dkehn has joined #openstack-infra13:20
*** pgadiya has quit IRC13:20
*** rvasilets___ has joined #openstack-infra13:21
*** furlongm_ has joined #openstack-infra13:21
*** furlongm_ has quit IRC13:22
*** matt-borland has joined #openstack-infra13:23
*** yamamoto has joined #openstack-infra13:24
*** dkehn_ has joined #openstack-infra13:24
anteayabookwar: this is the only channel we have to discuss jenkins job builder issues13:25
openstackgerritJeffrey Zhang proposed openstack-infra/project-config: Enable ubuntu-xenial for Kolla
strigazimordred: Any news on the broken etherpad?
openstackgerritOleh Baran proposed openstack-infra/project-config: Add non-voting unit test coverage job for murano-dashboard
*** zhurong has joined #openstack-infra13:27
*** amotoki has quit IRC13:27
mordredstrigazi: the content is definitely in the DB13:27
mordredstrigazi: so don't lose any sleep13:27
mordredstrigazi: recovering it is a thing I'd rather one of the other roots help with when they return13:27
zigomordred: pabelanger: One question here. Currently, there's a check, a gate and a post job that are all rebuilding packages. Would there be a way to build only once instead of 3 time?13:28
openstackgerritMerged openstack-infra/project-config: Switch to tripleo-centos-7 for tripleo-ci jobs
strigazimordred: thanks13:29
*** nmagnezi_ has quit IRC13:29
*** furlongm_ has joined #openstack-infra13:29
*** yamamoto has quit IRC13:29
mordredzigo: not in the short term - it sohuld be much easier to accomplish once we've rolled out zuul v313:30
*** zhurong has quit IRC13:31
*** masayukig has quit IRC13:31
*** zhurong has joined #openstack-infra13:32
*** eharney has joined #openstack-infra13:32
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Implement TripleO centos-7-nonha-multinode-updates
zigoShit, it's not publishing correctly, my bad, fixing...13:32
*** julim has joined #openstack-infra13:33
mnaserFYI about the etherpad issues13:33
*** Sudha has joined #openstack-infra13:33
*** ayoung has joined #openstack-infra13:34
*** masayukig has joined #openstack-infra13:34
*** matrohon has quit IRC13:35
anteayazigo: can you perhaps not swear in channel, please?13:35
*** esberglu has joined #openstack-infra13:36
*** ayoung has quit IRC13:37
*** amoralej|lunch is now known as amoralej13:37
*** zhoragh1 has joined #openstack-infra13:38
anteayamtreinish: are the mqtt client and the mostiquo client equivalent?13:39
*** jcoufal has joined #openstack-infra13:39
*** zhoragh has quit IRC13:42
*** ayoung has joined #openstack-infra13:43
*** moravec has joined #openstack-infra13:44
*** ccamacho is now known as ccamacho|lunch13:44
*** thorst_ has joined #openstack-infra13:44
*** masayukig has quit IRC13:45
anteayasorry mosquitto13:46
mordredanteaya: we should write something called mostiquo but have it not be related to mqtt13:46
*** masayukig has joined #openstack-infra13:46
anteayathat sounds like it will create hours of fun13:47
*** thorst has quit IRC13:48
anteayaso far on the firehose I belive I am seeing gerrit events and zuul events13:49
sdaguemnaser: you are with ovh, right?13:50
mtreinishanteaya: the only thing reporting to firehose is gerrit right now13:50
anteayasdague: vexxhost13:50
sdagueoh, damn13:50
sdagueanteaya: do we have ovh people in here regularly?13:50
anteayamtreinish: ah, then the status's I am seeing must also be coming from gerrit13:51
anteayasdague: if we do I don't have the cloud to nick mapping in my head13:51
anteayamordred: ^^13:51
sdaguebecause right now ovh's odd networking is preventing changing to neutron by default in devstack13:51
mordredsdague: yah - same as anteaya ... I think jean-daniel lurks in here13:51
*** matrohon has joined #openstack-infra13:51
mtreinishanteaya: the mosquitto clients are an implementation of a an mqtt client. mqtt is just a protocol specification and doesn't actually include an implementation13:51
mordredbut I do not know nick13:51
mordredttx: ^^ ?13:51
anteayamordred: jean-daniel, that's the person, yeah13:51
mtreinishso there are a bunch of projects that implement the same thing13:51
anteayayolanda: might now13:52
*** mriedem has joined #openstack-infra13:52
anteayamtreinish: ah thank you13:52
anteayamtreinish: cool, the debian mosquitto-clients seem to be flooding my terminal adequately13:52
mtreinishanteaya: I do have a WIP patche up to add logstash and also another to add gerritbot as a consumer: and
mordredsdague: woot on the email13:53
mordredsdague: otoh - that config doesn't work in ovh?13:53
sdagueovh is reflecting outbound packets back in13:54
mordredsdague: that seems odd13:54
matt-borlandHello...Asking for a second review of ; has one +2 from AJaeger.  I appreciate it, let me know if there are questions!  Thanks in advance.13:54
sdaguewhich causes a tempest test to fail because the it ends up making 2 guests talk to each other which normally shouldn't be able to13:54
sdaguemordred: agreed, if you read the thread, it's all massively confusing to everyone involved13:55
mtreinishthe only issue with adding other publishers right now is that mqtt without tls pushes passwords in plaintext on the wire, so we need to add encryption before let remote servers auth against the server13:55
anteayamtreinish: thank you13:55
matt-borlandThat patch I listed is a Horizon patch to create general integration jobs, two from a template rather than a single test job.13:55
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Change ipmi_username for all servers to HELIONADMIN
anteayamtreinish: ah yes, plaintext passwords are not good13:58
anteayamtreinish: do we have a patch up to add encryption?13:58
*** coolsvap_ is now known as coolsvap13:58
anteayathere it is 34803013:59
*** rbuzatu_ has joined #openstack-infra13:59
mtreinishanteaya: sorta: and
Zarais anyone able to take a look at ? it updates the actions for review-dev so that they do the same things that are planned for review.o.o, re: gerrit-storyboard plugin. that way we can check the behaviour before we merge gerrit-storyboard integration for the production systems.13:59
mtreinishbut I think I'll have to adjust how that all works, because I think the current approach there is a no-go13:59
*** Liuqing has joined #openstack-infra14:00
mtreinishanteaya: I'm thinking it might be better to store the certs and key contents in hiera and have puppet-mosquitto (and whatever publishers) manage the files14:00
*** rbrndt has joined #openstack-infra14:01
mtreinishbut I don't have a lot of experience doing tls stuff in the infra setup14:01
anteayawell storing secret things in hiera is kind of the established workflow, yeah14:01
anteayaI don't think you can go wrong if you take the heira road14:01
* anteaya shows herself to the door14:01
*** roxanaghe has joined #openstack-infra14:02
*** xarses_ has quit IRC14:02
*** rbuzatu has quit IRC14:02
*** elarson has joined #openstack-infra14:03
*** e0ne has quit IRC14:03
mordredmtreinish: yah - heira road. it's not terribly bad. basically, you make puppet-mosquitto take a parameter for each secret thing14:04
*** baoli_ has quit IRC14:04
mordredmtreinish: then in the top-level site.pp, when firehose.o.o is making a puppet-mosquitto, it'll do something like ::mosquitto(cert=hiera('firehose_cert'))14:05
*** matrohon has quit IRC14:06
*** roxanaghe has quit IRC14:06
*** Na3iL has quit IRC14:07
mtreinishmordred: ok, yeah I did that for the ssh keys. I guess it doesn't solve the cert generation, but I can leave that to you :)14:08
*** esker has joined #openstack-infra14:10
*** ajmiller has joined #openstack-infra14:10
*** woodster_ has joined #openstack-infra14:11
openstackgerritZara proposed openstack-infra/storyboard-webclient: Search for boards and worklists
*** zz_dimtruck is now known as dimtruck14:15
*** e0ne has joined #openstack-infra14:15
*** PollyZ has joined #openstack-infra14:16
*** spzala has joined #openstack-infra14:16
*** TheJulia has quit IRC14:16
*** mtanino has joined #openstack-infra14:18
*** permalac has joined #openstack-infra14:21
openstackgerritMonty Taylor proposed openstack-infra/shade: Allow creating a floating ip on an arbitrary port
*** matrohon has joined #openstack-infra14:22
mordredmtreinish: yah - fungi is good at making certs14:22
mordredShrews: ^^ now with tests14:23
fungimtreinish: yeah, i just need to buy and expense them. i can do that later today14:23
nibalizerfungi is the certlord14:24
*** matrohon has quit IRC14:24
fungii'd like to get the foundation to set up a startcom account so we can just regenerate certs as needed under a flat annual fee14:24
fungier, startssl14:25
jrollletsencrypt is now trusted directly in firefox :)14:25
* jroll doesn't remember what the argument against that was, but wanted to point it out14:25
fungiletsencrypt is not yet trusted directly by systems administrators14:25
sdaguemordred / fungi - where would I register a bug for this ovh behavior, as we're going to probably need to skip a tempest test temporarily until we get a work around14:26
fungii don't like the idea of some process periodically replacing certs and restarting services causing unanticipated breakage14:26
*** Sudha has quit IRC14:26
mordredfungi: why would a process periodically replace certs and restart services?14:26
mordredfungi: is there a letsencrypt SERVICE?14:27
nibalizermordred: yea its a bit wack14:27
fungimordred: tey call it a "client" but yes14:27
openstackgerritAndrey Nikitin proposed openstack-infra/project-config: Add one new Fuel's plugin
nibalizerit phones home and gets a new cert every 60 days14:27
fungiand letsencrypt forces a max 90 day expiration, so we'd end up replacing our certs a lot more often (right now we renew them every two years)14:27
*** beekneemech is now known as bnemec14:27
mordredI mean, I'm thrilled it makes it easier for other folks to add certs to things14:27
nibalizerit has bad and good parts14:27
mordredthat's a great benefit to the world14:27
pabelangermordred: unknown, I haven't done much work with etherpad14:27
mordredpabelanger: kk.14:27
nibalizershorter certs is also good in theory, its just a burden too14:28
mordredfungi: you konw how to fix borked etherpads, right?14:28
*** e0ne has quit IRC14:28
mordrednibalizer: I have no problem with frequently replacing certs ... I just want that to be under admin control :)14:28
*** e0ne has joined #openstack-infra14:28
nibalizermordred: you manage the cron that does it :P14:29
*** vponomaryov1 is now known as vponomaryov14:29
nibalizeri mean we already have puppet running regularly and swaping certs and restarting services and such14:29
*** zhoragh1 has left #openstack-infra14:29
fungimordred: the only solution i have is you manually construct the url to the history timeslider and then you can copy the content out of there (or can try to export it)14:29
nibalizerthat doesn't go out to a public service though, which is nice14:29
mordredfungi: ah - I thought there was an "edit the db" route14:30
funginibalizer: we don't have puppet restarting those services when we update the certs in hiera, and also the cert updating in hiera is a manual process so we're under control of when it gets done14:30
AJaegerfungi, sorry to ping again. I'm on vacation next week and would love to finish the constraints work this week in time for Newton. Could you review - especially 348764 and 349181, please?14:30
fungimordred: none i've had much luck with14:30
mordredfungi: nod14:30
nibalizerfungi: oh im just bringing up things, not trying to apples to apples compare14:31
fungiAJaeger: i have that query up in gertty right now. just trying to get to the point where i can do code review today. friday ended up being a whole day of me just fixing broken things14:31
nibalizerand I think we do have puppet restarting our httpd's, just not our more 'stateful' services14:31
fungisdague: we have an openstack-gate project in lp for that14:31
*** mrmartin has quit IRC14:31
fungisdague: (mostly there so we have a place for elastic-recheck to reference bugs about transient behaviors in systems)14:32
*** Sudha has joined #openstack-infra14:33
*** mrmartin has joined #openstack-infra14:33
AJaegerthanks, fungi.14:33
funginibalizer: pretty sure the service restarts aren't subscribed to the ssl key/cert files, but it's possible we do that in some places14:33
funginibalizer: we do it for configuration changes, yes14:33
fungibut even those are triggered by us merging some change, not by a timer14:34
*** amoralej is now known as amoralej|brb14:34
fungiwell, okay, ansible is running on a timer, so technically it is, but you know what i mean14:35
*** psachin has quit IRC14:35
*** Teresita-Warrior has joined #openstack-infra14:35
jrollmordred: updating LE certs is totally admin-initiated, it's just that the recommendation is to cron it (so you don't forget to renew)14:35
*** _ari_ has quit IRC14:35
jrollnothing happens without you running the client to generate a cert and then doing something with it14:36
fungisdague: do you have a more detailed writeup of the behavior besides "reflecting outbound packets back in"? i could try to flex the long-dormant network engineering parts of my brain14:36
*** PollyZ_ has joined #openstack-infra14:36
*** _ari_ has joined #openstack-infra14:36
openstackgerritAndrey Nikitin proposed openstack-infra/project-config: Add one new Fuel's plugin
fungijroll: yep, the cron is recommended so that you don't have to perform a manual operation every couple months because they don't let you do traditionally long cert expirations14:36
*** thiagop has joined #openstack-infra14:37
*** SudhaKR has joined #openstack-infra14:37
*** roxanaghe has joined #openstack-infra14:37
*** ccamacho|lunch is now known as ccamacho14:38
fungiso my point was that letsencrypt gives you the choice of either having your certs maintained through periodic automation or renewing certs ~10x as often14:38
*** Sudha has quit IRC14:38
*** xarses_ has joined #openstack-infra14:38
*** PollyZ has quit IRC14:38
fungisdague: oh, ick. we should have some sort of egress filtering to prevent our workers from leaking packets sourced from or destined to rfc-1918 address space14:40
nibalizeralso as I page back in the infra- stuff, um we have a message queue now?14:40
*** matrohon has joined #openstack-infra14:40
anteayaby message queue do you mean
funginibalizer: and experimental one, yes.
anteayayes that is emmiting gerrit events currently14:41
*** markvoelker has quit IRC14:41
fungier, an experimental one14:41
anteayathat should have been emitting14:42
anteayanibalizer: the docs patch on clients is merged but not published yet:
mordredstrigazi: ... your content is there - something about revision 999 borks things14:43
mordredstrigazi: I recommend going and grabbing your content from that url and putting it somewhere14:44
*** nherciu_ has quit IRC14:44
*** thorst has joined #openstack-infra14:45
fungiwe've seen a rash of etherpad page breaks lately (something like one a month that have come to my attention) but haven't updated teh version od etherpad we're running. i have to wonder if there's some browser plugin or something causing broken content to get injected under rare circumstances14:45
*** mdrabe has quit IRC14:46
*** psachin has joined #openstack-infra14:46
anteayawell a long time ago there was a browser plugin on windows that kept adding blank lines to the bottom of an open etherpad14:47
*** TheJulia has joined #openstack-infra14:47
anteayaI think that was HenryG's and we discovered it at the neutron tempest sprint in montreal14:47
anteayaso yeah, I think it is possible14:47
*** thorst_ has quit IRC14:47
anteayathe maintainer of the browser plugin eventually fixed the plugin, which was nice14:48
*** asettle has joined #openstack-infra14:48
openstackgerritMatthew Treinish proposed openstack-infra/puppet-mosquitto: Add support for configuring tls ports
*** burgerk has joined #openstack-infra14:50
anteayamtreinish: I think puppet keys with blank default values are '' instead of undef14:51
anteayahowever don't ever take my word for any puppet reviews14:51
*** SudhaKR has quit IRC14:51
*** mhickey has quit IRC14:52
*** mdrabe has joined #openstack-infra14:54
*** markvoelker has joined #openstack-infra14:54
fungiundef has a special behavior when used as a default, in that it can't override another default. sometimes that's a useful property14:54
anteayaah thank you14:54
anteayaI don't know which one is best in mtreinish's patch14:55
*** jaosorior has quit IRC14:56
mrmartinfungi: the puppet-askbot akismet patch looks ok now, I've fixed the syntax issue.14:58
fungii think there was a somewhat recent push to start switching our empty value defaults to undef for ease of debugging, but also puppet 4.x changed some nehaviors around empty strings that have made that more complicated14:58
anteayaah this I did not know14:58
openstackgerritMatthew Treinish proposed openstack-infra/puppet-mosquitto: Add support for configuring tls ports
fungimrmartin: oh, thanks! i only had enough time to skim the job log and notice my typos, but not enough time to submit a new patchset14:59
anteayacrinkle: have you any thoughts on the best way forward on mtreinish's patch,
*** amotoki has joined #openstack-infra14:59
*** e0ne has quit IRC14:59
anteayacrinkle: I have no idea if it should be '' or undef14:59
*** amotoki has quit IRC14:59
*** karthikp_ has joined #openstack-infra15:00
*** e0ne has joined #openstack-infra15:00
*** kragniz has quit IRC15:00
*** amotoki has joined #openstack-infra15:01
*** bin_ has joined #openstack-infra15:01
crinkleanteaya: it should be undef15:01
anteayacrinkle: thank you15:01
anteayamtreinish: you got it right, yay!15:02
mordredrcarrillocruz: we still havne't added any support to shade for magnum bays, right?15:02
rcarrillocruzniet, it's on my queue15:02
mordredrcarrillocruz: I ask - because apparently magnum is renaming bay to cluster15:02
*** psachin has quit IRC15:02
mordredwhich I like much better15:02
anteayacrinkle: do you have time to expand on that? I'm fine taking your word but if there is more, I will listen15:03
rcarrillocruzok, i'm glad i didn't get back to it then15:03
rcarrillocruzthx for the heads-up mordred15:03
mordredrcarrillocruz: yah. we should sync up with their new terminology and make shade's api use it - even if magnum doens't yet :)15:03
openstackgerritMatthew Treinish proposed openstack-infra/system-config: Add tls support to firehose
*** mriedem1 has joined #openstack-infra15:04
mtreinishanteaya: cool15:04
crinkleanteaya: the truthiness of the empty string changes between puppet 3 and puppet 4 so we have lint checks in most of the puppet modules to avoid using it entirely to avoid confusion15:05
*** karthikp_ has quit IRC15:05
anteayacrinkle: wow15:05
*** mriedem has quit IRC15:05
anteayacrinkle: okay thank you15:05
anteayacrinkle: have you a link for the empty string truthiness change documentation?15:06
mordredcrinkle: so tl;dr - empty string bad?15:06
*** amitgandhinz has quit IRC15:06
*** amitgandhinz has joined #openstack-infra15:06
anteayamordred: that is the way I'm taking it15:07
*** amoralej|brb is now known as amoralej15:07
crinkleanteaya: this is where most of our lint changes came from
*** baoli has joined #openstack-infra15:07
*** mriedem1 is now known as mriedem15:07
anteayacrinkle: thank you15:07
*** hurgleburgler has joined #openstack-infra15:08
*** amotoki has quit IRC15:08
crinklemordred: in practice it doesn't really matter that much as long as you're using `if $foo == '' {}` and not `if $foo {}`15:08
crinklewhich we usually do15:08
crinklebut the lint check isn't smart enough to find those15:08
*** matrohon has quit IRC15:10
anteayathanks crinkle15:11
anteaya<-- offline for a bit15:11
*** narayrak has quit IRC15:12
*** dprince has quit IRC15:14
*** amotoki has joined #openstack-infra15:14
*** amitgandhinz has quit IRC15:15
*** amitgandhinz has joined #openstack-infra15:16
*** amotoki has quit IRC15:16
*** amotoki has joined #openstack-infra15:16
*** zhurong has quit IRC15:18
*** nherciu has joined #openstack-infra15:18
*** rbuzatu_ has quit IRC15:18
*** amotoki_ has joined #openstack-infra15:20
*** amotoki has quit IRC15:21
*** amotoki_ has quit IRC15:22
*** vincentll has quit IRC15:22
*** rbuzatu has joined #openstack-infra15:23
*** matrohon has joined #openstack-infra15:27
openstackgerritMonty Taylor proposed openstack-infra/shade: Add a 'meta' passthrough parameter for glance images
*** cody-somerville has joined #openstack-infra15:30
*** yamahata has joined #openstack-infra15:32
*** mikelk has quit IRC15:32
*** matrohon has quit IRC15:32
pabelangerzigo: looks like we need to make reprepro import support multiple packages15:33
pabelangerzigo: working on patch now15:33
*** csomerville has joined #openstack-infra15:34
*** karthikp_ has joined #openstack-infra15:34
strigazimordred: Thanks, I copied  the content. In day one, in the last line, drinks with team someone had put a beer emoticon. Either Tom or Drago15:34
*** armax has joined #openstack-infra15:35
*** hichihara has quit IRC15:37
*** cody-somerville has quit IRC15:37
*** baoli has quit IRC15:38
*** baoli has joined #openstack-infra15:39
*** harlowja_at_home has joined #openstack-infra15:42
*** Liuqing has quit IRC15:43
*** thorst is now known as thorst_afk15:44
*** Ravikiran_K has quit IRC15:45
*** jlanoux has quit IRC15:45
openstackgerritPaul Belanger proposed openstack-infra/project-config: Update reprepro-import build to support multiple files
pabelangerzigo: mordred: ^first fix15:46
pabelangerI spotted another15:46
rcarrillocruzthey changed the nics cabling again15:46
rcarrillocruzeth0 is now eth215:47
rcarrillocruzwhich means15:47
rcarrillocruzmost likely15:47
rcarrillocruzthe macs we provision on our inventory will have to change15:47
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Only error for duplicate requirements when strict
*** matrohon has joined #openstack-infra15:49
mordredrcarrillocruz: sigh indeed15:49
crinklercarrillocruz: the mellanox nics don't have physical links so that eth0/eth1/eth2 stuff can't be changed by recabling afaik15:49
*** edmondsw has quit IRC15:50
*** Sukhdev has joined #openstack-infra15:50
*** weshay is now known as weshay_mtg15:55
*** amitgandhinz has quit IRC15:56
*** amitgandhinz has joined #openstack-infra15:56
zarobookwar: ask jjb questions here.  what's wrong with pipeline plugin?15:57
*** inc0 has joined #openstack-infra15:58
*** hemanthm has left #openstack-infra15:59
*** PollyZ_ has quit IRC15:59
*** amotoki has joined #openstack-infra15:59
*** matrohon has quit IRC15:59
*** karthikp_ has quit IRC15:59
*** amitgandhinz has quit IRC16:00
prometheanfireAJaeger: do you have another updated requirements update fail I can look at?16:00
prometheanfireAJaeger: freezer was on the list but it seems like they are working
*** amitgandhinz has joined #openstack-infra16:00
*** karthikp_ has joined #openstack-infra16:01
*** sshnaidm has quit IRC16:01
*** rcernin has quit IRC16:01
openstackgerritZara proposed openstack-infra/storyboard-webclient: WIP 'All Boards and Worklists' Page
*** amotoki has quit IRC16:02
*** amotoki has joined #openstack-infra16:02
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova unshelve race bug 1611008
openstackbug 1611008 in OpenStack Compute (nova) "ServersNegativeTestJSON.test_suspend_server_invalid_state fails with "NovaException: Unable to find host for Instance"" [Undecided,Confirmed]
AJaegerprometheanfire: yeah, monasco seems fine now!16:06
prometheanfireAJaeger: you have the review url?16:06
*** kzaitsev_mb has quit IRC16:06
AJaegerprometheanfire: I took the top-most change from the post queue in
*** piet has joined #openstack-infra16:07
prometheanfirek, thanks16:07
*** kzaitsev_mb has joined #openstack-infra16:08
AJaegerprometheanfire: line would have given us that information as well ;)16:08
AJaegerand the log file has the SHA1 in it - 6c3b431...16:08
openstackgerritZara proposed openstack-infra/storyboard-webclient: 'All Boards and Worklists' Page
prometheanfiresuppose so :P16:08
*** hichihara has joined #openstack-infra16:08
AJaegerlog URL I mean16:08
*** e0ne has quit IRC16:09
*** pcaruana has quit IRC16:11
openstackgerritMonty Taylor proposed openstack-infra/shade: Infer nova-net security groups better
*** szaher has joined #openstack-infra16:11
prometheanfireszaher: hi :D16:11
*** Ravikiran_K has joined #openstack-infra16:11
*** PollyZ has joined #openstack-infra16:11
*** permalac has quit IRC16:12
*** dimtruck is now known as zz_dimtruck16:12
*** adrian_otto has joined #openstack-infra16:12
mordredcrinkle: "the mellanox nics don't have physical links" is a set of words which confuse me - does one not plug cables into mellanox cards?16:12
*** Apoorva has joined #openstack-infra16:12
*** sshnaidm has joined #openstack-infra16:13
*** hichihara has quit IRC16:14
*** amotoki has quit IRC16:15
*** rvasilets___ has quit IRC16:17
*** _nadya_ has quit IRC16:18
*** xenogear_ has joined #openstack-infra16:19
szaherprometheanfire: Hi :D16:19
szaherprometheanfire: I think m3m0 is going to take of it :)16:20
prometheanfireyep, looks like it16:21
crinklemordred: er i meant they don't have more than one16:21
mordredcrinkle: phew. I understand better now :)16:22
crinklewords are hard16:23
*** ifarkas is now known as ifarkas_afk16:24
AJaegerbnemec, EmilienM, sshnaidm, pabelanger : I see now four reviews changing tripleo - and it seems some of these are in conflict or do the same way: and . Could you together come up with a plan on how to move forward, please?16:28
*** doug-fish has joined #openstack-infra16:29
*** harlowja_at_home has quit IRC16:29
* bnemec looks16:29
*** david-lyle_ has joined #openstack-infra16:29
AJaegerIt might be that all four need to go in - but I see some overlap at least in the descriptions...16:30
AJaegerbnemec: thanks16:30
EmilienMAJaeger: looking16:30
*** baoli has quit IRC16:30
bnemecYeah, there's at least some overlap between mine and Derek's.16:31
*** baoli has joined #openstack-infra16:31
EmilienMAJaeger: is something we agreed on ML16:31
*** asettle has quit IRC16:31
AJaegerEmilienM: bnemec's change uses the old name...16:32
*** asettle has joined #openstack-infra16:32
*** rbuzatu has quit IRC16:32
openstackgerritMarton Kiss proposed openstack-infra/groups: Custom membership fields and reporting
bnemecYeah, I should probably rebase that on EmilienM's.16:32
*** Teresita-Warrior has quit IRC16:32
*** david-lyle has quit IRC16:33
EmilienMbnemec: thank you16:33
sshnaidmAJaeger, bnemec  my patch ( is only about returning tempest job to experimental pipeline, no objection to have it included in other patch (e.g. of bnemec), but nobody duplicates it as far as I can see16:33
*** gyee has joined #openstack-infra16:33
AJaegerI'm happy to abandon Derek's for you if that's the best way forward - please review as a team those four changes and tell the way forward. I have to do some cooking now, will be back later and read backscroll.16:33
*** tonytan4ever has quit IRC16:34
*** tonytan4ever has joined #openstack-infra16:34
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add option to ensure get_test_runs_by_test_id() is ordered by date
*** rbuzatu has joined #openstack-infra16:35
*** Teresita-Warrior has joined #openstack-infra16:35
*** mrmartin has quit IRC16:36
*** zz_dimtruck is now known as dimtruck16:36
*** tonytan_brb has joined #openstack-infra16:37
bnemecsshnaidm: Agreed.  I left +1 on that one.16:38
*** Teresita-Warrior has quit IRC16:38
*** tonytan4ever has quit IRC16:38
*** Goneri has quit IRC16:40
*** Na3iL has joined #openstack-infra16:40
*** Teresita-Warrior has joined #openstack-infra16:40
*** thorst_a_ has joined #openstack-infra16:40
*** roxanaghe has quit IRC16:40
*** roxanaghe has joined #openstack-infra16:41
*** thorst_afk has quit IRC16:41
*** Teresita-Warrior has quit IRC16:41
*** Swami has joined #openstack-infra16:42
*** Teresita-Warrior has joined #openstack-infra16:43
*** Teresita-Warrior has quit IRC16:44
*** Teresita-Warrior has joined #openstack-infra16:45
*** Teresita-Warrior has quit IRC16:45
*** kzaitsev_mb has quit IRC16:46
bnemecAJaeger: Here's what it looks like to me: sshnaidm and EmilienM's patches are fine as-is.  I need to rebase my patch on EmilienM's, and pull in a few other things from Derek's patch.16:46
*** amoralej is now known as amoralej|off16:47
bnemecDerek is out this week, so I think the easiest thing would be to just switch to my patch for re-enabling the jobs.16:47
bnemecAlso, I really don't want to enable all three at once right off the bat.16:47
openstackgerritRoopak Parikh proposed openstack-infra/project-config: I have updated the review request with suggestions from Andreas, can someone review it again.
*** e0ne has joined #openstack-infra16:49
*** e0ne has quit IRC16:49
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Remove useless Puppet OpenStack  experimental jobs
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Remove useless Puppet OpenStack experimental jobs
*** jpich has quit IRC16:51
openstackgerritMerged openstack-infra/storyboard: Fix /v1/worklists/:id/items endpoint
*** matthewbodkin has quit IRC16:53
*** Sukhdev has quit IRC16:55
*** Sukhdev has joined #openstack-infra16:55
*** inc0 has quit IRC16:55
*** mrmartin has joined #openstack-infra16:55
*** tesseract- has quit IRC16:57
*** mat128|weekend is now known as mat12816:57
*** tphummel has joined #openstack-infra16:58
*** esker has quit IRC16:58
openstackgerritBen Nemec proposed openstack-infra/project-config: Re-enable all three jobs for TripleO
*** wfoster has quit IRC16:59
openstackgerritBen Nemec proposed openstack-infra/project-config: Move tripleo updates job back to check jobs
openstackgerritMerged openstack-infra/system-config: Change ipmi_username for all servers to HELIONADMIN
*** Sukhdev has quit IRC17:03
*** ganso has joined #openstack-infra17:04
openstackgerritMike Fedosin proposed openstack-infra/project-config: Glare
*** asettle has quit IRC17:05
*** javeriak has joined #openstack-infra17:06
*** esker has joined #openstack-infra17:08
*** zul_ has joined #openstack-infra17:09
*** e0ne has joined #openstack-infra17:09
*** zul has quit IRC17:09
*** javeriak_ has joined #openstack-infra17:10
*** javeriak has quit IRC17:11
openstackgerritPaul Belanger proposed openstack-infra/project-config: Fix issue with reprepro-import distribution
pabelangerzigo: mordred: another fix for autobackports ^17:13
*** esker has quit IRC17:13
*** sdake_ has joined #openstack-infra17:13
openstackgerritPaul Belanger proposed openstack-infra/project-config: Fix issue with reprepro-import distribution
*** sdake has quit IRC17:15
*** senk has joined #openstack-infra17:15
senksomeone here who can tell me how i can make my full name appear here: ?17:16
*** daemontool has joined #openstack-infra17:16
pabelangersenk: you can update your information in
*** ganesan has joined #openstack-infra17:17
senkah okay :) i'll try that17:18
*** asettle has joined #openstack-infra17:19
*** wfoster has joined #openstack-infra17:20
*** yamamoto has joined #openstack-infra17:21
*** BobBall is now known as BobBall_AWOL17:21
*** nmagnezi has joined #openstack-infra17:22
*** asettle has quit IRC17:24
ganesanI am building 3rd party CI system, I was able to proceed up to "Run masterless Puppet". But struck while starting zull service17:25
ganesanzuul service is not starting17:25
*** yamamoto has quit IRC17:27
*** thorst_a_ is now known as thorst17:27
*** karthikp_ has quit IRC17:27
*** sputnik13 has joined #openstack-infra17:28
*** Sukhdev has joined #openstack-infra17:28
*** mrmartin has quit IRC17:28
*** edmondsw has joined #openstack-infra17:28
fungiganesan: the zuul module for puppet doesn't automatically start the service since it can be dangerous to have it start when you're not ready to keep an eye on it17:29
fungiganesan: you'll want to `sudo service zuul start` (and probably also `sudo service zuul-merger start`)17:30
ganesanI did that, only zuul-merger is running17:30
*** Sukhdev has quit IRC17:30
ganesanzuul service is still not running17:30
*** thorst_ has joined #openstack-infra17:30
*** thorst has quit IRC17:32
fungiganesan: make sure there's not a stale /var/run/zuul/zuul.pid17:32
ganesanthere is no pid file create17:32
fungiganesan: also, check /var/log/zuul/debug.log for any clues as to why it's not starting17:32
ganesanonly merger.log and merger-debug.log17:33
ganesanis available17:33
*** asettle has joined #openstack-infra17:33
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Handle xenial/trusty and -nv jobs
fungiganesan: and you say you got through this step without any errors?17:35
AJaegerproject-config cores, we missed one case with the xenial/trusty switch, could you review the change above to handle -nv jobs as well, please? ^
AJaegerbnemec: So, should we abandon Derek's ?17:35
*** pt_15 has joined #openstack-infra17:36
ganesanfungi : yes17:36
bnemecAJaeger: That would be my preference.  It's also blocked because it comes after another patch that was conflicted.17:36
*** ihrachys has quit IRC17:37
bnemecApparently we're all bad about looking at Derek's patches before pushing others. :-(17:37
*** asettle has quit IRC17:37
*** thorst_ has quit IRC17:38
*** thorst has joined #openstack-infra17:39
harlowjaquestion for any jenkins job builder folks17:40
AJaegerpabelanger: Just gave a +2 on - if you want to +2A later...17:40
harlowjahow is it commonly done to run `jenkins-jobs -l DEBUG --conf ./jenkins_jobs.ini -u $USER -p $JENKINS_TOKEN  update` safely17:40
harlowja(or equivalent)17:40
harlowjabasically the command to update a live jenkins17:40
harlowjais there a sequence the folks here used to ensure that active jobs collasced (and were idle)17:41
fungiganesan: i'm not sure what to suggest next as i haven't tried running an all-in-one zuul+nodepool+jenkins server, but maybe asselin has some ideas17:41
*** Nakato has quit IRC17:41
fungiharlowja: the job configuration changes don't take effect for jobs which are already running afaik17:42
*** david-lyle_ has quit IRC17:42
*** Nakato has joined #openstack-infra17:42
fungiharlowja: there are a lot of people just triggering this exec whenever job configuration is changed
fungiharlowja: back when we were using jenkins, we never did anything to quiesce the jobs before updating configs anyway17:44
*** bin_ has quit IRC17:44
harlowjafungi kk17:44
harlowjafungi thx, good to know17:44
harlowjai'll probably use something similar then17:45
AJaegerbnemec, EmilienM, sshnaidm, pabelanger : Reviewed all tripleo changes, looks fine from my side now. Thanks for helping to sort it out. Now let's wait for jobs to pass...17:45
*** david-lyle has joined #openstack-infra17:45
*** PollyZ has quit IRC17:46
*** asettle has joined #openstack-infra17:47
*** e0ne has quit IRC17:47
*** acoles is now known as acoles_17:47
ganesanfungi: thanks17:48
openstackgerritKhai Do proposed openstack-infra/system-config: Upgrade gerrit on review-dev.o.o
harlowjafungi  back on that aspect of pipeline plugin and such17:50
harlowja (this thing)17:50
harlowjahas anyone produced a jjb (class?) for that17:50
fungii thought i saw someone talking about it in scrollback yesterday/earlier today17:50
*** PollyZ has joined #openstack-infra17:51
fungiharlowja: grep says bookwar asked about it in here today at 10:48:04 utc17:51
harlowjau are my personal grep, thx :-P17:52
*** asettle has quit IRC17:52
*** karthikp_ has joined #openstack-infra17:53
*** javeriak_ has quit IRC17:53
mordredharlowja: I mean, infra isn't particularly interested in the pipeline plugin, and it's worth noting that going down that road likely will make it harder for you to use zuul v3 in the future ... but if it does the thing you want I agree with fungi, people seem interested in adding jjb support for it17:54
harlowjaya, mordred i get that, though zuul v3 vs jenkins is a battle i don't really care to fight :-P17:54
openstackgerritKhai Do proposed openstack-infra/system-config: Upgrade gerrit on review.o.o
*** sdake has joined #openstack-infra17:55
mordredharlowja: sure. I tihnk of it more as "groovy vs. ansible" though, if that's a better positioning :)17:55
EmilienMAJaeger: thanks a lot17:55
*** mriedem has quit IRC17:55
*** tosky has quit IRC17:55
harlowjamordred fair, though the issue i think is more around people know jenkins, not many outside of openstack know zuul, thus jenkins is what companies have built (or are building) on17:56
harlowjaand changing what companies have built or are building on, especially at a fundamental level like this, is ummm, non trival :-P17:56
* mordred looks at systemd and disagrees17:56
*** mriedem has joined #openstack-infra17:56
*** sdake_ has quit IRC17:57
harlowjai mean, mordred obviously u will change the world, but the current state, ya... seems to lean (from what i've seen) to jenkins still17:57
mordredapparently one company can decide to change out the most fundamental underpinning of 40+ years of IT and it can get rolled out to every distro within three years with no remediation or end-user choice in the matter ... so I have a hunch that people choosing to use zuul instead of jenkins isn't THAT hard17:57
harlowjaya, mindshare is part of that17:58
*** mrmartin has joined #openstack-infra17:58
harlowjazuul needs to get more out of the 'openstack world' to do what u say imho17:58
*** pvaneck has joined #openstack-infra17:58
mordredworking on it17:58
*** mriedem1 has joined #openstack-infra17:58
mordredbut yeah - like I said, if jenkins pipelines make you happy in some way, awesome. they're not useful for us, but it turns out the world has differnet people with different needs17:58
mordredand monocultures are not good for anyone17:58
harlowjasorta happy?17:59
harlowjanot like exteremly17:59
*** karthikp_ has quit IRC17:59
mordredmore like happy in a stockholm syndrome kind of way? :)17:59
*** karthikp_ has joined #openstack-infra17:59
fungifwiw, most of the stuff developed under the openstack infra team umbrella is not meant to be openstack-community-specific18:00
mordredwell, hopefully one day we'll help you break free of your captors - but until then, hopefully the jjb folks can provide you with pleasure18:00
harlowjafungi  i totally get that :)18:00
harlowjafungi it just needs more PR to make that known18:00
harlowjaPR or something18:00
*** mriedem has quit IRC18:01
*** javeriak has joined #openstack-infra18:01
fungiyep, we're working on that too18:02
*** Sina_Maleki has joined #openstack-infra18:04
*** julim has quit IRC18:04
Sina_MalekiHi friends , i have 8 hp servers for testing openstack via canonical and maas , so which switch model if cisco do you offer ? Thanks18:05
harlowjafungi go do some press releases or something :-P18:05
*** doug-fish has quit IRC18:05
harlowjaor i guess that's mordred job, idk18:05
fungiSina_Maleki: i think you're probably in the wrong channel. who are you looking for?18:05
*** julim has joined #openstack-infra18:05
Sina_MalekiThis chanel is not about openstack ? Installation and deploying ? Hardware requirements?18:06
fungiSina_Maleki: this is the channel where we run the community infrastructure for the openstack project, yes18:06
fungiSina_Maleki: it's not a channel about installing and deploying openstack18:07
fungiSina_Maleki: but also as far as i know "canonical maas" is not official openstack software either, so you probably need to find canonical's support group18:07
*** dprince has joined #openstack-infra18:08
*** Sina_Maleki has left #openstack-infra18:08
*** hongbin has joined #openstack-infra18:09
*** lucasagomes is now known as lucas-dinner18:10
*** daemontool has quit IRC18:10
*** weshay_mtg is now known as weshay18:11
*** doug-fish has joined #openstack-infra18:13
*** degorenko is now known as _degorenko|afk18:13
AJaegerWhat an interesting commit message "I have updated the review request with suggestions from Andreas, can someone review it again. " -
*** adrian_otto has quit IRC18:16
openstackgerritJames E. Blair proposed openstack-infra/zuul: Add some documentation on functional testing tools
*** PollyZ has quit IRC18:16
asselinganesan, make sure tox -e zuul works for your project-config. Typically bad zuul configuration cause zuul to not start18:17
asselinganesan, probably should doc that if it's not already...seems to be a common failure18:18
*** jasondotstar has joined #openstack-infra18:18
asselinganesan, documented here, but maybe needs to be reinforced in the published docs:
ganesanasselin, Thanks, checking now...18:20
*** doug-fish has quit IRC18:20
fungiasselin: that sounds like something we should try to fix. i expect there's some sort of race where it dies parsing configuration before setting up logging. maybe that would be useful to error to stdout before daemonizing or something18:21
asselinganesan, fungi fyi you can see the actual zuul error by running zuul with -d (start in foreground)18:21
openstackgerritK Jonathan Harker proposed openstack-infra/project-config: Add integration tests between system-config and logstash-filters
asselinfungi, ++ actually I thought there was a patch to fix that at some point....18:21
ganesanI see error with zuul -d18:23
ganesanroot@openstack-ci:~# /usr/local/bin/zuul-server -d Traceback (most recent call last):   File "/usr/local/bin/zuul-server", line 10, in <module>     sys.exit(main())   File "/usr/local/lib/python2.7/dist-packages/zuul/cmd/", line 239, in main     server.main()   File "/usr/local/lib/python2.7/dist-packages/zuul/cmd/", line 148, in main     import zuul.trigger.timer   File "/usr/local/lib/python2.7/dist-pack18:23
fungiganesan: your traceback got cut off. paste it to and give us the link18:23
asselinfungi, fyi existing pid file for zuul should be fixed:
fungiasselin: excellent18:24
*** rbuzatu has quit IRC18:25
openstackgerritSam Yaple proposed openstack-infra/shade: Add update_endpoint()
*** rbuzatu has joined #openstack-infra18:25
*** ajmiller has quit IRC18:25
asselinganesan, no idea about that error...but please make sure `tox` passes in your project-config repo18:26
*** hurgleburgler1 has joined #openstack-infra18:26
*** hurgleburgler has quit IRC18:26
fungithough it does look like maybe we have an incompatible version od apscheduler installed18:27
fungiganesan: do a `pip freeze` and paste the results to and give us the link18:27
*** ayoung has quit IRC18:27
fungiit looks like apscheduler made a non-backward-compatible change in its api for 3.x18:27
*** apetrich has joined #openstack-infra18:28
fungiyeah, we're requiring apscheduler>=3.0 for zuul18:29
*** ajmiller has joined #openstack-infra18:29
funginevermind, it depends on the version of zuul. latest zuul release needs older apscheduler>=2.1.1,<3.018:29
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: fix release notes for final releases
openstackgerritDoug Hellmann proposed openstack-infra/project-config: add diff-start to release tag metadata
*** rbuzatu has quit IRC18:30
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: fix release notes for final releases
ganesanfungi, here is the result of 'pip freeze'
openstackgerritSam Yaple proposed openstack-infra/shade: Add update_endpoint()
*** rbuzatu has joined #openstack-infra18:31
openstackgerritDoug Hellmann proposed openstack-infra/project-config: add diff-start to release tag metadata
fungiasselin: ganesan: i suspect the issue is that the latest release of nodepool requires a version of apscheduler that is incompatible with the latest release of zuul
*** dahoo has joined #openstack-infra18:31
fungiyeah, looks like that's got a version of apscheduler suitable for nodepool, but incompatible with zuul18:32
fungiwhich hasn't been a problem for us because we run nodepool and zuul on separate servers18:33
jeblairfungi: don't they both require >=3?18:33
fungibut would be a problem for an all-in-one install18:33
openstackgerritSam Yaple proposed openstack-infra/shade: Add update_endpoint()
dahooclarkb: ping18:33
fungijeblair: we need a new zuul 2.x release for the apscheduler update18:33
asselinganesan, which versions of zuul & nodepool are you using? master or tag? which tag?18:33
jeblairfungi: ah18:33
*** dahoo is now known as jerryz18:33
fungijeblair: latest tag is prior to that18:33
jeblairfungi: i reckon we could issue one now18:33
fungilatest nodepool release plus zuul from master would likely work okay18:34
asselinganesan, nevermind...see it in your pip freeze paste output18:34
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE TESTING
*** greghaynes has quit IRC18:35
jeblairfungi: shall i tag 2.2.0 now?18:35
*** piet has quit IRC18:35
pabelangerfungi: yes, we've needed a new release for a while to fix that18:36
jeblairi had been stalling because i did not want to release with a known memory leak, but that seems better now.18:37
fungijeblair: i guess you don't want to go all in on the zuul v2.5 theme and tag it 2.5.0?18:37
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Make networking-generic-switch-dsvm-tempest voting
jeblairfungi: oh, heh.  i guess we could :)18:38
fungiat this point "zuul v2.5" has been discussed in enough places people might get confused by a new 2.2.0 release18:38
jeblairokay, i'll do 2.5.018:38
fungipossibly fewer than would be confused wondering what happened to 2.2, 2.3 and 2.418:39
jeblairasselin, ganesan: ^ zuul 2.5.0 should exist soon18:39
fungiganesan: if all goes well, in a few minutes your problem should be fixed by reapplying puppet18:39
asselinganesan, actually, you'll likely need to update the pin to 2.5.0 and re-run puppet18:40
ganesansure I will do that18:41
openstackgerritEmilien Macchi proposed openstack-infra/system-config: Added Gem Mirror to Infra
ganesanfungi, let me know once it is ready18:41
openstackgerritMerged openstack-infra/project-config: Remove tripleo-centos-7-ovb from nodepool
*** senk has quit IRC18:44
*** piet has joined #openstack-infra18:44
asselinganesan, would you like to propose the pin to puppet-openstackci?
*** rbuzatu has quit IRC18:45
fungiasselin: oh, right, i forgot we had pinned the version there explicitly rather than just making it latest tag18:46
*** greghaynes has joined #openstack-infra18:47
*** mriedem1 is now known as mriedem18:47
ganesanasselin, sorry I don't understand18:48
fungijeblair: asselin: ganesan: zuul 2.5.0 is up on pypi now
asselinganesan, the file I linked is the main puppet 'hiera' confiruration file you use to setup ci.18:49
*** tsufiev_ has joined #openstack-infra18:49
*** tsufiev has quit IRC18:49
*** doffm_ has joined #openstack-infra18:50
*** Anticime1 has joined #openstack-infra18:50
*** mtreinish_ has joined #openstack-infra18:50
*** sc`_ has joined #openstack-infra18:50
*** nibalize1 has joined #openstack-infra18:50
*** rderose_ has joined #openstack-infra18:50
*** gsilvis_ has joined #openstack-infra18:50
openstackgerritSean Dague proposed openstack-infra/project-config: make neutron the default integrated gate set of jobs
ganesanasselin, ohh yes18:50
*** ianw_ has joined #openstack-infra18:50
*** dulek_ has joined #openstack-infra18:50
*** sc68cal_ has joined #openstack-infra18:50
*** mbacchi_ has joined #openstack-infra18:50
*** roaet_ has joined #openstack-infra18:50
*** tswanson_ has joined #openstack-infra18:50
*** pcm_ has joined #openstack-infra18:50
*** ivyyy_ has joined #openstack-infra18:50
*** dulek has quit IRC18:50
*** swat30 has quit IRC18:50
*** igorbelikov has quit IRC18:50
*** ggherdov- has quit IRC18:50
*** zaro has quit IRC18:50
*** rkukura has quit IRC18:50
*** ianw has quit IRC18:50
*** tobiash has quit IRC18:50
*** mtreinish has quit IRC18:50
*** sc68cal has quit IRC18:50
*** ivyyy has quit IRC18:50
*** waynr has quit IRC18:50
*** pc_m has quit IRC18:50
*** mbacchi has quit IRC18:50
*** loquacities has quit IRC18:50
*** Anticimex has quit IRC18:50
*** taron has quit IRC18:50
*** nikhil has quit IRC18:50
*** doffm has quit IRC18:50
*** rderose has quit IRC18:50
*** roaet has quit IRC18:50
*** dabukalam has quit IRC18:50
*** sneti has quit IRC18:50
*** nibalizer has quit IRC18:50
*** sc` has quit IRC18:50
*** gsilvis has quit IRC18:50
*** Swanson has quit IRC18:50
*** antonym has quit IRC18:50
*** SlickNik has quit IRC18:50
*** dulek_ is now known as dulek18:50
*** sc`_ is now known as sc`18:50
*** mtreinish_ is now known as mtreinish18:50
ganesanfungi, so I update the zuul and rerun the puppet18:50
*** SlickNik has joined #openstack-infra18:50
*** ianw_ is now known as ianw18:50
*** waynr has joined #openstack-infra18:50
*** antonym has joined #openstack-infra18:50
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add DB API function to get list of runs from ids
*** rkukura has joined #openstack-infra18:50
*** swat30 has joined #openstack-infra18:50
*** igorbelikov has joined #openstack-infra18:50
*** zaro has joined #openstack-infra18:50
*** loquacity has joined #openstack-infra18:50
*** ggherdov- has joined #openstack-infra18:51
*** dabukalam has joined #openstack-infra18:51
*** tobiash has joined #openstack-infra18:51
asselinganesan, btw I posted this recently to the mailing list: instructions are the same:
*** gsilvis_ is now known as gsilvis18:51
*** Ravikiran_K has quit IRC18:51
openstackgerritJeremy Stanley proposed openstack-infra/puppet-openstackci: Update to latest Zuul on AIO configuration
fungiganesan: asselin was suggesting that ^18:52
*** nikhil has joined #openstack-infra18:52
sdaguefungi / jeblair - could we get this inserted into the gate queue early - - a bunch of projects are impacted by it18:52
*** taron has joined #openstack-infra18:52
sdaguethe gate queue is small enough, just being in the end is fine18:52
ganesanasselin, fungi, I will follow that18:52
jeblairsdague, fungi: i will do that18:52
fungithanks jeblair18:52
sdaguejeblair: thank you18:53
jeblairsdague: enqueud18:53
*** sneti has joined #openstack-infra18:53
fungiAJaeger: i'm going to start working through reviews of topic:constraints is:open now18:53
fungisorry it's taken me so long18:53
AJaegerthanks, fungi.18:53
*** asettle has joined #openstack-infra18:54
*** dprince has quit IRC18:54
AJaegerfungi, If you +2 348764, I'll remove the WIP...18:54
AJaegerso, please don't skip it ;)18:55
fungiAJaeger: got it. will look18:55
fungiAJaeger: oh, right, that one. this will be fun18:56
*** sc68cal_ is now known as sc68cal18:56
AJaegerfungi, I tried to keep it simple ;)18:56
*** javeriak has quit IRC18:59
fungiAJaeger: i see one bug19:02
AJaegerfungi, let me check and try fixing quickly ;)19:02
fungiit should be easy to fix ;)19:02
*** Sukhdev has joined #openstack-infra19:03
AJaegerit is - and thanks for spotting19:03
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Use constraints for translation sync
AJaegerfungi, ^19:03
fungithatks, will +2 in just a sec19:04
fungino need to re-wip19:04
openstackgerritMerged openstack-infra/project-config: Enable constraints for releasenotes
AJaegerthanks, that's what I hoped ;)19:04
*** roxanaghe has quit IRC19:04
fungiAJaeger: fwiw, i'm never sure on those and always end up looking at the xml diff19:05
fungiit can be easy to forget19:06
AJaegerfungi, indeed. For some of these I check as well - but sometimes forget - as in this case.19:06
*** HeOS has quit IRC19:07
*** fifieldt has quit IRC19:07
ganesanfungi,asselin, jeblair, Its working, now zuul is running19:07
fungiganesan: excellent news--thanks for bringing that bug to our attention!19:08
*** oanson has joined #openstack-infra19:08
asselinganesan, thanks...can you let me know what you did to fix it? Do you follow my e-mail suggestion or do something else? I'd like some feedback before sending another e-mail.19:09
*** ganesan has quit IRC19:09
*** ganesan has joined #openstack-infra19:10
*** ganesan has quit IRC19:11
*** karthikp_ has quit IRC19:11
*** ajmiller has quit IRC19:12
*** ganesan has joined #openstack-infra19:12
*** ajmiller has joined #openstack-infra19:14
*** sarob has joined #openstack-infra19:14
*** ganesan has quit IRC19:15
*** ganesan has joined #openstack-infra19:16
*** mrmartin has quit IRC19:16
anteayagerrit's ref-replicated event is mirroriing a merged patch out to the git farm and github, yes?19:16
fungianteaya: and to the local gerrit mirror on review.o.o's filesystem too19:17
anteayathank you19:17
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Use geard with keepalives
*** mdrabe has quit IRC19:18
*** ganso has left #openstack-infra19:18
*** mdrabe_ has joined #openstack-infra19:18
*** fifieldt has joined #openstack-infra19:18
AJaegerarmax, dougwig : IF you have time, could you review  - they all have two +2s and wait for Neutron Infra Liason review19:18
fungii'm not sure if you get one ref-replacated event per mirror or just one when the ref has finally been replicated to all mirrors19:18
fungianteaya: ^19:18
armaxAJaeger: ack19:18
fungibut if you don't see mirror specific metadata in the event, then it's presumably the latter19:18
AJaegerfungi, thanks for reviews!19:18
fungiAJaeger: thanks even more for your patches19:19
openstackgerritMike Fedosin proposed openstack-infra/project-config: Glare
AJaegerjeblair, mordred, krotscheck, fungi: Could you review the bindep change which I did based on Friday's discussion, please?
fungiAJaeger: on the release-specific macro, we're still planning to eventually try to collapse those into the normal upper-constraints builders, right?19:19
AJaegerthanks, armax19:20
funginot urgent, just trying to remember what we had discussed19:20
*** PollyZ has joined #openstack-infra19:20
anteayafungi: per mirror is what I see19:20
anteayaand I see that "/opt/lib/git/openstack-infra/project-config.git" is the internal gerrit mirror on review.o.o, yes?19:20
fungianteaya: wow, how verbose19:20
*** ajmiller has quit IRC19:21
anteayashall I paste what I see from firehose?19:21
fungianteaya: yes, that's the "local" one19:21
fungianteaya: no need. i can get plenty of them if i just connect19:21
AJaegerfungi, so merging zuul-git-prep-upper-constraints and zuul-release-git-prep-upper-constraints? Which means replacing zuul-git-prep-upper-constraints by zuul-release-git-prep-upper-constraints? let me do that next so that we don't forget it...19:21
anteayaah sorry19:21
anteayayes, very verbose19:22
pabelangeranteaya: Oh, nice!19:22
pabelangerI haven't tested firehose yet19:22
*** baoli has quit IRC19:22
fungiAJaeger: right. i expect we'd want to keep a very close eye on it once it merges because it could risk blowing up most tox-based jobs if we're overlooking something19:22
*** baoli has joined #openstack-infra19:23
anteayapabelanger: deb has a mosquitto clients package19:23
*** ajmiller has joined #openstack-infra19:23
AJaegerI'll put out a WIP change now - let's first settle the rest a bit. Thanks for reminder19:23
EmilienMhey there, could I have a last review on please?19:23
fungianteaya: pabelanger: yeah, that's what i used. apt-get install it and away i go19:23
openstackgerritMerged openstack-infra/project-config: Remove unneeded --branch from zuul-cloner
pabelangeranteaya: fungi: very nice19:24
anteayapabelanger: this patch merged: but isn't published on docs yet19:24
anteayanot sure why not19:24
anteayamerged this morning19:24
fungianteaya: big backlog in the post pipeline?19:24
fungianteaya: looks like we're going on 11 hours of backlog there19:24
pabelangerya, we're basically starved of nodes in post19:24
anteayawas just going to look19:25
anteayaah that would be the reason19:25
anteayanice long back log19:25
anteayathe release team will be pleased19:25
*** oanson has quit IRC19:25
anteaya160 in post19:25
pabelangerhotel question about QAInfraNewtonSprint, are we trying to group up at the same place or are people making a choice on personal preference?19:26
anteayacloudnull: I've lost track are you waiting on anything from us before our osic quota is increased?19:27
rcarrillocruzpabelanger: i guess it also depends on budget on what the employer travel tool allows19:28
rcarrillocruzi've picked...19:28
*** amitgandhinz has quit IRC19:28
fungianteaya: actually, that docs job _just_ finished 5 minutes ago19:28
anteayafungi: woooo19:28
rcarrillocruzoutlook365 is a bit slow19:28
fungianteaya: i see the new content on docs.o.o now19:28
*** mbacchi_ is now known as mbacchi19:28
anteayawonderful thank you19:28
*** yaume has quit IRC19:28
anteayafor me too19:28
*** amitgandhinz has joined #openstack-infra19:28
rcarrillocruzpabelanger: leonardo walendorf hotel19:29
rcarrillocruzif that helps at all19:29
anteaya is a much nicer url19:29
pabelangerrcarrillocruz: Thanks, that is what I am looking at now too19:29
rcarrillocruzyolanda and dmellado will be there too19:29
*** mdrabe_ is now known as mdrabe19:29
*** krtaylor has quit IRC19:29
pabelangerrcarrillocruz: flight into FRA Frankfurt right?19:30
rcarrillocruzpabelanger: not really in today, but was able to set all iLO IPs on all nodes last night19:30
rcarrillocruzand also confirmed we can ssh in to the baremetal host19:31
pabelangerrcarrillocruz: ya, seen that.19:31
pabelangercool, should see if puppetmaster is working again for it19:31
pabelangerI can check that shortly19:31
rcarrillocruzpabelanger: the IP range changed tho19:31
anteayacrinkle: so from the link you posted earlier: puppet-lint-undef_in_function-check: When passing the ‘undef’ value to a Puppet function, Puppet munges it into an empty string. This plugin checks for ‘undef’ as a function parameter.19:31
rcarrillocruzif you have the panel for dns handy or you're familiar with the command to update it19:32
rcarrillocruzthe new IP is:19:32
pabelangerrcarrillocruz: ah, right. I can do that19:32
rcarrillocruzfungi: they had to change the IP range slightly19:32
openstackgerritgreghaynes proposed openstack/diskimage-builder: Explain the difference between 'DIB_OFFLINE' and 'DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE'
anteayacrinkle: should I interpret that as we can't give an empty string anymore as a value to a key, we have to use undef because puppet will then turn it into an empty string?19:32
rcarrillocruzthe new public ip range is
rcarrillocruzwill update the infracloud docs to show that19:32
fungircarrillocruz: yeah, i had a feeling that was going to happen. regardless of their earlier assurances that they wouldn't need to do new address assignments19:33
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Merge zuul-git-prep-upper-constraints/zuul-release-git-prep-upper-constraints
rcarrillocruz'all is fine, everything will just work'19:33
crinkleanteaya: functions are different from classes, i don't think we're really concerned with that behavior19:33
*** kushal has quit IRC19:33
*** baoli has quit IRC19:33
anteayacrinkle: thank you19:33
fungircarrillocruz: it's like we're just moving a network cable to... oh wait, these switches are in an entirely different part of our network19:34
openstackgerritgreghaynes proposed openstack/diskimage-builder: Explain the difference between 'DIB_OFFLINE' and 'DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE'
AJaegerfungi, could you quickly check 352575, please? I'll mark it as WIP for now since we have changes in flight that need renames and I'd like to wait a few days - but like to know that this is the right thing19:34
fungiAJaeger: sure, will do as soon as i'm through the bindep.txt change19:34
*** kushal has joined #openstack-infra19:35
*** oanson has joined #openstack-infra19:37
*** ajmiller has quit IRC19:37
openstackgerritMerged openstack-infra/project-config: networking-odl: add coverage, translation and releasenotes
notmorganjeblair, mordred, Shrews: any idea what I'm doing wrong here? i cant figure out why the hell this is failing. All attempts to put real debugging in/capture output has resulted in somehow a NoneType ended up in a queue? and/or needing to kill -9 python19:38
*** baoli has joined #openstack-infra19:38
openstackgerritMerged openstack-infra/project-config: Make py35 voting for networking-midonet
openstackgerritMerged openstack-infra/project-config: Make gate-networking-ovn-dsvm-functional voting
*** amitgandhinz has quit IRC19:39
openstackgerritMerged openstack-infra/elastic-recheck: Add query for nova unshelve race bug 1611008
openstackbug 1611008 in OpenStack Compute (nova) "ServersNegativeTestJSON.test_suspend_server_invalid_state fails with "NovaException: Unable to find host for Instance"" [High,In progress] - Assigned to Matt Riedemann (mriedem)19:39
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: upgrades job is renamed to updates
*** amitgandhinz has joined #openstack-infra19:40
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton
docaedofungi: regarding app-catalog-bugs-team, sorry if I messed it up, following I thought I was setting the owner right19:40
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton
anteayaAJaeger: check out this paste:  I think there are 3 separate events in there19:40
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Implement non-ovb overcloud update job - Newton -> Newton
fungidocaedo: you didn't mess anything up. i've recently discovered that lp auto-adds the owner on new team creation. i may patch the documentation to ask people to remove it as a group member after creation19:41
*** mrmartin has joined #openstack-infra19:41
AJaegeranteaya: I'm missing context, did you really want me to look at it?19:41
docaedofungi: oh ok, cool - thanks!19:42
*** kzaitsev_mb has joined #openstack-infra19:42
anteayaAJaeger: that is from the firehose event stream19:42
*** edmondsw has quit IRC19:42
AJaegeranteaya: ah - sorry, can't dig into that now19:43
anteayaand perhaps I am wrong, perhaps one event merge triggered mirroring out to /refs/heads/master, /refs/notes/review both on project-config and as well openstack/openstack19:43
AJaegerfungi, shall I quickly fix the nit on bindep?19:43
anteayaAJaeger: oh you showed interest in the last paste so thought you wanted to know about firehose, sorry to bother you19:43
fungiAJaeger: sure. i'm happy to +2 again19:43
anteayaAJaeger: nothing needed19:43
fungianteaya: yes, gerrit is replicating git notes about the merged commit as well as the merged commit itself19:44
* notmorgan realizes openstack SV is happening so people might be less-available or some such.19:44
openstackgerritAndreas Jaeger proposed openstack-infra/bindep: Add bindep.txt as default filename
anteayafungi: ah the git notes, thank you19:44
AJaegerfungi, done ^19:44
*** jistr has quit IRC19:44
pabelangerrcarrillocruz: looks like we might have a DNS issue on ?19:44
anteayanotmorgan: what is openstack sv?19:44
pabelangerrcarrillocruz: nslookup fails to result google19:44
*** ajmiller has joined #openstack-infra19:45
AJaegerSV = Silican Valley?19:45
notmorganAJaeger: yes19:45
anteayaoh okay19:45
notmorgananteaya: :)19:45
zigopabelanger: mordred: For some reason, the POST job to publish backports didn't work. Is this related to your patch?19:45
pabelangerrcarrillocruz: using does19:45
mordrednotmorgan: looking19:45
pabelangerzigo: yes19:45
*** rbuzatu has joined #openstack-infra19:45
pabelangerzigo: there are 2 patches we need to land19:45
zigopabelanger: Ok. From my side everything works.19:46
pabelanger352489 and 35252419:46
rcarrillocruzpabelanger: you just set the record on rackdns?19:46
pabelangerrcarrillocruz: not yet, doing that now19:46
rcarrillocruzpabelanger: also, i think we want to have baremetal00.ic.openstack.org19:46
rcarrillocruzno chocolate, no vanilla19:46
zigopabelanger: It built 4 packages, as I expected.19:46
rcarrillocruzsince the baremetal host will be the same for both 'clouds'19:47
pabelangerthat make sense19:47
pabelangerzigo: great19:47
*** karthikp_ has joined #openstack-infra19:47
rcarrillocruzwe can then add cnames for vanilla/chocolate19:47
*** igorbelikov has quit IRC19:47
*** igorbelikov has joined #openstack-infra19:47
zigopabelanger: One thing which would be nice would be to have just the hostname of the repositories, so that I could just use it to compose sources.list within the sbuild chroot.19:47
*** zaro has quit IRC19:47
*** zaro has joined #openstack-infra19:47
pabelangermordred: AJaeger: do you mind helping on 352489 and 35252419:47
*** ggherdov- has quit IRC19:47
*** ggherdov- has joined #openstack-infra19:47
*** nikhil has quit IRC19:47
*** nikhil has joined #openstack-infra19:47
*** ganesan has quit IRC19:47
*** ganesan has joined #openstack-infra19:47
*** tsufiev_ is now known as tsufiev19:47
fungircarrillocruz: assuming it's one bastion to rule them all, that's fine by me... but if we end up with more regions later they'll have their own bastions due to not being privately networked to each other, right? what naming convention will those end up with?19:48
AJaegerpabelanger, rcarrillocruz, fungi, anteaya : Could I get two +2s on this change - it fixes the logic for when to run xenial/trusty to handle -nv jobs as well. clarkb missed that initially.19:48
AJaegerpabelanger: on it...19:48
*** jistr has joined #openstack-infra19:48
*** ajmiller has quit IRC19:48
*** ajmiller has joined #openstack-infra19:48
*** karthikp_ has quit IRC19:48
*** karthikp_ has joined #openstack-infra19:48
AJaegerpabelanger: see my trade in above ;)19:48
fungircarrillocruz: say red hat donates a region in one of their datacenters, for example19:48
amrithhi infra folks, I have a question re: the CI infrastructure and how requirements (especially upper-constraints.txt) are handled. I've posed my question in this short etherpad, would someone please provide me some pointers to where I could find more about how this part of the CI works. Thanks19:48
zigopabelanger: I need it in command lines, like for calling sbuild-createchroot.19:48
pabelangerAJaeger: +319:49
rcarrillocruzfungi: then we would add a bastion per region19:49
zigopabelanger: Currently, I'm using for sbuild, which is really sub-optimal.19:49
rcarrillocruzfungi: so you mean, could have clouds from various 'infraclouds' ?19:50
*** rbuzatu has quit IRC19:50
anteayaAJaeger: give me a minute to try them out19:50
fungircarrillocruz: that was my impression at least. and then we divvy them up under flavored subdomains of one or more per cloud19:51
*** yamahata has quit IRC19:51
pabelangerrcarrillocruz: I've kept for now19:51
fungircarrillocruz: when the hp east and west gear all got moved into one location that made this first deployment able to have one bastion, but additional deployments in other locations would presumably need separate bastions19:51
rcarrillocruzk, i guess we leave for now the baremetal to rule them all, add sub-domains bastions should we get more clouds (hope so!)19:51
pabelangerzigo: right, we can make optimizations as we go19:52
*** nmagnezi has quit IRC19:52
rcarrillocruzfungi: ++19:52
fungircarrillocruz: yeah, i definitely don't want to make anything on the first deployment harder just because we might have more deployments than this one down the road, but if there are places we can plan for that possibility without much additional effort...19:52
*** HeOS has joined #openstack-infra19:52
*** jistr has quit IRC19:53
rcarrillocruzfungi: btw, i've spotted at least 15 servers that were not present on hiera19:53
rcarrillocruzso moar compute19:53
*** klindgren_ has joined #openstack-infra19:53
rcarrillocruzi've also opened tickets for the issues i've encountered during the inventory check19:53
rcarrillocruzservers not showing stuff on textcons19:53
rcarrillocruzbut overall they seem to be o-k19:53
jeblairnotmorgan: i'm just getting back from lunch and was about to poke at your wip patch19:53
AJaegerthanks, rcarrillocruz , pabelanger and anteaya19:54
fungircarrillocruz: ooh! hopefully those are actually ours and not someone else's systems comingled on the same lan ;)19:54
AJaegerpabelanger: one +2 and one suggestion to improve comments...19:54
anteayaAJaeger: I didn't do much, I didn't get finished testing19:54
* anteaya moves on to something else19:54
rcarrillocruzfungi: hah, that's the very first question i made to the DC folks :-)19:54
AJaegeramrith: for which job did the upper-constraint not work as expected?19:55
*** karthikp_ has quit IRC19:55
*** itisha has joined #openstack-infra19:56
amrithAJaeger, in the trove job, which was depending on the requirements job, I don't see UPPER_CONSTRAINTS_FILE being set19:56
AJaegeramrith: which job exactly?19:56
*** PollyZ has quit IRC19:56
*** jistr has joined #openstack-infra19:56
amrithsorry, gate-trove-functional-dsvm-mysql for example19:56
amrithor gate-trove-scenario-functional-dsvm-<datastore name>19:57
amrithdatastore name could be mysql or redis19:57
AJaegeramrith: I'm asking since I'm currently enabling UPPER-constraints on the jobs - and not all of it yet.19:57
amrithI'm interested in knowing where in infra that gets setup19:57
AJaegeramrith: ;)19:57
*** adrian_otto has joined #openstack-infra19:57
openstackgerritMerged openstack-infra/project-config: Handle xenial/trusty and -nv jobs
AJaegeramrith: I'm currently focussing on the tox based jobs - so can't help with the others.19:58
AJaegerHope that somebody else can help.19:58
* AJaeger waves good-night19:58
*** rbuzatu has joined #openstack-infra19:58
anteayaAJaeger: good night19:59
amrithAJaeger, g'night, thanks for the pointer19:59
*** karthikp_ has joined #openstack-infra20:00
*** jcoufal has quit IRC20:03
*** Apoorva has quit IRC20:03
openstackgerritPaul Belanger proposed openstack-infra/project-config: Fix issue with reprepro-import distribution
*** roxanaghe has joined #openstack-infra20:06
fungiamrith: i _think_ the devstack-based jobs already use constraints if they're using devstack's pip_install function20:06
amrithfungi, how would I begin to track that down?20:08
*** Apoorva has joined #openstack-infra20:08
openstackgerritMerged openstack-infra/project-config: Implement TripleO centos-7-nonha-multinode-updates
fungiamrith: i'm trying to see if i can spot the problem20:08
amrithlet me begin at a more basic place, is there a beginners guide to our CI20:08
amrithI really need to learn more about it rather than the occasional hack at projects.yaml20:08
amrithor jjb20:08
fungiamrith: there are some audience-specific guides, i'm trying to figure out which one to recommend20:09
amriththx fungi20:10
fungithe surface area is so large that we tend to slice it by areas of interest20:10
*** rockyg has joined #openstack-infra20:10
*** ajmiller_ has joined #openstack-infra20:10
fungiso for example, we have project-specific documentation (zuul has documentation, nodepool has documentation, jenkins-job-builder has documentation, and so on)20:11
*** dprince has joined #openstack-infra20:11
fungiand then we have holistic documentation of the entire system targeted at people trying to deploy it to operate a third-party ci system20:11
fungiand we have operational documentation of the systems we're using to run our deployments of all of that software, targeted at people interested in helping run it20:12
*** apetrich has quit IRC20:12
*** yamahata has joined #openstack-infra20:12
fungiamrith: perhaps a good middle-ground is our operational documentation page on jjb
*** kzaitsev_mb has quit IRC20:13
amriththanks, let me start there20:13
*** nibalize1 is now known as nibalizer20:13
*** ajmiller has quit IRC20:15
fungiwe also have documentation in the infra-manual focused on several different audiences: one for developers interacting with our services (developers), one for people doing code review with approval rights (core reviewers), one for people managing projects administratively (project drivers), and one for people creating new git repos/projects (project creators)
*** tonytan_brb has quit IRC20:16
*** oanson has quit IRC20:18
*** rbuzatu has quit IRC20:19
*** rocky_g has joined #openstack-infra20:19
Shrewsnotmorgan: i can look in the morning if you still need me to. context switching to that now would kill my momentum20:19
Shrewsand hurt my feeble brain20:19
* mordred hands Shrews a box of spare brains he found over the weekend20:20
*** rockyg has quit IRC20:20
Shrewsmordred: well, if you aren't using them....20:20
mordredShrews: I'm not sure I've used a brain in years20:20
*** apetrich has joined #openstack-infra20:20
*** mhickey has joined #openstack-infra20:21
dhellmannfungi : are the scripts from jenkins/scripts/release_tools installed on all of the CI nodes? and have me thinking that we may want to move more release tools into the project-config repo so we can make these changes atomically20:24
*** rockyg has joined #openstack-infra20:24
dhellmannalthough those scripts have a pyyaml dependency, so I don't know20:24
dhellmannI think that's why we held off before20:24
*** rocky_g has quit IRC20:25
*** spzala has quit IRC20:25
fungidhellmann: we can install pyyaml from distro packages on our persistent job nodes if needed20:25
*** rbuzatu has joined #openstack-infra20:25
dhellmannok, there may be other things, I'd have to look at the list again to see if we have anything weird20:26
dhellmannthat repo is more organic than most :-)20:26
mordredfungi: wow, we have nodes on which pyyaml isn't installed?20:26
sdaguedims: (the requirements update that unblocks neutron / ironic / and others) failed on random ssh race20:26
fungidhellmann: and yes, those scripts are installed onto all job nodes (preinstalled in single-use node images by nodepool, immediately pushed onto persistent nodes by ansible/puppet)20:26
*** spzala has joined #openstack-infra20:27
fungisdague: need to get it reenqueued/promoted?20:27
sdaguefungi: yeh, it hasn't completely failed out yet20:27
dhellmannok. I think it may have been the delay that kept us from moving them before. Might be time to reconsider.20:27
sdaguebut it will20:27
dimssdague : looks like gate-grenade-dsvm-ubuntu-trusty20:27
fungisdague: oh, i can promote it then and it will be retested20:27
sdaguefungi: yeh, probably good20:27
fungiif you'd rather20:27
dhellmannfungi : how're things looking with launchpadlib on the signing node?20:28
fungidhellmann: haven't had more time to write up the solution for that yet20:28
*** tqtran has joined #openstack-infra20:29
mordredsdague: btw - I pushed up a devstack change to try to track down the failure better - I have a patch to neutronclient that I believe will fix things, but I haven't been able to reproduce the problem locally just running similar commands with the same tools against existing clouds20:29
mordredsdague: I'd rather verify that the reason I think it's failing is actually the reason it;s failing20:29
sdaguemordred: well, it reproduced consistently on grenade20:29
sdaguethat might be the thing to try to chase20:30
sdaguethere was no straight devstack reproduce, right?20:30
mordredsdague: yah. I'm just not set up on my laptop to run grenade - thus the patch to the devstack plugin where the error shows up20:30
fungisdague: it's promoted now20:30
mordredsdague: and yah - it's only on grenade20:30
sdaguefungi: thanks20:30
mordredI mostly want to see what the env vars are at the time that the fail happens20:30
sdaguemordred: gotcha20:30
mordredthat's the only piece I'm missing for local reproduction I THINK20:31
*** nwkarsten has joined #openstack-infra20:31
*** edtubill has joined #openstack-infra20:31
*** tongli has joined #openstack-infra20:32
*** dprince has quit IRC20:33
*** sdake has quit IRC20:33
*** inc0 has joined #openstack-infra20:33
openstackgerritPaul Belanger proposed openstack-infra/project-config: Prepare to replace jenkins user with zuul user
*** igorn has quit IRC20:34
pabelangerclarkb: ^ I _think_ that addresses your comments20:34
*** dprince has joined #openstack-infra20:35
*** Jeffrey4l has quit IRC20:35
jeblairnotmorgan: ping me when you are around; i think i'm making headway on that bug20:35
*** Jeffrey4l has joined #openstack-infra20:35
*** mrmartin has quit IRC20:36
pabelanger2016-08-08 20:26:30,286 p=17226 u=root | : ok=13   changed=7    unreachable=0    failed=020:37
pabelangerrcarrillocruz: fungi: ^20:37
*** vhosakot has joined #openstack-infra20:37
pabelangeransible runs on infracloud again!20:37
*** inc0 has quit IRC20:38
*** rossella_s has quit IRC20:38
*** rossella_s has joined #openstack-infra20:38
zigomordred: Could you please review these? and
mordredzigo: yup20:39
mordredrcarrillocruz: \o/20:39
pabelangerrcarrillocruz: looks like dns is fixed too, now that unbound is configured20:39
*** moravec1 has joined #openstack-infra20:40
mordredpabelanger, zigo: lgtm20:41
fungiamrith: looking at it seems like the oslo.context version referenced in the upper-constraints change you depended-on was installed in the system environment by devstack. i'm reading through how your integration tests work to see if i can figure out why they're not using it20:42
*** krtaylor has joined #openstack-infra20:42
sc`does not do https?20:42
sc`my browser seems to think so20:43
*** moravec has quit IRC20:43
fungisc`: it does not. it's hosted on rackspace cloudsites, not a server we maintain20:43
sc`fungi: okay. makes sense. my browser is just being too draconian20:44
fungiwe also have to push files to it via ftp. a quaint blast from the past20:44
*** moravec1 has quit IRC20:44
*** krtaylor has quit IRC20:46
*** gyee has quit IRC20:46
openstackgerritMerged openstack-infra/project-config: Update reprepro-import build to support multiple files
*** gyee has joined #openstack-infra20:47
jeblairfungi: we can do older than ftp!20:47
*** hichihara has joined #openstack-infra20:47
*** sarob has quit IRC20:48
fungijeblair: touché20:48
sc`not sneakernetting files to the castle?20:48
sc`such futuristic :)20:48
jeblairsc`: wow, 'sneakernet to the castle' does sound like some sort of weird scifi/fantasy thing.20:49
sc`it kind of does, doesn't it?20:50
*** rbuzatu has quit IRC20:50
*** vhosakot has quit IRC20:51
*** edmondsw has joined #openstack-infra20:51
mtreinishfungi: so do and look better now?20:51
fungisc`: we ship them reels of our content20:52
*** vhosakot has joined #openstack-infra20:52
*** tongli has quit IRC20:52
*** eharney has quit IRC20:53
openstackgerritMatthew Treinish proposed openstack-infra/puppet-logstash: WIP: Add mqtt output plugin to logstash indexers
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Add an example logging.conf for development
*** ayoung has joined #openstack-infra20:55
notmorganjeblair: o/20:56
pabelangerfungi: do you mind helping land:
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Add a fake-secure.conf
zigofungi: Could you please review that one? This should fix the POST job of deb-auto-backports:
fungipabelanger: zigo: yes, i'm looking at it already20:58
*** ihrachys has joined #openstack-infra20:58
sc`fungi: now that's the dry humor i've come to love out of openstack. /me golfclap21:00
*** gouthamr has quit IRC21:01
*** coolsvap has quit IRC21:01
*** baoli has quit IRC21:01
*** baoli has joined #openstack-infra21:02
*** dprince has quit IRC21:02
*** thorst has quit IRC21:03
pabelangerfungi: thanks for the help21:03
*** thorst has joined #openstack-infra21:03
*** raildo has quit IRC21:03
fungiamrith: maybe a clearer log entry to suggest that it's really getting the constrained version of the module
fungiamrith: you can see there that pip was called with -c /opt/stack/new/requirements/upper-constraints.txt when installing oslo.context onto the system21:04
*** vhosakot_ has joined #openstack-infra21:05
openstackgerritThomas Goirand proposed openstack-infra/project-config: Write /etc/nodepool_mirror_host
zigopabelanger: Should I wait before attempting to publish backport packages again?21:05
*** vhosakot has quit IRC21:05
*** jkilpatr has quit IRC21:05
zigo(I mean, after the merge happens...)21:06
*** acoles_ is now known as acoles21:06
*** thorst_ has joined #openstack-infra21:07
*** vhosakot_ has quit IRC21:07
pabelangerzigo: still a while of before merging21:07
*** thorst has quit IRC21:07
*** yamamoto has joined #openstack-infra21:09
*** _nadya_ has joined #openstack-infra21:09
*** weshay has quit IRC21:10
*** yamamoto has quit IRC21:10
*** _nadya_ has quit IRC21:10
*** yamamoto has joined #openstack-infra21:10
*** _nadya_ has joined #openstack-infra21:11
*** thorst_ has quit IRC21:11
*** hurgleburgler1 has quit IRC21:11
*** hurgleburgler has joined #openstack-infra21:11
*** _ari_ has quit IRC21:12
*** _nadya__ has joined #openstack-infra21:13
*** _nadya_ has quit IRC21:13
*** rockyg has quit IRC21:13
*** rockyg has joined #openstack-infra21:14
*** yamamoto has quit IRC21:15
*** tonytan4ever has joined #openstack-infra21:17
*** senk_ has quit IRC21:18
*** thiagop has quit IRC21:20
*** inc0 has joined #openstack-infra21:20
*** julim has quit IRC21:21
*** _nadya_ has joined #openstack-infra21:22
*** _nadya__ has quit IRC21:22
*** tonytan4ever has quit IRC21:22
*** pradk has quit IRC21:22
*** senk has joined #openstack-infra21:23
*** inc0 has quit IRC21:24
*** ldnunes has quit IRC21:24
*** rhallisey has quit IRC21:25
*** acoles is now known as acoles_21:26
*** mhickey has quit IRC21:27
*** ganesan has quit IRC21:28
*** matt-borland has quit IRC21:29
openstackgerritKhai Do proposed openstack-infra/system-config: Upgrade gerrit on review-dev.o.o
*** tonytan4ever has joined #openstack-infra21:35
*** piet has quit IRC21:37
*** _nadya_ has quit IRC21:37
*** _nadya_ has joined #openstack-infra21:38
*** Na3iL has quit IRC21:38
*** burgerk has quit IRC21:38
cloudnullanteaya: sorry been afk for most the day. We're not waiting on anything from infra, we have issues with IP space for the provider networks but I'm working on getting a new one, setup to get us some breathing room.21:39
*** _nadya_ has quit IRC21:42
anteayacloudnull: no worries and thank you21:42
anteayago you \o/21:42
*** ihrachys has quit IRC21:43
*** dimtruck is now known as zz_dimtruck21:43
*** zz_dimtruck is now known as dimtruck21:43
anteayalet us know if you need any emotional support on IP space for the provider networks21:43
*** ihrachys has joined #openstack-infra21:43
cloudnullwill do :)21:44
cloudnullI'm hoping to have v6 up and running by end of the week21:44
*** inc0 has joined #openstack-infra21:44
*** thorst has joined #openstack-infra21:44
anteayaawesome, thank yoiu21:45
mnaserI didn't know this.. "If you provided an IRC nic - do you prefer the IRC nic to be the most prominent identifier printed on your Summit name badge?"21:46
mnaser"based on popular demand"21:46
anteayamnaser: this is a new question when registering for summit21:47
anteayamordred: started the trend21:47
mnaserI remember seeing him with the "mordred" badge at Austin :p21:48
anteayaa few of us followed suit last summit then the point came up at the feedback session21:48
anteayamnaser: yeah, that was when it started21:48
mordredI hacked the one at the austin summit by going and having it reprinted with my first name as "mordred"21:48
anteayanow it is "official"21:48
*** apetrich has quit IRC21:48
anteayawhatever that means21:48
mordredyah. hats off to the folks at the foundation for making that change for us21:48
*** nherciu has quit IRC21:48
*** thorst has quit IRC21:49
*** spzala has quit IRC21:49
*** rfolco has quit IRC21:49
*** moravec has joined #openstack-infra21:50
*** rbuzatu has joined #openstack-infra21:50
openstackgerritIan Wienand proposed openstack-infra/project-config: Add Fedora 24 work-around kernel
*** nwkarsten has quit IRC21:52
*** dimtruck is now known as zz_dimtruck21:52
ianwpabelanger: ^ i think that's the final revision on that .... so many layers of bugs21:53
pabelangerianw: wow21:54
*** nwkarsten has joined #openstack-infra21:54
*** rbuzatu has quit IRC21:55
*** ihrachys_ has joined #openstack-infra21:55
*** mriedem has quit IRC21:55
*** xenogear_ has quit IRC21:55
pabelangerianw: any idea when f24 will update the kernel?21:56
ianwpabelanger: ahh, i lose track.  it goes into the kernel net tree, which gets pulled into the stable kernel tree periodically, which i think fedora update from periodically21:57
ianwso, yeah, about then21:57
*** baoli has quit IRC21:58
*** ihrachys has quit IRC21:58
*** baoli has joined #openstack-infra21:58
pabelangerit would be good to see if we can get it into updates-testing repo early some how21:58
*** nwkarsten has quit IRC21:58
pabelangerbut I have no idea how to do that21:58
ianwyeah, i will see who i can find who might know21:59
*** lewo has quit IRC22:00
*** lewo has joined #openstack-infra22:00
*** Apoorva has quit IRC22:00
openstackgerritPaul Belanger proposed openstack-infra/project-config: Migrate to gate-{name}-tox-db-{envlist}-{node}
*** Apoorva has joined #openstack-infra22:01
openstackgerritIsaku Yamahata proposed openstack-infra/project-config: networking-odl: cover more combinations of version
mtreinishfungi, rcarrillocruz: \o/ our mosquitto bug got tagged as something to fix for the next release:
mtreinishheh, of course I doubt we'll ever see it backported to the xenial package22:03
anteayaone can hope22:03
*** baoli has quit IRC22:05
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Fix queue for py35 Neutron unit test job
*** Apoorva has quit IRC22:05
*** Apoorva has joined #openstack-infra22:06
*** edtubill has quit IRC22:06
openstackgerritPaul Belanger proposed openstack-infra/project-config: Stack nodepool images targets in grafana
*** ihrachys_ has quit IRC22:08
*** esberglu has quit IRC22:09
*** hurgleburgler1 has joined #openstack-infra22:10
*** hurgleburgler has quit IRC22:10
*** amitgandhinz has quit IRC22:10
*** mdrabe has quit IRC22:10
*** hichihara has quit IRC22:11
fungiwell, it's not as if it's a problem for _us_ now that we know why it failed22:13
fungibut it's good to see they're responsive to bug reports22:13
*** krtaylor has joined #openstack-infra22:13
amrithfungi, sorry, wasn't paying attention. let me check22:17
*** julim has joined #openstack-infra22:19
amrithfungi, can I assume that if upper-constraints.txt exists, it will be in /opt/stack/new/requirements/upper-constraints.txt22:19
asselinzaro, how were JJB configuration scripts tested before merging?22:19
pabelangerfungi: we likely need to restart zuul again, our demand calculation is out again for ubuntu-trusty. We have 50+ nodes in ready state atm22:19
*** thorst has joined #openstack-infra22:22
*** thorst has quit IRC22:23
fungiamrith: for devstack-gate based jobs, like your functional test job, then yes22:23
amrithOK, in that case I'll just look for that file instead of the environment variable in my jobs22:24
amrithI'm doing this for
fungiamrith: devstack-gate copies relevant git repositories into /opt/stack/new and updates them there based on any $ZUUL_REF provided (which will integrate proposed changes in dependencies you've specified, like the depends-on in your example)22:25
amrithso, I think my 'big question' is this; is it safe to rely on the path or is there an environment variable that points to (for example) /opt/stack/new22:26
amriththat I can treat as being unchanging in the future22:26
amrithIf you have a minute, I'll explain the bigger issue22:26
amriththat I'm trying to deal with22:26
fungiamrith: yeah, the $UPPER_CONSTRAINTS_FILE envvar is something being done to make use of constraints files in tox-based jobs, but isn't relevant to devstack-gate based jobs22:26
*** yamamoto has joined #openstack-infra22:26
amrithOK, so I can't rely on that; explains why my test of didn't find that variable.22:27
fungithere's no separate envvar for that which i'm aware of, but i'll double-check the devstack source code for the pip_install function to make sure22:27
*** sputnik13 has quit IRC22:27
*** sputnik13_ has joined #openstack-infra22:27
zaroasselin: it's tested against openstack configs.
asselinzaro, I there an easy way to get a JJB configuraiton output to bash scripts that someone can use to manually upload and run on a held node?22:29
fungiamrith: so it's here where the constraints file path is being assembled inside devstack
* amrith looks22:30
amrithgreat, so I can just use $REQUIREMENTS_DIR22:31
zaroasselin: i think jjb provides a number of out parameters, one being output to files22:31
amrithperfect, $REQUIREMENTS_DIR/upper-constraints.txt should be what I'm after22:31
zaroasselin: i mean output all the confis to files so you can use upload it anywhere.22:31
fungiamrith: i'm trying to find if that'll be set in any useful context available to your gate_hook22:32
zaroasselin: i'm not really understanding what you want to do though22:32
amrithfungi, easy for me to try it out :)22:32
openstackgerritJerry Zhao proposed openstack-infra/devstack-gate: WIP: restart ntpd to sync time
fungiamrith: looks like it's set in stackrc here so maybe?22:33
fungiworth a shot22:33
asselinzaro, you gave me some ideas of what I can try...but basically before merging JJB configurations, how does the developer gain confidence that the jjb configuration will do what they expect it to do?22:33
*** raunak has joined #openstack-infra22:34
amriththx fungi will give it a shot right now22:34
fungiasselin: we run a ci job that generates an xml dump of the proposed job config change and diffs it against an xml dump of the parent commit in git22:34
*** gordc has quit IRC22:34
fungiasselin: so that we can see the resulting jenkins xml configuration which would be applied if that change were to merge22:35
*** chlong has quit IRC22:35
*** piet has joined #openstack-infra22:35
fungiasselin: also runnable locally with `tox -e jjb` in a project-config checkout22:36
asselinfungi, right...that tells you what changed, but not necessarily if the developer has an error in their script.22:36
zaroasselin: yup basically do exactly as openstack does, or
asselinI'm looking for a way to say, I want to output the bash for job gate_xyz_test so I can run it myself and find my typos and other errors22:37
zaroasselin: jjb unit tests do some of that checking22:37
fungiasselin: yeah, actually trying to run the job is risky for us, since some jobs can be configured to run on privileged nodes with access to sensitive credentials, so we don't want jobs to be test-run without review22:37
asselintrue, but I thinking more in terms of holding a node and run it in that environment as the jenkins user.22:38
*** asettle has quit IRC22:38
asselinso manual testing for a trusted developer22:38
*** tpsilva has quit IRC22:38
mordredasselin: so - for zuul 2.5 we translate the jjb into ansible playbooks22:38
asselinthe issue is that there are often many scripts and it's quite a bit of work to extract all of them from the jjb configurations22:38
*** sshnaidm is now known as sshnaidm|afk22:38
*** hurgleburgler1 has quit IRC22:39
*** hurgleburgler has joined #openstack-infra22:39
fungiasselin: oh, for that i bet our ansible-launcher parser could be leveraged to dump a role as one large script you could run22:39
mordredasselin: for the transition to 3, we need to write a conversion tool to turn the jjb into ansible ...22:39
asselinI with zuul 2.5 this probably basically goes away. devs can just run the playbook and debug on their own from there22:39
mordreda thing like fungi just mentioned should be faily simple to write at the moment22:39
mordredwell, with zuul 3 it goes away22:39
mordredfrom now until 3 is ready, the thing you describe is certainly a thing I could see being valued if someone wrote it22:40
*** rbrndt has quit IRC22:41
*** hurgleburgler1 has joined #openstack-infra22:41
*** hurgleburgler has quit IRC22:41
asselinfungi, do you have the link to ansible-launcher parser? I'd like to dig into that and see if that can be useful for this problem.22:41
*** Na3iL has joined #openstack-infra22:42
*** rockyg has quit IRC22:43
pabelangerfor the most part zuulv2.5 almost give a user to ability to re-run ansible on a local VM to reproduce the test results.  We'd have to change very few things to make that work today22:43
amrithfungi, I tagged you on a review for this change; will let you know how it works out. thx. hi mordred, bye mordred22:43
pabelangerupdate inventory file for example22:44
fungiamrith: cool--i'm curious to know how it goes and whether you need further help22:44
amrithI'm curious too, and I'm sure I need a lot of help. I was told that from a young age.22:45
anteayamorning jhesketh22:45
mordredmorning jhesketh22:45
fungiamrith: i'm still told it at an old age22:46
mordredfungi: maybe if people still tell us that, it means we're still young22:47
amrithsometimes I don't remember whether I'm old or not22:48
*** xyang1 has quit IRC22:51
*** rbuzatu has joined #openstack-infra22:52
openstackgerritMerged openstack-infra/project-config: Return tripleo tempest job to experimental pipe
openstackgerritPaul Belanger proposed openstack-infra/zuul: Use {{ ansible_host }} for ssh-keyscan
*** PollyZ has joined #openstack-infra22:55
harlowjadhellmann  if u want to work through a similar kind of bot22:55
harlowja(or others with bot ideas)22:55
harlowjathat's a plugin that i've created, will switch to MTQQ soon22:55
openstackgerritMerged openstack-infra/project-config: Fix issue with reprepro-import distribution
harlowjaand will figure out some things around using the older config files22:56
*** rbuzatu has quit IRC22:56
pabelangermordred: fungi: noop change to start making it easier for users to run our ansible playbooks locally if they want too22:56
pabelangerzigo: try your build now, patch landed22:59
openstackgerritMonty Taylor proposed openstack-infra/subunit2sql: Trim microseconds before passing to sqlalchemy
*** ams__ has quit IRC23:04
*** esberglu has joined #openstack-infra23:07
*** gouthamr has joined #openstack-infra23:08
*** gouthamr_ has joined #openstack-infra23:09
*** amitgandhinz has joined #openstack-infra23:11
*** gouthamr has quit IRC23:13
*** amitgandhinz has quit IRC23:15
*** roxanaghe has quit IRC23:18
openstackgerritMerged openstack-infra/puppet-zuul: Ansible launcher: Pin JJB to 1.6.1
*** esberglu has quit IRC23:18
*** sarob has joined #openstack-infra23:19
*** sdague has quit IRC23:20
openstackgerritJames E. Blair proposed openstack-infra/zuul: WIP: Re-enable test_connection tests
openstackgerritJames E. Blair proposed openstack-infra/zuul: Stop sharing Gerrit event queues in tests
openstackgerritJames E. Blair proposed openstack-infra/zuul: Update category mapping in tests
openstackgerritJames E. Blair proposed openstack-infra/zuul: WIP Fix addFailTest
jeblairjhesketh, notmorgan: ^23:21
mordredjeblair: ooh23:21
jeblairjhesketh, notmorgan: 352636 is the big change, and should be ready to go.  635 is the thing i ran into after that, and 637 is me right in the middle of making a change to fix the next thing.  but i'm going to EOD soon, so i wanted to push that up.  hopefully with that fixed, 949 should start working again23:22
*** PollyZ has quit IRC23:23
jeblairjhesketh, notmorgan: sorry, 635 is the big change, 636 is the thing after it.  :)23:23
*** thorst has joined #openstack-infra23:24
*** inc0 has quit IRC23:24
*** nwkarsten has joined #openstack-infra23:24
jeblairjhesketh: you also have a pair of reviews from mordred and i on 22333323:25
notmorganjeblair: ok looking23:26
mordredjhesketh: I was just talking to mtreinish about that actually (not that that is important information) ... but there is a thing I suggest that I think the 'right' way for me to suggest it is a follow on patch that you can squash into yours23:26
jheskethjeblair: awesome, thanks (will take a look)23:27
openstackgerritJim Rollenhagen proposed openstack-infra/elastic-recheck: Add bug 1610960
openstackbug 1610960 in neutron "Invalid input for external_gateway_info. Reason: '' is not a valid UUID." [Critical,Confirmed]
jheskethmordred: sure23:27
*** inc0 has joined #openstack-infra23:29
*** kien-ha has joined #openstack-infra23:29
mordredjroll: that is the evil bug from earlier?23:29
jrollmordred: it is23:30
*** roxanaghe has joined #openstack-infra23:30
jrollthe patch merged a couple hours ago23:30
jrollthe patch we rechecked to test failed earlier, though, so -.-23:30
JayFjroll: if my metrics on the agent client can fail a tempest test, our metrics lib is f'd23:30
mordredjroll: cool. I have some patches up to test fixes to neutronclient23:31
JayFjroll: especially with the noop metric logger23:31
*** roxanaghe has quit IRC23:31
mordredjroll: but- given gate load, it might take me a couple of days to cycle through them to be confident of a real fix23:31
mordredjroll: so the pin is definitely the right choice23:31
jrollmordred: talking in -qa about fixing the query, btw23:31
*** sarob has quit IRC23:32
*** sarob has joined #openstack-infra23:33
*** zz_dimtruck is now known as dimtruck23:34
openstackgerritJim Rollenhagen proposed openstack-infra/elastic-recheck: Add bug 1610960
openstackbug 1610960 in neutron "Invalid input for external_gateway_info. Reason: '' is not a valid UUID." [Critical,Confirmed]
*** nwkarsten has quit IRC23:37
*** Julien-zte has quit IRC23:38
*** dingyichen has joined #openstack-infra23:38
*** ggillies has joined #openstack-infra23:38
*** markvoelker has quit IRC23:39
*** mtanino has quit IRC23:40
*** inc0 has quit IRC23:42
*** hurgleburgler has joined #openstack-infra23:42
*** hurgleburgler1 has quit IRC23:44
*** furlongm_ has quit IRC23:44
*** xarses_ has quit IRC23:45
*** zhurong has joined #openstack-infra23:46
*** nwkarsten has joined #openstack-infra23:46
*** rfolco has joined #openstack-infra23:46
*** PollyZ has joined #openstack-infra23:50
*** rbuzatu has joined #openstack-infra23:53
openstackgerritKien Ha proposed openstack-infra/jenkins-job-builder: Update docker build publish to use convert xml
*** hurgleburgler has quit IRC23:55
*** zhurong has quit IRC23:55
*** hurgleburgler1 has joined #openstack-infra23:55
*** sdake has joined #openstack-infra23:56
*** sarob has quit IRC23:57
JayFI'd love a review on for ds-gate; already has a +1 from mtreinish and sambetts (from Ironic). Pretty simple devstack-gate change to enable us changing a default in devstack. (And yes, we do have an open bug currently being worked to remove ds-g as an intermediary for those vars that don't need it)23:58
*** rbuzatu has quit IRC23:58

Generated by 2.14.0 by Marius Gedminas - find it at!