Tuesday, 2016-01-05

bknudson_gerrit goes to sleep for a few minutes00:03
anteayabknudson_: how do you mean?00:05
bknudson_anteaya: it took 2 minutes to post a change. happens at 0 UTC00:06
anteayabknudson_: backup time00:08
*** piet has quit IRC00:09
*** gordc has quit IRC00:10
*** ddieterly has quit IRC00:11
*** otter768 has joined #openstack-infra00:13
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit diff timeout options  https://review.openstack.org/26348500:13
*** asselin_ has joined #openstack-infra00:15
asselin_Thelo, double check your configuration setting for ssh access00:17
*** bharathm_ has joined #openstack-infra00:17
*** bharathm has quit IRC00:17
*** otter768 has quit IRC00:18
TheloI just did00:19
Thelonow I am having issue to restart nodepool00:19
Thelosomehow it does not start00:19
asselin_Thelo, you using the nodepool user? anything in the log file?00:20
*** kzaitsev_mb has quit IRC00:20
lifelesshello infra! looks like we're going to want to do a 0.11.1 release of pbr, because kilo :/. But we have no branch for kilo. Whats the best way to get one (either called feature/0.11, like our 0.10 branch, or stable/kilo - either is fine IMO)00:20
*** bharathm_ has quit IRC00:22
Theloasselin_: Yes ( http://paste.openstack.org/show/482989/)00:22
*** bharathm has joined #openstack-infra00:22
*** bharathm has quit IRC00:22
openstackgerritmelanie witt proposed openstack-infra/elastic-recheck: Remove query for bug 1501435  https://review.openstack.org/26348800:22
openstackbug 1501435 in python-novaclient "osc 1.7 no longer can boot a server from volume" [Critical,Fix released] https://launchpad.net/bugs/1501435 - Assigned to melanie witt (melwitt)00:22
*** bharathm has joined #openstack-infra00:23
*** ashleighfarnham has quit IRC00:23
jheskethjeblair: so noting the zuul github work has come out of WIP... is that something we want in parallel to v3? I feel like there's been a lot of work there and we should consider merging it sooner rather than later00:23
asselin_Thelo, that's usually because of a config file issue. Look at the command being run by the service and run it manually as non-daemon and you should be able to get the error00:24
*** hdd has quit IRC00:25
asselin_Thelo, I think it's -d optino00:25
mordredjhesketh, jeblair: I think it would be useful for other things if it existed - but I do not know what the impact on v3 is/would be00:25
jeblairjhesketh: i wasn't planning on looking at it until after v3; it's a slippery slope which is why i decided 6 months ago that the connections stuff is where i'd draw the line00:25
*** Qiming has joined #openstack-infra00:25
Theloasselin_: How can I know the command, the logs print nothing00:26
*** Swami_ has quit IRC00:26
jeblair(i'm keen on the idea, just have to draw the line somewhere)00:26
jheskethagreed... I guess it depends how far the reporters/triggers get refactored00:26
jheskethmaybe it'll be easy for it to come along for the v3 ride00:26
asselin_Thelo, I believe you can find it in /etc/init.d/nodepool or a file there. That's what service nodepool start calls itself.00:27
jeblairjhesketh: i'm hoping not a lot in spirit (even if we move the code around a bit, their basic inputs and outputs should be about the same i think, so hopefully it will be a smallish rebase)00:27
*** bhunter71 has quit IRC00:27
asselin_Thelo, also, I think you should be root to run it...I'm not sure nodepool user has sudo access00:28
waynrzaro: would you mind if I add you as a reviewer on the commits in https://review.openstack.org/#/q/topic:jjb-2.0.0-api ? also, I keep forgetting other JJB contributor's IRC nicks so who else should I ping here?00:28
anteayawaynr: https://review.openstack.org/#/admin/groups/194,members00:29
openstackgerritKhai Do proposed openstack-infra/system-config: Increase Gerrit's diff timeout  https://review.openstack.org/26349000:29
*** sabeen3 has quit IRC00:30
waynranteaya: cool, thanks!00:30
anteayawaynr: in the gerrit navigation at the top, people > list groups > enter group name (usually repo based) in search text field00:31
*** rbrndt_ has quit IRC00:31
waynranteaya: huh, I only see 5 membes on that group, but when I try to add the group as a reviewer on the review it warns that there are 14 people in the group00:31
waynroh right, it includes infra-core00:31
anteayalook at included groups00:31
*** rbrndt_ has joined #openstack-infra00:32
*** Sukhdev has quit IRC00:32
waynrwould it be annoying if I added that whole group to each review in the topic?00:32
*** dims has joined #openstack-infra00:33
*** dims_ has quit IRC00:34
openstackgerritSachi King proposed openstack/requirements: Run Kilo with PBR latest  https://review.openstack.org/26349400:34
rm_workanyone aware of something funky with tempest changing recently that would cause "tempest_lib.exceptions.InvalidCredentials: Invalid Credentials" on everything?00:35
Theloasselin_: when I run the command manually (start-stop-daemon --start --quiet --pidfile $PIDFILE -c $USER --exec $DAEMON --test ) I get the following message ( Would start /usr/local/bin/nodepoold  (as user nodepool[1002], and group [1002]).), but still no deamon running00:36
jheskethjeblair: yep, if that's the case though do we want to look at it merging sooner (realising this goes against the line in the sand) for two reasons. 1) There's a lot of work there so the less rebasing the better, and by having it in tree it'll make v3 more concious of changes that would make the rebasing difficult. 2) it's a feature some may like to use00:36
jhesketh(I'm leaning towards sticking with the line in the sand and waiting to post v3 fwiw)00:37
asselin_Thelo, I think it's the next line. This one is --test00:38
asselin_but you don't want to use the start-stop-daemon00:38
asselin_you want to run it outside of daemon context. so it would be /usr/local/bin/nodepoold with the args specified in that file00:38
rm_work... is there a tempest channel?00:39
JayFrm_work: probably #openstack-qa00:39
anteayawaynr: I would find that annoying00:43
*** tzn has joined #openstack-infra00:43
*** jpr has joined #openstack-infra00:44
*** bharathm has quit IRC00:48
*** zz_dimtruck is now known as dimtruck00:49
*** tzn has quit IRC00:50
openstackgerritkevin proposed openstack-infra/project-config: New Project - jacket  https://review.openstack.org/26350000:51
waynranteaya: yeah I just added a comment to the first commit indicating that there are other commits to follow00:52
openstackgerritMerged openstack-infra/shade: Normalize server objects  https://review.openstack.org/26262700:52
anteayawaynr: okay00:52
waynri guess y'all are pretty savvy with this gerrit review thing and would realize that anyway00:53
anteayait appears that 263500 is a repo proposed to be for other drivers, which doesn't make a whole lot of sense to me00:53
anteayawaynr: using the same topic for a series is very helpful00:53
anteayawaynr: and if the commits are children of a parent commit, gerrit tells us that, so does gertty00:54
waynryeah these are all jjb-2.0.0-api topic00:54
anteayaa common topic makes them easy to find00:54
anteayaas a group00:54
waynryeah i really like that about gerrit, makes it easy to see related changes across projects00:56
*** eil397 has quit IRC00:57
anteayathe gerrit devs work hard at creating multiple ways for patches to be grouped and discovered, it is true00:57
*** baoli has joined #openstack-infra01:00
*** ybathia has quit IRC01:04
*** ybathia has joined #openstack-infra01:04
*** baoli has quit IRC01:06
*** mtanino has quit IRC01:08
*** eliqiao has quit IRC01:12
*** eliqiao has joined #openstack-infra01:13
anteaya163.com appears to be Korean gmail01:14
*** pvaneck has quit IRC01:15
openstackgerritKhai Do proposed openstack-infra/system-config: Show toggle CI button for Gerrit URLs with patch set ranges  https://review.openstack.org/26111001:16
*** fawadkhaliq has quit IRC01:18
mordredShrews: yay! the normalize patch landed!01:18
*** Sukhdev has joined #openstack-infra01:18
dahooanteaya: it is chinese01:19
anteayadahoo: ah thank you01:19
Shrewsmordred: yeah. left a comment about a followup that is non-urgent01:19
anteayagoogle translate led me astray01:19
mordredShrews: thanks for mentioning - I nevr would see that comment otherwise now that it's landed01:20
mordredbut I have now seen it01:20
dahooanteaya: could you help me with the gerrit web login issue01:20
dahooanteaya: openid doesn't work. i can login to review-dev.o.o though01:21
*** otter768 has joined #openstack-infra01:21
anteayawell I think clarkb was helping you earlier01:21
anteayaand as far as I could tell following along, I had thought you were able to remember your password and log in01:22
anteayais that not the case?01:22
dahooclarkb: could you think of other cause to the openid issue with gerrit01:22
dahooanteaya: ssh worked, but not openid01:22
anteayaoh I had thought you had gotten into your account01:23
dahooanteaya: unfortunately no01:23
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Allow puppet to set Gerrit sshd options to control connections  https://review.openstack.org/26282001:24
dahooanteaya: did gerrit change to a new server also?01:24
dahooanteaya: i noticed the fingerprint changed01:24
anteayano we upgraded gerrit in place01:24
*** sputnik13_ has quit IRC01:24
anteayawe used the same server as we had been using01:25
anteayawe did change the server last spring01:25
anteayawhen we upgraded the operating system01:25
TheloHi, I get an import error in nodepool. is anyone familiar with that (http://paste.openstack.org/show/482994/)01:27
anteayadahoo: http://lists.openstack.org/pipermail/openstack-infra/2015-February/002425.html01:28
openstackgerritSpencer Krum proposed openstack-infra/shade: Allow inventory filering by cloud name  https://review.openstack.org/26351301:29
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure the Gerrit's sshd.batchThreads option  https://review.openstack.org/26205301:29
*** ybathia has quit IRC01:31
*** otter768 has quit IRC01:31
openstackgerritSpencer Krum proposed openstack-infra/shade: Allow inventory filering by cloud name  https://review.openstack.org/26351301:33
*** dimtruck is now known as zz_dimtruck01:34
*** asselin_ has quit IRC01:34
*** piet has joined #openstack-infra01:36
*** weshay has quit IRC01:36
*** Liuqing has joined #openstack-infra01:37
TheloDoes anyone know how nodepool connect to a node? I guess through the private key defined in nodepool.yaml, however when I check the vm I see that it does not have any key.01:41
pleia2Thelo: have you seen the nodepool docs? they may be helpful to you :) http://docs.openstack.org/infra/nodepool/01:43
* mordred is excited to see nibalizer shade patch01:43
*** claudiub has quit IRC01:44
*** amotoki has quit IRC01:45
anteayanibalizer does cast a large shadow it is true01:45
nibalizermordred: its wip because I don't know how to write tests for it yet01:45
*** tzn has joined #openstack-infra01:46
nibalizerbut Shrews has agreed to help me with that tomorrow01:46
*** sam_wan has joined #openstack-infra01:49
*** tlian has quit IRC01:49
*** thorst has joined #openstack-infra01:49
*** thorst_ has joined #openstack-infra01:51
*** piet has quit IRC01:53
*** thorst has quit IRC01:53
*** piet has joined #openstack-infra01:54
*** piet has quit IRC01:54
*** piet has joined #openstack-infra01:55
*** tlian has joined #openstack-infra01:56
Thelopleia2: Thanks, I see that I need to specify the private key, however I don't see where I should specify the public key01:59
*** fawadkhaliq has joined #openstack-infra02:00
*** bpokorny_ has joined #openstack-infra02:00
*** ddieterly has joined #openstack-infra02:02
*** sridhar_ram has quit IRC02:02
*** bpokorny has quit IRC02:03
*** zz_dimtruck is now known as dimtruck02:04
*** bpokorny_ has quit IRC02:05
openstackgerritkevin proposed openstack-infra/project-config: New Project - jacket  https://review.openstack.org/26350002:05
*** zhurong has joined #openstack-infra02:09
*** gsagie has quit IRC02:14
openstackgerritLars Kellogg-Stedman proposed openstack-infra/shade: correct rpmlint errors  https://review.openstack.org/26352602:15
*** baoli has joined #openstack-infra02:16
*** rfolco_ has joined #openstack-infra02:18
*** rfolco_ has quit IRC02:18
*** baoli has quit IRC02:18
*** baoli has joined #openstack-infra02:18
*** unicell has joined #openstack-infra02:18
*** tzn has quit IRC02:19
*** unicell has quit IRC02:20
*** hdd has joined #openstack-infra02:21
lifelessmordred: so ping02:22
*** dims has quit IRC02:25
*** gsagie has joined #openstack-infra02:27
*** thorst_ has quit IRC02:27
*** unicell has joined #openstack-infra02:28
*** thorst has joined #openstack-infra02:28
*** baoli has quit IRC02:28
*** unicell1 has joined #openstack-infra02:28
*** unicell has quit IRC02:28
*** thorst has quit IRC02:29
*** thorst has joined #openstack-infra02:29
*** amuller has quit IRC02:31
openstackgerritMerged openstack-infra/zuul: Disable most tests  https://review.openstack.org/25501902:34
*** Sukhdev has quit IRC02:34
openstackgerritMerged openstack/requirements: Bump notifier up to 1.0.3  https://review.openstack.org/25690002:36
openstackgerritMerged openstack-infra/zuul: Add tenants  https://review.openstack.org/25445702:45
*** dims has joined #openstack-infra02:46
*** dimtruck is now known as zz_dimtruck02:47
*** zz_dimtruck is now known as dimtruck02:48
openstackgerritMerged openstack-infra/zuul: Update the test config with a function  https://review.openstack.org/25541302:49
*** jyuso1 has joined #openstack-infra02:50
*** dimtruck is now known as zz_dimtruck02:55
*** zz_dimtruck is now known as dimtruck02:56
*** dimtruck is now known as zz_dimtruck03:00
*** zz_dimtruck is now known as dimtruck03:06
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/requirements: Keep py3.X compatibility for urllib  https://review.openstack.org/26119003:09
openstackgerritMerged openstack-infra/zuul: Merge configurations with multiple layout files  https://review.openstack.org/25541403:09
*** tlian has quit IRC03:10
*** dims has quit IRC03:14
*** tlian has joined #openstack-infra03:15
*** tzn has joined #openstack-infra03:16
*** darvon has quit IRC03:18
*** piet has quit IRC03:19
*** Qiming has quit IRC03:21
*** piet has joined #openstack-infra03:26
*** pcrews has quit IRC03:29
*** pcrews has joined #openstack-infra03:30
*** otter768 has joined #openstack-infra03:32
*** yamahata has quit IRC03:33
*** hdd has quit IRC03:33
*** signed8bit has joined #openstack-infra03:35
openstackgerritJohn L. Villalovos proposed openstack-infra/devstack-gate: Update README.rst on how users can reproduce gate issues  https://review.openstack.org/26354803:35
*** Qiming has joined #openstack-infra03:41
*** david-lyle_ has joined #openstack-infra03:49
*** tzn has quit IRC03:49
*** bharathm has joined #openstack-infra03:50
*** Sukhdev has joined #openstack-infra03:50
*** links has joined #openstack-infra03:52
*** tlian has quit IRC03:52
*** bharathm has quit IRC03:54
*** amotoki has joined #openstack-infra03:54
*** unicell1 has quit IRC03:57
*** amotoki has quit IRC03:58
*** signed8bit has quit IRC03:59
*** amotoki has joined #openstack-infra03:59
*** apoorvad has joined #openstack-infra04:00
*** yamahata has joined #openstack-infra04:01
*** flepied has quit IRC04:01
*** asselin_ has joined #openstack-infra04:03
asselin_Thelo, http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/manifests/single_node_ci.pp#n4604:05
*** bpokorny has joined #openstack-infra04:08
*** appprod0 has left #openstack-infra04:09
openstackgerritMerged openstack-infra/zuul: Load in-repo configuration  https://review.openstack.org/25560104:09
openstackgerritMerged openstack-infra/zuul: Remove stop method from reporters, sources, triggers  https://review.openstack.org/25606704:09
openstackgerritMerged openstack-infra/zuul: Remove metajobs  https://review.openstack.org/25819704:09
*** bpokorny has quit IRC04:10
*** bpokorny has joined #openstack-infra04:11
*** sam_wan has quit IRC04:20
*** hdd has joined #openstack-infra04:22
*** sam_wan has joined #openstack-infra04:22
*** armax has joined #openstack-infra04:22
*** piet has quit IRC04:24
*** harlowja_at_home has joined #openstack-infra04:27
*** thorst has quit IRC04:27
*** thorst has joined #openstack-infra04:28
*** aeng has quit IRC04:34
*** hdd has quit IRC04:34
oomichihi, I tried to release a new python library(tempest-lib 0.13.0) yesterday, but https://pypi.python.org/pypi/tempest-lib doesn't show it still now.04:35
oomichihttp://tarballs.openstack.org/tempest-lib/ contains the tar file(tempest-lib-0.13.0.tar.gz), can someone help me?04:36
*** thorst has quit IRC04:36
*** nelsnels_ has joined #openstack-infra04:40
openstackgerritJoão Vale proposed openstack-infra/jenkins-job-builder: Add view management functionality  https://review.openstack.org/20617804:40
clarkboomichi: pull up the jobs logs for the sha1 of the tag at http://logs.openstack.org/first2charsofsha1/sha104:41
*** tsg_ has quit IRC04:42
*** nelsnelson has quit IRC04:42
*** dahoo has quit IRC04:42
openstackgerritMerged openstack-infra/zuul: Add job inheritance and start refactoring  https://review.openstack.org/25819804:45
*** tzn has joined #openstack-infra04:46
*** treaki__ has joined #openstack-infra04:46
oomichiclarkb: the jobs logs is the operation log of `git push gerrit <tag number>` ?  if so, I lost it because it happened yesterday..04:47
*** tzn has quit IRC04:50
*** treaki_ has quit IRC04:51
clarkboomichi: yes, you can get those logs from the url I showed you04:52
clarkbyou just have to replace the sha104:53
oomichiclarkb: thanks, will try it04:53
*** fawadkhaliq has quit IRC04:54
*** fawadkhaliq has joined #openstack-infra04:55
*** asselin_ has quit IRC04:57
*** asselin_ has joined #openstack-infra04:58
*** ddieterly has quit IRC05:01
clarkboomichi: http://logs.openstack.org/f0/f0b44f3e4292353b186ee3b32d9b89f9bfe9d894/release/tempest-lib-pypi-both-upload/92ab2a5/console.html05:01
*** signed8bit has joined #openstack-infra05:01
clarkboomichi: pypi errored05:01
clarkbonce we know pypi is working again we can reschedule the upload job05:01
oomichiclarkb: thanks!05:02
oomichiclarkb: this kind of thing happens usually?05:02
clarkbI don't think so05:03
*** PsionTheory has quit IRC05:03
oomichiclarkb: ok, I was unlucky05:03
oomichiclarkb: thanks anyways :)05:04
*** Liuqing has quit IRC05:05
*** zhurong has quit IRC05:05
*** zhurong has joined #openstack-infra05:05
*** Liuqing has joined #openstack-infra05:06
*** signed8__ has joined #openstack-infra05:06
*** Sukhdev has quit IRC05:07
*** Sukhdev has joined #openstack-infra05:09
*** signed8bit has quit IRC05:09
*** david-lyle_ has quit IRC05:09
*** sdake has quit IRC05:10
*** Sukhdev has quit IRC05:11
*** Ramanjaneya has joined #openstack-infra05:12
*** Sukhdev has joined #openstack-infra05:13
*** otter768 has quit IRC05:16
*** dahoo has joined #openstack-infra05:18
*** Sukhdev has quit IRC05:20
*** Sukhdev has joined #openstack-infra05:21
*** abregman has joined #openstack-infra05:23
*** aeng has joined #openstack-infra05:24
*** julim has joined #openstack-infra05:24
*** piet has joined #openstack-infra05:30
*** nikil89_ has joined #openstack-infra05:31
*** unicell has joined #openstack-infra05:34
*** thorst has joined #openstack-infra05:34
*** Thelo has quit IRC05:35
*** jaosorior has joined #openstack-infra05:37
*** Thelo has joined #openstack-infra05:38
*** harlowja_at_home has quit IRC05:39
*** nikil89_ is now known as naggappan05:39
*** bpokorny has quit IRC05:40
*** thorst has quit IRC05:42
*** signed8bit has joined #openstack-infra05:43
*** piet has quit IRC05:45
*** signed8__ has quit IRC05:46
*** tzn has joined #openstack-infra05:47
*** Sukhdev has quit IRC05:47
openstackgerritAndreas Jaeger proposed openstack-infra/gearman-plugin: Use git.openstack.org  https://review.openstack.org/26230805:48
*** apoorvad has quit IRC05:51
*** Liuqing has quit IRC05:51
*** tzn has quit IRC05:51
openstackgerritAndreas Jaeger proposed openstack-infra/gearman-plugin: Remove broken scm section  https://review.openstack.org/26356605:55
openstackgerritAndreas Jaeger proposed openstack-infra/gearman-plugin: Remove broken scm section  https://review.openstack.org/26356605:56
openstackgerritAndreas Jaeger proposed openstack-infra/zmq-event-publisher: Remove broken scm section  https://review.openstack.org/26356805:58
*** sputnik13 has joined #openstack-infra05:59
*** signed8bit has quit IRC05:59
*** sputnik13 has quit IRC06:01
*** gyee has quit IRC06:01
*** ddieterly has joined #openstack-infra06:02
*** coolsvap|away is now known as coolsvap06:02
*** jaosorior has quit IRC06:05
*** sputnik13 has joined #openstack-infra06:06
*** ddieterly has quit IRC06:07
*** sputnik13 has quit IRC06:09
*** unicell has quit IRC06:11
*** unicell has joined #openstack-infra06:14
*** coolsvap is now known as coolsvap|away06:17
*** coolsvap|away is now known as coolsvap06:20
openstackgerritEinst Crazy proposed openstack-infra/zuul: Use assertTrue/False instead of assertEqual(T/F)  https://review.openstack.org/26357506:22
*** ildikov_afk is now known as ildikov06:22
*** sputnik13 has joined #openstack-infra06:23
*** sputnik13 has quit IRC06:24
*** hdd has joined #openstack-infra06:25
*** jaosorior has joined #openstack-infra06:26
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/26357606:26
*** hdd has quit IRC06:29
*** flepied has joined #openstack-infra06:32
*** MarkAtwood has quit IRC06:32
dougwigpabelanger: have you gotten any reports of the apscheduler bump in zuul causing issues?  i just spent the day figuring out why my zuul was hanging at start, and it narrowed down to that.06:38
*** thorst has joined #openstack-infra06:39
*** aopincar has joined #openstack-infra06:46
*** thorst has quit IRC06:47
*** tzn has joined #openstack-infra06:47
*** dizquierdo has joined #openstack-infra06:48
*** apoorvad has joined #openstack-infra06:51
*** tzn has quit IRC06:52
*** unicell1 has joined #openstack-infra06:53
*** unicell has quit IRC06:54
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/26359206:59
*** apoorvad has quit IRC06:59
*** ddieterly has joined #openstack-infra07:04
*** ddieterly has quit IRC07:09
openstackgerritQiming Teng proposed openstack/requirements: Bump python-senlinclient version to 0.2.1  https://review.openstack.org/26359807:10
openstackgerritAlexandre Viau proposed openstack-dev/pbr: exclude from files in _find_git_files  https://review.openstack.org/26329707:13
*** otter768 has joined #openstack-infra07:17
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/26357607:19
*** otter768 has quit IRC07:22
*** oomichi has quit IRC07:24
*** dahoo has quit IRC07:29
*** Hal has joined #openstack-infra07:32
*** Hal is now known as Guest7525007:32
*** irenab has quit IRC07:34
*** flepied has quit IRC07:35
*** e0ne has joined #openstack-infra07:43
*** salv-orlando has joined #openstack-infra07:44
*** nmagnezi has joined #openstack-infra07:44
*** flepied has joined #openstack-infra07:47
*** tzn has joined #openstack-infra07:48
*** samuelBartel has joined #openstack-infra07:52
*** _nadya_ has joined #openstack-infra07:52
openstackgerrityolanda.robla proposed openstack-infra/project-config: Split puppet installation into its own element  https://review.openstack.org/19841707:52
*** tzn has quit IRC07:53
*** _nadya_ has quit IRC07:53
*** aopincar has quit IRC07:53
*** apoorvad has joined #openstack-infra07:56
openstackgerrityolanda.robla proposed openstack-infra/project-config: Split puppet installation into its own element  https://review.openstack.org/19841707:58
*** coolsvap is now known as coolsvap|away08:00
*** salv-orl_ has joined #openstack-infra08:01
*** apoorvad has quit IRC08:02
*** jtomasek has joined #openstack-infra08:02
*** ddieterly has joined #openstack-infra08:04
*** salv-orlando has quit IRC08:05
*** salv-orl_ has quit IRC08:05
*** irenab has joined #openstack-infra08:06
*** ifarkas has joined #openstack-infra08:07
*** markus_z has joined #openstack-infra08:08
*** ddieterly has quit IRC08:09
*** claudiub has joined #openstack-infra08:12
openstackgerritMerged openstack-infra/project-config: Remove python3 checks from Dragonflow gate  https://review.openstack.org/26298108:15
*** arxcruz has joined #openstack-infra08:22
*** sam_wan has quit IRC08:23
*** saggi has joined #openstack-infra08:23
saggiHi, we have the smaug meeting every two weeks on even weeks. This month there is a string of two odd weeks. Do we have the slot today (since it's been two weeks) or do we have it next week (since it's an odd week)?08:25
*** sam_wan has joined #openstack-infra08:26
*** matrohon has joined #openstack-infra08:26
*** armax has quit IRC08:28
*** shardy has joined #openstack-infra08:33
*** mpaolino has joined #openstack-infra08:35
*** fhubik has joined #openstack-infra08:37
*** AJaeger has joined #openstack-infra08:37
AJaegersaggi, there were some mails on the openstack-dev list about these odd weeks - and there's a calendar exported for your meeting on eavesdrop that should guide you08:38
AJaegerso, the meeting will be in the next even week08:38
*** flepied has quit IRC08:42
*** fhubik is now known as fhubik_brb08:44
AJaegeryolanda: happy new year! if you have time later for reviews, could you look at https://review.openstack.org/#/c/261828/ and https://review.openstack.org/261743 , please?08:45
*** treaki__ has quit IRC08:45
*** treaki has joined #openstack-infra08:45
*** bexelbie_ has joined #openstack-infra08:47
*** bexelbie_ is now known as bexelbie08:48
*** dingyichen has quit IRC08:48
*** jlanoux has joined #openstack-infra08:48
*** thorst has joined #openstack-infra08:49
*** tzn has joined #openstack-infra08:49
saggiAJaeger: The ics files[1] don't specify even\odd week so they say that we have a meeting today and Neutron next week and not the other way around. If you say our meeting is next week the ICS files should be fixed. http://eavesdrop.openstack.org/#Smaug_Project_Meeting08:50
*** sfinucan has joined #openstack-infra08:52
*** tzn has quit IRC08:53
*** _nadya_ has joined #openstack-infra08:53
*** fawadkhaliq has quit IRC08:56
*** thorst has quit IRC08:56
*** _nadya_ has quit IRC08:58
*** fawadkhaliq has joined #openstack-infra08:59
*** hashar has joined #openstack-infra09:03
AJaegersaggi: Read http://lists.openstack.org/pipermail/openstack-dev/2016-January/083216.html09:03
*** ddieterly has joined #openstack-infra09:05
*** jordanP has joined #openstack-infra09:05
*** tzn has joined #openstack-infra09:05
*** femnad has joined #openstack-infra09:06
*** _nadya_ has joined #openstack-infra09:06
*** _nadya_ has quit IRC09:08
*** zeih has joined #openstack-infra09:08
*** thiagop has quit IRC09:09
*** ddieterly has quit IRC09:09
*** apoorvad has joined #openstack-infra09:10
*** apoorvad has quit IRC09:10
*** apoorvad has joined #openstack-infra09:10
yolandaAJaeger, thanks for the review. I picked an old version of my patch to fix the rebase, so i repeated my typos09:11
openstackgerrityolanda.robla proposed openstack-infra/project-config: Split puppet installation into its own element  https://review.openstack.org/19841709:12
AJaegerRun git review -d XXX to download first next time ;)09:12
yolandayep, clever advice :)09:13
*** abregman has quit IRC09:13
odyssey4meHNY yolanda :) if you have a moment, please help by doing the last review on https://review.openstack.org/263341 and https://review.openstack.org/26336009:14
odyssey4memorning AJaeger :)09:14
*** abregman has joined #openstack-infra09:14
*** flepied has joined #openstack-infra09:15
AJaegermorning, odyssey4me09:15
yolandahi odyssey4me, i'll take a look09:15
*** kzaitsev_mb has joined #openstack-infra09:15
*** fhubik_brb is now known as fhubik09:16
*** coolsvap|away is now known as coolsvap09:17
*** otter768 has joined #openstack-infra09:18
*** tzn has quit IRC09:20
*** nihilifer has quit IRC09:20
*** jistr has joined #openstack-infra09:21
*** nihilifer has joined #openstack-infra09:21
*** hashar has quit IRC09:21
*** jcoufal has joined #openstack-infra09:22
*** zhurong has quit IRC09:22
*** otter768 has quit IRC09:22
*** kzaitsev_mb has quit IRC09:24
AJaegeryolanda: did you see my request earlier for review as well?09:27
yolandaAJaeger, no , i was out09:27
* yolanda is still on a long holiday09:27
yolandabut i see now, just a sec09:27
*** gampel has joined #openstack-infra09:29
AJaegeryolanda: in that case: go back to your holiday!09:30
AJaegeryolanda: this can wait...09:31
yolandai approved first one, for the second one i set a +2, but as I saw anteaya added some comments, i'd prefer to wait for her feedback to approve09:31
AJaegeryolanda: thanks!09:32
openstackgerritMerged openstack-infra/project-config: Merge openstack-ansible skip-if to resolve conflict  https://review.openstack.org/26336009:32
yolandano problem. I try to come back slow to reality, after that long period of holiday :)09:32
openstackgerritMerged openstack-infra/project-config: Skip openstack-ansible-docs gate/post jobs for juno branch  https://review.openstack.org/26334109:33
gampelHi all,I need help uploading the lost Smaug weekly meeting log due to my mis typing added space to the start; Can you please direct me to who could help me09:33
*** jyuso1 has quit IRC09:33
AJaegergampel: any of the infra-roots can help you - best come back during US morning...09:33
yolandahi gampel, if you have preserved the transcription somehow, you can send me and i can add it manually09:34
gampel yolanda:  thx, I have  preserved the log, whats your e-mail to send it ?09:35
yolandasend it to infra mailing list as an attach, i can pick it09:35
*** devvesa has joined #openstack-infra09:35
gampel yolanda: Thanks I will send it now09:36
openstackgerritMerged openstack-infra/project-config: check that osc plugins do not break openstackclient  https://review.openstack.org/26182809:36
*** apoorvad has quit IRC09:37
*** yaume has joined #openstack-infra09:40
nibalizerhi yolanda AJaeger :)09:41
*** fhubik is now known as fhubik_brb09:42
yolandahi nibalizer, how are you?09:42
nibalizergoing to bed soon but figured I'd say hi09:42
nibalizerhow was your holiday?09:42
*** devvesa has left #openstack-infra09:42
yolandastill enjoying the last 2 days. Very looong09:42
*** jaosorior has quit IRC09:44
*** jaosorior has joined #openstack-infra09:44
*** nmagnezi has quit IRC09:45
*** thorst has joined #openstack-infra09:54
*** jaosorior has quit IRC09:56
*** zeih has quit IRC09:56
*** jaosorior has joined #openstack-infra09:56
*** fhubik_brb is now known as fhubik09:56
AJaegernibalizer: hi!09:57
*** nmagnezi has joined #openstack-infra09:57
*** Guest75250 has quit IRC09:57
*** devvesa has joined #openstack-infra09:58
devvesaHi AJaeger09:58
AJaegerHi devvesa09:58
devvesaIt's about this comment: https://review.openstack.org/#/c/260093/2/gerrit/projects.yaml09:59
devvesaI understand that you don't want to push all the tags09:59
devvesaBut we prefer to keep them in the original repo09:59
AJaegerdevvesa: We will not delete your tags and branches in our imported version if you don't need them in the future - and we will not touch the original repo either ;)10:00
devvesaIt is acceptable if I fork the midonet/puppet-midonet to my GitHub namespace and I remove the tags there10:00
*** mdbooth has quit IRC10:00
AJaegerdevvesa: that'S what many do if they want to create a cleaned up repo for import10:00
AJaegerso, yes, perfectly fine.10:00
devvesaAJaeger: great. Thanks!10:01
*** mdbooth has joined #openstack-infra10:01
*** rguillebert has joined #openstack-infra10:01
AJaegerdevvesa: you're welcome10:01
*** thorst has quit IRC10:01
*** boris-42 has quit IRC10:03
*** ddieterly has joined #openstack-infra10:06
*** hashar has joined #openstack-infra10:07
AJaegerdevvesa: and then change gerrit/projects.yaml to point to the new repo10:08
devvesaAJaeger: Yup. On it10:08
*** _nadya_ has joined #openstack-infra10:09
*** mdbooth_ has joined #openstack-infra10:09
*** yamahata has quit IRC10:10
*** mdbooth has quit IRC10:10
*** mdbooth_ is now known as mdbooth10:10
*** ddieterly has quit IRC10:10
*** yuanying has quit IRC10:11
*** yuanying has joined #openstack-infra10:11
*** isaacb has joined #openstack-infra10:14
*** _nadya_ has quit IRC10:14
*** tzn has joined #openstack-infra10:14
openstackgerritMatthew Thode proposed openstack/diskimage-builder: Initial add of gentoo support for diskimage-builder  https://review.openstack.org/26334910:14
*** tzn has quit IRC10:14
*** tzn has joined #openstack-infra10:15
*** flepied has quit IRC10:15
*** Hal has joined #openstack-infra10:16
*** Hal is now known as Guest6886210:17
openstackgerritMerged openstack/requirements: Remove oslo-incubator from projects.txt  https://review.openstack.org/26114810:17
e0nehi team. I'm trying to add  patch on review for a new project: http://paste.openstack.org/show/483011/10:18
e0neAJaeger: hi. could you please help me ^^10:18
*** mdbooth has quit IRC10:19
openstackgerritTimur Sufiev proposed openstack-infra/project-config: Use scripts from horizon repo in integration job test hooks  https://review.openstack.org/26345710:19
*** flepied1 has joined #openstack-infra10:19
AJaegere0ne: that repo does not include a gitreview file ;(10:19
AJaegere0ne: oh, it does...10:20
openstackgerritJaume Devesa proposed openstack-infra/project-config: Add puppet-midonet to OpenStack infrastructure  https://review.openstack.org/26009310:20
*** zeih has joined #openstack-infra10:20
AJaegere0ne: it's broken, let me send a patch ;)10:20
e0neAJaeger: ok, thanks for the help!10:21
* e0ne waits for the patch link10:21
e0neoh.. we have to replace "brick-python-cinderclient-ext" with "python-brick-cinderclient-ext" in .gitreview10:22
AJaegere0ne: git review -d 263662 ;git review -s and then start again...10:22
AJaegere0ne: exactly - approve 263662 if you can ;)10:22
AJaegeror ask for cinder core review help10:23
e0neAJaeger: thanks for the help. it was my mistake during on if the cycles of renames10:23
*** Qiming has quit IRC10:23
AJaegerthis block everybody contributing to that repo...10:23
AJaegere0ne: I suggest you ask for additional reviews yourself to push this through...10:24
e0neAJaeger: yes, I'll ask somebody else to approve it10:24
e0neonce CI passed10:24
*** sshnaidm_ has joined #openstack-infra10:25
AJaegere0ne: thanks10:25
*** zeih has quit IRC10:25
*** mdbooth has joined #openstack-infra10:27
*** dtantsur|afk is now known as dtantsur10:28
*** yuanying has quit IRC10:28
*** yuanying has joined #openstack-infra10:29
*** abregman is now known as abregman|brb10:30
*** fhubik is now known as fhubik_brb10:31
*** zhurong has joined #openstack-infra10:33
*** abregman|brb is now known as abregman10:34
*** tzn has quit IRC10:35
*** mdbooth has quit IRC10:35
*** tzn has joined #openstack-infra10:35
*** tzn has quit IRC10:36
*** zeih has joined #openstack-infra10:38
*** apoorvad has joined #openstack-infra10:39
*** kzaitsev_mb has joined #openstack-infra10:42
*** salv-orlando has joined #openstack-infra10:44
*** fhubik_brb is now known as fhubik10:44
*** dims has joined #openstack-infra10:47
*** masber has joined #openstack-infra10:55
*** yamamoto has joined #openstack-infra10:56
*** apoorvad has quit IRC10:57
*** samuelBartel has quit IRC10:58
*** thorst has joined #openstack-infra10:59
*** cdent has joined #openstack-infra11:02
*** kzaitsev_mb has quit IRC11:02
*** lucas-dinner is now known as lucasagomes11:03
*** yamamoto has quit IRC11:04
*** thorst has quit IRC11:06
*** ddieterly has joined #openstack-infra11:07
*** apoorvad has joined #openstack-infra11:07
*** apoorvad has quit IRC11:08
openstackgerritJulien Danjou proposed openstack-infra/project-config: tooz: gate on etcd  https://review.openstack.org/26367811:09
*** _nadya_ has joined #openstack-infra11:10
*** ddieterly has quit IRC11:11
*** salv-orlando has quit IRC11:13
*** salv-orlando has joined #openstack-infra11:14
*** _nadya_ has quit IRC11:15
*** yuanying has quit IRC11:15
*** kzaitsev_mb has joined #openstack-infra11:15
*** salv-orlando has quit IRC11:16
*** salv-orlando has joined #openstack-infra11:17
*** salv-orlando has quit IRC11:18
*** BobBall has quit IRC11:18
*** otter768 has joined #openstack-infra11:19
*** salv-orlando has joined #openstack-infra11:20
*** salv-orl_ has joined #openstack-infra11:21
*** salv-orlando has quit IRC11:21
*** fhubik is now known as fhubik_brb11:22
*** salv-orl_ has quit IRC11:23
*** salv-orlando has joined #openstack-infra11:23
*** BobBall has joined #openstack-infra11:23
*** otter768 has quit IRC11:24
*** salv-orlando has quit IRC11:24
*** dimtruck is now known as zz_dimtruck11:25
*** salv-orlando has joined #openstack-infra11:26
*** zeih has quit IRC11:28
*** salv-orlando has quit IRC11:29
*** salv-orlando has joined #openstack-infra11:29
*** salv-orlando has quit IRC11:30
*** kzaitsev_mb has quit IRC11:31
*** Qiming has joined #openstack-infra11:31
*** salv-orlando has joined #openstack-infra11:32
*** salv-orlando has quit IRC11:35
*** daemontool has joined #openstack-infra11:35
*** salv-orlando has joined #openstack-infra11:36
*** dizquierdo has quit IRC11:39
*** sam_wan has quit IRC11:41
*** zeih has joined #openstack-infra11:42
*** tzn has joined #openstack-infra11:43
*** salv-orlando has quit IRC11:43
*** salv-orl_ has joined #openstack-infra11:43
*** ericksonsantos has joined #openstack-infra11:43
*** bradjones has joined #openstack-infra11:44
*** bradjones has quit IRC11:44
*** bradjones has joined #openstack-infra11:44
*** salv-orl_ has quit IRC11:46
*** yaume has quit IRC11:46
*** salv-orlando has joined #openstack-infra11:47
*** salv-orlando has quit IRC11:47
*** salv-orl_ has joined #openstack-infra11:51
*** salv-orl_ has quit IRC11:53
*** coolsvap is now known as coolsvap|away11:54
*** ihrachys has joined #openstack-infra11:55
*** salv-orl_ has joined #openstack-infra11:55
*** salv-orl_ has quit IRC11:56
*** salv-orlando has joined #openstack-infra11:57
*** salv-orlando has quit IRC11:58
*** mpaolino has quit IRC12:00
*** salv-orlando has joined #openstack-infra12:04
*** zz_dimtruck is now known as dimtruck12:04
*** thorst has joined #openstack-infra12:04
sdagueAJaeger: so zantana is going to strip out the authors now - https://review.openstack.org/#/c/252197/35/nova/locale/ja/LC_MESSAGES/nova.po  ?12:05
*** salv-orlando has quit IRC12:05
sdagueI just want to make sure all that looks like you expect before I approve it12:06
AJaegerlet me double check, sdague12:06
*** erlon has joined #openstack-infra12:06
*** Ramanjaneya has quit IRC12:07
*** ddieterly has joined #openstack-infra12:07
*** salv-orlando has joined #openstack-infra12:10
*** sfinucan has quit IRC12:10
*** sfinucan has joined #openstack-infra12:10
*** rhallisey has joined #openstack-infra12:11
*** thorst has quit IRC12:11
openstackgerritPaul Michali proposed openstack-infra/project-config: Constraint based experimental jobs for neutron-vpnaas  https://review.openstack.org/26339812:12
*** salv-orlando has quit IRC12:12
*** salv-orl_ has joined #openstack-infra12:12
*** ddieterly has quit IRC12:12
*** bexelbie has quit IRC12:12
*** salv-orl_ has quit IRC12:13
*** fhubik_brb is now known as fhubik12:13
AJaegersdague: let me investigate this a bit more - thanks for spotting this.12:13
*** kushal has quit IRC12:14
*** fawadkhaliq has quit IRC12:15
AJaegersdague: I'll -1 the change and will ask StevenK and pleia2 for help via email...12:15
*** dimtruck is now known as zz_dimtruck12:15
*** fawadkhaliq has joined #openstack-infra12:16
*** tzn has quit IRC12:16
*** piet has joined #openstack-infra12:20
*** fhubik has quit IRC12:21
*** devvesa has quit IRC12:32
*** bexelbie has joined #openstack-infra12:36
bexelbiejoin #centos-docs12:36
openstackgerritFausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client  https://review.openstack.org/25534912:36
*** rfolco has joined #openstack-infra12:39
*** gongysh has joined #openstack-infra12:40
*** andreykurilin__ has joined #openstack-infra12:40
*** aopincar has joined #openstack-infra12:42
*** piet has quit IRC12:42
openstackgerritFausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client  https://review.openstack.org/25534912:43
*** trown|outttypeww is now known as trown12:43
*** piet has joined #openstack-infra12:44
*** salv-orlando has joined #openstack-infra12:47
*** fawadkhaliq has quit IRC12:47
*** thorst has joined #openstack-infra12:47
*** fawadkhaliq has joined #openstack-infra12:48
*** salv-orlando has quit IRC12:48
*** bexelbie has quit IRC12:51
*** devvesa has joined #openstack-infra12:51
*** kzaitsev_mb has joined #openstack-infra12:52
*** baoli has joined #openstack-infra12:53
*** dprince has joined #openstack-infra12:54
*** moravec has quit IRC12:54
*** aopincar has quit IRC12:55
*** moravec has joined #openstack-infra12:56
*** devvesa has quit IRC12:59
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve usability of the "New Board" modal  https://review.openstack.org/26371713:00
*** bexelbie has joined #openstack-infra13:01
*** amuller has joined #openstack-infra13:02
*** piet has quit IRC13:04
*** zz_dimtruck is now known as dimtruck13:07
*** masber has quit IRC13:08
*** ddieterly has joined #openstack-infra13:08
*** mattt has joined #openstack-infra13:08
*** links has quit IRC13:10
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: ovh cirros crashing bug  https://review.openstack.org/26372213:11
*** _nadya_ has joined #openstack-infra13:12
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add tests for stack search API  https://review.openstack.org/25972713:12
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Fix filtering in search_stacks()  https://review.openstack.org/26372413:12
*** zhurong has quit IRC13:12
*** gordc has joined #openstack-infra13:12
*** ddieterly has quit IRC13:12
*** jpr has quit IRC13:15
*** _nadya_ has quit IRC13:16
*** dimtruck is now known as zz_dimtruck13:20
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve usability of the "New Worklist" modal  https://review.openstack.org/26372813:21
*** amuller_ has joined #openstack-infra13:21
*** moravec has quit IRC13:22
*** hashar has quit IRC13:23
*** _amrith_ is now known as amrith13:24
*** samuelBartel has joined #openstack-infra13:24
*** piet has joined #openstack-infra13:24
*** moravec has joined #openstack-infra13:25
*** amuller has quit IRC13:25
*** edmondsw has joined #openstack-infra13:27
*** pradk has joined #openstack-infra13:27
*** pradk has quit IRC13:27
*** nmagnezi has quit IRC13:28
*** hashar has joined #openstack-infra13:29
*** kgiusti has joined #openstack-infra13:30
*** sdake has joined #openstack-infra13:31
*** kzaitsev_mb has quit IRC13:31
*** weshay_xchat has joined #openstack-infra13:34
*** weshay_xchat is now known as weshay13:34
openstackgerritMerged openstack-infra/elastic-recheck: ovh cirros crashing bug  https://review.openstack.org/26372213:36
*** ddieterly has joined #openstack-infra13:41
*** moravec has quit IRC13:42
*** Daisy has joined #openstack-infra13:43
*** tzn has joined #openstack-infra13:44
*** nmagnezi has joined #openstack-infra13:44
*** shardy has quit IRC13:47
*** tlian has joined #openstack-infra13:48
*** tzn has quit IRC13:48
*** shardy has joined #openstack-infra13:49
*** mpaolino has joined #openstack-infra13:52
*** bcourt has joined #openstack-infra13:52
*** dtantsur is now known as dtantsur|brb13:54
*** bexelbie- has joined #openstack-infra13:55
*** bexelbie has quit IRC13:55
*** bexelbie- is now known as bexelbie13:55
*** tzn has joined #openstack-infra13:55
*** piet has quit IRC13:56
*** piet has joined #openstack-infra13:57
*** kushal has joined #openstack-infra14:00
*** lucasagomes is now known as lucas-hungry14:00
*** amotoki has quit IRC14:00
*** signed8bit has joined #openstack-infra14:01
*** Daisy has quit IRC14:02
*** dkranz has joined #openstack-infra14:02
*** dslev has joined #openstack-infra14:02
*** zz_dimtruck is now known as dimtruck14:04
*** fawadkhaliq has quit IRC14:04
yolandagampel, ping. This will be the first meeting for smaug? i don't see the folder created on eavesdrop14:04
*** yaume has joined #openstack-infra14:06
*** signed8bit has quit IRC14:07
*** amuller_ is now known as amuller14:07
*** ddieterly has quit IRC14:07
*** kzaitsev_mb has joined #openstack-infra14:09
gampelyolanda: yes thanks it was the first one14:11
*** Anticimex has joined #openstack-infra14:11
*** signed8bit has joined #openstack-infra14:11
Anticimexis there anyone around to help me change my gerrit (review.openstack.org) username? (created it before i understood it should === linux username)14:12
AJaegerAnticimex: it does not need to be the same as your linux username14:13
AJaegerAnticimex: please follow http://docs.openstack.org/infra/manual/developers.html#account-setup14:13
Anticimexfine, but i have conflicts due to other review.* systems, so i end up having to configure it manually per repo which is a hassle14:13
*** devvesa has joined #openstack-infra14:14
*** sdake has quit IRC14:14
AJaegerAnticimex: I have to leave now and can't help with that - but this is a one time setup per repo...14:14
AJaegerfungi, clarkb, could you review https://review.openstack.org/#/c/263678/ ? We removed etcd from our images and now they download it via a script, I like a review on that approach, please14:15
*** Sukhdev has joined #openstack-infra14:15
AnticimexAJaeger: ack, i guess i could just script "make me new openstack gerrit folder"14:16
*** AJaeger has quit IRC14:16
*** krtaylor has joined #openstack-infra14:17
fungiAnticimex: you can set per-destination-host default usernames in your ~/.ssh/config14:17
Anticimexfungi: now that's an excellent fix, i do that already.. great :)14:18
fungiyeah, just create a Host review.openstack.org section in there and tell it what your gerrit username is14:19
*** woodster_ has joined #openstack-infra14:19
*** zeih has quit IRC14:21
*** ericksonsantos has quit IRC14:22
openstackgerritmathieu bultel proposed openstack-infra/shade: Add normalize stack function for heat stack_list  https://review.openstack.org/25433714:23
*** darvon has joined #openstack-infra14:24
*** links has joined #openstack-infra14:24
*** darvon has quit IRC14:25
*** darvon has joined #openstack-infra14:25
*** jsavak has joined #openstack-infra14:25
yolandagampel, i created here http://eavesdrop.openstack.org/meetings/smaug/2015/14:26
yolandanaming and formatting may not be the best...14:26
*** Sukhdev has quit IRC14:26
openstackgerritMonty Taylor proposed openstack-dev/pbr: Split changelog on nulls instead of (  https://review.openstack.org/26374814:27
*** Sukhdev has joined #openstack-infra14:27
pabelangerdougwig: No reports yet. Do you have more details on the issue?14:27
*** nmagnezi has quit IRC14:27
mordredfungi: ^^ that pbr patch fixes a thing that's bugged me for quite a while14:27
*** zeih has joined #openstack-infra14:28
*** Sukhdev has quit IRC14:29
*** dimtruck is now known as zz_dimtruck14:29
*** _nadya_ has joined #openstack-infra14:29
*** sdake has joined #openstack-infra14:30
*** kzaitsev_mb has quit IRC14:30
*** jsavak has quit IRC14:33
openstackgerritMonty Taylor proposed openstack-dev/pbr: Split changelog on nulls instead of (  https://review.openstack.org/26374814:33
*** sdake has quit IRC14:33
*** jsavak has joined #openstack-infra14:34
pabelangerdougwig: trying to reproduce locally now14:34
*** regXboi has joined #openstack-infra14:34
*** sdake has joined #openstack-infra14:34
*** esikachev has joined #openstack-infra14:35
gampelyolanda: thank you very much14:37
*** irenab has quit IRC14:39
*** denisra has joined #openstack-infra14:40
*** signed8b_ has joined #openstack-infra14:42
*** esikachev has quit IRC14:43
*** ddieterly has joined #openstack-infra14:43
*** dtantsur|brb is now known as dtantsur14:44
*** signed8bit has quit IRC14:45
*** moravec has joined #openstack-infra14:46
*** erlon has quit IRC14:46
*** Qiming has quit IRC14:48
*** mriedem_away is now known as mriedem14:48
Anticimexfungi: so i've done the .ssh/config entry, but "git review -s" initialization in a repo overrides that with "git config --get gitreview.username" for its ssh'ing14:49
isaacbyolanda: Hi, can you please review: https://review.openstack.org/#/c/262466/14:49
*** links has quit IRC14:49
*** dkranz has quit IRC14:49
Anticimexbut i guess i can work around this by some git review args or just manually setting up the remote, thx again14:49
*** dkranz has joined #openstack-infra14:50
*** hashar_ has joined #openstack-infra14:50
*** hashar has quit IRC14:53
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve usability of the "New Board" modal  https://review.openstack.org/26371714:53
*** zz_dimtruck is now known as dimtruck14:53
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve usability of the "New Worklist" modal  https://review.openstack.org/26372814:54
*** signed8b_ has quit IRC14:55
*** zeih has quit IRC14:57
*** ddieterl_ has joined #openstack-infra14:58
*** amitgandhinz has joined #openstack-infra14:58
*** jsavak has quit IRC15:00
*** jsavak has joined #openstack-infra15:01
*** ddieterly has quit IRC15:01
*** sigmavirus24_awa is now known as sigmavirus2415:02
*** isaacb has quit IRC15:02
*** lucas-hungry is now known as lucasagomes15:05
*** jsavak has quit IRC15:06
openstackgerritIhar Hrachyshka proposed openstack-infra/project-config: Add neutron on-steroids job template for advanced features  https://review.openstack.org/24769715:06
*** zeih has joined #openstack-infra15:06
*** fawadkhaliq has joined #openstack-infra15:07
*** peristeri has joined #openstack-infra15:08
fungiAnticimex: if it's not possible to make git-review fall back on ssh's username default (and omit the username@ in its git remotes) then i'd consider that a bug worth fixing15:09
EmilienMnibalizer: on meeting-415:09
EmilienMnibalizer: 25-26-27 of January would work for you for the puppet sprint?15:09
Anticimexfungi: yeah, doesn't appear to work, but, git-openstack.sh to the rescue15:10
Anticimexfungi: git-review falls back on appending $USER if it doesn't get anything from the "git config --get gitreview.username"15:11
pabelangerdougwig: works for me.  I'd be interested in hearing what issues you found15:11
Anticimex(which is clearly redundant...)15:11
*** zeih has quit IRC15:12
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve usability of "Edit board" form  https://review.openstack.org/26376115:12
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve the usability of the "Edit project" form  https://review.openstack.org/26376215:13
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Bug fix: Cinder v2 returns bools now  https://review.openstack.org/26376415:14
*** dizquierdo has joined #openstack-infra15:14
Shrewsmordred: ^^^15:14
*** eharney has joined #openstack-infra15:15
*** isaacb has joined #openstack-infra15:16
*** peristeri has quit IRC15:17
*** bhunter71 has joined #openstack-infra15:17
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve usability of the "Edit project" form  https://review.openstack.org/26376215:18
*** ddieterl_ has quit IRC15:18
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Improve usability of the "Edit board" form  https://review.openstack.org/26376115:18
*** Ramanjaneya has joined #openstack-infra15:18
*** otter768 has joined #openstack-infra15:20
*** wznoinsk has quit IRC15:21
amullerquick Q - anyone remember offhand where's the bit of code that forbids direct 'sudo' usage (without rootwrap)? devstack-gate?15:21
Anticimexfungi: done: https://storyboard.openstack.org/#!/story/200045315:22
*** gampel has quit IRC15:23
*** jsavak has joined #openstack-infra15:23
*** bexelbie has quit IRC15:23
*** bexelbie has joined #openstack-infra15:23
*** bcourt has quit IRC15:24
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: add bug for subnode failing to come up  https://review.openstack.org/26376915:25
*** otter768 has quit IRC15:25
mordredShrews: woot15:26
mordredShrews: in return, I just put up an OCC patch to add IBM Cloud15:26
*** changbl has quit IRC15:27
fungiamuller: yes, in devstack-vm-gate-wrap.sh it defaults DEVSTACK_GATE_REMOVE_STACK_SUDO to 1 and then uses that to decide whether devstack-vm-gate.sh should remove the /etc/sudoers.d/50_stack_sh file, but you should be able to pass in a 0 if you need to undo that for some reason (i'd recommend consulting the devstack maintainers in #openstack-qa though to find out if there's a saner solution to15:27
fungiwhatever you're trying to do there)15:27
*** arxcruz has quit IRC15:28
*** bhunter71 has quit IRC15:29
*** ganso has quit IRC15:29
*** bhunter71 has joined #openstack-infra15:30
amullerfungi: thanks now I know what to grep for :) I just noticed that some of the neutron functional tests are running with sudo directly and passing15:31
*** jsavak has quit IRC15:31
fungiAnticimex: thanks! i left a comment that it may need some additional consideration for non-ssh protocol urls, but otherwise i agree 100% with the recommendation there15:31
Anticimexgood catch15:32
*** jsavak has joined #openstack-infra15:32
*** kzaitsev_mb has joined #openstack-infra15:32
*** piet has quit IRC15:32
*** peristeri has joined #openstack-infra15:34
fungiamuller: oh, well the rootwrap enforcement is for the "stack" user specifically, to make sure that devstack isn't calling sudo directly. that doesn't prevent tests (running as the "jenkins" or "tempest" users) from possibly calling sudo15:35
*** Thelo has quit IRC15:35
fungiso depending on what uses of sudo you're seeing, that may or may not be relevant15:35
*** dslev has quit IRC15:35
*** shardy has quit IRC15:35
amullerfungi: thanks I'll look in to it15:36
fungiand when i say devstack using sudo, of course i really mean things running in the context of devstack (so nova, neutron, cinder, whatever) are being forced to go through rootwrap15:36
*** arxcruz has joined #openstack-infra15:37
*** ociuhandu has joined #openstack-infra15:37
*** Thelo has joined #openstack-infra15:37
openstackgerritPaul Belanger proposed openstack-infra/zuul: Update to statsd 3.0  https://review.openstack.org/22923215:39
*** ayoung has quit IRC15:41
*** naggappan has quit IRC15:41
openstackgerritPaul Belanger proposed openstack-infra/zuul: Update to statsd 3.0  https://review.openstack.org/22923215:41
*** cdent has quit IRC15:42
*** signed8b_ has joined #openstack-infra15:43
*** signed8b_ has quit IRC15:43
*** piet has joined #openstack-infra15:44
*** shardy has joined #openstack-infra15:44
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: use send-announcements-to metadata for deliverables  https://review.openstack.org/26228115:44
openstackgerritmathieu bultel proposed openstack-infra/shade: Add normalize stack function for heat stack_list  https://review.openstack.org/25433715:45
*** Daisy has joined #openstack-infra15:46
*** piet has quit IRC15:46
*** piet has joined #openstack-infra15:47
*** apoorvad has joined #openstack-infra15:48
*** signed8b_ has joined #openstack-infra15:49
*** Daisy has quit IRC15:51
*** bcourt has joined #openstack-infra15:52
*** apoorvad has quit IRC15:52
*** pblaho has quit IRC15:53
*** cdent has joined #openstack-infra15:53
openstackgerritMerged openstack-infra/elastic-recheck: add bug for subnode failing to come up  https://review.openstack.org/26376915:56
*** bgmccollum has left #openstack-infra15:57
pleia2good morning15:57
*** weshay_xchat has joined #openstack-infra15:58
*** baoli has quit IRC15:59
fungiyay! the best thing about a script you only run once every few months is that it's a guaranteed source of work to update every time you do run it15:59
openstackgerritPaul Michali proposed openstack-infra/project-config: Skip if doc only change for neutron-lib  https://review.openstack.org/26378616:00
fungithe atcs script needs updating to deal with gerrit 2.11 database schema differences, it appears16:00
*** baoli has joined #openstack-infra16:00
*** mtanino has joined #openstack-infra16:00
clarkbcan probably completely rest it now? of course the rest api changes too so :(16:01
*** weshay has quit IRC16:01
*** devvesa has quit IRC16:03
pleia2sdague: re: translations script pulling out authors, we have a mechanism in our script to update the headers for the year and this is the first change of the year for nova translations, I *believe* that's what is wiping out the old translation author headers, we'll have to dig into it16:04
sdaguepleia2: mostly I want to know if it's desired behavior16:05
sdagueif I'm told yes, that's fine, I'll approve that through16:05
sdaguebut it was notable enough I figured I'd ask16:05
pleia2sdague: I don't really know why we track authors in the header, so I'm going to chat with AJaeger about it16:05
* pleia2 nods16:05
pleia2good catch16:05
pabelangermorning peeps! have 2 reviews open related to nodepool requirements.  Mostly uncapping or bumping them to latest releases: https://review.openstack.org/#/q/status:open+project:openstack-infra/nodepool+branch:master+topic:fedora-packaging This is part of my effort to package our tools into Fedora / CentOS16:05
openstackgerritMerged openstack-infra/subunit2sql: Stop using implict joins  https://review.openstack.org/26177116:06
openstackgerritAlexandre Viau proposed openstack-dev/pbr: exclude from files in _find_git_files  https://review.openstack.org/26329716:06
clarkbpabelanger: have you tested https://review.openstack.org/#/c/262019/2/requirements.txt works with jenkins?16:07
*** pblaho has joined #openstack-infra16:07
anteayafungi: yay summit passes16:07
fungiclarkb: i started looking into that and there's still some user account info not presented through the rest api even in 2.1116:08
fungii have a partway completed attempt at reimplementing it via rest api16:09
*** isaacb has quit IRC16:09
*** cdent has quit IRC16:09
fungii believe where i got stuck is that there's no detailed change owner information in the query results so i can't map it to an account16:09
*** tonytan4ever has joined #openstack-infra16:09
*** femnad has quit IRC16:10
openstackgerritAlexandre Viau proposed openstack-dev/pbr: exclude from files in _find_git_files  https://review.openstack.org/26329716:10
*** pblaho has quit IRC16:10
fungiif i find time i'll write up a feature request for gerrit to add that (i doubt i'd have time to figure out how to try to implement it in gerrit myself)16:11
*** pblaho has joined #openstack-infra16:11
*** femnad has joined #openstack-infra16:11
*** matrohon has quit IRC16:11
fungiif memory serves, it only provides an owner name but no option for including uniquely identifying details (e-mail address, username, anything) which would allow me to then do an account lookup query16:12
clarkbno account id?16:12
fungithat would be ideal16:13
*** matrohon has joined #openstack-infra16:13
fungiit's possible i'm wrong about that. now i can't recall if i tested against 2.11 on review-dev at the time or simply relied on their api docs (the latter could be lagging behind) but it was definitely not working on 2.816:14
*** austin81 has joined #openstack-infra16:14
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Bug fix: Cinder v2 returns bools now  https://review.openstack.org/26376416:14
*** signed8b_ is now known as signed8bit_ZZZzz16:15
openstackgerritMerged openstack-infra/tripleo-ci: Don't build everything in periodic jobs  https://review.openstack.org/24451916:15
openstackgerritMerged openstack-infra/tripleo-ci: Set DELOREAN_REPO_URL for the master periodic job  https://review.openstack.org/24452616:15
jeblairfungi: i think gertty maps changes to owners via id lemme dbl check16:15
jeblairfungi: account = session.getAccountByID(remote_change['owner']['_account_id']16:16
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Bug fix: Cinder v2 returns bools now  https://review.openstack.org/26376416:16
jeblairfungi: naturally, the underscore is there because the mysql db is only temporary until it's all in git, so the account id will go away when they figure out how to make it go away :)16:16
*** dslev has joined #openstack-infra16:17
*** signed8bit_ZZZzz is now known as signed8b_16:17
*** pblaho has quit IRC16:17
pabelangerclarkb: Ya, I ran it locally before christmas.  Let me reconfirm16:17
*** pblaho has joined #openstack-infra16:17
jeblairfungi: but that's via a rest api query of the change, not an ssh query16:18
*** hdd has joined #openstack-infra16:19
fungijeblair: yeah, i want to say that the problem is that if you query for a set of changes you can't get them to include that and so you end up needing to make a rest api call for every single change id to find the owner? or maybe i just didn't explore that avenue--it's been months since i tried. i'll give it another shot16:19
fungisort of odd that you can have it include detailed account info for reviewers on each change but not the owner16:20
*** pblaho has quit IRC16:20
*** pblaho has joined #openstack-infra16:20
jeblairfungi: it shows up in gertty's queries (which are big batches) with &o=DETAILED_ACCOUNTS16:20
fungihuh. i could just be misremembering what i ended up stuck on in that case16:21
*** pblaho has quit IRC16:22
jeblairi wonder of the method of querying (and recombining results after paging, etc, in gertty is now complicated enough to move it into gerritlib)16:22
*** pblaho has joined #openstack-infra16:22
jeblairfungi: oh look!  https://review.openstack.org/Documentation/rest-api-accounts.html#list-account-emails16:22
jeblairfungi: i _think_ that means that we can make the script entirely rest with no local db access!16:23
fungiyeah, i'm paging back in where i left off to see what exactly i was stuck on16:23
jeblair(since as i recall, we needed the local db in order to get the non-primary addresses)16:24
fungioh, that may have been where i actually got stuck16:24
*** hdd has quit IRC16:24
fungitrying to find non-primary addresses16:24
*** zeus has quit IRC16:24
fungiin 2.816:24
jeblairyeah, no idea if that was there then16:24
*** PsionTheory has joined #openstack-infra16:25
jeblairfungi: do you want me to take a stab at resting this?  i recently refreshed the query code in gertty so it's fresh in my mind.  but i don't want to steal your fun if you were looking forward to it.16:26
pabelangerclarkb: anything specific I should try? nodepool is able to delete / launch new nodes to jenkins without an issue locally.  This is using pyzmq==15.1.016:26
*** woodster_ has quit IRC16:26
*** pradk has joined #openstack-infra16:26
clarkbpabelanger: just that it can receive events ok, so nodes should go ready -> used -> delete based on jenkins events16:27
fungijeblair: i'd welcome the help. i'll check how far i got and push it up as a wip if it's not trivial to finish now16:27
*** vgridnev has joined #openstack-infra16:27
jeblairfungi: cool.  i'll eat breakfast, and i bet we can have something well before the meeting.16:27
pabelangerclarkb: Yup, nodepool works as expected16:28
*** zeus has joined #openstack-infra16:29
*** zeus is now known as Guest5433216:30
fungijeblair: aha, i am definitely getting owner details from change queries now16:30
*** rbrndt__ has joined #openstack-infra16:31
*** rbrndt_ has quit IRC16:32
*** vgridnev_ has joined #openstack-infra16:32
*** vgridnev has quit IRC16:32
openstackgerritMerged openstack-infra/reviewstats: Fix --start with existing cache  https://review.openstack.org/26103916:32
openstackgerritMerged openstack-infra/reviewstats: Add release cycle management tools to projects  https://review.openstack.org/26125416:33
openstackgerritMerged openstack-infra/reviewstats: Update keystone projects  https://review.openstack.org/26126816:33
*** ddieterly has joined #openstack-infra16:33
openstackgerritMerged openstack-infra/reviewstats: Add rpm packaging to projects  https://review.openstack.org/26127316:35
*** ayoung has joined #openstack-infra16:35
openstackgerritMerged openstack-infra/reviewstats: Add OpenStack ansible to projects  https://review.openstack.org/26125516:35
openstackgerritMerged openstack-infra/reviewstats: Add manila to projects  https://review.openstack.org/26125216:36
openstackgerritMerged openstack-infra/reviewstats: Add i18n to projects  https://review.openstack.org/26126216:36
openstackgerritMerged openstack-infra/reviewstats: Add rally to projects  https://review.openstack.org/26126016:36
openstackgerritMerged openstack-infra/reviewstats: Update glance projects  https://review.openstack.org/26125116:36
openstackgerritMerged openstack-infra/reviewstats: Add deb packaging to projects  https://review.openstack.org/26127116:36
*** Guest68862 has quit IRC16:37
openstackgerritMerged openstack-infra/reviewstats: Add zaqar to projects  https://review.openstack.org/26125816:37
*** harlowja_at_home has joined #openstack-infra16:37
anteayayolanda: I understand you are on holidays still, I have reviewed https://review.openstack.org/#/c/261743/9 thanks for giving me time to do so, I have left a +1 if you want to approve16:37
*** sridhar_ram has joined #openstack-infra16:37
openstackgerritMerged openstack-infra/reviewstats: Add refstack to projects  https://review.openstack.org/26126116:37
openstackgerritMerged openstack-infra/reviewstats: Add congress to projects  https://review.openstack.org/26126716:37
openstackgerritMerged openstack-infra/reviewstats: Update heat project list  https://review.openstack.org/26126516:37
openstackgerritMerged openstack-infra/reviewstats: Add OpenStack Client to projects  https://review.openstack.org/26126316:39
*** piet has quit IRC16:39
*** piet has joined #openstack-infra16:40
openstackgerritJesse Pretorius proposed openstack-infra/reviewstats: Add additional repositories to OpenStack-Ansible  https://review.openstack.org/26379916:41
*** yaume has quit IRC16:42
*** ashleighfarnham has joined #openstack-infra16:43
*** weshay__xchat has joined #openstack-infra16:44
*** pblaho has quit IRC16:44
*** pblaho has joined #openstack-infra16:44
*** dprince has quit IRC16:45
openstackgerritDavanum Srinivas (dims) proposed openstack/os-testr: Support comments in whitelist files  https://review.openstack.org/26380316:45
*** weshay_xchat has quit IRC16:47
rajinirasselin: Figured the nodepool ssh timeout issue. The node takes more than 10 mins to boot but it is marked as active so ssh timesout16:48
clarkbthats really slow16:49
clarkbare you using qemu or real virt?16:49
*** dprince has joined #openstack-infra16:50
rajinirclarkb:  Qemu.Its devstack on a vm. I'm planning to setup the openstack on physical node(using kvm)16:51
rajinirclarkb: In the meantime the boot-timeout and launch-timeout in the nodepool.yaml is not affecting the ssh timeout16:52
*** armax has joined #openstack-infra16:53
*** unicell1 has quit IRC16:54
*** matrohon has quit IRC16:54
*** hdd has joined #openstack-infra16:54
clarkbboot-timeout is for that16:54
*** pblaho has quit IRC16:54
clarkbit is in seconds16:54
*** pblaho has joined #openstack-infra16:54
*** markus_z has quit IRC16:55
*** ericksonsantos has joined #openstack-infra16:55
sdagueso, if anyone is up for poking at a thing with ovh people - http://status.openstack.org//elastic-recheck/#153115516:56
sdaguecirros guests are crashing some times on boot, only on ovh nodes16:56
*** pblaho has quit IRC16:56
*** pblaho has joined #openstack-infra16:57
sdagueand, curiously, the subnode allocation failure looks to be rax-iad 100% of the time - http://status.openstack.org//elastic-recheck/#153118716:57
clarkbsdague could that be nested virt?16:57
sdagueclarkb: don't know16:57
*** kzaitsev_mb has quit IRC16:57
sdaguedid they enable nested kvm?16:58
clarkbyes its enabled because distros enable by default now16:58
asselin_rajinir, yeah, you'll want to run it on bare metal if you want the vm to do any useful work16:58
greghayneswhoa its enabled in rax?16:58
clarkbin ovh only16:58
greghaynesoh, ok16:58
jeblairclarkb: did we land a d-g change to use that?16:58
clarkbjeblair no, devstack uses it by default16:59
*** kzaitsev_mb has joined #openstack-infra16:59
clarkbso we would have to explicitly disable16:59
jeblairclarkb: i know we putzed around with something locally and decided it didn't make a huge difference16:59
greghaynesah, I was crossing the streams16:59
clarkbya not a big difference for performance16:59
rajinirasselin: thanks, will probably take that route. But not sure why boot-time is not effective16:59
jeblairclarkb: ah yeah, we turned it off sometimes16:59
rajinirasselin: I meant boot-timeout in the nodepool.yaml17:00
openstackgerritMerged openstack/os-testr: Add tool to create a subunit stream  https://review.openstack.org/26346517:00
*** gongysh has quit IRC17:00
jeblairclarkb, sdague: so i think that means we're using it on ovh.  if we add that line to d-g, we should disable it everywhere.  may be worth doing and seeing if the problem disappears17:00
clarkbrajinir it works on our integration tests which have long boot due to qemu17:00
greghaynesclarkb: so those nodepool builder changes - if I recheck them they will in theory 'just merge' - is that fine to do?17:00
sdaguejeblair: wfm17:00
*** ttx has quit IRC17:00
*** unicell has joined #openstack-infra17:01
*** Guest54332 is now known as zeus17:01
clarkbgreghaynes uh, would be good to line up a service restart too17:01
jeblairsdague, clarkb: patch in progress17:01
clarkbso we run the new code17:01
greghaynesclarkb: ok, thats why I was kind of holding off ;)17:01
*** ttx has joined #openstack-infra17:01
*** zeus has quit IRC17:01
*** zeus has joined #openstack-infra17:01
clarkbI have puppet thing today and doctors next two days, maybe friday or find another root?17:02
greghaynesclarkb: sure, so should I WIP the bottom of the stack or?17:02
anteayaclarkb: memory usage on logstashworker20 seems to have increased a bit but has leveled off and is sitting at 1G compared to 3G on logstashworker0117:02
dougwigpabelanger: i've since wiped the hang by reverting my zuul, to get my system back online, but it was hanging on this line, when importing the timer trigger:17:03
clarkbgreghaynes ya probably safest17:03
dougwigpabelanger: i expect some package combo is not happy.17:03
*** ayoung has quit IRC17:03
openstackgerritThomas Morin proposed openstack-infra/project-config: net-bagpipe: remove now unneeded -liberty jobs  https://review.openstack.org/26380917:04
asselin_rajinir, yeah...maybe there's a bug with that? You can try to investigate and see what the root cause it.17:04
mtreinishfungi, clarkb: btw, I finally got around to fixing that issue you reported on health way back on the first prototype deployment: https://review.openstack.org/#/c/259646/17:04
*** bexelbie has quit IRC17:05
mtreinishI did try using a histogram instead of a line graph, but it didn't look quite right because of the number of samples17:05
rajinirasselin: sure, will post a bug when I confirm the root cause17:05
*** jlanoux has quit IRC17:05
*** bexelbie has joined #openstack-infra17:06
openstackgerritJames E. Blair proposed openstack-infra/devstack-gate: Use qemu everywhere  https://review.openstack.org/26381017:07
jeblairclarkb, sdague: ^17:07
*** bexelbie has quit IRC17:07
*** bexelbie has joined #openstack-infra17:07
*** pblaho has quit IRC17:07
*** pblaho has joined #openstack-infra17:07
jeblairfungi: want to hand me the baton on the script or you still enthralled?17:08
fungijeblair: still poking at what i have. i think it's doable now17:08
*** gyee has joined #openstack-infra17:08
fungimostly just working out the logic to retain per project team atcs lists and handle extra atcs parsing17:09
*** zz_ja is now known as zz_zz_ja17:09
jeblairfungi: okay, i'll go elsewhere :)17:09
clarkbjeblair need to write it to localrc right?17:09
jeblairclarkb: it's already there17:09
jeblairclarkb: it was apparently a pattern-breaking variable that existed in the main script but not documented with a default in the wrap :(17:10
*** e0ne has quit IRC17:11
*** jsavak has quit IRC17:11
*** peristeri has quit IRC17:11
*** jsavak has joined #openstack-infra17:12
pleia2need to take a sick kitty to the vet, hopefully will make it back for the meeting, but depends on many vet timing things17:12
anteayapleia2: I hope kitty has good health17:12
pleia2anteaya: thanks17:12
jeblairclarkb: added in this change https://review.openstack.org/#/c/134348/317:12
*** Ramanjaneya has quit IRC17:13
jesusaurusclarkb: logstash-worker-20 seems to be doing well with memory \o/17:17
clarkbjesusaurus: ya I think switching to having logstash do the load balancing is going to be a good improvement17:17
dhellmannfungi : I'd like to find a time this week to talk about next steps for the release automation work. I think we have all of the scripts we need written, but they're going to rely on the artifact signing work and I need to know how to send email from the CI nodes. Do you have time this week to discuss?17:17
jesusaurusyep, looks like it17:17
clarkbjeblair: +3'd at least locally17:18
*** sfinucan has quit IRC17:18
*** samuelBartel has quit IRC17:18
clarkbmtreinish: nice17:18
*** moravec has quit IRC17:18
*** kzaitsev_mb has quit IRC17:19
*** jistr has quit IRC17:19
*** AJaeger has joined #openstack-infra17:19
clarkbthe vanilla tempest run that ran on bluebox yesterday passed17:19
clarkbjeblair: mordred: I think we should go ahead and revert the max-servers back to 39 or whatever it was17:19
jeblairclarkb: ++17:19
jeblairdhellmann: does today between 19:00 and 20:00 work for you? :)17:20
fungijeblair: the new api call to list account emails does in fact work as desired. it's only available as an authenticated endpoint method, so this script will still need credentials, but that's better than nothing17:20
mordredclarkb: ++17:20
clarkbjeblair: mordred cool let me get the patch up for you to review17:21
*** femnad has quit IRC17:21
fungidhellmann: i second jeblair's question--we can put it on today's meeting agenda17:21
jeblairfungi: see tools/maintain-infra-groups.py where i inexplicably have a script read in .gertty.yaml :)17:21
*** otter768 has joined #openstack-infra17:21
dhellmannfungi , jeblair : sure, we can do that, I thought it might be too short notice17:21
fungijeblair: heh. tempting. i have it currently just prompting with getpass()17:22
fungibut gertty.yaml can certainly be a first check17:22
*** baoli has quit IRC17:22
fungiand would now be consistent with an existing tool in the same repo ;)17:23
*** dizquierdo has quit IRC17:23
openstackgerritClark Boylan proposed openstack-infra/project-config: Run Bluebox at full IP address capacity  https://review.openstack.org/26381517:23
nibalizerclarkb: for bluebox?17:23
clarkbnibalizer: yup ^17:23
dhellmannjeblair , fungi : added17:24
nibalizerdid we get some confirmation they swapped cpus or hamsters or whatever they were going to do?17:24
fungidhellmann: thanks--looking forward to discussing!17:24
*** dahoo has joined #openstack-infra17:24
jeblairnibalizer: mordred confirmed they did17:24
*** baoli has joined #openstack-infra17:24
mordredyes. they have swapped cpus17:24
mordredthey have reported that they have swapped CPUs17:24
openstackgerritMerged openstack/os-testr: Support comments in whitelist files  https://review.openstack.org/26380317:24
jeblairthey mey have _actually_ swapped hamsters17:24
*** vgridnev_ has quit IRC17:24
mordredI have not independently verified that new cpus exist17:24
greghaynesthese hamsters have racing stripes17:25
*** fawadkhaliq has quit IRC17:25
*** ddieterly has quit IRC17:25
jeblairthey painted little stripes on them?  how cute!17:25
clarkbgerrit us up to mean heap use of 9GB17:25
AJaegerpleia2: good morning, do you have time for the authors discussion?17:26
clarkbmordred: neither have I fwiw, I have learned to not trust what VMs tell me17:26
*** otter768 has quit IRC17:26
clarkbgerrit GC time has spiked a few times in the lsat day but nothing too crazy17:26
mordredclarkb: I think my testing was going to be "look at performance graphs and see if it's faster"17:26
greghaynesyea, if they dont lie a bit then live migration is a pain17:26
pabelangerdougwig: if I am looking correctly, it seems related to this commit: https://github.com/openstack-infra/zuul/commit/352264b3c22b501ae53c1643e31df2ccdfc8938817:27
pabelangerdougwig: would be interesting to try to reproduce it17:27
*** moravec has joined #openstack-infra17:27
*** moravec has quit IRC17:27
*** dims has quit IRC17:29
anteayaAJaeger: she took her sick kitty to the vet17:29
*** baoli has quit IRC17:29
anteayaAJaeger: she hopes to be back in time for the team meeting but isn't sure17:29
AJaegerthanks, anteaya . Ok, I'll wait17:30
anteayaAJaeger: awesome17:30
krotscheckjeblair, mordred: I couldn't help but notice that the AFS cell you two set up already had a /mirror directory in it that was populated. Is that a thing that is already working, oooor....17:31
jeblairkrotscheck: tested a bit but not in prod; we need to land a change to bandersnatch to use it for pypi, and need to get around to it for git.17:31
*** baoli has joined #openstack-infra17:32
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual: Add limitations and caveats to cross dependencies  https://review.openstack.org/26174317:33
*** mudassirlatif has joined #openstack-infra17:33
AJaegerjeblair: Here's another revision of my cross dependencies change incorporating a suggestion by anteaya (thanks, anteaya !) - could you review it again, please?17:34
*** shardy has quit IRC17:34
anteayathank you, sorry you lost your 2 +2's to incorporate my suggestion, but thank you for doing so17:35
AJaegeranteaya: I rather get this one really clear and have a few questions less to answer ;)17:35
clarkbfungi: I just added a meeting topic that I think is becoming important as we draw near to the end of the month17:36
*** dims has joined #openstack-infra17:36
clarkbfungi: particularly since some of us will be semi afk traveling around then17:36
anteayaAJaeger: agreed, you have worked hard on it and is very good now, thank you17:36
*** shardy has joined #openstack-infra17:36
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Add heat lbaas specific test for v1 and v2  https://review.openstack.org/23715617:37
clarkbzaro: good morning, small gerrit memory use update in scrollback. TL;DR is mean heap use is now 9GB and we are seeing some graphable GC time17:37
AJaegerclarkb: could you review these, please? https://review.openstack.org/263566 and https://review.openstack.org/263566 The zmq-event-publisher change was your original change, I think this maven scm plugin will not work with OpenStack17:37
*** unicell has quit IRC17:37
AJaegerthanks, jeblair and anteaya for the reviews!17:38
clarkbAJaeger: thats the same change twice :) I think that was mostly just copy pasta from the example plugin17:39
zaroclarkb: from yesterday, correct?17:39
clarkbzaro: from this morning17:39
*** moravec has joined #openstack-infra17:39
clarkbAJaeger: zaro the pom.xml scm information is for things like eclipse or whatever to push and pull from the upstream?17:40
*** moravec1 has joined #openstack-infra17:40
*** austin81 has quit IRC17:40
AJaegerhttps://review.openstack.org/263568 - same change again, different repo and change ;)17:40
*** austin81 has joined #openstack-infra17:40
zarowaynr: np, the other jjb cores are zxiiro, hashar, electrofelix and of course infra-core17:41
clarkbAJaeger: if the tests for those manage to build properly I think its fine, but zaro should weigh in too I think he groks that the best17:41
AJaegerclarkb: sorry for wrong change - 263568 is the other one17:41
clarkbyup see it now thanks17:41
AJaegerclarkb: tests passed...17:42
*** dtantsur is now known as dtantsur|afk17:42
AJaegerzaro, can you check 263568 and 263566, please?17:42
clarkbgah, I fial at reading new change screen17:42
krotscheckjeblair: What change in bandersnatch?17:42
hashar_zaro: though I have been barely reviewing anything for months :-((( ( I should do my review duty ...17:42
bhunter71Is there something.. that limits the amount of usable RAM on a devstack-trusty/running dsvm instance?   That is, when I make a 16GB devstack-trusty, I see some evidence that Nova is only seeing ~7GB of free memory.  cgroups in play here?17:42
clarkbbhunter71: kernel boot parameter17:43
jeblairkrotscheck: https://bitbucket.org/pypa/bandersnatch/pull-requests/12/add-option-to-dir-hash-index-files/diff17:43
bhunter71thanks clarkb!17:43
krotscheckjeblair: ah-hah!17:43
jeblairkrotscheck: seems like they're ok with the change, i just need to figure out how to update an hg pr17:43
anteayahashar_: we miss you17:44
*** jaosorior has quit IRC17:44
*** moravec has quit IRC17:44
*** fawadkhaliq has joined #openstack-infra17:44
hashar_anteaya: yeah I do to. Been to busy in 2015 with misc random stuff at wikimedia :-D17:44
anteayahashar_: yay random stuff17:44
*** jaosorior has joined #openstack-infra17:44
clarkbbhunter71: its intentional to keep memory use semi sane across providers regardless of the underlying flavor size17:44
clarkbbhunter71: will probably go away at the end of the month when hpcloud is turned off since that is the only place we really need it right now17:45
*** jaosorior has quit IRC17:45
*** jaosorior has joined #openstack-infra17:45
clarkbAJaeger: approved the gearman plugin one and +2'd the zmq event plugin17:45
zaroAJaeger: reviewed17:45
*** bdemers has quit IRC17:45
AJaegerclarkb, zaro: Thanks!17:46
*** abregman has quit IRC17:46
*** amuller is now known as amuller_afk17:47
pleia2AJaeger: at the vet with my kitty this morning17:47
*** hashar_ has quit IRC17:47
krotscheckjeblair: So, your vision to set up an AFS mirrors is to have some form of a mirror-master, and then a bunch of mirror slaves?17:47
zxiirozaro: happy new year! I'm working on getting another student for the summer to work on JJB again.17:48
krotscheckI'm guessing only one privileged mirror master, because bandersnatch might step on its own feet17:48
AJaegerpleia2: understood - let's talk once you're back if I'm still around...17:48
*** bdemers has joined #openstack-infra17:48
pleia2AJaeger: yep, sur17:48
krotscheckI'm just exploring this right now. Am going to get the wheel-thingie working first, then start thinking about mirror things.17:48
mordredkrotscheck: kinda - there will be one read-write volume17:49
jeblairkrotscheck: yeah, run the mirroring script (eg bandersnatch) on one host -- it writes to a read-write mounted afs volume.  when complete, it does a vos-release which replicates that volume to multiple read-only replicas.  the actual 'mirrors' are basically just apache servers that serve out the read-only volumes17:49
*** Swami has joined #openstack-infra17:49
*** yamahata has joined #openstack-infra17:49
mordredwhat jeblair said17:49
*** sputnik13 has joined #openstack-infra17:50
jeblairkrotscheck: (they will have large caches, so as they are used, they eventually converge on just serving the active set of data from their local cache)17:50
zarofungi: take a look at https://gerrit-documentation.storage.googleapis.com/Documentation/2.11/rest-api-changes.html#detailed-accounts17:50
openstackgerritMerged openstack-infra/infra-manual: Add limitations and caveats to cross dependencies  https://review.openstack.org/26174317:50
* krotscheck goes to read things about vos-release17:50
*** ddieterly has joined #openstack-infra17:51
clarkbhpcloud is http 500'ing and there are nodes in ovh-bhs1 entering error state17:52
clarkbI am going to guess hpcloud may have leaked ports/fips17:52
clarkbwill work on sorting that out17:52
fungizaro: helpful--thanks!17:53
zaroclarkb: i can't find anything in today's scrollback regarding gerrit memory.  could you please provide recap17:53
*** trown is now known as trown|lunch17:54
clarkbzaro: mean use is 9GB, max is still 12GB (so we are spiking and hitting that often ish), and gc time occasionally is spiking but not to 10% more like 1%17:54
openstackgerritMerged openstack-infra/gearman-plugin: Remove broken scm section  https://review.openstack.org/26356617:54
zaroclarkb: that seems to suggest may just a little more memory would be ideal?17:55
clarkbzaro: if it continues to use more and more I agree, I still have a small amount of hope that it has reached a steady state17:55
*** lezbar has quit IRC17:56
openstackgerritMerged openstack-infra/project-config: Run Bluebox at full IP address capacity  https://review.openstack.org/26381517:57
*** jordanP has quit IRC17:59
anteayalennyb_: this also might be helpful for you for your patch to make lastcomment work with 2.11: https://gerrit-documentation.storage.googleapis.com/Documentation/2.11/rest-api-changes.html#detailed-accounts17:59
*** tlian has quit IRC18:00
clarkbwe seem to have only leaked ~7 fips in hpcloud, I will go ahead and delete them18:00
clarkbthat may account for the errors since we are basically at capacity18:01
*** bharathm has joined #openstack-infra18:01
pabelangerclarkb: any reason not to make nodepool more aggressive in deleting them?  Some sort of crontab every few minutes18:02
clarkbpabelanger: yes, 1) cloud should not leak these resources 2) it would require managing pools of a second resource which gets complicated18:02
*** signed8b_ is now known as signed8bit_ZZZzz18:02
clarkbpabelanger: I worry that we would invest a lot of effort into a thing that should just be fixed in neutron/nova18:03
clarkbthen they would fix it and our work would be redundant18:03
*** sabeen1 has joined #openstack-infra18:04
*** signed8bit_ZZZzz is now known as signed8b_18:04
clarkbhrm do any of the clouds other than hpcloud require floating IPs?18:04
clarkbthat may be another reason to ust ignore it for now18:04
*** e0ne has joined #openstack-infra18:05
pabelangerclarkb: Ya, fair enough.  I often heard about leaked FIPs but was curiously why we manually dealt with deleting them.18:05
mordredclarkb: ibm does, as does bluebox18:06
mordredclarkb, pabelanger: this will be better when we land the shade patch18:06
clarkbpabelanger: stubborness and a hope that the issue goes away? its actually a bit unfun to fix properly since you have to keep track of what fips map to what hosts and whether or not that host is still valid18:06
*** tlian has joined #openstack-infra18:07
mordredas the mechanism for attaching floating ips is different (using neutron api)18:07
greghaynesclarkb: By managing a second pool of resources you mean manage fips as a pool also?18:07
mordredwhich means that all unattached floating ips will always be a leak, and never an in-process fip18:07
clarkbgreghaynes: yes18:07
mordredwe do not need to do that18:07
clarkbgreghaynes: beacuse neutron and nova don't keep track for you when it breaks18:07
mordredwe do not have to keep track of anything18:07
clarkbmordred: how do you know what is safe to delete or not then?18:08
mordredwe just have to creat/attache the fips differently18:08
greghaynesmordred: what is the api youre talking about?18:08
mordredclarkb: it is ALWAYS safe to delete an unattached fip18:08
mordredin the shade codepath18:08
clarkbmordred: it isn't due to races on the back end18:08
mordredbecuase shade creates the fip with a port id18:08
greghaynesmordred: there is some atomic "booot and attach"?18:08
mordredthat's what shade does instead of how nodepool does it18:08
clarkblast I checked there is no atomic api call18:08
clarkband you can't properly detect errors in each step of the multi step attach18:09
mordredthe floating ip is created with the server's port id18:09
clarkb(or delete)18:09
mordredthere is no multi-step attach18:09
mordredthat's what I'm trying to say - it's a single step attach18:09
clarkbmordred: its create + attach18:09
mordredit's not18:09
mordrednot in the shade path18:09
clarkbthen where does the fip come from?18:09
mordredlet me try this again18:09
mordredwhen you create eh floating ip - you create it with the port_id of the server - this is a single step of "create-and-attac" from the FIP perspective, not from the server perspective18:10
mordredso you can have a server without a fip18:10
mordredbut not a fip without a server18:10
mordredthus, fips without a server are fips where the server has been deleted but the fip hasn't18:10
Sam-I-Amyou can have non-attached fips18:10
mordredyes you can18:10
mordredbut they will always be fips to be deleted in our case18:10
*** unicell has joined #openstack-infra18:11
mordredbecuase we do not re-use them18:11
mordredthis is a diffrent workflow than our current sequence18:11
*** tonytan4ever has quit IRC18:11
clarkbmordred: iirc, the issue on the create side is attach returns 200, but nova/neutron actually don't manage to attach18:11
AJaegermordred: could you review https://review.openstack.org/198417 and tell us whether you're happy with this updated version, please?18:11
mordredwhich has the characteristics clarkb was talking about18:11
clarkbmordred: so from nova's perspective there is no attach18:11
mordredclarkb: we do not run an attach call18:11
mordredclarkb: there is literally never a time when we call attach18:11
clarkbmordred: are you sure? when I read the code its distinct api calls18:12
lifelessmordred: hi18:12
*** armax has quit IRC18:12
clarkbmordred: the client fudges over it poorly18:12
mordredclarkb: yes. very very  much so18:12
lifelessmordred: we need to do a 0.11.1 pbr18:12
*** Adri2000 has quit IRC18:12
dougwigpabelanger: i reverted to the commit immediately before the apscheduler upgrade and it all started working, which is three commits (apsched, webob removal, tox tweak)18:12
mordredlifeless: did you see my latest pbr patch?18:12
Sam-I-Amneutron floatingip-create <net> will just create a fip without attaching it18:12
clarkbmordred: which is part of the problem because different bits will fail but you don't know what bit and iether way you don't get enough info to rollback properly18:12
*** changbl has joined #openstack-infra18:12
mordredSam-I-Am: yes. that's not what we're talking about18:12
Sam-I-Amoh, ok18:12
* Sam-I-Am fades out again18:12
clarkbthis was made better in response to our original complaint of the issue, but not completely fixed18:12
clarkb(the racey code was minimized)18:13
*** Adri2000 has joined #openstack-infra18:13
lifelessmordred: https://review.openstack.org/#/c/263748/ ?18:13
greghaynesmordred: I think clarkb is just saying that there is a failure case in nova/neutron which can leave an unattached fip18:13
greghayneswhich seems plausible18:14
greghaynesbut IMO that is a better failure case since we can say "whenever that happens it is a nova/neutron bug"18:14
mordredclarkb: http://developer.openstack.org/api-ref-networking-v2-ext.html#createFloatingIp18:14
clarkbmordred: ya I am reading that there is no server id18:14
mordredclarkb: port_id18:14
mordredclarkb: you pass the port_id of the server18:14
*** mpaolino has quit IRC18:14
mordredof the fixesd ip of the server18:14
clarkbmordred: right so the issue is this18:14
pabelangerdougwig: Hmm. keep my updated please. Interested to see what you find18:15
clarkball of that is neutron construct that nova is unaware of iirc. So neutron willreturn 200 and say its all golden18:15
clarkbnova on the other hand never found out about the floating IP18:15
clarkbthis means when you go to delete the VM that floating IP which was "attached" does not exist and you cannot delete it18:15
clarkbbecause you don't know its there18:15
pabelangerdougwig: only thing I can think of is maybe stale dependencies?18:15
clarkbif you want to track that state outside of nova and neutron it will work18:15
clarkbbut you have to track the state yourself18:15
mordredso, that's all valid - but the thing is that it doesn't matter and you don't have to track the state18:16
greghaynesclarkb: I think we wouldnt need to track state there though? if something is unattached we'd just delete18:16
clarkbor otherwise do reverse lookup listings that are expense etc18:16
clarkbgreghaynes: no there is a race18:16
jeblairclarkb: but i think mordred is saying that because of the different way of creating them it's safe to simply periodically "delete all unattached fips"18:16
mordredclarkb: because when you delete the server, it will delete the port18:16
clarkbgreghaynes: because internally neutron will have fip's that are not yet attached that will be attached18:16
mordredwhich means that the fip will then not have a fixed port_id18:16
AJaegerpabelanger: our currently oldest untouched change in project-config is https://review.openstack.org/186441  -could you have a look at it or should I abandon it?18:16
mordredwhich means that you can ALWAYS run a sweep operation that says "get me all the fips that do not have fixed port ids and delete them"18:16
mordredand that will ALWAYS be safe to do18:17
greghaynesclarkb: uhhh18:17
greghaynesclarkb: wat18:17
greghaynesclarkb: that is bad18:17
clarkbmordred: so that relies on the underlying non floating IP port going away?18:17
mordredclarkb: no. we will NEVER have fips that are not yet attached but _will_ be attached. if we have fips tht are not attached, they will never be attached18:17
mordredthat's the difference in the workflows18:17
mordredthey both have failure conditions18:17
mordredbut you can infer different information in the new one18:18
greghaynesI mean, that is a plausable assertion... I cant know without reading code.18:18
mordredbecaue there is not a valid state of "exists pending attachment"18:18
clarkbmordred: ok, that was definitely the case when I read the server and client code when this first popped up18:18
clarkbso I am confused how shade can do it differently18:18
mordredit's a _completely_ different code path we'll be using in shade18:18
mordredwe use neutron api not nova api18:19
clarkbright the issue is in neutron api18:19
mordredwhich gives us the ability to create a floaitng ip wiht the server fixed port id18:19
mordredyou're still talking about the old attach-based api18:19
mordredthat is not what we're talking about here18:19
mordredwe're talking about creating the floating ip with a pre-existing fixed port target at create time18:19
mordredthere is no attach18:19
greghaynesmordred: I think clarkb is just asserting that the neutron api is not atomic and can raced with neutron list calls18:20
mordredyou are correct that if we used attach with neutron api we'd still be in the broken place18:20
pabelangerAJaeger: Ya.  Sadly that project did work as I expected.  Now that deb-packaging group is a thing, we should either ask if they want to maintain it or maybe just close it off.  FWIW: I'm actually working directly in fedora packaging team for zuul, nodepool, etc.18:20
greghaynesI can just read the codes and see18:20
pabelangerAJaeger: add to that, we never landed our packaging spec this year and sadly it just lagged behind18:20
pabelangerAJaeger: I'll abandon for now to keep things easy18:21
mordredgreghaynes: I have not been able to produce a race, where the other one that clarkb talks about produces a race _very_ trivially18:21
mordred(exists in a constant state of race really)18:21
greghaynesAbsolutely. I havent actually read the code so *shrug*18:21
AJaegerpabelanger: in that case you can tell the rpm-packaging guys as well . thanks for abandoning it18:21
greghaynesThey would just need some extra internal state for fips to not show them while attempting an attach, so that should be fixable if it doesnt exist.18:22
mordredI will concede that it's possible that there will be a moment where a list shows a floating ip without a fixed port - but I have not been able to produce such a state in the new code path18:22
pabelangerAJaeger: Ya. Once I have some spare cycles, I want to get packaging builds going again.  I had it working, but our lack of spec made it hard to land18:22
clarkbmordred: also, I think you are suggesting periodic cleanup rather than correct deletes right?18:22
clarkber correct deletes at VM delete18:22
AJaegerpabelanger: understood.18:22
clarkbbecause we can't do that either way without a ton of extra api calls?18:22
mordredclarkb: well, I'm suggesting correct deletes at VM delete - but also safe automatic sweeps becuase E_CLOUD18:23
*** _nadya_ has quit IRC18:23
mordredin any case, we should absolutely not maintain our own floating ip pool state :)18:24
mordreda point on which I believe we both violently agree18:24
clarkbyes :)18:24
AJaegerpabelanger: do you want to continue on 208088, 258682, 205673 as well?18:24
*** tonytan4ever has joined #openstack-infra18:24
pabelangerAJaeger: Ya, just looking at my outstanding reviews now18:24
*** eil397 has joined #openstack-infra18:25
sputnik13can someone with +2 for requirements please take a look at https://review.openstack.org/#/c/261074/, it already has a +2 from dims18:25
AJaegerpabelanger: thanks18:25
jlvillalI know basically nothing about Puppet, but trying to setup an environment to simulate the devstack-gate. https://github.com/openstack-infra/devstack-gate  When doing 'puppet apply ...' I get an error about not being able to upgrade tox.18:25
sputnik13just bumping taskflow minimum version18:25
*** angdraug has joined #openstack-infra18:25
jlvillalMy theory is that the 'pip' module in puppet doesn't know how to go through a proxy server. Or I somehow have to configure it. I do have http_proxy/https_proxy set.18:26
*** dslev has quit IRC18:26
jlvillalThe error generated seems to come from: /etc/modules/pip/lib/puppet/provider/package/pip3.rb18:27
sputnik13jlvillal afaik http_proxy/https_proxy settings don't always carry over when doing sudo18:27
jrolljlvillal: https://tickets.puppetlabs.com/browse/PUP-5212 :(18:27
jlvillalsputnik13: I am running this as root, inside the VM18:27
jlvillaljroll: You are a font of sad news :P  But thanks, that is helpful!18:28
*** hdd has quit IRC18:28
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add ansible-role-functional-jobs-(centos|trusty)-nv  https://review.openstack.org/25868218:28
* jlvillal sobs softly...18:28
*** hdd has joined #openstack-infra18:28
sputnik13jlvillal maybe setting proxy in pip config would work?18:28
mordredlifeless: heya18:28
jlvillalsputnik13: It isn't actually pip that is failing. It is the puppet module. pip itself works fine through the proxy. But thanks for the idea!18:29
jrolljlvillal: probably not released yet, but https://github.com/puppetlabs/puppet/pull/450318:29
sputnik13kk, good luck18:29
mordredlifeless: sorry, you tried to talk to me about a pbr release while clarkb and I were talking about fips (yay)18:29
mgagnejeblair I'm back from holidays. Checking at created instances by nodepool at Internap and I see nodes are created and deleted in less then 30s. Is it the normal behavior or is nodepool going crazy?18:29
jlvillaljroll: I could try hacking that in there locally, maybe...  I will look at it.18:29
mordredlifeless: yes, that is the patch I was talking about, would love to get it in if we're cutting something, but it's also by no means essentially18:30
mordred(just ugly in changelogs)18:30
lifelessmordred: yes, we need a 0.11.1 release18:30
*** _nadya_ has joined #openstack-infra18:30
lifelessmordred: which means we need a branch for that18:30
lifelessmordred: for which I believe I need infra help18:30
clarkbmordred: I am rereading fip implementation (for dvr because its what I found) and I think there is still the issue let me link some code18:30
anteayamgagne: I do believe we prefer for created nodes to last a little longer than 30s18:30
lifelessmordred: [before we can put the patch up or anything useful]18:31
*** signed8b_ is now known as signed8bit_ZZZzz18:31
anteayamgagne: is that the behaviour you are seeing on all your nodes created by our nodepool18:31
mgagneanteaya I'm looking for an explanation for the (faulty or not) behavior  =)18:31
mgagneanteaya I don't have access to Nova logs/database of other providers18:32
jeblairmgagne, clarkb, mordred: this change was intended to address the problem that mgagne is seeing: https://review.openstack.org/25200018:32
jeblairanteaya: ^18:32
clarkbmordred: actually I see whree it adds the fixed port now18:32
anteayaI'm trying to draw attention to your request for an explaination by engaing you18:32
anteayaengaging you18:32
clarkbmordred: it does the updates in two different location buts apperas to be a single transaction18:32
clarkbso it may be ok18:32
*** andreykurilin__ has quit IRC18:32
jrolljlvillal: good luck :)18:33
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Add wheel mirror to mirrors  https://review.openstack.org/25352118:33
mordredclarkb: good. that's what the experiene I've had is - but I could totally imagine a server-side race :)18:33
mgagnejeblair is there anything I can do to speed up merge of this change?18:33
jeblairclarkb, mgagne, mordred, greghaynes: it has a minor conflict with the builders series which is approved but failed jenkins.18:33
greghaynesclarkb: I say we switch to that style, if we end up with unattached fips then we know it is a nova/neutron bug18:33
*** aopincar has joined #openstack-infra18:33
jlvillaljroll: Looks fairly simple. Famous last words...18:33
*** signed8bit_ZZZzz is now known as signed8b_18:33
jeblairclarkb, mgagne, mordred, greghaynes: we should either merge the builders changes immediately or approve that change and rebase builders.18:33
mordredgreghaynes: yes. we will be changing to that style when we land the shade patch18:33
*** ifarkas has quit IRC18:33
mordredgreghaynes: which I will start working on again once builders has landed18:34
greghaynesjeblair: Builders are ready to merge and have a +A, they just failed to merge due to occ bug which is now fixed18:34
clarkbjeblair: I have committed to helping crinkle with the puppet-mysql update after the meeting so not really in a place to approve and restart nodepool and check it all18:34
clarkbbut if someone else has time that would be great18:34
clarkbits ready to go and tests should pass after the occ release18:34
greghaynesYep, just waiting on that18:34
krotscheckgreghaynes: I'm messing around with your wheel patches.18:34
krotscheckgreghaynes: Just fyi18:34
jeblairgreghaynes: you have the root of the series in WIP.18:34
greghayneskrotscheck: fun18:34
greghaynesjeblair: Yep, I was asking this morning if I should recheck or wait, I got a 'wait until we are ready to deploy' so I did that18:35
pabelangerya, looking forward to the nodepool builder stuff landing, set aside some time already to play with it :)18:35
jeblairgreghaynes: well, if we're waiting then we need to merge the internap-fixing change and rebase builders.  that is not my preference but we must do one or the other.18:35
mgagneok so status is: waiting for free cycle to merge and babysit the deployment18:36
greghaynesjeblair: Do you have a link to that change? I am curious how bad it conflicts18:36
jeblairgreghaynes: it is minor18:36
jeblairgreghaynes: https://review.openstack.org/25200018:36
*** dslev has joined #openstack-infra18:36
jeblairmgagne: no, when we last discussed this we agreed that we would only delay https://review.openstack.org/252000 for builders as long as we were waiting for quota from internap.  we have it now, so we should proceed.18:37
mgagnejeblair ok I have some free time this week to work with you guys to increase quota if things go well18:38
greghaynesjeblair: Ya, we can go ahead with merging that and i'll rebase18:38
jeblairgreghaynes: i'm perfectly happy to restart nodepool.  what other work would be involved?18:38
anteaya22 minutes to team meeting18:38
*** ganso has joined #openstack-infra18:38
greghaynesjeblair: in theory no, I dont want to get in to a firedril on bugfixing right now if something does pop up though18:38
fungijeblair: pretty close now... i have a script dumping yaml of gerrit change owner e-mail addresses per governance project-team based on an arbitrary change query. artificially constraining it to the past month of changes for repos under the cinder and nova teams, it completes in ~60 seconds so probably not much worse than the mysql/ssh-based solution18:39
jeblairgreghaynes: okay, we can always revert, but your call.18:39
greghaynesjeblair: The rebase shouldn't be too bad so lets do that18:39
*** jsavak has quit IRC18:39
mgagnejeblair I'm also looking at graphite.o.o and I can't login (getting a python error). Is it a known behavior? Should I be able to login and saves personal dashboards and charts?18:40
AJaegersdague: could you review https://review.openstack.org/#/c/258642 , please? I'm not sure whether that works as expected...18:40
*** piet has quit IRC18:40
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Bug fix: Cinder v2 returns bools now  https://review.openstack.org/26376418:40
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Add 'public' attribute to 'networks' entry  https://review.openstack.org/25200018:40
*** terriyu has joined #openstack-infra18:40
jeblairmgagne: no login needed18:40
*** fawadkhaliq has quit IRC18:40
mgagnejeblair all stored in cookies?18:40
mgagneor local storage.18:41
jeblairmordred, greghaynes, clarkb: if you could approve https://review.openstack.org/252000 that would be great.18:41
greghaynesAlso, there are tests, so I am happy :)18:41
jeblairmgagne: it's mostly stateless, graphs are just long urls18:42
pabelangerIf reviewers are around, I wouldn't mind help getting statsd 3.0 for zuul landed: https://review.openstack.org/#/c/229232/ Then maybe get a new release of zuul later today18:42
mgagnejeblair ok, I was more or less looking for a way to avoid copy/paste the URLs somewhere just to save them if I can have then saved online under my personal account18:43
jeblairmgagne: we're starting to make dashboards in grafana.openstack.org18:43
*** jaosorior has quit IRC18:43
mgagnejeblair I'm selfish, I don't see an internap provider centric dashboard :D18:44
jeblairmgagne: we can go ahead and make one there :)18:44
AJaegercould I get another pair of eyes on mordred's change for run-pep8 in project-config, please? https://review.openstack.org/#/c/255466 It has two +2s but was not approved, so if somebody wants to check, it would be nice - otherwise I'll approve tomorrow...18:44
jeblairmgagne: i've been going around and getting okays from our existing providers (ovh and bluebox are both okay, still need to talk to rax) which is why we haven't done it yet18:44
mgagnejeblair right, just looking for a way to get more details about how upstream uses our resources =)18:44
*** tzn has quit IRC18:45
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Add wheel_mirror_slaves that use pypi-mirror  https://review.openstack.org/16524018:45
waynrzxiiro zaro, either of you have any insight into why commits targeting the feature/2.0.x branch of jenkins-job-builder would be taking in the range of 14-20 minues on the jenkins-job-builder-compare-xml test?18:46
*** sabeen1 has quit IRC18:46
zxiirowaynr: I have no idea18:46
*** ihrachys has quit IRC18:47
*** ayoung has joined #openstack-infra18:47
clarkbjeblair: ya paging it back in for review18:48
jlvillalSo on puppet. Are the modules it executes in /etc/puppet/ ?  I updated the code in there and I get the same error message. So then I updated the error message but I don't see my change.18:49
jlvillalJust wondering if the modules put in /etc/puppet/ are just there to confuse people like me :)18:49
jlvillalI had tried modifying /etc/modules/pip/lib/puppet/provider/package/pip3.rb18:49
clarkbjlvillal: /etc/puppet/modules18:50
jlvillalclarkb: Aha, I found it :)  /usr/lib/ruby/vendor_ruby/puppet/provider/package/pip.rb18:50
jlvillalclarkb: Sorry I meant to say: /etc/puppet/modules/pip/lib/puppet/provider/package/pip3.rb18:51
jlvillalThat code is ignored.18:51
jlvillalclarkb: Changing /usr/lib/ruby/vendor_ruby/puppet/provider/package/pip.rb made it work18:51
clarkbpip3 is only for python318:51
clarkband you ahve to set it as the provider explicitly iirc18:51
jlvillalclarkb: Ah, thanks for the info. First time ever running puppet.18:52
*** rbrndt__ has quit IRC18:52
* krotscheck is pondering how to rsync built wheels into an AFS-and-pip-sensitive folder structure without having to hit every file :/18:52
*** arxcruz has quit IRC18:53
AJaegerthanks, anteaya !18:53
anteayaAJaeger: welcome, it looks fairly straightward from what I understand, if I am wrong, then I get to learn what I should have done18:54
mordredkrotscheck: hitting every file in the read-write volume should be fine - since the volume in question should be local18:54
mordredkrotscheck: and it shouldn't mess with the caches on the readonly volumes or hosts18:54
krotscheckmordred: I'm trying to land wheel pre-afs18:54
krotscheckBecause faster things are awesome, and I don't have a schedule on that bandersnatch patch.18:55
mordredkrotscheck: nod. mainly just trying to say I don't think you have to be _too_ careful about not hitting every file - unless I'm missing an important (which is quite possible)18:55
*** rbrndt__ has joined #openstack-infra18:55
clarkbmordred: did we figure out what the slow stating of large dirs was about?18:55
clarkbrsync may bump into that depending on dir structure18:55
mordredclarkb: I have not poked in to it more18:56
krotscheckmordred: My current theory is building the desired folder structure with symlinks.18:56
krotscheckAnd then rsyncing that.18:56
*** tonytan_brb has joined #openstack-infra18:57
krotscheckTHe way these wheel slaves are currenlty set up, they don't clean their output directory18:57
*** hdd_ has joined #openstack-infra18:57
*** hdd has quit IRC18:58
mordredlifeless: what branch do we need / why?18:58
*** sshnaidm_ has quit IRC18:58
mordredlifeless: there is a stable/liberty branch already and a master branch18:59
*** tonytan4ever has quit IRC18:59
*** isaacb has joined #openstack-infra18:59
*** baoli has quit IRC18:59
*** jsavak has joined #openstack-infra19:00
fungimeeting time! see you all in #openstack-meeting19:00
openstackgerritMerged openstack-infra/project-config: Do the sdist check in a virtualenv  https://review.openstack.org/25546619:00
*** baoli has joined #openstack-infra19:00
*** tonytan_brb has quit IRC19:02
*** _nadya_ has quit IRC19:02
*** tonytan4ever has joined #openstack-infra19:03
*** amuller_afk is now known as amuller19:04
*** signed8b_ is now known as signed8bit_ZZZzz19:04
openstackgerritMathieu Gagné proposed openstack-infra/system-config: gerrit: Add borders to test result table  https://review.openstack.org/26386619:06
*** erikwilson has joined #openstack-infra19:06
*** jsavak has quit IRC19:07
*** jsavak has joined #openstack-infra19:07
*** ociuhandu has quit IRC19:09
lifelessmordred: yeah, kilo19:10
lifelessmordred: setuptools 19.2 has a fix that means pbr's 'patch all the commands' code actually starts working properly19:10
mordredlifeless: ah - we need a stable/kilo - can you give me a sha from which to start it?19:11
lifelessmordred: of course, pbr's testr glue command (in kilo) breaks if testr isn't installed19:11
lifelessmordred: 0.1119:11
*** harlowja_at_home has quit IRC19:11
lifelessmordred: erm 0.11.019:11
mordredlifeless: kk19:12
*** gampel has joined #openstack-infra19:12
mordredlifeless: done19:12
lifelessmordred: thanks. Nakato ^19:13
*** rbrndt__ has quit IRC19:13
*** harlowja_at_home has joined #openstack-infra19:14
*** thiagop has joined #openstack-infra19:17
krotscheckmordred, jeblair: Either /a/a-versionthing.whl or /a/b/ablongerthanonecharacter-versionthing.whl?19:19
krotscheckor is it /a/{packagename}/{packagename}-{version}.whl, I don't quite recall.19:19
mordredkrotscheck: I think jeblair was doing /a/b//ablonger19:19
*** trown|lunch is now known as trown19:20
krotscheckOk, so it's the first character, second character if it exists, packagename as dir, and then package file itself. Check.19:20
*** rbrndt__ has joined #openstack-infra19:21
*** davideagnello has quit IRC19:21
*** otter768 has joined #openstack-infra19:22
mordredkrotscheck: I _think_19:24
jeblairkrotscheck, mordred: yep19:24
mordredjeblair: does your rewrite take in to account single-character things?19:24
*** bdemers has left #openstack-infra19:24
mordredjeblair: like /a/a.whl ?19:24
jeblairmordred: yep19:24
jeblaira/b/abcdef.whl and a/a.whl19:24
*** otter768 has quit IRC19:27
*** aopincar has quit IRC19:27
greghaynesclarkb: did you say friday might be ok for you to do builders?19:27
greghaynesclarkb: trying to figure out when is a potential good time for me...19:27
clarkbI hvae doctors next two days19:28
greghaynesNext two days a pretty bad for me too19:28
greghaynesclarkb: so try for friday morning? I have family in town but I should be able to get stuff done in the morning19:28
nibalizerill be plane19:31
nibalizeri mean I'll be on a plane or getting ready to be on one friday mornign19:31
nibalizerbut I think yall said the puppet stuff isn't actually required?19:32
greghaynesnibalizer: this bit is no puppet19:32
clarkbyup first step is no puppet, next step is puppet19:32
*** salv-orlando has joined #openstack-infra19:33
sdagueAJaeger: that part seems fine on it's own19:34
sdagueif folks feel they have an answer to the rest of it, it's fine19:34
AJaegerthanks, sdague19:34
AJaegerwill you +2?19:35
AJaegerah, you did - thanks19:35
*** salv-orlando has quit IRC19:38
*** daemontool has quit IRC19:39
*** daemontool has joined #openstack-infra19:41
*** dslev has quit IRC19:42
*** eharney_ has joined #openstack-infra19:43
*** baoli has quit IRC19:43
*** eharney has quit IRC19:45
*** sabeen1 has joined #openstack-infra19:46
*** jcoufal has quit IRC19:46
*** Sukhdev has joined #openstack-infra19:47
*** davideagnello has joined #openstack-infra19:47
*** hashar has joined #openstack-infra19:47
*** sabeen3 has joined #openstack-infra19:47
*** daemontool has quit IRC19:47
*** daemontool has joined #openstack-infra19:48
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Bug fix: Cinder v2 returns bools now  https://review.openstack.org/26376419:48
*** sabeen1 has quit IRC19:50
*** baoli has joined #openstack-infra19:50
*** daemontool has quit IRC19:51
*** devananda has quit IRC19:54
*** harlowja_at_home has quit IRC19:54
openstackgerritMathieu Gagné proposed openstack-infra/system-config: gerrit: Add borders to test result table  https://review.openstack.org/26386619:55
openstackgerritMerged openstack-infra/project-config: fix stable/liberty telemetry integration job  https://review.openstack.org/25864219:57
krotscheckjeblair: Case sensitive? /B/a/Babel is different from /b/a/babel19:57
clarkbkrotscheck: I want to say packages themselves are not case sensitive19:58
clarkbbecuase windows19:58
*** ddanielroche has joined #openstack-infra19:58
krotscheckjeblair: Also, pip prefers /packagename/packagename-version-things.whl. Would that then map to /p/a/packagename/packagename-version-things.whl, or does the package-named directory need to go?19:58
clarkbso pip install Babel and pip install babel should install the same thing19:58
jeblairkrotscheck: i think my regexes just pass through whatever, so i think the rewrite is case sensitive (but the tool may not be)19:59
*** weshay__xchat is now known as weshay19:59
*** lucasagomes is now known as lucas-dinner19:59
*** ayoung has quit IRC19:59
*** devananda has joined #openstack-infra20:00
jeblairkrotscheck: i am not 100% positive on that20:00
anteayaI'm in favour of slowly weaning ourselves off of hpcloud so the sunset is not abrupt20:01
krotscheckWell, URL's are case sensitive.20:01
*** eharney_ is now known as eharney20:01
fungiclarkb: i'd be fine with turning down hpcloud sooner, especially if it's being problematic and needing intervention frequently anyway20:01
*** jsavak has quit IRC20:01
*** gampel1 has joined #openstack-infra20:02
*** jsavak has joined #openstack-infra20:02
mordredanteaya: ++ to weaning20:02
AJaegerlooking at our nodes stats, we're maxing out, don't we? So, if it works, I would leave it running - but if maintenance is expensive...20:02
clarkbI want us off of it before the 31st20:03
anteayaclarkb: want to offer a series of patches pulling us back with suggested dates for merging?20:03
clarkbif we want to wean off of it do a chagne a week until 0 of equal increments?20:03
nibalizerI don't see what the value of that would be20:03
fungii don't known what a slow turn-down buys us20:03
anteayaAJaeger: well all that will happen if we leave it at max until it is abruptly shut off is that we are flooded with angry devs week of feb 120:03
fungiyeah, what nibalizer said20:03
*** gampel has quit IRC20:03
anteayaAJaeger: I'm not in favour20:03
*** ddanielroche has left #openstack-infra20:04
anteayait communicates a change to the community20:04
fungiturning it off sooner than the deadline so it's under our control and not a zomgemergency sounds good, but slowly decreasing it doesn't seem like it's much use20:04
anteayaeither way20:04
nibalizerwe could send some emails to the list I guess20:04
fungiwe already did20:04
anteayamostly if we are going to have a flood I'd like to be prepared20:05
fungimonths ago20:05
nibalizerHPE has done a pretty good job of publicising that its being turned off20:05
fungibut a reminder might make sense20:05
AJaegerand getting it in the weekly newsletter ;)20:05
nibalizerI don't want to promise that infra-cloud can provide nodes by jan 31st but it is getting really close now20:05
fungia pingee for thingee perhaps20:05
clarkbya I too think just turning it off all at once makes the most sense and being prepared to do that20:06
clarkbthat way wwe can take advantage of the resources until that time20:06
funginibalizer: well, we've got ovh ramped up, bb close, internap on the way...20:06
nibalizerthose are pretty small node counts though rightt?20:06
*** Eva-i has quit IRC20:06
clarkbone thing to note is that none of the not rax and not hpcloud clouds run unittests20:06
nibalizercollectively they aren't even half of hpe iirc20:06
fungitogether they're fairly substantial20:06
anteayanibalizer: well we have been in the onboard phase for at least a month20:07
anteayaeach provider has their own quirks and we have to build a relationship20:07
clarkbwhat I really want to avoid is nodepool having a bad day on february 1st20:07
*** rguillebert has quit IRC20:07
clarkbso just want to make sure we control the shutoff on our side20:07
thingeefungi, clarkb ack20:08
nibalizerI think we should communicate to the devs what is happening, I don't think we should turn it down ahead of schedule, I do think we should turn it off ourselves instead of waiting for it to dissapear20:08
AJaegerclarkb: agreed. So, it should be done at least on Friday the 29th, so planninga few days buffer, I would say 25th as the latest date.20:09
clarkbI am looking at a calendar, jan 29th is friday20:09
clarkbAJaeger is ahead of me20:09
*** Eva-i has joined #openstack-infra20:09
AJaegerclarkb: we had the same idea - counting backwards20:09
clarkbya so maybe write the change, then make a note for infra-root to get it merged. I and fungi I think are afkish 24th-28th20:10
clarkband I think others are headed to LCA early feb20:10
nibalizerand fosdem20:10
*** ayoung has joined #openstack-infra20:10
nibalizerok sounds like a plan?20:11
fungii'm not super worried about a sudden drop in quota. we frequently end up consuming whatever quota we have for as long as necessary to work through the backlog. this will be no different from having an increase in workload or an extended hpcloud outage (no offense to anyone, but we have seen some of those in the past)20:11
*** jsavak has quit IRC20:11
*** hdd_ has quit IRC20:11
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Bug fix: Cinder v2 returns bools now  https://review.openstack.org/26376420:11
nibalizerfungi: good point20:11
*** jsavak has joined #openstack-infra20:11
clarkbya, I will go ahead and write the change to set max-servers to -1 and a follow up to remove the config from nodepool.yaml and clouds.yaml20:12
clarkbthen WIP them20:12
fungiat the end of the day, we have whatever capacity we have. sometimes that means devs wait a while for job results. situation normal20:12
fungithanks clarkb! i'll leave it on the meeting agenda so we can revisit next tuesday20:12
krotscheckjeblair: Ok, so if pip is expecting /packagename/packagename-version.whl, which of these two would you prefer? http://paste.openstack.org/show/483066/  http://paste.openstack.org/show/483067/20:12
clarkbthen we can merge the max-server change a few days early, giving us time to clean it up and remove it from the config completely20:12
*** purp_too is now known as purp20:13
anteayaclarkb: are you lca'ing this year?20:13
clarkbanteaya: no I have a foundation thing20:13
fungias do i20:14
* anteaya is not as going from nova mid-cycle to au seemed like to much this year20:14
anteayaah, well happy foundation thing then20:14
clarkband decided trying to do both and leaving larissa at home with babies for that long would end up with bad things happening when I got home20:14
*** signed8bit_ZZZzz is now known as signed8b_20:14
krotscheckclarkb: I feel your pain.20:14
fungianteaya: yeah, since our staff are spread out and normally only see each other twice a year at summits, we try to pick a few days in between summits to have other quarterly in-person meetings20:15
crinkleclarkb: hey, ready for wiki/nodepool downtime at 2100?20:15
krotscheckclarkb: btw, my wife & baby are joining me for Barcelona, but probably not Austin. Are you bringing the fam, and if so, can we collaborate on childcare?20:15
anteayaclarkb: sounds like you are making good choices20:15
anteayafungi: sounds like a recipe for sanity to me20:15
anteayafungi clarkb where are you frolicing this year?20:16
krotscheckrbradfor may also be interested in joint childcare arrangments @ summit things.20:16
clarkbcrinkle: yes, should I go ahead and disable puppet on wiki.o.o and nodepool.o.o so I can approve the change to bump the mysql version?20:16
*** mudassirlatif has quit IRC20:17
crinkleclarkb: sure, we can also approve https://review.openstack.org/#/c/258193/ first without hurting anything20:17
*** Sukhdev has quit IRC20:17
clarkbcrinkle: then I can more gracefully stop services, do db backup, run puppet, start services20:17
*** jsavak has quit IRC20:17
fungianteaya: the yucatan coast of mexico, gulf side, mostly because it's cheap and easy to get to/from20:17
*** mudassirlatif has joined #openstack-infra20:17
anteayaI like it there, or did when I was there long ago20:18
*** otter768 has joined #openstack-infra20:18
*** mudassirlatif has quit IRC20:18
fungiwell, it reduces the risk of any of us stuck when nyc or chicago area airports inevitably end up buried in white stuff20:18
anteayaindeed it does20:18
*** ociuhandu has joined #openstack-infra20:19
*** krtaylor has quit IRC20:19
clarkbcrinkle: let me stop puppet first because the nI can approve whatever without worrying20:19
clarkbcrinkle: I am doing that now20:19
*** mudassirlatif has joined #openstack-infra20:19
crinkleclarkb: kk20:19
fungiwhich befell me when we did one in utah. all flights through chicago were cancelled, and the passenger backlog meant the earliest they could rebook me into a seat was going to be a couple days later20:19
*** otter768 has quit IRC20:19
anteayabasically missing the in person event20:20
*** mudassirlatif has quit IRC20:20
funginah, that was at least after the event, so it was delaying my ability to get home20:21
anteayaby getting to know many airport staff and passengers whilst typing which you do such a great job of at home20:21
anteayayou do have some great getting home after event stories, I must say20:21
*** annegentle has joined #openstack-infra20:21
clarkbok those two should no longer have ansible run puppet happen20:21
clarkbthe ansible emergency hosts file is updated to add them20:22
clarkbkrotscheck: I have a feeling that it may just be me20:22
fungii ended up getting them to find me an open seat on a flight to phx, where i slept in a chair in one of the terminals and found a four-stop redeye the next morning to get me from phx to near home that night20:22
Clintfour-stop redeye, you say20:22
clarkbkrotscheck: though I need to do more research on what our seat options are. With twins apparently most of the time you end up in different rows due to oxygen mask availability or something20:22
fungiso slightly quicker than waiting two days for another same-day itenerary from slc to home20:23
*** arxcruz has joined #openstack-infra20:23
fungiClint: yeah, one of those insane commuter hoppers that goes from one mid-sized regional airport to another working its way across the country20:23
*** peristeri has joined #openstack-infra20:23
Clinti don't know about those!20:24
clarkbkrotscheck: they are definitely getting a lot easier to deal with as they get older20:24
clarkbbut that could all change tomorrow20:24
krotscheckclarkb: Yeah I know. We did Hawaii a few weeks ago though, and S. was super, super easy20:24
*** signed8b_ is now known as signed8bit_ZZZzz20:24
*** gyee has quit IRC20:25
fungiClint: er, not a redeye. departed early in the morning and got in late at night, so the opposite of a redeye. a blackeye?20:25
krotscheckUntil he got there. Then he was all "What is this 'warm weather' thing, gimme my seattle rain!"20:25
Clintthat makes me hungry20:25
krotscheckclarkb: LMK if y'all end up deciding to make it a family vacation. I'll reach out to the foundation about childcare.20:25
anteayafungi: wow, I just I don't know what to say20:27
anteayaI do like pbx airport though, but never had to spend the night there20:27
clarkbcrinkle: and we aren't doing the backwrad forward compat thing with the non test slave manifests?20:27
fungianteaya: well, they had good wifi, so i mostly stayed up and did code review if memory serves. it was very quiet there overnight!20:28
crinkleclarkb: no, since it's all in system-config it should be all atomic so there should be no need20:28
*** e0ne has quit IRC20:28
fungialso managed to find a lounge where they'd left one of the televisions on all night, set to a channel with classic cartoons, so it wasn't so bad20:29
clarkboh right I remember asking the same question before, sorry20:29
anteayayay cartoons20:29
clarkbcrinkle: the change you linked is approved, looking at the system-config change now20:29
anteayawhich do you consider the classics?20:29
anteayabugs bunny?20:29
fungianteaya: sure20:29
fungianything from ~1960..1980 i guess'20:29
*** AJaeger has quit IRC20:30
* anteaya makes a note20:30
anteayaI found a television station in utah that played old televison shows20:30
anteayaleave it to beaver, my three sons20:30
anteayaI couldn't remember last time I had seen any of those20:31
clarkbcrinkle: then my plan is do nodepool first since its data is mostly ephemeral20:31
anteayait was awesome20:31
clarkbI worry less about it :)20:31
*** signed8bit_ZZZzz is now known as signed8b_20:31
fungiyeah, nodepool outages are also relatively painless for us20:31
crinkleclarkb: okay20:31
fungiunless it goes on for hours, we're generally the only ones who know it's down20:31
fungihowever, it does usually destroy our image updates for the day20:32
*** rfolco has quit IRC20:32
fungiso if you don't kill it until after the dib uploads have finished, that's even less impact20:32
*** Sukhdev has joined #openstack-infra20:33
*** yolanda has quit IRC20:33
*** yolanda has joined #openstack-infra20:34
clarkbwell we are scheduled for start at 210020:34
*** sparkyco_ has joined #openstack-infra20:36
*** mudassirlatif has joined #openstack-infra20:36
clarkband system config chagne is approved now20:37
*** andreykurilin__ has joined #openstack-infra20:37
*** mudassirlatif has quit IRC20:37
*** weshay has quit IRC20:38
*** terriyu has quit IRC20:40
clarkbcrinkle: so now we wait20:40
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Add heat lbaas specific test for v1 and v2  https://review.openstack.org/23715620:40
crinkleclarkb: mmk20:40
*** signed8b_ is now known as signed8bit_ZZZzz20:41
*** piet has joined #openstack-infra20:41
openstackgerritMerged openstack-infra/puppet-nodepool: Add logic to work with old and new mysql module  https://review.openstack.org/25819320:42
pleia2cat tended to and me fed20:42
anteayapleia2: yay20:42
anteayapleia2: welcome back20:42
pleia2anteaya: thanks20:42
isaacbclarkb: Hi, following our talk last week about timing the cron job in puppet master, can you please look in https://review.openstack.org/#/c/262466/20:42
*** signed8bit_ZZZzz is now known as signed8b_20:44
openstackgerritMerged openstack-infra/system-config: Upgrade mysql module to 3.6.1  https://review.openstack.org/25764020:44
*** piet has quit IRC20:45
*** piet has joined #openstack-infra20:45
*** kzaitsev_mb has joined #openstack-infra20:45
clarkbisaacb: when I am done with the puppet-mysql trainsition20:45
clarkbcrinkle: and you said that mysql will flap as part of this right?20:46
*** mudassirlatif has joined #openstack-infra20:46
isaacbclarkb: thanks.20:46
clarkbcrinkle: it won't do that on every puppet run just as part of the initial transition because of hte ocnfig file changes?20:46
clarkb(this was how I understood it at least)20:46
crinkleclarkb: correct, the newer module is going to rearrange some things in my.cnf, it will only do it once20:47
*** Sukhdev has quit IRC20:47
*** jsavak has joined #openstack-infra20:49
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add option to use the wall time instead of the sum to run_time_meta  https://review.openstack.org/23176420:52
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: WIP: Kinda hacky double sorted run time meta  https://review.openstack.org/23449420:52
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add new graph time for variance of individual tests  https://review.openstack.org/23061120:52
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add new graph type to show run_time variance by metadata  https://review.openstack.org/21056920:52
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add a db api call to provide a rough wall time for a given run  https://review.openstack.org/23176220:52
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add run_time_meta arge to specify values to use in graph  https://review.openstack.org/23176320:52
*** weshay has joined #openstack-infra20:53
*** eharney has quit IRC20:53
rbradforkrotscheck, maybe, that would enable them to come, likewise, probably barcelona would be a good goal.20:55
pabelangerfungi: nibalizer: pleia2: do you mind reviewing: https://review.openstack.org/#/c/229232/20:55
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add option to use the wall time instead of the sum to run_time_meta  https://review.openstack.org/23176420:56
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: WIP: Kinda hacky double sorted run time meta  https://review.openstack.org/23449420:56
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add new graph time for variance of individual tests  https://review.openstack.org/23061120:56
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add new graph type to show run_time variance by metadata  https://review.openstack.org/21056920:56
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add a db api call to provide a rough wall time for a given run  https://review.openstack.org/23176220:56
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add run_time_meta arge to specify values to use in graph  https://review.openstack.org/23176320:56
*** piet has quit IRC20:57
pleia2pabelanger: sure, having a look20:57
*** tzn has joined #openstack-infra20:59
clarkbcrinkle: it is time, I will be stopping nodepool, dumping the db, and running puppet momentarily21:00
*** baoli has quit IRC21:00
clarkb(I also confirmed that the new version of mysql module is on the master)21:00
crinkleclarkb: cool21:00
clarkbnibalizer: mordred fungi puppet agent --test is still the way to run puppet ya?21:00
clarkbkk thanks21:01
*** krtaylor has joined #openstack-infra21:01
*** kzaitsev_mb has quit IRC21:02
*** ayoung has quit IRC21:02
clarkbok service is stopped and mysqldump is made, running puppet now21:03
clarkbnodepool is running again21:05
crinkleit looks good in puppetboard21:06
waynrzaro zxiiro hashar I think this review is ready to go: https://review.openstack.org/#/c/133465/21:06
fungiclarkb: it's what i ended up doing on paste.o.o on sunday night, at least21:06
clarkbyup sees to be working so far21:06
clarkbI didn't see any tracebacks about nodepool complaining about mysql things21:07
clarkbI am going to rerun puppet to prove that it won't flap myslq again21:07
zxiirowaynr: looking21:07
waynrnot sure if we should wait for Darragh/electrofelix (and maybe i shouldn't have +2'd it since I last updated it)21:09
clarkbcrinkle: hrm ps seems to indicate mysqld has been running since 201521:09
clarkbso it may not actually flap the service21:09
jeblairmordred: do you have time to approve https://review.openstack.org/252000 or should i just do it?21:09
clarkbmordred: does mysql do config reloads on the fly?21:09
mordredjeblair: lookig21:09
mordredclarkb: no21:09
crinkleclarkb: that is strange21:10
mordredclarkb: some settings you can set on the fly21:10
mordredsome you cannot - the mysql docs indicate which can be set live21:10
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add option to use the wall time instead of the sum to run_time_meta  https://review.openstack.org/23176421:10
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: WIP: Kinda hacky double sorted run time meta  https://review.openstack.org/23449421:10
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add new graph time for variance of individual tests  https://review.openstack.org/23061121:10
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add a db api call to provide a rough wall time for a given run  https://review.openstack.org/23176221:10
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add run_time_meta arge to specify values to use in graph  https://review.openstack.org/23176321:10
*** piet has joined #openstack-infra21:10
mordredclarkb: do you have a setting that has been changed?21:10
crinkleclarkb: it looks like whether to restart is controlled in puppet, https://github.com/puppetlabs/puppetlabs-mysql/blob/master/manifests/params.pp#L621:11
crinkleclarkb: i'm not sure why it restarted when i was testing it then21:11
clarkbmordred: puppet has rewritten the mysql.conf so mostly just want to sanity check it comes back with the new config that is euqivalent21:11
openstackgerritMerged openstack-infra/jenkins-job-builder: Reorder imports to match hacking guidelines  https://review.openstack.org/13346521:11
mordredclarkb: so - fwiw: show global variables like 'innodb_buffer_pool_size';21:11
clarkbcrinkle: do you think I should stop nodepool again, bounce mysqld, then start nodepool?21:12
mordredis how you verify a config setting is in the running mysql21:12
clarkbmordred: thanks21:12
crinkleclarkb: yes21:12
mordredbut in this case bouncing both might be easier21:12
clarkbya I will go ahead and do that now21:12
*** hdd has joined #openstack-infra21:13
openstackgerritSpencer Krum proposed openstack-infra/system-config: Rename puppet-community to voxpupuli  https://review.openstack.org/26391021:14
clarkbthats done and seems to be working21:14
clarkbI will be sure to check mysqld on wiki before starting apache and fast cgi things21:14
*** ajmiller has quit IRC21:14
*** rockyg has joined #openstack-infra21:14
clarkbnow I want to say in the psat we have stopped php-fpm and apache both in order to stop the wiki service21:16
clarkbfungi: ^ is that what you remember?21:16
fungiclarkb: yeah, php-fpm tends to cache the php source21:16
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: use default from address for emails  https://review.openstack.org/26391221:16
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: add command to actually send email announcements  https://review.openstack.org/26391321:16
*** ajmiller has joined #openstack-infra21:17
fungiso restarting it is good if you need some stuff reloaded from disk21:17
clarkbI don't think I need tat but better safe than sorry21:17
*** kgiusti has left #openstack-infra21:18
*** rhallisey has quit IRC21:18
clarkbok stopping apache and php-fpm on wiki now21:18
*** amitgandhinz has quit IRC21:19
clarkbwaiting it says21:19
*** amitgandhinz has joined #openstack-infra21:20
clarkbmysql dump is dumping21:20
*** chlong has quit IRC21:22
*** zzzeek has quit IRC21:22
*** zzzeek has joined #openstack-infra21:22
clarkb(and still going this db is fairly large I guess that is to be expected)21:22
clarkband its done, running puppet agent --test now21:23
*** annegentle has quit IRC21:24
clarkband thats still going21:25
clarkbError: Could not retrieve catalog; skipping run21:26
rajinirasselin: Nodepool successfully added the node to the jenkins and slave is ready. But when I make commits to the ci-sandbox, zuul is only sending emails (Silient check ones), jenkins jobs are not triggered. Any ideas?21:26
*** weshay has quit IRC21:26
jeblairmordred: thanks.  waiting on you to review then i'll land and restart nodepool.21:26
clarkbcrinkle: nibalizer ^ is that just a retry error or a go digging on the puppetmaster error?21:26
crinkleclarkb: that makes it seem like it can't contact the puppetmaster or isn't in site.pp21:27
crinkleretry first21:27
clarkbcrinkle: it did get the new plugin stuff at the start fwiw21:27
*** aeng has quit IRC21:27
crinklethat's weird that it would get part of the catalog21:28
clarkbthere it goes, maybe just contention with the master21:28
clarkband mysqld was not restarted so restarting it now21:28
mordredjeblair: did you see dougwig's comment?21:28
mordredjeblair: (I'm happy to think about that later - just wanted to make sure you saw it)21:29
mriedemclarkb: $100 if you pull a tonya harding on russell wilson's leg before sunday21:29
mriedemeasy money21:29
*** kzaitsev_mb has joined #openstack-infra21:29
*** claudiub has quit IRC21:29
clarkbok starting wiki and php fpm now21:29
clarkbmriedem: but I want him to win21:30
jlvillalI am (still) trying to do 'Simulating Devstack Gate Tests'   https://github.com/openstack-infra/devstack-gate  I got to the part of doing: ./safe-devstack-vm-gate-wrap.sh21:30
jeblairmordred: yep, responded21:30
jlvillalBut it errors because stuff isn't in /opt/stack/21:30
asselinrajinir, not sure.....what's in the zuul debug.log file?21:30
*** Sukhdev has joined #openstack-infra21:30
*** gyee has joined #openstack-infra21:30
clarkbok please to test wiki21:30
crinklelooks good in puppetboard and i can pull up the infra agenda21:31
clarkbhome page loads for me, I am going to try editing something21:31
jeblairsearching works21:31
anteayahere too21:31
anteayaI can log in21:32
jeblairi added a thing to the infra meeting agenda for zuulv321:32
jlvillalThis is the output of trying to do the devstack-gate simulation: http://paste.openstack.org/show/483076/21:32
mordredjeblair: cool. +A21:32
nibalizerjlvillal: n.b. https://git.openstack.org/cgit/openstack-infra/devstack-gate is the cannonical url21:32
clarkbI can make an edit21:32
jlvillalnibalizer: Those are the instructions I'm following :)21:32
clarkbcool so looks good to everyone?21:32
clarkbI am going to rerun puppet really quickly to confirm it noops now21:33
dougwigjeblair: heh, i saw the note in the commit message, i did not see a why. it's likely just my latent dislike of how inconsistent openstack is with accepting id, name, or id/name.21:33
jlvillalJust wondering what devstack-gate is expecting to be in /opt/stack/ so I can also set that up too.21:34
jeblairdougwig: yeah, wasn't sure if you're question was "why?" or "why why?" :)21:34
jeblairdougwig: i'm not thrilled either :)21:34
*** gyee has quit IRC21:34
clarkblooks like the rerun may timeout again21:35
anteayaI can edit21:35
clarkbbut once I get that to noop I can reenable puppet21:35
anteayawiki seems to be wiki'ing for me21:36
clarkbyup it nooped21:37
clarkbcrinkle: from what you can see it looks good?21:37
clarkbif so I am ready to reenable puppeting on these two Vms21:37
crinkleclarkb: as far as i can tell21:37
pleia2crinkle: nice work!21:38
clarkbthen I can trigger a bare-trusty image build to check that puppet still works for the slave builds21:38
rajinirasselin: http://paste.openstack.org/show/483078/21:38
*** hashar has quit IRC21:39
clarkbpuppet is reenabled21:39
rajinirasselin: http://paste.openstack.org/show/483079/ - layout.yaml21:39
crinkleshould i send an all-clear to the ml?21:40
*** weshay has joined #openstack-infra21:40
clarkbya  Ithink we are done (I just started the nodepool image build in rax-dfw21:40
*** aeng has joined #openstack-infra21:40
crinklecool thank you clarkb21:40
*** signed8b_ is now known as signed8bit_ZZZzz21:41
openstackgerritMerged openstack-infra/nodepool: Add 'public' attribute to 'networks' entry  https://review.openstack.org/25200021:41
*** gyee has joined #openstack-infra21:41
anteayamgagne: ^21:41
mgagnethanks, will wait for deployment if not done already =)21:42
mordredjeblair: nodepool change landed21:42
anteayaI think that a change to nodepool requires a nodepool restart21:42
mgagneI guess puppet needs to run first to update sources21:43
mgagnebut I don't know all the details of how it's deployed now21:43
anteayato be honest neither do I21:43
jeblairyep, puppet then manual restart21:43
clarkband puppet is running again21:44
nibalizercrinkle: clarkb excellent!21:44
pleia2yes, very nice to have mysql from this era :)21:46
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add option to use the wall time instead of the sum to run_time_meta  https://review.openstack.org/23176421:46
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: WIP: Kinda hacky double sorted run time meta  https://review.openstack.org/23449421:46
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add new graph time for variance of individual tests  https://review.openstack.org/23061121:46
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add new graph type to show run_time variance by metadata  https://review.openstack.org/21056921:46
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add a db api call to provide a rough wall time for a given run  https://review.openstack.org/23176221:46
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add run_time_meta arge to specify values to use in graph  https://review.openstack.org/23176321:46
pleia2s/mysql/mysql module21:46
asselinrajinir, seems you have a chain of patches, that should work, but it makes debugging a bit more tricky. Try using a single patch without any dependencies.21:47
asselinalso, try telnet localhost 4730 and type 'status' and paste the output21:47
*** matrohon has joined #openstack-infra21:47
rajinirasselin: status in telnet http://paste.openstack.org/show/483082/21:48
mordredjeblair: fwiw: Forbidden: IP allocation over quota in pool external. (HTTP 403) (Request-ID: req-d99b99f2-e32e-4243-9243-130e8668f0a4)21:49
mordredwe're getting floating ip quota issues in bluebox - I'm looking in to it21:49
*** signed8bit_ZZZzz is now known as signed8b_21:50
openstackgerritEmilien Macchi proposed openstack-infra/project-config: zuul/layout: disable puppet-tuskar jobs on master  https://review.openstack.org/26392321:51
asselinrajinir, so none of the jobs are registered. you know this b/c none of them from your layout.yaml are listed there. Try deleting a nodepool node and let one get recreated and get the updated status.21:54
clarkbasselin: rajinir you can just offline online in jenkins21:54
clarkbor disable enable a job to dp that21:54
*** annegentle has joined #openstack-infra21:55
sputnik13hate to keep asking, but this is gating a much needed feature for cue so...21:55
sputnik13can someone with +2 for requirements please review https://review.openstack.org/#/c/261074/21:55
greghaynesodyssey4me: Hey - did new images fix your resizing issue?21:56
*** eharney has joined #openstack-infra21:56
*** arxcruz has quit IRC21:57
*** bhunter71 has quit IRC21:58
*** sparkyco_ has quit IRC22:00
mordredgreghaynes: any chance you know how to set neutron quota?22:00
mordredgreghaynes: or, rather, how as a user I can check what my neutron quota is22:00
greghaynesmordred: no idea :(22:00
greghaynesmordred: neutron quota-update doesnt work?22:01
greghaynesthats what help says22:01
mordredgreghaynes: I'm sure that will work - but wasn't sure how I'd check as a user what my quota qas22:01
mordredto verify it had worked22:01
greghaynesquota-show will show for a tenant22:01
greghaynesno idea user22:02
jeblairrestarting nodepool22:02
greghayneser, no idea for user22:02
mordredgreghaynes: ok seems you can do that for your tenant-id as a normal user22:02
*** matrohon has quit IRC22:03
mordredjeblair, clarkb, greghaynes: we were seeing a non-zero number of failures on bluebox beause the floating ip quota on the openstackjenkins user was set to 10 and 50 in neutron22:03
mordredI'm guessing it was the nova limit that was borking us22:04
mordredI have raised the quotas22:04
greghaynesah, awesome22:04
jeblairmordred: 10 and 50?22:04
clarkbmordred: does a quota of 50 mean we have more available IPs?22:04
mordredjeblair: I think those were just the default values22:04
mordredand that when we got new ips we didn't get quota updates?22:04
jeblairmordred: i don't understand why there are 2 numbers?22:04
mordredanywho - we do have 2x /27 networks22:05
mordredjeblair: because nova has a number and neutron has a number22:05
*** sparkyco_ has joined #openstack-infra22:05
jeblairmordred: so nova was 10 and neutron was 50?22:05
*** jsavak has quit IRC22:05
mordredbecause nova had one from history because nova-network, but also neutron has one because you actually create floating ips via neutron22:05
jeblairmordred: when i set up the tenant, there was a fip quota option22:05
jeblairand i set it22:05
*** jsavak has joined #openstack-infra22:05
jeblairusing the gui22:05
jeblairso perhaps there's a bug that causes that to only set one thing?22:06
mordredyou probably wold have set it to 50, yeah?22:06
jeblairmordred: yeah, i think so22:06
mordredwell, let's see if our bluebox error rate goes down now22:06
*** matrohon has joined #openstack-infra22:06
jeblairmordred: ok.  also, surely we've had more than 10 in use though?22:06
openstackgerritSachi King proposed openstack-dev/pbr: Make setup.py --help-commands work without testrepository  https://review.openstack.org/26392822:07
mordredjeblair: totally unsure22:07
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Add heat lbaas specific test for v1 and v2  https://review.openstack.org/23715622:07
mordredjeblair: yes. in the past we definitely have had more than 1022:07
mordredNotFound: No more floating ips in pool external. (HTTP 404) (Request-ID: req-ffae274d-3143-4851-8ca4-34339753681b)22:08
openstackgerritJames E. Blair proposed openstack-infra/project-config: Nodepool: Identify public internap network  https://review.openstack.org/26392922:09
jeblairclarkb, mgagne, mordred, anteaya: ^ that should now correct our use of internap (no further restart needed)22:09
*** trown is now known as trown|outttypeww22:09
*** moravec1 has quit IRC22:09
*** rguillebert has joined #openstack-infra22:09
mordredjeblair: ah - actually, we seem to have leaked a bunch of fips there22:09
jeblairmordred: it'll be nice to be in the shade22:09
*** harlowja has quit IRC22:09
greghaynesalways throwing shade at the clouds22:10
*** harlowja has joined #openstack-infra22:10
anteayajeblair: +222:10
*** sparkyco_ has quit IRC22:10
greghaynesmordred: does that mean the nova quota might have been a red herring?22:10
greghaynesmordred: it seems really wierd to me that nova would care about that when using neutron22:10
*** thiagop has quit IRC22:11
*** dslev has joined #openstack-infra22:12
mordredgreghaynes: quite possibly22:12
isaacbclarkb: reminder, following our talk last week about timing the cron job in puppet master, can you please look in https://review.openstack.org/#/c/262466/22:13
anteayaclarkb: different topic what is the status of the rax/intel <insert acronym here> thing? I just thought of it, sorry to interrupt22:13
clarkbanteaya: last I heard they were trying to have something we could tlak to by mid month22:13
anteayaclarkb: cool, thank you22:14
anteayaI wish acronyms were hieroglyphics instead, I could remember "draw me a picture"22:15
clarkbanteaya: with a quota of 100 test VMs22:16
*** dslev_ has joined #openstack-infra22:17
*** moravec has joined #openstack-infra22:17
*** moravec has quit IRC22:17
anteayathat is usable22:18
* anteaya draws a quota of 100 test vms in hieroglyphs22:18
NakatoThanks mordred, lifeless: https://review.openstack.org/26392822:19
*** boris-42 has joined #openstack-infra22:20
*** otter768 has joined #openstack-infra22:20
*** dims has quit IRC22:20
*** masber has joined #openstack-infra22:20
*** dslev has quit IRC22:20
*** regXboi has quit IRC22:22
openstackgerritMatthew Thode proposed openstack/diskimage-builder: Initial add of gentoo support for diskimage-builder  https://review.openstack.org/26334922:22
*** ybathia has joined #openstack-infra22:23
*** kzaitsev_mb has quit IRC22:24
*** peristeri has quit IRC22:24
*** thorst has quit IRC22:24
*** otter768 has quit IRC22:25
*** thorst has joined #openstack-infra22:25
*** dims has joined #openstack-infra22:25
*** annegentle has quit IRC22:28
*** erikwilson has quit IRC22:28
mordredNakato, lifeless: looks great! wanna review https://review.openstack.org/#/c/263748/ ?22:29
isaacbclarkb: thanks.22:30
*** thorst has quit IRC22:30
*** moravec has joined #openstack-infra22:31
*** signed8b_ is now known as signed8bit_ZZZzz22:32
clarkbisaacb: np22:32
*** moravec has quit IRC22:32
*** edmondsw has quit IRC22:33
*** dkranz has quit IRC22:34
Nakatomordred: Taking a look22:34
*** signed8bit_ZZZzz is now known as signed8b_22:34
clarkbcrinkle: bare-trusty build in rax-dfw completed successfully22:36
*** edmondsw has joined #openstack-infra22:37
*** isaacb has quit IRC22:37
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: toci_gate_test/overcloud/non-ha: enable monitoring  https://review.openstack.org/25478722:37
openstackgerritMerged openstack-infra/system-config: Dynamically schedule for updatepuppet master cron job  https://review.openstack.org/26246622:37
*** edmondsw has quit IRC22:37
crinkleclarkb: \o/22:38
*** dprince has quit IRC22:40
*** piet has quit IRC22:40
reedshould the linux kernel resolution be on http://governance.openstack.org/resolutions/index.html ?22:41
*** Sukhdev has quit IRC22:41
mordredreed: yes. it seems neither thing today has published docs22:41
mordredreed: I have not been able to look in to it yet22:42
reedthe draft job doesn't have it either22:42
*** annegentle has joined #openstack-infra22:42
jlvillalclarkb: I am trying to simulate the devstack-gate. If I don't have: /etc/nodepool/sub_nodes_private is that a problem? As the ansible command seems to be using that.22:43
*** krtaylor has quit IRC22:43
jlvillalBasically wondering what I should put in that file.22:43
jlvillalclarkb: Asking you because git blame says you wrote that part of the code :)22:43
*** aeng has quit IRC22:43
clarkbno its not a problem the error is higher in your paste22:44
mordredreed: it's there when I build it locally :(22:44
clarkbit should write out an empty inventory which is why ansible nooped there22:44
jlvillalclarkb: Okay. I will look higher in my pastebin and see what I can discover.22:46
jlvillalclarkb: But if you have any hints it would be appreciated...22:46
*** amitgandhinz has quit IRC22:47
*** amitgandhinz has joined #openstack-infra22:48
*** annegent_ has joined #openstack-infra22:48
*** annegentle has quit IRC22:49
*** thorst has joined #openstack-infra22:49
clarkbits says the main script has failed please refer to such and such log for details22:49
jlvillalclarkb: Thanks. I think I found an issue.22:50
openstackgerritMerged openstack-infra/project-config: Nodepool: Identify public internap network  https://review.openstack.org/26392922:50
anteayajeblair: ^22:51
*** Qiming has joined #openstack-infra22:51
jeblairanteaya: woot, should just start working after the next puppeting22:52
docaedoAny infra cores care to merge https://review.openstack.org/#/c/260230/ ?  It festively adorned with +2's and +1's :)22:52
*** thorst has quit IRC22:53
openstackgerritMerged openstack-infra/shade: Bug fix: Cinder v2 returns bools now  https://review.openstack.org/26376422:54
*** aeng has joined #openstack-infra22:56
anteayajeblair: yay22:57
*** amitgandhinz has quit IRC22:57
anteayaclarkb: I have a question on 263923, the jobs will be thus according to the layout output: http://logs.openstack.org/23/263923/1/check/gate-project-config-layout/51a593d/console.html#_2016-01-05_22_26_15_18622:57
anteayaclarkb: to me it looks like that master will only run with noop thus avoiding the issue where a patch has both noop jobs and other jobs on it22:58
anteayaclarkb: do you see the same?22:58
anteayaEmilienM: btw somehow puppet-tuskar still has two icehouse jobs configured for it: http://logs.openstack.org/23/263923/1/check/gate-project-config-layout/51a593d/console.html#_2016-01-05_22_26_15_18622:59
EmilienManteaya: we don't support icehouse anymore22:59
EmilienManteaya: we stop after juno22:59
*** matrohon has quit IRC22:59
anteayaI know22:59
anteayabut if you look at the jobs in the link, somehow there are still two icehouse jobs configured23:00
anteayaEmilienM: I don't know where they are coming from23:00
*** daemontool has joined #openstack-infra23:00
anteaya<Job gate-puppet-tuskar-puppet-syntax-4-dsvm-centos7> ^(?!stable/(icehouse)).*$23:00
anteayaand -3 from the line above23:01
openstackgerritJohn L. Villalovos proposed openstack-infra/devstack-gate: Fix misspelling of connectivity  https://review.openstack.org/26393823:01
EmilienManteaya: that's weird. This module does not have stable/icehouse so it will never match23:02
anteayaEmilienM: right, I also find it weird23:02
*** matrohon has joined #openstack-infra23:02
anteayaEmilienM: thought I would share what I saw with you23:02
*** yuanying has joined #openstack-infra23:03
*** jsavak has quit IRC23:04
*** rhallisey has joined #openstack-infra23:04
*** otter768 has joined #openstack-infra23:05
EmilienManteaya: I can add icehouse if you want to the branch line23:07
EmilienMbut it's useless, since we don't have this branch online23:07
*** bhunter71 has joined #openstack-infra23:07
anteayano no no23:07
*** woodster_ has joined #openstack-infra23:07
anteayaI failed to communicate what I tried to do23:07
clarkbanteaya: le me see23:08
anteayaI am saying that our configuration believes it needs to configure an icehouse job for that repo23:08
anteayaclarkb: thanks23:08
anteayaEmilienM: which both you and I know it does not need to do23:08
anteayaEmilienM: there is a ghost in the configuration somewhere and it is thinking it needs to configure a job for that repo that it doesn't need to23:09
*** dingyichen has joined #openstack-infra23:09
anteayaEmilienM: and we didn't catch it before because that non-existant job will never run23:09
anteayamy dinner is ready and I will be back in a bit23:11
clarkbanteaya: so that will run the noop jobs on all branches, and restrict the other jobs to the stable branches. This is problematic because then that project ends up in the noop job queue with real jobs23:11
clarkbyou can have a project specific noop job instead23:11
clarkband use that only against master23:11
anteayaclarkb: thanks for taking a look, that was exactly the feedback I was looking for, thanks23:12
anteayadinner now, back in a few23:12
clarkbif it wree me I would just deprecate the project23:12
*** chlong has joined #openstack-infra23:12
clarkbif upstreams is saying no more then its easy downstream to follow along23:12
*** chlong has quit IRC23:14
clarkbhttps://jenkins01.openstack.org/job/gate-tempest-dsvm-neutron-full/17978/console is running on internap23:15
clarkbjeblair: ^ fyi23:15
*** doug-fis_ has joined #openstack-infra23:15
*** angdraug has quit IRC23:16
clarkbmgagne: ^ you are probably interested as well23:17
mgagnewas about to check =)23:17
mgagnethanks for the link23:17
*** doug-fish has quit IRC23:17
*** dims has quit IRC23:20
*** matrohon has quit IRC23:22
*** krtaylor has joined #openstack-infra23:22
*** dims has joined #openstack-infra23:25
*** doug-fis_ has quit IRC23:25
TheloHi all, if I do two commits, one in os_brick the other one in nova, is there any way to tell the CI that I want my nova patch to be tested with the path for os_brick (which is installed by pip)?23:26
JayFI'm having all kinds of trouble building upper-constraints.txt for the requirements repo. Everytime I run the generate-constraints command, I get failures that look like a malformed krb5 build string23:27
asselinThelo, yes, there's a env variable in devstack: libs_from_git or something like that23:27
anteayaEmilienM: any chance you can just deprecate the project as clarkb suggests?23:27
clarkbJayF: have a paste?23:27
JayFjust got finished pasting it23:28
samueldmqasselin: isn't he wanting to get it tested via openstack ci? if so, isn't that what Depends-On does ?23:28
EmilienMclarkb: the project is deprecated23:28
JayFclarkb: ^ the weird thing is it looks like some error message is getting passed as options to gcc23:28
EmilienMclarkb: I don't mind dropping all jobs23:28
anteayaEmilienM: then let's do that23:28
asselinsamueldmq, depend-on won't work for a lib afaik23:28
*** sridhar_ram has quit IRC23:28
anteayajust remove the jobs23:28
EmilienManteaya: noop23:29
EmilienMso we can still patch stable branches23:29
EmilienMjust in case23:29
anteayaEmilienM: well if you like23:29
*** dimtruck is now known as zz_dimtruck23:29
asselinsamueldmq, it will check it out, but sill install via pip. if you set libs-from-git to include os-brick, it will install based on the depends-on23:29
asselinThelo, ^^23:29
*** austin81 has left #openstack-infra23:29
*** shardy has quit IRC23:30
clarkbJayF: looks like you ar emissing C headers23:30
*** chlong has joined #openstack-infra23:30
JayFbuild-essential is installed, as is the output of bindep -b23:31
*** weshay has quit IRC23:31
clarkbyou need krb-config23:31
Thelowhere do I put the "libs from git"23:31
Theloin the commit message?23:31
openstackgerritEmilien Macchi proposed openstack-infra/project-config: zuul/layout: disable puppet-tuskar jobs  https://review.openstack.org/26392323:31
JayFclarkb: it's installed.23:31
JayFclarkb: that was the first thing I tried upon seeing the error23:31
clarkbJayF: the -dev headers package?23:31
clarkboh looks like there is no header package for that on trusty at least23:32
asselinThelo, no, you need a special ci job. here's one for example: https://review.openstack.org/#/c/188156/23:32
samueldmqasselin: sure, I thought it wasn't about testing it locally or not needing to create a new ci job23:32
JayFclarkb: libkrb5-dev existed and wasn't installed23:32
JayFclarkb: so rerunning now23:32
samueldmqasselin: great thanks for clarifying23:32
*** annegent_ has quit IRC23:32
jrollJayF: that should be added to other-requirements.txt, AIUI23:33
jrollJayF: is this running on master of openstack/requirements or with additional requirements added?23:33
JayFit's running on openstack/requirements w/mimic 2.0 added23:33
*** gordc has quit IRC23:33
JayFI guess I should've run against master first23:33
asselinsamueldmq, Thelo http://git.openstack.org/cgit/openstack-dev/devstack/tree/doc/source/configuration.rst#n32423:33
jrollJayF: yeah, you need to, to get a baseline, but sounds like you'll need to add libkrb5-dev to other-requirements.txt to the patch23:34
JayF10-4 can do that23:34
*** sigmavirus24 is now known as sigmavirus24_awa23:35
asselininfra-root http://codesearch.openstack.org/ seems broken now23:36
asselinnevermind...working again.23:36
nibalizerasselin: it works  http://codesearch.openstack.org/?q=asselin&i=nope&files=&repos=23:36
asselinthre was a message "server broke down" followed by a refresh will a 500 error23:36
*** mriedem has quit IRC23:37
nibalizerill look at some logs23:37
*** otter768 has quit IRC23:37
*** oomichi has joined #openstack-infra23:37
*** signed8b_ is now known as signed8bit_ZZZzz23:39
*** moravec has joined #openstack-infra23:39
Theloasselin: But when you push a patch for review how do you sy you want to add this option in the gate local.conf?23:39
*** tonytan4ever has quit IRC23:39
anteayaasselin nibalizer: jeblair has said that there is a strange intermittent failure with codesearch, that usually goes away on a refresh23:39
anteayaI don't think a bug has been filed23:40
*** daemontool has quit IRC23:40
asselinyeah, sound like this23:40
anteayaif nibalizer can find any evidence in logs mayhap it is time for a bug report?23:40
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Fix filtering in search_stacks()  https://review.openstack.org/26372423:41
*** signed8bit_ZZZzz is now known as signed8b_23:41
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add tests for stack search API  https://review.openstack.org/25972723:41
asselinThelo, yeah I see, I don't think that's possible right now. You can however test your os-brick patch with a depends-on nova patch....23:41
nibalizerfrom the logs it looks like it spends almost a minute down while starting new searchers23:41
anteayanibalizer: hmmm23:42
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: correct rpmlint errors  https://review.openstack.org/26352623:42
anteayanibalizer: how does it start new searchers?23:42
anteayaha ha ha23:42
*** thorst has joined #openstack-infra23:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud modules to modules.env  https://review.openstack.org/20961723:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud controller node  https://review.openstack.org/20969823:42
nibalizerit also seems to wipe out its log every time it restarts23:42
nibalizerso thats cool23:42
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud compute node definition  https://review.openstack.org/21398023:43
anteayadoes it start new searchers after it re-indexes?23:43
anteayais that the trigger?23:43
anteayaor I guess if it takes out the logs it would be tough to find out23:43
*** amuller has quit IRC23:44
*** sabeen3 has quit IRC23:45
samueldmqasselin: thanks for the link23:46
openstackgerritSpencer Krum proposed openstack-infra/puppet-hound: Append to logfile instead of overwriting  https://review.openstack.org/26395323:46
*** dslev_ has quit IRC23:46
nibalizerthat should fix the log issue23:46
Theloasselin: thanks23:46
openstackgerritMichael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror  https://review.openstack.org/16492723:48
anteayanibalizer: thanks23:48
*** sridhar_ram has joined #openstack-infra23:48
krotscheckjeblair: Do you have a pastebin with those rewrite rules of yours? I'd like to add them to https://review.openstack.org/#/c/253521/23:49
anteayajhesketh: jenkins is unhappy on these: https://review.openstack.org/#/q/status:open+topic:enable_swift+project:openstack-infra/project-config23:49
*** rbrndt__ has quit IRC23:49
*** signed8b_ has quit IRC23:49
nibalizerya it kinda looks like whenever hound restarts its just down for a little bit while it figure its life out23:49
nibalizerfinds new repos, clones them, ettc23:49
nibalizerasselin: did you get a proxy nott responding error?23:49
nibalizerthat seems to be the error apache has23:49
asselinnibalizer, no23:49
anteayaso whenever it finds a new repo, it restarts itself?23:50
nibalizerwhat did you get23:50
*** annegentle has joined #openstack-infra23:50
asselinshould've taken a screenshot. First was a nicely formated page with the words "server broke down"23:50
anteayaha ha ha23:50
nibalizerthat sounds like the kind of error that the etsy people would write (making me think it came from hound)23:51
krotscheckgreghaynes: I think I've managed to combine both the folder structure concerns from the last comment, and the desire to have the folder structure work with AFS, into ^^.23:51
* anteaya pictures a server by the side of the road, steam issuing from under the hood23:51
asselinI refreshed and got the standard server not available message23:51
asselinI refreshed again and it was up and ready to go23:52
asselinyeah I think it's from hound too23:52
taronhmm, this is what I was afraid of when I set hound up that way, though I don't have any brilliant alternatives off the top of my head23:56
*** tzn has quit IRC23:56
anteayataron: so when hound encounters a new repo it restarts itself?23:57
taronthe restart is triggered through puppet23:57
anteayataron: can you link me to where it is triggered?23:57
taronyep, one sec23:57
*** piet has joined #openstack-infra23:58
*** eil397 has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!