| *** baoli has joined #openstack-infra | 00:01 | |
| *** markvoelker has quit IRC | 00:01 | |
| *** asselin_ has joined #openstack-infra | 00:02 | |
| *** kzaitsev_mb has quit IRC | 00:02 | |
| *** hongbin has quit IRC | 00:04 | |
| *** yamada-h has joined #openstack-infra | 00:05 | |
| *** asselin_ has quit IRC | 00:07 | |
| *** Sukhdev has joined #openstack-infra | 00:07 | |
| *** yamada-h has quit IRC | 00:09 | |
| *** ffrank has quit IRC | 00:11 | |
| *** ffrank has joined #openstack-infra | 00:11 | |
| *** rcernin has quit IRC | 00:13 | |
| *** baoli has quit IRC | 00:13 | |
| *** dingyichen has joined #openstack-infra | 00:16 | |
| *** amitgandhinz has joined #openstack-infra | 00:19 | |
| *** zeih has joined #openstack-infra | 00:21 | |
| *** baoli has joined #openstack-infra | 00:23 | |
| *** amitgandhinz has quit IRC | 00:24 | |
| *** zeih has quit IRC | 00:25 | |
| *** baoli has quit IRC | 00:28 | |
| *** sflanigan has joined #openstack-infra | 00:28 | |
| *** kzaitsev_mb has joined #openstack-infra | 00:29 | |
| *** chlong has joined #openstack-infra | 00:36 | |
| *** lvdongbing has joined #openstack-infra | 00:38 | |
| *** roxanaghe has joined #openstack-infra | 00:38 | |
| *** roxanaghe has quit IRC | 00:42 | |
| *** kzaitsev_mb has quit IRC | 00:55 | |
| *** thorst has joined #openstack-infra | 01:01 | |
| *** thorst has quit IRC | 01:04 | |
| *** mriedem has quit IRC | 01:04 | |
| *** thorst has joined #openstack-infra | 01:05 | |
| *** NM has joined #openstack-infra | 01:06 | |
| *** lvdongbing has quit IRC | 01:07 | |
| *** ffrank has quit IRC | 01:08 | |
| *** thorst has quit IRC | 01:13 | |
| *** zhurong has joined #openstack-infra | 01:16 | |
| *** amitgandhinz has joined #openstack-infra | 01:20 | |
| *** lvdongbing has joined #openstack-infra | 01:22 | |
| *** cfarquhar has quit IRC | 01:24 | |
| *** amitgandhinz has quit IRC | 01:24 | |
| *** thorst has joined #openstack-infra | 01:26 | |
| *** thorst has quit IRC | 01:26 | |
| *** thorst has joined #openstack-infra | 01:27 | |
| openstackgerrit | wangxiyuan proposed openstack-infra/project-config: Enable zaqar tempest test https://review.openstack.org/328670 | 01:27 |
|---|---|---|
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Update zuul layout for puppet-openstack_extras https://review.openstack.org/328808 | 01:29 |
| *** cfarquhar has joined #openstack-infra | 01:29 | |
| *** cfarquhar has quit IRC | 01:29 | |
| *** cfarquhar has joined #openstack-infra | 01:29 | |
| *** thorst has quit IRC | 01:31 | |
| *** david-lyle has quit IRC | 01:34 | |
| lvdongbing | tonyb, Are you there? | 01:34 |
| loquacities | lvdongbing: it's a public holiday in australia, so i imagine tonyb is doing something !work | 01:36 |
| lvdongbing | Oh, got it, thank you! | 01:36 |
| *** kengo_sakai has joined #openstack-infra | 01:36 | |
| *** kengo_sakai has quit IRC | 01:36 | |
| *** kengo_sakai has joined #openstack-infra | 01:37 | |
| *** baoli has joined #openstack-infra | 01:38 | |
| *** baoli has quit IRC | 01:40 | |
| *** timothyb89 has quit IRC | 01:41 | |
| *** timothyb89 has joined #openstack-infra | 01:42 | |
| *** Daisy has joined #openstack-infra | 01:47 | |
| *** hongbin has joined #openstack-infra | 01:47 | |
| zxiiro | are logs on logs.openstack.org stored forever or do they get cleaned up sometimes? | 01:49 |
| lvdongbing | clarkb, Could you please help review this commit: https://review.openstack.org/#/c/326271/ | 01:50 |
| *** PollyZ has joined #openstack-infra | 01:51 | |
| *** kzaitsev_mb has joined #openstack-infra | 01:51 | |
| *** kengo_sakai has quit IRC | 01:53 | |
| *** xnox has quit IRC | 01:54 | |
| fungi | zxiiro: we have a cron job which runs this weekly http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/files/log_archive_maintenance.sh | 01:54 |
| fungi | zxiiro: so ~120-127 days retention right now | 01:54 |
| zxiiro | fungi: cool that's what I was looking for, thanks! | 01:55 |
| fungi | zxiiro: that results in the pretty waveform pattern you see at http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=717&rra_id=all | 01:55 |
| *** kzaitsev_mb has quit IRC | 01:56 | |
| *** markvoelker has joined #openstack-infra | 01:58 | |
| *** baoli has joined #openstack-infra | 01:58 | |
| *** baoli has quit IRC | 01:59 | |
| *** PollyZ_ has joined #openstack-infra | 02:00 | |
| *** PollyZ has quit IRC | 02:00 | |
| *** xnox has joined #openstack-infra | 02:01 | |
| *** markvoelker has quit IRC | 02:02 | |
| *** yamamoto_ has joined #openstack-infra | 02:05 | |
| *** rhallisey has quit IRC | 02:06 | |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/project-config: Add gates for deploying kolla-kubernetes https://review.openstack.org/328814 | 02:06 |
| *** zz_dimtruck is now known as dimtruck | 02:06 | |
| *** xnox has quit IRC | 02:08 | |
| *** r-mibu has quit IRC | 02:12 | |
| *** sputnik13 has joined #openstack-infra | 02:12 | |
| *** sputnik13 has quit IRC | 02:13 | |
| *** PollyZ has joined #openstack-infra | 02:14 | |
| *** PollyZ_ has quit IRC | 02:14 | |
| lvdongbing | fungi, Could you please help review this commit: https://review.openstack.org/#/c/326271/ | 02:15 |
| *** r-mibu has joined #openstack-infra | 02:17 | |
| *** amitgandhinz has joined #openstack-infra | 02:20 | |
| *** zeih has joined #openstack-infra | 02:21 | |
| *** amitgandhinz has quit IRC | 02:26 | |
| *** zeih has quit IRC | 02:26 | |
| *** PollyZ has quit IRC | 02:29 | |
| *** thorst has joined #openstack-infra | 02:30 | |
| *** thorst has quit IRC | 02:30 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Bring baremetal00.vanilla.ic.openstack.org online https://review.openstack.org/327314 | 02:30 |
| *** thorst has joined #openstack-infra | 02:30 | |
| pabelanger | rcarrillocruz: fungi: I believe that will enroll baremetal00 into ansible again | 02:31 |
| pabelanger | spent some time going through our commit history | 02:31 |
| *** NM has quit IRC | 02:31 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Bring baremetal00.vanilla.ic.openstack.org online https://review.openstack.org/327314 | 02:32 |
| *** PollyZ has joined #openstack-infra | 02:32 | |
| *** roxanaghe has joined #openstack-infra | 02:32 | |
| *** yamamoto_ has quit IRC | 02:33 | |
| *** zhurong has quit IRC | 02:35 | |
| *** thorst has quit IRC | 02:36 | |
| *** yamamoto_ has joined #openstack-infra | 02:37 | |
| openstackgerrit | wangxiyuan proposed openstack-infra/project-config: Switch the voting for zaqar tempest test https://review.openstack.org/328670 | 02:40 |
| *** PollyZ has quit IRC | 02:46 | |
| *** yamamoto_ has quit IRC | 02:46 | |
| *** zhurong has joined #openstack-infra | 02:48 | |
| *** ayoung has quit IRC | 02:48 | |
| *** yamamoto_ has joined #openstack-infra | 02:48 | |
| *** PollyZ has joined #openstack-infra | 02:49 | |
| *** yamamoto_ has quit IRC | 02:50 | |
| mordred | pabelanger: I think you're the new me | 02:50 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-infracloud: Copy hpuswest to vanilla https://review.openstack.org/327332 | 02:51 |
| mordred | pabelanger: I cannot remember the last time I thought "pabelanger clearly must be asleep right now" | 02:51 |
| *** wanghua has joined #openstack-infra | 02:51 | |
| pabelanger | mordred: not sure I'll every be the new mordred | 02:51 |
| pabelanger | move computer into new room in the condo, more access to it now! | 02:52 |
| pabelanger | with is a good and bad thing | 02:52 |
| mordred | ahhhh | 02:52 |
| mordred | yes | 02:52 |
| mordred | I know this very well | 02:52 |
| pabelanger | I need to get SteamOS going on my linux box again | 02:52 |
| *** Guest39128 has quit IRC | 02:52 | |
| mordred | of course, when I was in NY, there really was only the one room | 02:52 |
| pabelanger | then I can burn brain cells on stream | 02:52 |
| *** kzaitsev_mb has joined #openstack-infra | 02:53 | |
| pabelanger | mordred: no doubt | 02:53 |
| nibalizer | pabelanger: steam or stream? | 02:53 |
| nibalizer | because 'pabalanger plays infra' on twitch.tv would be fun | 02:53 |
| pabelanger | nibalizer: lawl | 02:53 |
| mordred | nibalizer: zomg | 02:54 |
| mordred | nibalizer: I think I actually got that reference | 02:54 |
| mordred | nibalizer: does that mean I'm getting younger again? | 02:54 |
| wanghua | Hi all, when I use diskimage-builder to build a centos image with elements centos-minmal vm simple-init openstack-repos nodepool-base node-devstack puppet cache-bindep growroot infra-package-needs, there is an error in 95-chown-jenkins: chown: invalid user: 'jenkins:jenkins'. I want to know when jenkins user is added? | 02:55 |
| nibalizer | mordred: yes | 02:55 |
| pabelanger | wanghua: puppet element, which runs puppet will add the jenkins user. | 02:56 |
| pabelanger | wanghua: looking at your debug.log, you'll likely see something is failing to apply correctly. I cannot remember what specifically, but I do remember the error | 02:57 |
| wanghua | pabelanger: thank you for your help | 02:58 |
| *** NM has joined #openstack-infra | 02:58 | |
| *** roxanaghe has quit IRC | 02:58 | |
| *** zhurong has quit IRC | 02:58 | |
| *** kzaitsev_mb has quit IRC | 02:59 | |
| mordred | nibalizer: woot! I've been trying to de-age for foreer | 02:59 |
| *** zhurong has joined #openstack-infra | 02:59 | |
| *** roxanaghe has joined #openstack-infra | 03:04 | |
| *** ramishra has joined #openstack-infra | 03:06 | |
| *** aviau has quit IRC | 03:09 | |
| *** aviau has joined #openstack-infra | 03:09 | |
| *** gouthamr has quit IRC | 03:12 | |
| *** amitgandhinz has joined #openstack-infra | 03:21 | |
| *** Daisy has quit IRC | 03:25 | |
| *** Daisy has joined #openstack-infra | 03:26 | |
| *** amitgandhinz has quit IRC | 03:26 | |
| *** PollyZ has quit IRC | 03:32 | |
| *** PollyZ has joined #openstack-infra | 03:33 | |
| *** thorst has joined #openstack-infra | 03:34 | |
| *** yamamoto_ has joined #openstack-infra | 03:34 | |
| *** Daisy has quit IRC | 03:35 | |
| *** Daisy has joined #openstack-infra | 03:35 | |
| *** yamamoto_ has quit IRC | 03:36 | |
| *** Daisy has quit IRC | 03:40 | |
| *** thorst has quit IRC | 03:41 | |
| *** PollyZ has quit IRC | 03:42 | |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config: Disable convergence for jobs with legacy engine https://review.openstack.org/327560 | 03:49 |
| *** PollyZ has joined #openstack-infra | 03:54 | |
| *** kzaitsev_mb has joined #openstack-infra | 03:55 | |
| *** PollyZ has quit IRC | 03:58 | |
| *** Wei_Liu has joined #openstack-infra | 03:58 | |
| *** markvoelker has joined #openstack-infra | 03:59 | |
| *** kzaitsev_mb has quit IRC | 04:00 | |
| *** Sukhdev has quit IRC | 04:02 | |
| *** roxanaghe has quit IRC | 04:02 | |
| *** markvoelker has quit IRC | 04:03 | |
| *** bpokorny has quit IRC | 04:04 | |
| *** dimtruck is now known as zz_dimtruck | 04:06 | |
| notmorgan | nibalizer: can we just do "twitch runs openstack infra" for a day? | 04:08 |
| notmorgan | nibalizer: no wait... lets not. | 04:08 |
| notmorgan | nibalizer: i just realized what i said. | 04:08 |
| *** roxanaghe has joined #openstack-infra | 04:10 | |
| *** Sukhdev has joined #openstack-infra | 04:10 | |
| *** Daisy has joined #openstack-infra | 04:13 | |
| *** psachin has joined #openstack-infra | 04:13 | |
| *** roxanaghe has quit IRC | 04:14 | |
| *** PollyZ has joined #openstack-infra | 04:15 | |
| *** zz_dimtruck is now known as dimtruck | 04:16 | |
| *** Daisy has quit IRC | 04:17 | |
| *** Daisy has joined #openstack-infra | 04:21 | |
| *** amitgandhinz has joined #openstack-infra | 04:22 | |
| *** zeih has joined #openstack-infra | 04:22 | |
| *** PollyZ has quit IRC | 04:23 | |
| *** roxanaghe has joined #openstack-infra | 04:23 | |
| *** Daisy has quit IRC | 04:25 | |
| *** Daisy has joined #openstack-infra | 04:25 | |
| *** dimtruck is now known as zz_dimtruck | 04:26 | |
| *** Daisy has quit IRC | 04:26 | |
| *** Daisy has joined #openstack-infra | 04:26 | |
| *** amitgandhinz has quit IRC | 04:26 | |
| *** links has joined #openstack-infra | 04:26 | |
| *** zeih has quit IRC | 04:27 | |
| *** hongbin has quit IRC | 04:28 | |
| *** Daisy has quit IRC | 04:31 | |
| *** yamamoto_ has joined #openstack-infra | 04:33 | |
| *** esikachev has joined #openstack-infra | 04:36 | |
| *** thorst has joined #openstack-infra | 04:39 | |
| *** esikachev has quit IRC | 04:40 | |
| *** yamada-h has joined #openstack-infra | 04:42 | |
| *** thorst has quit IRC | 04:46 | |
| *** asselin_ has joined #openstack-infra | 04:51 | |
| *** kengo_sa_ has joined #openstack-infra | 04:54 | |
| *** kzaitsev_mb has joined #openstack-infra | 04:55 | |
| *** kengo_sa_ has quit IRC | 05:00 | |
| *** kzaitsev_mb has quit IRC | 05:00 | |
| *** zeih has joined #openstack-infra | 05:02 | |
| *** lvdongbing has quit IRC | 05:07 | |
| *** zeih has quit IRC | 05:07 | |
| *** jaosorior has joined #openstack-infra | 05:11 | |
| *** nadya has joined #openstack-infra | 05:16 | |
| *** ilyashakhat has joined #openstack-infra | 05:20 | |
| *** salv-orlando has joined #openstack-infra | 05:21 | |
| *** salv-orl_ has quit IRC | 05:21 | |
| openstackgerrit | Doug Wiegley proposed openstack/gertty: Prevent more than one gertty from running at a time https://review.openstack.org/328583 | 05:22 |
| *** amitgandhinz has joined #openstack-infra | 05:23 | |
| *** pgadiya has joined #openstack-infra | 05:26 | |
| *** amitgandhinz has quit IRC | 05:27 | |
| *** salv-orl_ has joined #openstack-infra | 05:29 | |
| *** salv-orlando has quit IRC | 05:31 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Set debugging flag to testenv-client script https://review.openstack.org/328843 | 05:32 |
| *** kushal has joined #openstack-infra | 05:34 | |
| *** savihou has joined #openstack-infra | 05:43 | |
| *** thorst has joined #openstack-infra | 05:43 | |
| *** jianghuaw has quit IRC | 05:45 | |
| *** infra-red has joined #openstack-infra | 05:47 | |
| *** kushal has quit IRC | 05:47 | |
| *** thorst has quit IRC | 05:51 | |
| *** lvdongbing has joined #openstack-infra | 05:53 | |
| *** hashar has joined #openstack-infra | 05:54 | |
| *** infra-red has quit IRC | 05:54 | |
| *** hashar is now known as hasharAway | 05:55 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:57 | |
| *** markvoelker has joined #openstack-infra | 05:59 | |
| *** cody-somerville_ has quit IRC | 06:01 | |
| *** kzaitsev_mb has quit IRC | 06:02 | |
| *** kushal has joined #openstack-infra | 06:02 | |
| *** chlong has quit IRC | 06:03 | |
| *** markvoelker has quit IRC | 06:04 | |
| *** sridhar_ram is now known as sridhar_ram_ooo | 06:11 | |
| *** rcernin has joined #openstack-infra | 06:14 | |
| *** chlong has joined #openstack-infra | 06:15 | |
| *** ccamacho has joined #openstack-infra | 06:16 | |
| *** rcernin has quit IRC | 06:18 | |
| *** rcernin has joined #openstack-infra | 06:19 | |
| *** zeih has joined #openstack-infra | 06:22 | |
| *** nadya has quit IRC | 06:22 | |
| *** ilyashakhat has quit IRC | 06:22 | |
| *** amitgandhinz has joined #openstack-infra | 06:24 | |
| *** yolanda has joined #openstack-infra | 06:24 | |
| *** nmagnezi has joined #openstack-infra | 06:25 | |
| *** nmagnezi has quit IRC | 06:25 | |
| *** ilyashakhat has joined #openstack-infra | 06:26 | |
| yolanda | good morning | 06:26 |
| *** zeih has quit IRC | 06:27 | |
| *** nmagnezi has joined #openstack-infra | 06:27 | |
| *** amitgandhinz has quit IRC | 06:28 | |
| *** zeih has joined #openstack-infra | 06:30 | |
| *** kushal has quit IRC | 06:33 | |
| AJaeger | morning, yolanda | 06:34 |
| openstackgerrit | Merged openstack-infra/project-config: Install-Guide: Fix script https://review.openstack.org/328767 | 06:35 |
| *** ilyashakhat has quit IRC | 06:36 | |
| *** abregman has joined #openstack-infra | 06:38 | |
| *** flip217 is now known as flip214 | 06:42 | |
| *** flip214 has quit IRC | 06:42 | |
| *** flip214 has joined #openstack-infra | 06:42 | |
| Wei_Liu | morning, yolanda | 06:42 |
| yolanda | hi | 06:43 |
| openstackgerrit | Merged openstack-infra/project-config: Update zuul layout for puppet-openstack_extras https://review.openstack.org/328808 | 06:43 |
| *** andreas_s has joined #openstack-infra | 06:45 | |
| *** tkelsey has joined #openstack-infra | 06:47 | |
| *** thorst has joined #openstack-infra | 06:50 | |
| *** claudiub has quit IRC | 06:51 | |
| openstackgerrit | Merged openstack-infra/project-config: Switch the voting for zaqar tempest test https://review.openstack.org/328670 | 06:53 |
| *** e0ne has joined #openstack-infra | 06:53 | |
| *** yamada-h has quit IRC | 06:55 | |
| *** thorst has quit IRC | 06:56 | |
| *** yamada-h has joined #openstack-infra | 06:59 | |
| *** amotoki has joined #openstack-infra | 07:00 | |
| *** javeriak has joined #openstack-infra | 07:00 | |
| *** tesseract has joined #openstack-infra | 07:00 | |
| *** infra-red has joined #openstack-infra | 07:06 | |
| *** salv-orl_ has quit IRC | 07:07 | |
| *** javeriak has quit IRC | 07:07 | |
| *** salv-orlando has joined #openstack-infra | 07:08 | |
| *** Jeffrey4l has joined #openstack-infra | 07:09 | |
| *** mhickey has joined #openstack-infra | 07:12 | |
| *** yolanda has quit IRC | 07:15 | |
| *** chlong has quit IRC | 07:15 | |
| *** nadya has joined #openstack-infra | 07:16 | |
| *** matrohon has joined #openstack-infra | 07:16 | |
| *** arxcruz has joined #openstack-infra | 07:18 | |
| wanghua | pabelanger: Do you know when puppet modules are installed, system-config/install_modules.sh only git clone all the modules. | 07:19 |
| *** aviau has quit IRC | 07:20 | |
| openstackgerrit | Lukas Bednar proposed openstack-infra/jenkins-job-builder: Builders: Add ansible-playbook builder https://review.openstack.org/322243 | 07:20 |
| *** aviau has joined #openstack-infra | 07:21 | |
| *** yolanda has joined #openstack-infra | 07:22 | |
| *** ihrachys has joined #openstack-infra | 07:23 | |
| *** tphummel has quit IRC | 07:24 | |
| *** amitgandhinz has joined #openstack-infra | 07:24 | |
| openstackgerrit | Illia Khudoshyn proposed openstack-infra/project-config: Add Rally job against deployment with existing users https://review.openstack.org/328249 | 07:26 |
| *** flepied2 has quit IRC | 07:26 | |
| *** abregman_ has joined #openstack-infra | 07:27 | |
| *** Sukhdev has quit IRC | 07:27 | |
| *** amoralej|off is now known as amoralej | 07:28 | |
| *** amitgandhinz has quit IRC | 07:29 | |
| *** sorantis has joined #openstack-infra | 07:29 | |
| *** abregman has quit IRC | 07:29 | |
| *** sorantis has quit IRC | 07:29 | |
| *** jlanoux has joined #openstack-infra | 07:30 | |
| *** sorantis has joined #openstack-infra | 07:31 | |
| *** yolanda has quit IRC | 07:31 | |
| *** abregman_ is now known as abregman | 07:31 | |
| *** yolanda has joined #openstack-infra | 07:32 | |
| *** AJaeger has quit IRC | 07:34 | |
| *** e0ne has quit IRC | 07:37 | |
| *** jaosorior is now known as jaosorior_brb | 07:38 | |
| *** mikelk has joined #openstack-infra | 07:38 | |
| *** vincentll has joined #openstack-infra | 07:39 | |
| *** mikelk has quit IRC | 07:41 | |
| *** abregman has quit IRC | 07:43 | |
| *** dtantsur|afk is now known as dtantsur | 07:43 | |
| *** gibi has joined #openstack-infra | 07:44 | |
| *** nadya has quit IRC | 07:46 | |
| *** mikelk has joined #openstack-infra | 07:47 | |
| *** yolanda has quit IRC | 07:48 | |
| *** dguitarbite has joined #openstack-infra | 07:49 | |
| dguitarbite | hello, I would like to delete a branch in training-labs | 07:49 |
| dguitarbite | could someone help me with the same? | 07:49 |
| *** thorst has joined #openstack-infra | 07:53 | |
| *** e0ne has joined #openstack-infra | 07:55 | |
| *** dguitarbite has quit IRC | 07:56 | |
| *** oanson has joined #openstack-infra | 07:56 | |
| *** abregman has joined #openstack-infra | 07:57 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:58 | |
| *** kengo_sa_ has joined #openstack-infra | 07:59 | |
| *** e0ne has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** permalac has joined #openstack-infra | 08:00 | |
| *** markvoelker has joined #openstack-infra | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** thorst has quit IRC | 08:01 | |
| *** ifarkas has joined #openstack-infra | 08:03 | |
| *** kzaitsev_mb has quit IRC | 08:03 | |
| *** markvoelker has quit IRC | 08:05 | |
| *** electrofelix has joined #openstack-infra | 08:06 | |
| *** javeriak has joined #openstack-infra | 08:08 | |
| *** dguitarbite has joined #openstack-infra | 08:09 | |
| *** yanyanhu has joined #openstack-infra | 08:11 | |
| *** jtomasek_ has joined #openstack-infra | 08:13 | |
| yanyanhu | hi, guys, I found openstackgerrit has disappeared from #senlin channel for about two days, how can we get it back... | 08:15 |
| *** yamada-h has quit IRC | 08:16 | |
| *** dizquierdo has joined #openstack-infra | 08:17 | |
| *** shardy has joined #openstack-infra | 08:18 | |
| *** hasharAway has quit IRC | 08:19 | |
| *** jtomasek_ has quit IRC | 08:19 | |
| *** hashar has joined #openstack-infra | 08:19 | |
| *** javeriak has quit IRC | 08:21 | |
| *** shardy has quit IRC | 08:21 | |
| *** yamada-h has joined #openstack-infra | 08:22 | |
| *** salv-orlando has quit IRC | 08:23 | |
| *** salv-orlando has joined #openstack-infra | 08:24 | |
| *** amitgandhinz has joined #openstack-infra | 08:25 | |
| *** yamada-h has quit IRC | 08:26 | |
| *** yolanda has joined #openstack-infra | 08:29 | |
| *** amitgandhinz has quit IRC | 08:29 | |
| *** javeriak has joined #openstack-infra | 08:29 | |
| *** javeriak has quit IRC | 08:30 | |
| *** javeriak has joined #openstack-infra | 08:30 | |
| *** chem has joined #openstack-infra | 08:30 | |
| *** yamada-h has joined #openstack-infra | 08:34 | |
| dguitarbite | fungi: Can you please delete stable/mitaka branch for training-labs asap. I used the short # to create the branch and it created the release branch from the wrong commit. I want to re-create the same and this time I would be using the complete commit ID/#. It would be great if you could do the same asap. Training-labs team lead (Pranav Salunke) | 08:35 |
| *** dingyichen has quit IRC | 08:36 | |
| *** derekh has joined #openstack-infra | 08:38 | |
| *** jaosorior_brb has quit IRC | 08:39 | |
| *** jaosorior_brb has joined #openstack-infra | 08:39 | |
| *** dmk0202 has joined #openstack-infra | 08:44 | |
| *** kengo_sa_ has quit IRC | 08:52 | |
| *** PollyZ has joined #openstack-infra | 08:52 | |
| *** mrmartin has joined #openstack-infra | 08:53 | |
| *** mrmartin has quit IRC | 08:53 | |
| *** jaosorior_brb is now known as jaosorior | 08:54 | |
| *** mrmartin has joined #openstack-infra | 08:54 | |
| *** flepied has joined #openstack-infra | 08:57 | |
| *** PollyZ has quit IRC | 08:57 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:59 | |
| *** cdent has joined #openstack-infra | 08:59 | |
| *** gongysh has joined #openstack-infra | 09:00 | |
| *** thorst has joined #openstack-infra | 09:00 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: [Manila] Add new experimental job https://review.openstack.org/328364 | 09:01 |
| *** esikachev has joined #openstack-infra | 09:02 | |
| *** javeriak has quit IRC | 09:03 | |
| *** javeriak has joined #openstack-infra | 09:03 | |
| *** cdent has left #openstack-infra | 09:04 | |
| *** kzaitsev_mb has quit IRC | 09:04 | |
| *** acoles_ is now known as acoles | 09:04 | |
| *** thorst has quit IRC | 09:06 | |
| *** abregman is now known as abregman|afk | 09:16 | |
| *** nadya has joined #openstack-infra | 09:17 | |
| *** markusry has joined #openstack-infra | 09:17 | |
| *** e0ne has joined #openstack-infra | 09:22 | |
| *** dizquierdo has quit IRC | 09:22 | |
| *** yanyanhu has quit IRC | 09:24 | |
| *** mrmartin has quit IRC | 09:25 | |
| *** ilyashakhat has joined #openstack-infra | 09:25 | |
| *** mrmartin has joined #openstack-infra | 09:26 | |
| *** amitgandhinz has joined #openstack-infra | 09:26 | |
| *** pcaruana has joined #openstack-infra | 09:27 | |
| *** amitgandhinz has quit IRC | 09:31 | |
| *** amotoki has quit IRC | 09:31 | |
| *** rossella_s has joined #openstack-infra | 09:34 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: [Manila] Add new experimental job https://review.openstack.org/328364 | 09:34 |
| *** snarwade has quit IRC | 09:38 | |
| *** sree has joined #openstack-infra | 09:39 | |
| *** tosky has joined #openstack-infra | 09:42 | |
| *** lvdongbing has quit IRC | 09:51 | |
| *** abregman|afk is now known as abregman | 09:53 | |
| *** snarwade has joined #openstack-infra | 09:54 | |
| *** kengo_sa_ has joined #openstack-infra | 09:55 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Remove stray ";" from purge env command https://review.openstack.org/328933 | 09:59 |
| *** kzaitsev_mb has joined #openstack-infra | 10:00 | |
| *** HeOS has joined #openstack-infra | 10:01 | |
| *** ilyashakhat has quit IRC | 10:01 | |
| *** nadya has quit IRC | 10:02 | |
| *** kengo_sa_ has quit IRC | 10:03 | |
| *** zhurong has quit IRC | 10:03 | |
| *** thorst has joined #openstack-infra | 10:04 | |
| *** kzaitsev_mb has quit IRC | 10:05 | |
| *** thorst has quit IRC | 10:11 | |
| *** Daisy has joined #openstack-infra | 10:12 | |
| *** javeriak has quit IRC | 10:13 | |
| *** snarwade has quit IRC | 10:13 | |
| *** Daisy has quit IRC | 10:16 | |
| *** Daisy has joined #openstack-infra | 10:17 | |
| *** sree has quit IRC | 10:17 | |
| *** yamamoto_ has quit IRC | 10:19 | |
| *** amotoki has joined #openstack-infra | 10:19 | |
| *** Daisy has quit IRC | 10:21 | |
| *** Daisy has joined #openstack-infra | 10:24 | |
| *** yamada-h has quit IRC | 10:24 | |
| *** Daisy has quit IRC | 10:26 | |
| *** sorantis has quit IRC | 10:26 | |
| *** javeriak has joined #openstack-infra | 10:26 | |
| *** Daisy has joined #openstack-infra | 10:26 | |
| *** amitgandhinz has joined #openstack-infra | 10:27 | |
| *** armax has joined #openstack-infra | 10:28 | |
| *** snarwade has joined #openstack-infra | 10:29 | |
| *** Daisy has quit IRC | 10:30 | |
| *** amitgandhinz has quit IRC | 10:31 | |
| *** armax has quit IRC | 10:33 | |
| *** dizquierdo has joined #openstack-infra | 10:39 | |
| *** mikelk has quit IRC | 10:41 | |
| *** armax has joined #openstack-infra | 10:42 | |
| *** sorantis has joined #openstack-infra | 10:46 | |
| *** pilgrimstack has joined #openstack-infra | 10:50 | |
| *** armax has quit IRC | 10:50 | |
| *** armax has joined #openstack-infra | 10:50 | |
| *** sputnik13_ has joined #openstack-infra | 10:55 | |
| *** armax has quit IRC | 10:55 | |
| *** sputnik13_ has quit IRC | 11:00 | |
| *** kzaitsev_mb has joined #openstack-infra | 11:01 | |
| *** ociuhandu has joined #openstack-infra | 11:03 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/ansible-role-puppet: Syslog puppet output on Ansible control machine https://review.openstack.org/327789 | 11:04 |
| *** mikelk has joined #openstack-infra | 11:04 | |
| *** esikachev has quit IRC | 11:05 | |
| *** kzaitsev_mb has quit IRC | 11:06 | |
| *** salv-orlando has quit IRC | 11:07 | |
| rcarrillocruz | yolanda: i put the syslog output in tthe always block as we discussed the other day ^ | 11:07 |
| rcarrillocruz | nibalizer: the change for the syslogging ansible-puppet runs on puppetmaster ^ | 11:07 |
| *** armax has joined #openstack-infra | 11:08 | |
| *** thorst has joined #openstack-infra | 11:08 | |
| *** salv-orlando has joined #openstack-infra | 11:08 | |
| yolanda | rcarrillocruz, nice. I'll take a look | 11:09 |
| rcarrillocruz | tried on my vm passing puppetdb=lol puppet_report_as=loltoo and it enters the inner block on the always too | 11:10 |
| mordred | rcarrillocruz: nested blocks work? | 11:11 |
| mordred | neat | 11:11 |
| rcarrillocruz | yep, seems so | 11:11 |
| mordred | that's great | 11:11 |
| *** thorst has quit IRC | 11:15 | |
| openstackgerrit | Antonio Ojea proposed openstack-infra/project-config: Remove nova boot from rally-ci.sh script https://review.openstack.org/328953 | 11:16 |
| *** markusry has quit IRC | 11:18 | |
| *** markusry has joined #openstack-infra | 11:19 | |
| *** pilgrimstack has quit IRC | 11:20 | |
| *** NM has joined #openstack-infra | 11:24 | |
| *** yamada-h has joined #openstack-infra | 11:25 | |
| *** tlian has joined #openstack-infra | 11:25 | |
| *** rhallisey has joined #openstack-infra | 11:25 | |
| *** ccamacho is now known as ccamacho|lunch | 11:26 | |
| *** sorantis_ has joined #openstack-infra | 11:27 | |
| *** esikachev has joined #openstack-infra | 11:27 | |
| *** amitgandhinz has joined #openstack-infra | 11:27 | |
| *** salv-orl_ has joined #openstack-infra | 11:28 | |
| *** yamada-h has quit IRC | 11:29 | |
| *** sorantis has quit IRC | 11:30 | |
| *** sorantis_ is now known as sorantis | 11:30 | |
| *** nadya has joined #openstack-infra | 11:31 | |
| *** salv-orlando has quit IRC | 11:32 | |
| *** amitgandhinz has quit IRC | 11:32 | |
| *** thorst has joined #openstack-infra | 11:33 | |
| *** yamamoto has joined #openstack-infra | 11:33 | |
| *** esikachev has quit IRC | 11:33 | |
| dguitarbite | mordred: Can you delete a branch for my repo in openstack?? | 11:34 |
| *** zhurong has joined #openstack-infra | 11:36 | |
| *** psilvad has joined #openstack-infra | 11:38 | |
| *** gordc has joined #openstack-infra | 11:38 | |
| *** nadya has quit IRC | 11:40 | |
| *** yamamoto has quit IRC | 11:44 | |
| *** ldnunes has joined #openstack-infra | 11:46 | |
| *** elo has quit IRC | 11:46 | |
| *** pilgrimstack has joined #openstack-infra | 11:47 | |
| *** baoli has joined #openstack-infra | 11:47 | |
| *** baoli_ has joined #openstack-infra | 11:48 | |
| *** Jeffrey4l has quit IRC | 11:48 | |
| *** amotoki has quit IRC | 11:49 | |
| *** sdague has joined #openstack-infra | 11:49 | |
| *** Jeffrey4l has joined #openstack-infra | 11:49 | |
| *** mrmartin has quit IRC | 11:49 | |
| *** pilgrimstack has quit IRC | 11:50 | |
| *** baoli has quit IRC | 11:52 | |
| *** yamamoto has joined #openstack-infra | 11:53 | |
| *** weshay has joined #openstack-infra | 11:56 | |
| *** aysyd has joined #openstack-infra | 11:56 | |
| *** moravec1 has quit IRC | 11:59 | |
| mordred | dguitarbite: which branch and repo? | 11:59 |
| *** nadya has joined #openstack-infra | 12:01 | |
| *** raildo-afk is now known as raildo | 12:01 | |
| *** markvoelker has joined #openstack-infra | 12:01 | |
| *** moravec has joined #openstack-infra | 12:02 | |
| *** NM has quit IRC | 12:04 | |
| rcarrillocruz | eugh | 12:04 |
| *** moravec has quit IRC | 12:05 | |
| rcarrillocruz | the puppetlabs-puppetdb that we install does not accept ssl-cert and ssl-key params | 12:05 |
| rcarrillocruz | thus i assume that was set by hand on the host | 12:05 |
| rcarrillocruz | nibalizer: when you're around, i wanna chat with you about bumping the module to at least https://github.com/puppetlabs/puppetlabs-puppetdb/tree/4.1.0 | 12:05 |
| rcarrillocruz | crinkle: ^ | 12:05 |
| *** pradk has joined #openstack-infra | 12:06 | |
| *** markvoelker has quit IRC | 12:06 | |
| *** Daisy has joined #openstack-infra | 12:07 | |
| *** elo has joined #openstack-infra | 12:08 | |
| *** ilyashakhat has joined #openstack-infra | 12:08 | |
| *** Daisy has quit IRC | 12:09 | |
| *** Daisy has joined #openstack-infra | 12:09 | |
| *** pilgrimstack has joined #openstack-infra | 12:09 | |
| *** flaper87 has quit IRC | 12:09 | |
| rcarrillocruz | i'll simulate an upgrade of system-config from current to 4.1.0, that upgrade implies a bunch of other dependencies upgrade per release notes | 12:10 |
| *** thorst has quit IRC | 12:11 | |
| *** mtanino has joined #openstack-infra | 12:11 | |
| *** gongysh has quit IRC | 12:11 | |
| *** Daisy has quit IRC | 12:12 | |
| *** Daisy has joined #openstack-infra | 12:12 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/puppet-openstackci: Add puppetdb class https://review.openstack.org/328228 | 12:13 |
| *** yamamoto has quit IRC | 12:13 | |
| *** thorst has joined #openstack-infra | 12:15 | |
| *** flaper87 has joined #openstack-infra | 12:15 | |
| *** flaper87 has quit IRC | 12:15 | |
| *** flaper87 has joined #openstack-infra | 12:15 | |
| *** e0ne has quit IRC | 12:15 | |
| *** vincentll has quit IRC | 12:15 | |
| *** e0ne has joined #openstack-infra | 12:15 | |
| *** sdague has quit IRC | 12:16 | |
| *** sdague has joined #openstack-infra | 12:16 | |
| *** amotoki has joined #openstack-infra | 12:16 | |
| *** zhenguo_ has quit IRC | 12:21 | |
| *** kgiusti has joined #openstack-infra | 12:21 | |
| *** Daisy has quit IRC | 12:21 | |
| *** Daisy has joined #openstack-infra | 12:23 | |
| *** zhs has joined #openstack-infra | 12:24 | |
| zhs | hi, everyone. i am trying to login the wiki.openstack.org with my OpenID, but it returned "Nonce already used or out of range". Have anyone encountered this? | 12:24 |
| *** markvoelker has joined #openstack-infra | 12:25 | |
| vponomaryov | zhs: no, login works ok for me | 12:26 |
| *** srinivas has joined #openstack-infra | 12:26 | |
| *** ramishra has quit IRC | 12:27 | |
| zhs | this is the first time i try to log in wiki, do i have to do anything before loging in? | 12:27 |
| srinivas | Hai. For fc pass through i got the error as internal error: unable to execute QEMU command 'device_add': Device initialization failed. | 12:27 |
| *** amitgandhinz has joined #openstack-infra | 12:28 | |
| *** Daisy has quit IRC | 12:29 | |
| *** ramishra has joined #openstack-infra | 12:29 | |
| *** vincentll has joined #openstack-infra | 12:30 | |
| *** elo has quit IRC | 12:30 | |
| *** pgadiya has quit IRC | 12:30 | |
| *** Qiming_ has joined #openstack-infra | 12:31 | |
| *** scottda has joined #openstack-infra | 12:32 | |
| *** jcoufal has joined #openstack-infra | 12:32 | |
| *** tpsilva has joined #openstack-infra | 12:32 | |
| *** pilgrimstack has quit IRC | 12:32 | |
| *** amitgandhinz has quit IRC | 12:33 | |
| srinivas | <asselin_>Hai. For fc pass through i got the error as internal error: unable to execute QEMU command 'device_add': Device initialization failed. | 12:33 |
| *** pilgrimstack has joined #openstack-infra | 12:34 | |
| *** Qiming_ has quit IRC | 12:35 | |
| *** javeriak has quit IRC | 12:36 | |
| *** Daisy has joined #openstack-infra | 12:38 | |
| *** amotoki_ has joined #openstack-infra | 12:38 | |
| *** kashyap has left #openstack-infra | 12:39 | |
| *** amitgandhinz has joined #openstack-infra | 12:39 | |
| *** amitgandhinz has quit IRC | 12:39 | |
| *** amitgandhinz has joined #openstack-infra | 12:40 | |
| *** amotoki has quit IRC | 12:41 | |
| *** julim has joined #openstack-infra | 12:42 | |
| *** yamamoto has joined #openstack-infra | 12:46 | |
| *** yamamoto has quit IRC | 12:47 | |
| *** amotoki_ has quit IRC | 12:48 | |
| *** e0ne has quit IRC | 12:48 | |
| *** yolanda has quit IRC | 12:49 | |
| *** e0ne has joined #openstack-infra | 12:49 | |
| *** ccamacho|lunch is now known as ccamacho | 12:50 | |
| *** alekhya has joined #openstack-infra | 12:50 | |
| alekhya | hi there | 12:51 |
| *** Julien-zte has joined #openstack-infra | 12:51 | |
| *** ilyashakhat has quit IRC | 12:51 | |
| *** andreas_s has quit IRC | 12:52 | |
| *** Daisy has quit IRC | 12:52 | |
| alekhya | I am facing issue with fc-passthrough in ubutu 14.04.4 while attaching device for instance in devstack | 12:52 |
| alekhya | error: Failed to attach device from /tmp/tmp.n7wsJrrRLO_fcoe.xml error: internal error: unable to execute QEMU command 'device_add': Device initialization failed. | 12:53 |
| alekhya | Can anybody help me with error | 12:53 |
| *** alekhya has quit IRC | 12:54 | |
| *** yolanda has joined #openstack-infra | 12:54 | |
| *** PollyZ has joined #openstack-infra | 12:54 | |
| srinivas | Hi.I am facing issue with fc-passthrough in ubutu 14.04.4 while attaching device for instance in devstack. error: Failed to attach device from /tmp/tmp.n7wsJrrRLO_fcoe.xml error: internal error: unable to execute QEMU command 'device_add': Device initialization failed. | 12:54 |
| *** xyang1 has joined #openstack-infra | 12:55 | |
| *** vincentll has quit IRC | 12:55 | |
| *** pilgrimstack has quit IRC | 12:56 | |
| *** yamamoto has joined #openstack-infra | 12:56 | |
| *** yamamoto has quit IRC | 12:57 | |
| *** chlong has joined #openstack-infra | 12:58 | |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Support GitHub PR webhooks https://review.openstack.org/163117 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Make the string representation of change transparent https://review.openstack.org/238948 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Lower the log level in tests https://review.openstack.org/263364 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Merge pull requests from github reporter https://review.openstack.org/243250 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Encapsulate determining the event purpose https://review.openstack.org/247487 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Fix job hierarchy bug. https://review.openstack.org/192457 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: support github pull reqeust labels https://review.openstack.org/247421 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Better merge message for GitHub pull reqeusts https://review.openstack.org/280667 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Add 'push' and 'tag' github webhook events. https://review.openstack.org/191207 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Add 'pr-comment' github webhook event https://review.openstack.org/239203 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Support for dependent pipelines with github https://review.openstack.org/247500 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Configurable SSH access to GitHub https://review.openstack.org/239138 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: GitHub file matching support https://review.openstack.org/292376 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Allow github trigger to match on branches/refs https://review.openstack.org/258448 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Log GitHub API rate limit https://review.openstack.org/292377 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Set filter according to PR/Change in URL https://review.openstack.org/325300 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Allow using webapp from connections https://review.openstack.org/215642 | 12:59 |
| openstackgerrit | Jan Hruban proposed openstack-infra/zuul: Allow list values in template parameters. https://review.openstack.org/191208 | 12:59 |
| *** armax has quit IRC | 13:00 | |
| *** armax_ has joined #openstack-infra | 13:00 | |
| *** burgerk has joined #openstack-infra | 13:01 | |
| *** kengo_sa_ has joined #openstack-infra | 13:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 13:02 | |
| *** garyk has joined #openstack-infra | 13:04 | |
| *** pradk has quit IRC | 13:04 | |
| *** armax has joined #openstack-infra | 13:04 | |
| *** oanson has quit IRC | 13:05 | |
| *** armax_ has quit IRC | 13:05 | |
| *** marekd has quit IRC | 13:05 | |
| *** kengo_sa_ has quit IRC | 13:06 | |
| *** eharney has joined #openstack-infra | 13:07 | |
| *** srinivas has quit IRC | 13:09 | |
| *** ayoung has joined #openstack-infra | 13:09 | |
| *** zeih has quit IRC | 13:09 | |
| *** armax_ has joined #openstack-infra | 13:10 | |
| *** vincentll has joined #openstack-infra | 13:10 | |
| *** zeih has joined #openstack-infra | 13:10 | |
| *** hdaniel has joined #openstack-infra | 13:11 | |
| *** armax has quit IRC | 13:11 | |
| *** armax_ is now known as armax | 13:11 | |
| *** yamamoto has joined #openstack-infra | 13:13 | |
| *** moravec has joined #openstack-infra | 13:14 | |
| *** jcoufal has quit IRC | 13:14 | |
| *** armax has quit IRC | 13:14 | |
| *** zeih has quit IRC | 13:15 | |
| *** edmondsw has joined #openstack-infra | 13:15 | |
| *** gouthamr has joined #openstack-infra | 13:15 | |
| openstackgerrit | Yuriy Zveryanskyy proposed openstack/diskimage-builder: [EXPERIMENTAL] Ansible deploy ramdisk for Ironic https://review.openstack.org/239447 | 13:16 |
| *** zeih has joined #openstack-infra | 13:16 | |
| *** yamamoto has quit IRC | 13:16 | |
| *** mdrabe has joined #openstack-infra | 13:17 | |
| *** burgerk has quit IRC | 13:19 | |
| *** pblaho has joined #openstack-infra | 13:19 | |
| *** Hal has joined #openstack-infra | 13:19 | |
| *** Hal is now known as Guest47159 | 13:20 | |
| openstackgerrit | Yaroslav Lobankov proposed openstack-infra/project-config: [Rally] Enable Ceilometer for one of the 'rally verify' jobs https://review.openstack.org/327157 | 13:20 |
| *** denisra has quit IRC | 13:20 | |
| *** zeih has quit IRC | 13:21 | |
| *** sshnaidm has quit IRC | 13:21 | |
| *** sshnaidm has joined #openstack-infra | 13:21 | |
| *** sshnaidm has quit IRC | 13:21 | |
| *** sshnaidm has joined #openstack-infra | 13:22 | |
| *** matt-borland has joined #openstack-infra | 13:24 | |
| *** _ari_ has joined #openstack-infra | 13:25 | |
| *** moravec1 has joined #openstack-infra | 13:26 | |
| *** moravec has quit IRC | 13:27 | |
| pabelanger | morning | 13:28 |
| *** yamamoto has joined #openstack-infra | 13:28 | |
| *** burgerk has joined #openstack-infra | 13:30 | |
| *** AJaeger has joined #openstack-infra | 13:31 | |
| *** mriedem has joined #openstack-infra | 13:31 | |
| *** yamamoto has quit IRC | 13:31 | |
| *** yamamoto has joined #openstack-infra | 13:32 | |
| pabelanger | yolanda: fungi: I believe we can move forward with 325988 today. Starting of the process to remove tripleo-f22 | 13:33 |
| samueldmq | good morning | 13:34 |
| *** esberglu has joined #openstack-infra | 13:34 | |
| samueldmq | stackalytics.o.o seems to be unavailable, may anyone confirm ? | 13:35 |
| samueldmq | pabelanger: ^ maybe CPU is at 100% again ? | 13:36 |
| samueldmq | pabelanger: (I remember you had a patch to log such info for stackalytics.o.o, but I don't know where I may find that info) | 13:36 |
| pabelanger | samueldmq: I can check | 13:36 |
| pabelanger | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=13 should also give some stats | 13:36 |
| AJaeger | samueldmq: works for me - and stackalayitics shouldn't be criticial, it's just some statistics... | 13:37 |
| *** permalac has quit IRC | 13:37 | |
| *** zeih has joined #openstack-infra | 13:37 | |
| samueldmq | pabelanger: thanks | 13:37 |
| pabelanger | samueldmq: Yup apache2 process pinned CPU again. | 13:39 |
| *** jcoufal has joined #openstack-infra | 13:39 | |
| *** HeOS has quit IRC | 13:39 | |
| pabelanger | Would love some help to figure out why that is | 13:39 |
| fungi | loquacities: dguitarbite is requesting deletion of the stable/mitaka branch on openstack/training-labs | 13:39 |
| fungi | so he can recreate it | 13:39 |
| fungi | any objection? | 13:39 |
| pabelanger | uwsgi must be affecting it | 13:39 |
| samueldmq | AJaeger: sure, just making sure to report unavailability of a service that is expected to be up | 13:39 |
| samueldmq | AJaeger: up to you (infra folks) to prioritize fixing it within the amount of work you all have (and I know that a lot!) | 13:40 |
| openstackgerrit | Antoine Cabot proposed openstack-infra/project-config: Add release notes jobs for watcher https://review.openstack.org/329007 | 13:40 |
| AJaeger | samueldmq: sure - and you reported stackalytics.o.o and I checked .com ;( Sorry. | 13:40 |
| samueldmq | AJaeger: that's okay :) | 13:40 |
| fungi | zhs: if you're still around, i can enable new account creation temporarily. we have it disabled at the moment to thwart spammers | 13:40 |
| fungi | just let me know when you're ready to create your account so i can enable that briefly | 13:41 |
| pabelanger | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=177 is the right page | 13:41 |
| pabelanger | you can see our load issue from apache here: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2904&rra_id=all | 13:41 |
| *** rodrigods has quit IRC | 13:43 | |
| *** rodrigods has joined #openstack-infra | 13:43 | |
| samueldmq | pabelanger: yes I can confirm CPU usage is at 100%, system is always about 10% in worst case. user pinning it at 100% :( | 13:44 |
| *** andreas_s has joined #openstack-infra | 13:46 | |
| *** tosky has quit IRC | 13:48 | |
| *** yujunz has joined #openstack-infra | 13:48 | |
| *** yamamoto has quit IRC | 13:48 | |
| *** dtantsur is now known as dtantsur|brb | 13:48 | |
| pabelanger | cloudnull: checking to see if you are around this week. Maybe follow up about the SSH issues we are seeing in OSIC | 13:49 |
| *** yujunz has quit IRC | 13:49 | |
| *** hdaniel has quit IRC | 13:50 | |
| openstackgerrit | Antoine Cabot proposed openstack-infra/project-config: Add translation jobs to watcher https://review.openstack.org/329012 | 13:51 |
| *** tosky has joined #openstack-infra | 13:51 | |
| *** rbrndt has joined #openstack-infra | 13:52 | |
| openstackgerrit | Merged openstack-infra/project-config: Set min-ready 0 for tripleo-f22 https://review.openstack.org/325988 | 13:54 |
| pabelanger | fungi: I think we can also land 328395 too (removal of vexxhost) | 13:54 |
| *** yamamoto has joined #openstack-infra | 13:54 | |
| *** links has quit IRC | 13:54 | |
| *** nmagnezi has quit IRC | 13:56 | |
| *** kengo_sakai has joined #openstack-infra | 13:56 | |
| *** tonytan4ever has joined #openstack-infra | 13:57 | |
| rcarrillocruz | is vexxhost dead? | 13:58 |
| *** Liuqing has joined #openstack-infra | 13:58 | |
| *** eharney has quit IRC | 13:58 | |
| *** mriedem1 has joined #openstack-infra | 13:59 | |
| *** infra-red has quit IRC | 13:59 | |
| pabelanger | rcarrillocruz: no, they are still alive. I think we are moving to use them more for control plane servers then nodepool | 13:59 |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/project-config: Add gates for deploying kolla-kubernetes https://review.openstack.org/328814 | 13:59 |
| rcarrillocruz | oh nice | 13:59 |
| rcarrillocruz | the moar providers the better | 14:00 |
| *** infra-red has joined #openstack-infra | 14:00 | |
| fungi | yeah, their scheduler doesn't handle our nodepool profile very efficiently, but they're happy to add scheduler hints for long-lived servers instead | 14:00 |
| fungi | just hard to do that with nodepool because the instances are so short-lived | 14:01 |
| *** pblaho has quit IRC | 14:01 | |
| *** kengo_sakai has quit IRC | 14:01 | |
| SpamapS | fungi: is that documented, or just some feedback they're giving you? | 14:01 |
| SpamapS | because.. nodepool is a very typical workload on clouds | 14:01 |
| *** ikhudoshyn has joined #openstack-infra | 14:01 | |
| *** mriedem has quit IRC | 14:02 | |
| fungi | SpamapS: feedback from mnaser. he can almost certainly go into detail. i think he said they're hoping to take advantage of some additional scheduler options in liberty when they upgrade | 14:02 |
| *** woodster_ has joined #openstack-infra | 14:02 | |
| *** gongysh has joined #openstack-infra | 14:03 | |
| fungi | SpamapS: the end result was that when we spiked up and booted lots of nodes, nova wanted to stick them all on the same controllers rather than spreading them out, and so they fought for available cpu | 14:03 |
| clarkb | thr issue as I understand it is nova scheduler is memory based but workload is cpu based | 14:03 |
| *** pblaho has joined #openstack-infra | 14:03 | |
| openstackgerrit | Merged openstack-infra/project-config: Add releasenotes jobs for python-tackerclient https://review.openstack.org/328598 | 14:04 |
| *** e0ne has quit IRC | 14:04 | |
| *** r-daneel has joined #openstack-infra | 14:05 | |
| *** e0ne has joined #openstack-infra | 14:05 | |
| fungi | also, if anyone sees Rocky pop into irc, let her know i've temporarily blocked her wiki account until she gets an opportunity to reset her password for launchpad sso | 14:05 |
| openstackgerrit | Merged openstack-infra/system-config: define a release-job-failures mailing list https://review.openstack.org/328480 | 14:05 |
| *** oanson has joined #openstack-infra | 14:05 | |
| *** yujunz has joined #openstack-infra | 14:06 | |
| clarkb | fungi should we consider other uses of that account compromised too? gerrit etc | 14:06 |
| *** mriedem1 is now known as mriedem | 14:06 | |
| *** rvasilets___ has joined #openstack-infra | 14:06 | |
| *** gongysh has quit IRC | 14:06 | |
| pabelanger | fungi: I took a peak at the wiki over the weekend, nice work on purging the spam. | 14:07 |
| fungi | clarkb: probably, yes. however we have a good audit trail for activity and she can check to see if any changes were pushed/approved by that account | 14:07 |
| fungi | pabelanger: thanks. i meant to suggest that people should search the wiki for any spam keywords they can think of to spot any spammer accounts i might have missed | 14:08 |
| *** krtaylor has quit IRC | 14:09 | |
| fungi | pabelanger: i approved 327332 but it failed the trusty rspec job in the gate | 14:10 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute009.uswest metadata for better tracking https://review.openstack.org/327575 | 14:11 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute024.uswest metadata for better tracking https://review.openstack.org/327605 | 14:11 |
| *** PollyZ has quit IRC | 14:11 | |
| pabelanger | fungi: checking | 14:11 |
| pabelanger | fungi: ya, I think there is a race condition there. But would need crinkle to confirm. Going to recheck again | 14:12 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute035.uswest metadata for better tracking on hiera https://review.openstack.org/327537 | 14:12 |
| *** gongysh has joined #openstack-infra | 14:12 | |
| *** pradk has joined #openstack-infra | 14:12 | |
| *** abregman has quit IRC | 14:13 | |
| *** eharney has joined #openstack-infra | 14:13 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove vexxhost-ca-ymq-1 from nodepool https://review.openstack.org/328395 | 14:14 |
| *** zz_dimtruck is now known as dimtruck | 14:14 | |
| *** xarses has quit IRC | 14:14 | |
| *** dimtruck is now known as zz_dimtruck | 14:15 | |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run novaclient functional tests in Identity v3-only https://review.openstack.org/313588 | 14:15 |
| *** ajmiller has joined #openstack-infra | 14:15 | |
| *** PollyZ has joined #openstack-infra | 14:16 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute032.uswest metadata for better tracking https://review.openstack.org/327576 | 14:17 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute016.uswest metadata for better tracking https://review.openstack.org/327593 | 14:17 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute030.uswest metadata for better tracking https://review.openstack.org/327603 | 14:17 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute007.uswest metadata for better tracking https://review.openstack.org/327567 | 14:18 |
| *** kzaitsev_mb has quit IRC | 14:20 | |
| openstackgerrit | Lukas Bednar proposed openstack-infra/gerritlib: bulk_query: added extra parameters https://review.openstack.org/235891 | 14:21 |
| *** permalac has joined #openstack-infra | 14:21 | |
| *** Goneri has joined #openstack-infra | 14:21 | |
| *** Goneri has quit IRC | 14:21 | |
| *** xyang1 has quit IRC | 14:22 | |
| openstackgerrit | Merged openstack-infra/system-config: Add baremetal01 metadata for better tracking https://review.openstack.org/327597 | 14:22 |
| *** nadya has quit IRC | 14:23 | |
| *** yolanda has quit IRC | 14:23 | |
| *** pcrews_ has quit IRC | 14:23 | |
| *** yamamoto has quit IRC | 14:24 | |
| *** xyang1 has joined #openstack-infra | 14:25 | |
| raildo | AJaeger: ping, do you have a minute to take a look on this patch? :) https://review.openstack.org/#/c/313588/ | 14:25 |
| *** pcrews has joined #openstack-infra | 14:25 | |
| *** zhurong has quit IRC | 14:25 | |
| *** yamamoto has joined #openstack-infra | 14:25 | |
| *** yolanda has joined #openstack-infra | 14:27 | |
| *** kzaitsev_mb has joined #openstack-infra | 14:27 | |
| *** sorantis has quit IRC | 14:28 | |
| *** yamamoto has quit IRC | 14:28 | |
| *** waynr has quit IRC | 14:30 | |
| *** waynr has joined #openstack-infra | 14:30 | |
| *** zeih has quit IRC | 14:30 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove jenkins02 from nodepool https://review.openstack.org/328521 | 14:32 |
| *** yamamoto has joined #openstack-infra | 14:32 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:32 | |
| openstackgerrit | Antoni Segura Puimedon proposed openstack-infra/irc-meetings: kuryr: Make wiki and ics match https://review.openstack.org/329023 | 14:32 |
| *** sridhar_ram_ooo is now known as sridhar_ram | 14:34 | |
| *** claudiub has joined #openstack-infra | 14:35 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add new reference to puppet-certmonger to mirror list https://review.openstack.org/329025 | 14:35 |
| *** yamamoto has quit IRC | 14:35 | |
| *** inc0 has joined #openstack-infra | 14:37 | |
| *** cody-somerville has joined #openstack-infra | 14:37 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add new reference to puppet-certmonger to mirror list https://review.openstack.org/329025 | 14:37 |
| *** zz_dimtruck is now known as dimtruck | 14:37 | |
| corvus | clarkb: will you have time to review this? https://review.openstack.org/327422 | 14:39 |
| corvus | pabelanger: i have reviewed your zuul changes | 14:39 |
| *** corvus is now known as jeblair | 14:39 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add new reference to puppet-certmonger to mirror list https://review.openstack.org/329025 | 14:40 |
| *** mdrabe has quit IRC | 14:41 | |
| *** garyk has quit IRC | 14:41 | |
| clarkb | jeblair: I shpuld. eating breakfast now then catchup on email and reviews | 14:42 |
| *** dtardivel has joined #openstack-infra | 14:42 | |
| *** rook has joined #openstack-infra | 14:42 | |
| jeblair | nibalizer: http://logs.openstack.org/28/328628/1/gate/gate-openstackci-beaker-ubuntu-trusty/7f5354e/console.html failed due to pypi errors -- i have a vague memory of talk of modifying the python pip module honor pip.conf -- does that ring a bell? | 14:43 |
| rook | pabelanger hey i see browbeat got sucked in! However browbeat-core has no users and I am unable to add users -- is there a ACL I am missing? | 14:43 |
| jeblair | clarkb: okay, i would like to put that into production this morning | 14:43 |
| *** annegentle has joined #openstack-infra | 14:43 | |
| *** jamesmcarthur has joined #openstack-infra | 14:45 | |
| pabelanger | jeblair: thanks, I am revisiting 328649 | 14:45 |
| fungi | rook: just need one of us to add you as the initial group member and then you can add others as needed | 14:46 |
| fungi | rook: what was the change which added that repo? | 14:46 |
| fungi | for reference | 14:46 |
| *** Goneri has joined #openstack-infra | 14:47 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add python version to wheel builders https://review.openstack.org/329036 | 14:47 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Add POST_FAILURE status https://review.openstack.org/327978 | 14:47 |
| rook | fungi: https://review.openstack.org/#/c/327255/ | 14:48 |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run Neutron functional tests in Identity v3-only https://review.openstack.org/314604 | 14:48 |
| *** nadya has joined #openstack-infra | 14:49 | |
| *** yamamoto has joined #openstack-infra | 14:49 | |
| fungi | rook done | 14:49 |
| dguitarbite | fungi: Anything from lana? | 14:50 |
| dguitarbite | fungi: Not sure if we need Lana's approval for the same. | 14:50 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Use dlrn instead of delorean for delorean-build command https://review.openstack.org/325823 | 14:50 |
| *** jlanoux has quit IRC | 14:50 | |
| AJaeger | raildo: please let sdague or any other nova core review it | 14:51 |
| *** inc0 has quit IRC | 14:51 | |
| rook | fungi++ | 14:51 |
| *** amotoki has joined #openstack-infra | 14:51 | |
| raildo | AJaeger: ok, thanks | 14:51 |
| rcarrillocruz | lol | 14:51 |
| rcarrillocruz | puppetdb upgrade from packages 2.x to 3.x is horribly broken | 14:51 |
| raildo | AJaeger: btw, I made the same work on swift, and we already have a review from a swift core: https://review.openstack.org/#/c/313659/5 | 14:52 |
| rcarrillocruz | i'll try standing up a fresh one directly on 3.x | 14:52 |
| *** yamamoto has quit IRC | 14:53 | |
| *** mriedem1 has joined #openstack-infra | 14:53 | |
| *** nadya has quit IRC | 14:54 | |
| *** oanson has quit IRC | 14:54 | |
| *** kzaitsev_mb has quit IRC | 14:55 | |
| *** mriedem has quit IRC | 14:55 | |
| AJaeger | raildo: it adds to devstack, so QA PTL (oomichi) or sdague should review 313659 as well | 14:55 |
| *** yamamoto has joined #openstack-infra | 14:56 | |
| *** anteaya has joined #openstack-infra | 14:58 | |
| pabelanger | jeblair: okay, based on your comments for 328649. I think I see an issue with our timeout logic, let me see if I can explain | 14:58 |
| *** permalac has quit IRC | 14:58 | |
| *** sarob has joined #openstack-infra | 14:58 | |
| pabelanger | looking at the ansible-playbooks would actually be easier... | 14:58 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Implement UI for managing team membership https://review.openstack.org/320037 | 14:58 |
| *** mdrabe has joined #openstack-infra | 14:58 | |
| *** yujunz has quit IRC | 14:59 | |
| *** julim has quit IRC | 14:59 | |
| *** inc0 has joined #openstack-infra | 14:59 | |
| fungi | dguitarbite: just trying to make sure we've got ptl acknowledgement when we're deleting anything official, though if AJaeger as docs liaison wants to say it's good i'm happy to take that rather than wait for australia to wake up | 14:59 |
| pabelanger | jeblair: we are currenting setting our timeout based on the build-timeout wrapper, which defaults to 30 | 14:59 |
| jeblair | pabelanger: yeah, i'm eager to have the playbooks uploaded. | 14:59 |
| *** yujunz has joined #openstack-infra | 14:59 | |
| jeblair | pabelanger: yep (but is frequently overridden) | 15:00 |
| pabelanger | jeblair: then we pass said timeout into our _makeBuilderTask (which we actually don't use in the function) | 15:00 |
| *** tonytan4ever has quit IRC | 15:00 | |
| *** Liuqing has quit IRC | 15:00 | |
| *** julim has joined #openstack-infra | 15:00 | |
| jeblair | pabelanger: er, the default timeout is 2 hours | 15:00 |
| dguitarbite | fungi: It would be great, I have been already waiting for America to wake up!!! | 15:01 |
| dguitarbite | AJaeger: I want to delete one of the branches from training-labs and re-create the same. | 15:01 |
| jeblair | pabelanger: oh, the jjb default is 30 mins. | 15:01 |
| jeblair | pabelanger: gotcha. | 15:01 |
| pabelanger | currently _makeBuilderTask uses the values from -e timeout= (ansible-playbook) for running zuul_launcher | 15:01 |
| dguitarbite | AJaeger: stable/mitaka. Its pointing to the wrong commit (kilo) | 15:02 |
| openstackgerrit | Merged openstack-infra/storyboard: Make is_superuser a public field for users https://review.openstack.org/324629 | 15:02 |
| jeblair | pabelanger: agreed re makebuildertaks | 15:02 |
| pabelanger | but, it is possible for zuul_launcher to be run multiple times on a host, with that in mind, shouldn't we be decrementing our ansible-playbook timeout based on the previous tasks length of time? Other wise, each zuul_launcher run still simply default to 2hour, which always grows over our watchdog timeout? | 15:03 |
| *** hongbin has joined #openstack-infra | 15:03 | |
| *** gongysh has quit IRC | 15:03 | |
| pabelanger | bascially, I think we've confused our timeout in ansiblelauncher.py | 15:03 |
| jeblair | pabelanger: that is exactly what the zuul timeout callback plugin does | 15:03 |
| *** kushal has joined #openstack-infra | 15:04 | |
| jeblair | pabelanger: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/ansible/plugins/callback_plugins/timeout.py#n53 | 15:04 |
| *** Daisy has joined #openstack-infra | 15:04 | |
| pabelanger | jeblair: Oooh, this is a new file for me. I have missed it | 15:04 |
| *** yamamoto has quit IRC | 15:05 | |
| anteaya | pabelanger: can I ask you why you decided to merge this without talking to me about it? https://review.openstack.org/#/c/327065/ | 15:06 |
| anteaya | third party documentation is changed rarely | 15:06 |
| jeblair | mtreinish pointed out a missing subunit2sql result; i'm going to check the logstash system to see if the zmq connections are okay | 15:07 |
| *** yamamoto has joined #openstack-infra | 15:07 | |
| *** kushal has quit IRC | 15:07 | |
| *** andreas_s has quit IRC | 15:07 | |
| *** Daisy has quit IRC | 15:09 | |
| anteaya | pabelanger: this patch actually goes against infra policy on recheck syntax | 15:09 |
| anteaya | as far as I understood infra policy | 15:09 |
| pabelanger | anteaya: based on the commit message, there was a ML discussion about it. I was following that discussion as grounds for consensus | 15:09 |
| pabelanger | anteaya: sure, if there is an issue we can revert. | 15:10 |
| *** yamamoto has quit IRC | 15:10 | |
| jeblair | starting nodepool | 15:10 |
| anteaya | pabelanger: would you like to offer the revert or shall I? | 15:10 |
| anteaya | pabelanger: and I do feel I should have been pinged about the patch prior to it being merged | 15:10 |
| anteaya | since it changes third party documenation | 15:11 |
| *** david-lyle has joined #openstack-infra | 15:11 | |
| *** Alekhya has joined #openstack-infra | 15:12 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Revert "Update third party recheck syntax recommendations" https://review.openstack.org/329049 | 15:12 |
| *** EricGonczer_ has joined #openstack-infra | 15:12 | |
| anteaya | pabelanger: thank you | 15:12 |
| pabelanger | anteaya: welcome | 15:12 |
| *** yamamoto has joined #openstack-infra | 15:12 | |
| *** permalac has joined #openstack-infra | 15:13 | |
| *** ajmiller_ has joined #openstack-infra | 15:13 | |
| *** Alekhya has quit IRC | 15:13 | |
| pabelanger | jeblair: okay, now I understand why we pass -e timeout to ansible-playbook. Let me wrap my brain around this and how it relates to 'retries' | 15:14 |
| *** piet has joined #openstack-infra | 15:14 | |
| *** ajmiller has quit IRC | 15:16 | |
| Shrews | pabelanger: i left you a comment on 328662. i don't think that will work | 15:17 |
| *** yamamoto has quit IRC | 15:17 | |
| *** krtaylor has joined #openstack-infra | 15:17 | |
| *** vhosakot has joined #openstack-infra | 15:17 | |
| *** Swami has joined #openstack-infra | 15:17 | |
| *** Swami_ has joined #openstack-infra | 15:18 | |
| *** Swami_ has quit IRC | 15:18 | |
| *** Swami has quit IRC | 15:18 | |
| *** Swami has joined #openstack-infra | 15:18 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Add lftp to zuul-launcher https://review.openstack.org/328628 | 15:20 |
| *** jlanoux has joined #openstack-infra | 15:20 | |
| clarkb | jeblair: reviewing that change the first thing that comes to mind is maybe gear should support this directly? | 15:21 |
| *** dtantsur|brb is now known as dtantsur | 15:21 | |
| jeblair | mtreinish: it looks like the log processor is not receiving events from zl01 (but is from two other zuul launchers). I suspect this is a typical zmq connection fubar situation and will restart the daemon. | 15:21 |
| clarkb | specifically sendMassDo | 15:21 |
| jeblair | clarkb: perhaps, but that'd be a protocol change | 15:21 |
| mtreinish | jeblair: hmm, ok. Thanks | 15:21 |
| *** xarses has joined #openstack-infra | 15:22 | |
| clarkb | jeblair: oh right gearman doesn't support this does it? | 15:22 |
| *** psachin has quit IRC | 15:22 | |
| jeblair | clarkb: no, it's a private protocol enhancement because our service is on fire | 15:22 |
| Shrews | ;b 24 | 15:22 |
| Shrews | doh | 15:22 |
| *** mriedem1 is now known as mriedem | 15:22 | |
| fungi | what's the current state of gear deviations from the gearman protocol spec for features on which zuul depends? | 15:22 |
| *** lennyb has quit IRC | 15:22 | |
| jeblair | fungi: there are none | 15:23 |
| jeblair | fungi: the only reason we aren't using the C gearman server (which might also be a solution to this problem) is... | 15:23 |
| *** arxcruz has quit IRC | 15:24 | |
| jeblair | fungi: that we are unsure about whether it has a FIFO mode. apparently there's an option that might do that, but we haven't tested that it works | 15:24 |
| pabelanger | Shrews: Thanks, I was worried about that | 15:24 |
| pabelanger | Shrews: adding raw now | 15:24 |
| jeblair | if someone wants to test that, we might be able to switch | 15:24 |
| Shrews | i find it humorous that the gear docs say to not use the server in production :) | 15:24 |
| clarkb | jeblair: I left a comment that points out a small bug otherwise seems fine to me | 15:24 |
| *** lennyb has joined #openstack-infra | 15:25 | |
| jeblair | Shrews: yep! there are much better servers! | 15:25 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Add a modal to create Teams https://review.openstack.org/320517 | 15:25 |
| openstackgerrit | Curtis Collicutt proposed openstack-infra/irc-meetings: Change meeting time to 1500 UTC https://review.openstack.org/329057 | 15:26 |
| *** kzaitsev_mb has joined #openstack-infra | 15:26 | |
| *** weshay is now known as weshay_lunch_brb | 15:26 | |
| *** zhs has quit IRC | 15:27 | |
| anteaya | fungi: when you have an opportunity, third party recheck syntax is a topic again and many do not understand the original decision behind "recheck" as a recommendation | 15:27 |
| *** zhs has joined #openstack-infra | 15:27 | |
| anteaya | fungi: if you would be so kind as to view https://review.openstack.org/#/c/329049/1 and share your thoughts with me, I would be grateful | 15:27 |
| *** mtanino has quit IRC | 15:28 | |
| *** hashar is now known as hasharAway | 15:28 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Bootstrap worker logs with zuul information https://review.openstack.org/328662 | 15:28 |
| *** mhickey has quit IRC | 15:28 | |
| pabelanger | jeblair: Shrews: ^ | 15:28 |
| *** mrmartin has joined #openstack-infra | 15:28 | |
| mmedvede | anteaya: I reread the meeting discussion, and I think I made it confusing. When I said "new recheck syntax for third-party CI", I only meant the part that is used to only trigger particular third-party CI system. global "recheck" remains | 15:28 |
| clarkb | mmedvede: yes we had previously said don't do that and just honor recheck | 15:29 |
| anteaya | mmedvede: thanks, I also have an objection to changing third party requirements | 15:29 |
| anteaya | as I state in the comment, requirements carry a meaning | 15:29 |
| clarkb | I think the reasoning was to avoid confusion over additional syntax and to avoid "locking in" results | 15:29 |
| clarkb | eg recheck each bit until each one passes then move on to the next | 15:30 |
| anteaya | clarkb: thanks, I remember parts of the decision but you state it so well | 15:30 |
| crinkle | rcarrillocruz: pabelanger hi i'm around | 15:31 |
| pabelanger | crinkle: great | 15:31 |
| *** dguitarbite has quit IRC | 15:31 | |
| rcarrillocruz | crinkle: wb, hope you had a nice trip | 15:31 |
| *** coreycb has quit IRC | 15:31 | |
| crinkle | rcarrillocruz: it was fantastic | 15:32 |
| *** coreycb has joined #openstack-infra | 15:32 | |
| pabelanger | crinkle: rcarrillocruz: I think we have baremetal00.vanilla add back into ansible, just confirming now | 15:32 |
| rcarrillocruz | good | 15:33 |
| mmedvede | clarkb: that makes sense, but it leaves out the use case when third-party CI was legitimately broken. So it forces a lot of resource waste in that case. So e.g. when my non-voting CI reports failure I should recheck and force extra 4 hours of wait? | 15:33 |
| rcarrillocruz | i believe we should consolidate the mechanism for doing deploys | 15:34 |
| *** tonytan4ever has joined #openstack-infra | 15:34 | |
| rcarrillocruz | crinkle: you started a process that implied a second dhcp, which predated the changes we put into glean about being able to set up provisioning ip address | 15:34 |
| clarkb | mmedvede: the issue is that distinguishing between broken CI and failing test is really hard and most people don't do it, they just recheck until the lights go green | 15:34 |
| clarkb | I don't think there is a good answer here but that was the choice made back when | 15:35 |
| crinkle | rcarrillocruz: right | 15:35 |
| *** javeriak has joined #openstack-infra | 15:35 | |
| rcarrillocruz | well, the provisioning ip thing was put into bifrost rather | 15:35 |
| *** harlowja_at_home has joined #openstack-infra | 15:35 | |
| cinerama | SpamapS: ohai. about that glean release... | 15:36 |
| *** mikelk has quit IRC | 15:37 | |
| *** ddieterly has joined #openstack-infra | 15:37 | |
| pabelanger | crinkle: do you have a moment to confirm why we are failing on: http://logs.openstack.org/32/327332/3/gate/gate-puppet-infracloud-puppet-beaker-rspec-ubuntu-trusty/32b75e1/ I think we have a race condition? | 15:38 |
| SpamapS | cinerama: Yeah I'm the wrong person to ask.. not sure who is able to make those tags. | 15:38 |
| rcarrillocruz | crinkle: i also wanted to ask you about bumping puppetlabs-puppetdb. I'm working on bringing up a new puppetdb on 3.x and i realized that the current version of the puppet module lacks the ability to configure ssl certs, which were put on version 4.1 | 15:38 |
| crinkle | pabelanger: it failed again after a recheck so i don't think it's a race condition | 15:38 |
| rcarrillocruz | https://github.com/puppetlabs/puppetlabs-puppetdb/tree/4.1.0 | 15:38 |
| cinerama | anyone else here know? there's a change in glean we'd really like to have | 15:39 |
| rcarrillocruz | that makes me think the values were put by hand on jetty.ini | 15:39 |
| crinkle | pabelanger: spinning up my machine to check it out | 15:39 |
| anteaya | mmedvede: we can't stop you from implimenting whatever syntax you want, we can shut you down for the minimal things stated in requreiments, and the only requirement infra has had and continues to have regarding recheck syntax is "recheck" | 15:39 |
| *** tesseract has quit IRC | 15:39 | |
| pabelanger | crinkle: http://logs.openstack.org/32/327332/3/gate/gate-puppet-infracloud-puppet-beaker-rspec-ubuntu-trusty/32b75e1/console.html#_2016-06-13_13_47_18_891 is the failure, I think because the iamges status is currently saving? | 15:39 |
| jeblair | clarkb, mtreinish: the firewall on zl01 had an old ip address of logstash.o.o in it. i suspect we forgot to restart iptables on that host after moving logstash. i've restarted it, and it's seeing zmq events from zl01 now. | 15:39 |
| anteaya | requirements are what infra requires a third party ci to have ton keep its account, that is all | 15:40 |
| crinkle | pabelanger: yeah so best case scenario we just need to increase the sleep time to wait for the images to save, but i think it's more likely that something is wrong and the image is failing to upload | 15:40 |
| Shrews | pabelanger: that works. +1'd | 15:41 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Private embargoed vulnerability announcement list https://review.openstack.org/327699 | 15:41 |
| mtreinish | jeblair: ok, thanks | 15:41 |
| crinkle | rcarrillocruz: that's possible, or it might have been managed with separate ini_file resources, or the puppet module is managing different versions of puppetdb itself and ssl was configured differently, i don't know the puppetdb module that well | 15:42 |
| jeblair | mtreinish: thanks for the report; that narrowed it down quite quickly :) | 15:42 |
| fungi | anteaya: i'll take a look but yeah in addition to what clarkb said, the prevailing sentiment was that in the future we'd like to be able to get away from triggering jobs with gerrit comments as it's a terrible interface for that sort of thing, and if we design some formal syntax we're endorsing that behavior even more and baking it into policies | 15:42 |
| pabelanger | crinkle: ya, looks like we only wait 10seconds for the image upload. Likely hitting cases where our node is running slower then expected. | 15:43 |
| *** pblaho has quit IRC | 15:43 | |
| *** javeriak_ has joined #openstack-infra | 15:44 | |
| anteaya | fungi: thank you for being willing to take a look and to share your position and recollection | 15:44 |
| crinkle | pabelanger: hmm looks like both of the last fails were on osic, maybe they are slower? | 15:44 |
| *** javeriak has quit IRC | 15:44 | |
| anteaya | I was caught by surprise by this so had a bit of a gap whilst I dusted off my memories of the discussion | 15:44 |
| anteaya | I only had enough to remember there was one | 15:44 |
| anteaya | and this patch wasn't it | 15:44 |
| crinkle | pabelanger: oh nm the last was on rax | 15:44 |
| *** ociuhandu has quit IRC | 15:45 | |
| clarkb | crinkle: pabelanger: osic nodes are our quickest | 15:45 |
| openstackgerrit | Merged openstack-infra/project-config: Disable tripleo-f22 in nodepool https://review.openstack.org/325989 | 15:45 |
| *** weshay_lunch_brb is now known as weshay | 15:45 | |
| *** xiangxinyong has quit IRC | 15:46 | |
| *** xiangxinyong has joined #openstack-infra | 15:46 | |
| *** bpokorny has joined #openstack-infra | 15:47 | |
| pabelanger | crinkle: not sure. I think the issue is we can't assume 10sec is long enough for all nodes. We could poll openstack image list until we get active or ! saving image, then return success based on that. Means we'd wait at max for image upload process timeout ( I don't know what that is) | 15:48 |
| clarkb | pabelanger: that is what devstack does iirc | 15:48 |
| pabelanger | clarkb: crinkle: actually, we should do what we do for nodepool devstack test, I think we check the same process | 15:48 |
| clarkb | with some semi large timeout | 15:48 |
| pabelanger | clarkb: ya | 15:48 |
| dhellmann | has anyone asked for the os-brick 1.4.0 sdist upload job to be retriggered, yet? | 15:48 |
| *** vhosakot has quit IRC | 15:49 | |
| *** vhosakot has joined #openstack-infra | 15:49 | |
| fungi | dhellmann: ahh, not to my knowledge. what was the cause of failure? | 15:49 |
| dhellmann | fungi, it looks like the usual 503 error but this time it was for real? http://logs.openstack.org/7f/7f62af1be2ed6266e2bb6ba413f81d4ae0c7f3b8/release/os-brick-pypi-both-upload/e94ebcd/console.html.gz | 15:49 |
| *** nadya has joined #openstack-infra | 15:50 | |
| dhellmann | oh, no. maybe this was from before the recent fixes? | 15:50 |
| clarkb | I guess that means we should add the check back in and write it properly this time? | 15:50 |
| dhellmann | the py2-none-any wheel is there but that upload reported the 503 | 15:50 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-infracloud: Increase image upload timeout in beaker tests https://review.openstack.org/329067 | 15:51 |
| dhellmann | clarkb: this is from 9 June, but I don't know when the upload job changes were happening | 15:51 |
| *** dmk0202 has quit IRC | 15:51 | |
| crinkle | pabelanger: ^ we can see if the timeout is the issue | 15:51 |
| *** infra-red has quit IRC | 15:51 | |
| fungi | dhellmann: clarkb: according to dstufft pypi was under a denial of service attack and so the upload via warehouse failed, but that this shouldn't be a frequent occurrence like it was with uploads directly to pypi | 15:51 |
| crinkle | pabelanger: if it is we can change it to poll | 15:52 |
| *** piet has quit IRC | 15:52 | |
| clarkb | fungi: searching for some of the keywords that had spam on the wiki before I don't see anything glaring anymore. Will let you knwo if I find anything that needs cleaning | 15:52 |
| dhellmann | fungi : oh, right, I forgot about the DOS from last week | 15:52 |
| *** milan has joined #openstack-infra | 15:52 | |
| fungi | clarkb: thanks! | 15:52 |
| anteaya | clarkb: fungi yay! | 15:52 |
| *** piet has joined #openstack-infra | 15:52 | |
| *** Apoorva has joined #openstack-infra | 15:52 | |
| milan | hi Infra! I've got a question w/r dnsmasq vs resolv.conf in ubuntu: is there some resolving workaround applied in the infra gates? (http://www.stgraber.org/2012/02/24/dns-in-ubuntu-12-04/) | 15:53 |
| dhellmann | fungi : is it safe/possible to retrigger the upload, or do we need a new tag? I don't know if that job relies on a local file, or uses the one from tarballs.o.o | 15:53 |
| * milan needs to install dnsmasq because of ironic-inspector and out of sudden, no name resolution for him | 15:53 | |
| *** Sukhdev has joined #openstack-infra | 15:54 | |
| *** Swami has quit IRC | 15:55 | |
| fungi | dhellmann: since the wheel uploaded and the sdist did not, i'll need to manually upload the sdist and trigger the announcement job | 15:55 |
| *** nadya has quit IRC | 15:55 | |
| clarkb | milan: no, the only thing we do is run unbound pointed at google to avoid using potentially broken cloud dns | 15:55 |
| fungi | dhellmann: if the wheel hadn't uploaded successfully, i could just reenqueue the tag in zuul | 15:55 |
| *** mrmartin has quit IRC | 15:55 | |
| pabelanger | crinkle: sure | 15:55 |
| dhellmann | fungi : makes sense | 15:55 |
| *** gongysh has joined #openstack-infra | 15:55 | |
| *** Swami has joined #openstack-infra | 15:56 | |
| *** derekh has quit IRC | 15:56 | |
| anteaya | I'm about to request that Zara and SotK have Admin permissions granted to them on the storyboard-dev instance | 15:56 |
| anteaya | if anyone has any objections to having these permissions granted to Zara and SotK please do speak up | 15:57 |
| *** pblaho has joined #openstack-infra | 15:57 | |
| cinerama | mordred: hi, can you help us get a glean release? | 15:57 |
| *** mrmartin has joined #openstack-infra | 15:57 | |
| fungi | dhellmann: sdist has been uploaded, working on retriggering the announcement job now | 15:57 |
| dhellmann | fungi : cool, thanks | 15:57 |
| anteaya | fungi: how can Zara and SotK be granted Admin permissions on storyboard-dev if that is not objectionable to grant | 15:57 |
| dhellmann | fungi : it looks like the system-config patch to set up that release job failure mailing list has merged, let me know how you'd like to transmit a password to me so we can move ahead with https://review.openstack.org/#/c/328486/ | 15:58 |
| *** rcernin has quit IRC | 15:59 | |
| anteaya | on a different topic, in the last scientific working group meeting someone was asked to join in because they have been working on the blazar project: http://eavesdrop.openstack.org/meetings/scientific_wg/2016/scientific_wg.2016-06-08-06.59.log.html | 16:01 |
| fungi | anteaya: no objection whatsoever, though i don't recall off the top of my head how it's set. it looks like there's a ::storyboard::load_superusers class into which we need to pass a yaml file like http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/storyboard/superusers.yaml | 16:01 |
| *** jamesmcarthur has quit IRC | 16:01 | |
| anteaya | I've contacted them via email and hopefully someone will pop into channel at some point with questions about how to upstream their work | 16:01 |
| anteaya | so fyi | 16:01 |
| clarkb | fungi: https://wiki.openstack.org/wiki/Ccc | 16:02 |
| milan | clarkb, thanks, sounds like I need a workaround some place between stack.sh and inspector's plugin.sh | 16:02 |
| anteaya | fungi: awesome thank you | 16:02 |
| fungi | dhellmann: mailman should have sent an initial password e-mail to the address configured as the list admin | 16:02 |
| fungi | clarkb: ooh, great find! | 16:02 |
| anteaya | Zara: SotK so it looks like if one of you wants to patch that storyboard/superusers.yaml file and add yourselves that would be the way to begin getting the two of you Admin permissions | 16:02 |
| clarkb | fungi: pabelanger says it was created by pabelanger ? | 16:03 |
| dhellmann | fungi : ah, it did, and it was filed into a mailbox I hadn't read yet | 16:03 |
| fungi | clarkb: yes, i just saw that | 16:03 |
| pabelanger | clarkb: fungi: I think I did create that | 16:03 |
| milan | clarkb, actually is the unbound running on the instance? | 16:03 |
| milan | when the stack.sh is executed? | 16:03 |
| pabelanger | clarkb: fungi: I was testing some spamfilter addons | 16:03 |
| Zara | okay, and I'll let SotK know when he's better (he's ill atm) | 16:04 |
| pabelanger | fungi: clarkb: let me delete it | 16:04 |
| clarkb | pabelanger: ok thanks | 16:04 |
| fungi | pabelanger: oh, awesome. please do! | 16:04 |
| *** ayoung has quit IRC | 16:04 | |
| jeblair | pabelanger was the spammer all along! ;) | 16:04 |
| fungi | hah | 16:04 |
| pabelanger | I have been exposed! | 16:04 |
| *** sarob has quit IRC | 16:05 | |
| *** jamesmcarthur has joined #openstack-infra | 16:07 | |
| clarkb | othre than that page I am not really finding anything | 16:07 |
| *** vincentll has quit IRC | 16:07 | |
| *** matrohon has quit IRC | 16:08 | |
| *** yujunz has quit IRC | 16:08 | |
| fungi | clarkb: thanks for hunting (and for finding one i somehow missed) | 16:08 |
| fungi | i probably didn't search for 1-800 or quickbooks | 16:09 |
| *** gongysh has quit IRC | 16:09 | |
| *** dizquierdo has quit IRC | 16:09 | |
| *** srinivas has joined #openstack-infra | 16:10 | |
| anteaya | Zara: oh no :( | 16:10 |
| openstackgerrit | Paul Belanger proposed openstack-infra/infra-specs: Mark 'Use Diskimage Builder in Nodepool' finished https://review.openstack.org/329080 | 16:10 |
| pabelanger | \o/ ^ | 16:10 |
| pabelanger | long time coming | 16:10 |
| *** gongysh has joined #openstack-infra | 16:10 | |
| *** gyee has joined #openstack-infra | 16:10 | |
| clarkb | yay | 16:11 |
| fungi | the biggest bulk of pages i needed to pour through was a search for "phone" | 16:11 |
| anteaya | wow | 16:11 |
| srinivas | Hai. i am using fc pass through in 14.04.4 ubuntu. it is giving me an error as qemuMonitorJSONCheckError:354 : internal error: unable to execute QEMU command 'device_add': Device initialization failed | 16:11 |
| openstackgerrit | Merged openstack-infra/system-config: Decomission jenkins02 https://review.openstack.org/328523 | 16:11 |
| fungi | we have lots of legitimate hits, but a solid 95+% were spam pages | 16:11 |
| *** electrofelix has quit IRC | 16:11 | |
| *** hongbin has quit IRC | 16:12 | |
| clarkb | ya I have searched for amazon, kindle, quickbooks 1-800 and friends, customer, support, etc | 16:13 |
| *** mtanino has joined #openstack-infra | 16:13 | |
| *** mrmartin has quit IRC | 16:16 | |
| rcarrillocruz | so yeah | 16:16 |
| rcarrillocruz | even upgrading current puppetdb to lowest 3 version, 3.0.0, it's broken | 16:16 |
| *** rwsu has joined #openstack-infra | 16:16 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for os-brick privsep sudo bug 1592043 https://review.openstack.org/329084 | 16:16 |
| openstack | bug 1592043 in os-brick "os-brick 1.4.0 increases volume setup failure rates" [Undecided,Confirmed] https://launchpad.net/bugs/1592043 | 16:16 |
| rcarrillocruz | init script issues, install dir not the one expected etc | 16:16 |
| *** rvasilets___ has quit IRC | 16:16 | |
| *** gongysh has quit IRC | 16:16 | |
| fungi | srinivas: sounds like you're looking for general openstack help? this is where we discuss services we run for the openstack community, so it's probably not an ideal venue for troubleshooting an openstack deployment | 16:17 |
| rcarrillocruz | yay testing upgrades paths | 16:17 |
| *** Swami has quit IRC | 16:17 | |
| *** srinivas has left #openstack-infra | 16:18 | |
| *** robot has quit IRC | 16:19 | |
| openstackgerrit | Merged openstack-infra/project-config: Delete tripleo-f22 https://review.openstack.org/325990 | 16:19 |
| *** Julien-zte has quit IRC | 16:19 | |
| *** Wei_Liu has quit IRC | 16:19 | |
| *** ayoung has joined #openstack-infra | 16:19 | |
| openstackgerrit | amrith proposed openstack-infra/project-config: Add trove period jobs with oslo master https://review.openstack.org/329086 | 16:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add python version to wheel builders https://review.openstack.org/329036 | 16:22 |
| *** matrohon has joined #openstack-infra | 16:22 | |
| *** denisra has joined #openstack-infra | 16:24 | |
| nibalizer | rcarrillocruz: heya | 16:24 |
| nibalizer | you're looking at upgrading the puppetdb module? I tend to agree with that strategy | 16:24 |
| jeblair | #status log jenkins02.openstack.org has been deleted | 16:24 |
| openstackstatus | jeblair: finished logging | 16:24 |
| jeblair | infra-root: ^ | 16:24 |
| pabelanger | jeblair: woot | 16:24 |
| nibalizer | the same people who write puppetdb also write the module, and they tend to just major-release-bump the module whenever they change how puppetdb works | 16:25 |
| nibalizer | this is annoying for folks that want to use puppet to manage two instances for a little ibt | 16:25 |
| *** salv-orl_ has quit IRC | 16:25 | |
| nibalizer | jeblair: re: pip/pypi and puppet, crinkle fixed that | 16:25 |
| *** salv-orlando has joined #openstack-infra | 16:25 | |
| fungi | jeblair: i'll drink to that! | 16:26 |
| clarkb | jeblair: have you seen my comment on 327422? | 16:26 |
| nibalizer | jeblair: i think we just need to pipe 'provider => openstack_pip' through on the resources that don't have it yet | 16:27 |
| *** ddieterly is now known as ddieterly[away] | 16:27 | |
| *** PollyZ_ has joined #openstack-infra | 16:28 | |
| openstackgerrit | Zara proposed openstack-infra/system-config: Add Zara to StoryBoard Admin https://review.openstack.org/329092 | 16:28 |
| crinkle | nibalizer: is this for openstackci or somewhere else? i have https://review.openstack.org/#/c/305499/ waiting | 16:29 |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-openstackci: Use the openstack pip provider https://review.openstack.org/329093 | 16:29 |
| *** e0ne has quit IRC | 16:29 | |
| nibalizer | haha! | 16:29 |
| jeblair | crinkle: awesome! thanks! | 16:29 |
| nibalizer | crinkle: yep that is exactly it | 16:29 |
| crinkle | yay | 16:30 |
| *** mdrabe has quit IRC | 16:30 | |
| *** mdrabe has joined #openstack-infra | 16:30 | |
| *** PollyZ has quit IRC | 16:30 | |
| *** rcernin has joined #openstack-infra | 16:31 | |
| *** ds has joined #openstack-infra | 16:31 | |
| ds | Hai. i am using fc passthrough in ubuntu14.04.4. it is giving me an error as qemuMonitorJSONCheckError:354 : internal error: unable to execute QEMU command 'device_add': Device initialization failed. can anyone help me | 16:31 |
| *** amitgandhinz has quit IRC | 16:31 | |
| *** ddieterly[away] is now known as ddieterly | 16:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add python3 wheel-builds https://review.openstack.org/329094 | 16:32 |
| pabelanger | fungi: ^ should address our python3 wheel-builds | 16:32 |
| anteaya | <-- offline for a bit | 16:32 |
| pleia2 | woo, jenkins02 | 16:32 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: add private gearman function https://review.openstack.org/327422 | 16:33 |
| pabelanger | jenkins03 here we come! | 16:33 |
| *** PollyZ has joined #openstack-infra | 16:33 | |
| jeblair | clarkb: thanks, fixed ^ (and re-tested) | 16:33 |
| *** PollyZ_ has quit IRC | 16:34 | |
| zaro | morning | 16:34 |
| *** Swami has joined #openstack-infra | 16:34 | |
| *** rcernin has quit IRC | 16:35 | |
| *** trown is now known as trown|lunch | 16:35 | |
| *** rcernin has joined #openstack-infra | 16:36 | |
| *** gongysh has joined #openstack-infra | 16:36 | |
| jeblair | pabelanger, Shrews: i +3d all the v25 changes that are ready; i will work on creating zl03 and also restarting the others after they land | 16:37 |
| *** sputnik13_ has joined #openstack-infra | 16:38 | |
| pabelanger | jeblair: great | 16:38 |
| *** pcaruana has quit IRC | 16:38 | |
| *** Wei_Liu has joined #openstack-infra | 16:38 | |
| *** amitgandhinz has joined #openstack-infra | 16:39 | |
| *** jaosorior has quit IRC | 16:39 | |
| fungi | ds: sounds like you're looking for general openstack help? this is where we discuss services we run for the openstack community, so it's probably not an ideal venue for troubleshooting an openstack deployment | 16:39 |
| openstackgerrit | Merged openstack-infra/zuul: Add POST_FAILURE status https://review.openstack.org/327978 | 16:39 |
| *** jaosorior has joined #openstack-infra | 16:39 | |
| ds | <fungi> i am using in jenkins jobs. | 16:40 |
| openstackgerrit | Merged openstack-infra/zuul: Delegate FTP publisher to 127.0.0.1 https://review.openstack.org/328638 | 16:40 |
| openstackgerrit | Merged openstack-infra/zuul: Update SCPTask to also use delegate_to https://review.openstack.org/328641 | 16:40 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Use our custom pip provider for keyring package https://review.openstack.org/305499 | 16:40 |
| *** savihou has quit IRC | 16:41 | |
| fungi | ds: that would have been useful additional context. however, it sounds like maybe this would be better directed to the nova developers in #openstack-nova or the devstack devs in #openstack-qa | 16:41 |
| ds | <fungi> ok thanks | 16:41 |
| *** ds has quit IRC | 16:41 | |
| phschwartz | jeblair: question for you. Looking at the puppet module for nodepool it looks like it installs apache, but when I use that on a local system to setup nodepool I never get apache installed. Or am I missing it | 16:42 |
| openstackgerrit | Merged openstack-infra/zuul: Don't create .retry files when our playbooks fail https://review.openstack.org/328642 | 16:43 |
| openstackgerrit | Merged openstack-infra/zuul: Bootstrap worker logs with zuul information https://review.openstack.org/328662 | 16:44 |
| phschwartz | jeblair: ah, nvm. I see. I need to enable image logs | 16:44 |
| fungi | phschwartz: yeah, that's the only (current) use nodepool has for a web service: to publish logs | 16:44 |
| phschwartz | fungi: for my needs I am installing a wsgi container with a microservice on our nodepool hosts, but wasn't getting apache and missed that | 16:45 |
| phschwartz | fungi: Is there anything I should worry about security wise exposing the image logs? | 16:45 |
| openstackgerrit | Merged openstack-infra/zuul: Fix typo with ansible-playbook process https://review.openstack.org/328763 | 16:46 |
| *** ociuhandu has joined #openstack-infra | 16:46 | |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: add private gearman function https://review.openstack.org/327422 | 16:46 |
| *** annegentle has quit IRC | 16:47 | |
| fungi | phschwartz: the image build logs are pretty sanitary for our deployment at least (it's just dib output basically) | 16:47 |
| fungi | phschwartz: we don't currently publish the nodepool service logs, though we've discussed that's _probably_ safe now that we're fully on shade | 16:47 |
| clarkb | fungi: phschwartz though if you were to start baking secrets into images as discussed previously they could potentially be exposed via those logs depending on how you set up the build stuff | 16:47 |
| fungi | correct | 16:47 |
| fungi | (and why i added the caveat "for our deployment at least") | 16:48 |
| fungi | ymmv | 16:48 |
| phschwartz | clarkb: currently we have held off on doing that and moved more to a matching system to upstream | 16:49 |
| *** johnny_ has joined #openstack-infra | 16:49 | |
| *** tosky has quit IRC | 16:51 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: switch beaker/xenial jobs in gate https://review.openstack.org/329108 | 16:53 |
| *** niska has quit IRC | 16:56 | |
| *** gongysh has quit IRC | 16:56 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add zl03 https://review.openstack.org/329109 | 16:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add zl04 https://review.openstack.org/329110 | 16:56 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING https://review.openstack.org/316436 | 16:56 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Fix tempest configuration https://review.openstack.org/326055 | 16:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add zl03 to nodepool https://review.openstack.org/329111 | 16:57 |
| *** nadya has joined #openstack-infra | 16:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add zl03 to nodepool https://review.openstack.org/329111 | 16:57 |
| *** mrmartin has joined #openstack-infra | 16:57 | |
| *** smcginnis has quit IRC | 16:58 | |
| *** smcginnis has joined #openstack-infra | 16:58 | |
| *** harlowja_at_home has quit IRC | 16:59 | |
| jeblair | infra-root, infra-core and project-config-core: i'm going to continue to just self-approve the zuul-launcher and jenkins config changes as we roll them in and out | 16:59 |
| pleia2 | wfm | 16:59 |
| notmorgan | jeblair: omg... ZL03 and zl04! | 16:59 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/infra-specs: Add spec for unified message bus https://review.openstack.org/329115 | 16:59 |
| fungi | jeblair: sounds great, no need to block that on rubber-stamping onepliners | 16:59 |
| mtreinish | fungi: ^^^ I got a start on that, I left some sections for you to fill out :) | 16:59 |
| notmorgan | jeblair: :) | 16:59 |
| jeblair | notmorgan: :) | 16:59 |
| fungi | er, one-liners | 17:00 |
| *** aviau has quit IRC | 17:00 | |
| fungi | mtreinish: thanks! | 17:00 |
| *** aviau has joined #openstack-infra | 17:00 | |
| notmorgan | jeblair: so, chasing down the zuul cloner... looks like on the failures i've seen... it's actually just slow | 17:00 |
| notmorgan | jeblair: 30s might be too aggressive of a timeout. fwiw. | 17:01 |
| pabelanger | jeblair: roll on! | 17:01 |
| *** jamesmcarthur has quit IRC | 17:01 | |
| notmorgan | jeblair: i'm still looking though. [since i was looking at this over the weekend when ordering desk-like-things and chairs] | 17:01 |
| *** rossella_s has quit IRC | 17:01 | |
| jeblair | notmorgan: oh how about that. i guess lots of git operations, io contention, etc... | 17:01 |
| notmorgan | jeblair: yeah | 17:02 |
| notmorgan | jeblair: nodepool limits at 60s, we might just want to mirror that timeout | 17:02 |
| *** jaosorior has quit IRC | 17:02 | |
| *** jamesmcarthur has joined #openstack-infra | 17:02 | |
| *** nadya has quit IRC | 17:02 | |
| notmorgan | jeblair: also: https://review.openstack.org/#/c/328516/ (cc pabelanger ) should be good for eyes / approval | 17:02 |
| notmorgan | i'll start working with lifeless to address the underlying issue this week (or that is my plan) | 17:02 |
| *** sarob has joined #openstack-infra | 17:02 | |
| *** jlanoux has quit IRC | 17:03 | |
| crinkle | pabelanger: you're right it looks like the timeout was just too short https://review.openstack.org/#/c/329067/ | 17:03 |
| *** sarob has quit IRC | 17:03 | |
| *** ayoung has quit IRC | 17:04 | |
| jeblair | notmorgan: https://review.openstack.org/328516 lgtm; clarkb may be interested in that. | 17:05 |
| * notmorgan nods. | 17:05 | |
| *** ikhudoshyn has quit IRC | 17:05 | |
| * notmorgan would be happy to have a clarkb review on it. | 17:05 | |
| *** infra-red has joined #openstack-infra | 17:05 | |
| *** sarob has joined #openstack-infra | 17:06 | |
| *** ihrachys has quit IRC | 17:06 | |
| jeblair | notmorgan: picking a random recent success, cloner.test_upgrade takes 17 seconds, which doesn't sound *too* close to the timeout... | 17:06 |
| jeblair | that was on ovh | 17:06 |
| notmorgan | jeblair: right. it seems to be random in contention, but where i've seen the failure... it looks like it's just moving slow on io ops | 17:07 |
| notmorgan | let me find one. | 17:07 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Don't try to get stack details if the stack doesn't exist https://review.openstack.org/319337 | 17:07 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: TEST: Delete the overcloud when finished https://review.openstack.org/297328 | 17:07 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Don't pipe yes to a function https://review.openstack.org/317730 | 17:07 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Test overcloud deletion in periodic job https://review.openstack.org/296765 | 17:07 |
| jeblair | i'm going to restart zuul | 17:07 |
| notmorgan | oh ugh, don't have a test w/ the log capture handy... doh. | 17:08 |
| jeblair | we'll get one soon after 516 lands | 17:08 |
| clarkb | notmorgan: is that overloading private methods in fixtures.Timeout? | 17:09 |
| jeblair | pabelanger: oh, what's the status of the skip registration check? | 17:09 |
| notmorgan | clarkb: it is, unfortunately. i might be able to move it to setup directly? | 17:09 |
| clarkb | (this seems like not a good idea because private) | 17:09 |
| pabelanger | jeblair: just need to land puppet modules | 17:09 |
| pabelanger | jeblair: let me setup topic | 17:09 |
| *** ifarkas has quit IRC | 17:09 | |
| jeblair | pabelanger: might be nice to get that into this restart | 17:10 |
| notmorgan | clarkb: this is planned to be a short-term thing - because if it isn't... i'm not doing my job. | 17:10 |
| notmorgan | clarkb: i can look at respinning and see if it can be made into calling setUp. | 17:10 |
| *** PollyZ has quit IRC | 17:10 | |
| pabelanger | jeblair: -t v25 now | 17:11 |
| jeblair | pabelanger: not recheck! | 17:11 |
| clarkb | notmorgan: I am reading fixtures.Timeout and I think you can just delete your _setUP | 17:11 |
| jeblair | pabelanger: +workflow! | 17:11 |
| clarkb | notmorgan: it will use the signal_handler you wrote which should be sufficient? | 17:11 |
| pabelanger | jeblair: ah, right | 17:11 |
| pabelanger | jeblair: sorry about that | 17:11 |
| notmorgan | clarkb: no, it needs to not have the conditional | 17:12 |
| jeblair | pabelanger: i will directly enqueue it | 17:12 |
| clarkb | notmorgan: just hard set gentle then | 17:12 |
| clarkb | since that is public interface | 17:12 |
| pabelanger | jeblair: keep forgetting +workflow will fix depends-on | 17:12 |
| pabelanger | and not recheck | 17:12 |
| notmorgan | clarkb: oh. hmm. yeah -- that might be better | 17:12 |
| jeblair | pabelanger: recheck will, but it goes to the back of the line | 17:12 |
| notmorgan | clarkb: hmm. TimeoutFixture(no_i_really_mean_gentle=True) :P | 17:13 |
| jeblair | clarkb, fungi: +3 https://review.openstack.org/327952 ? | 17:13 |
| clarkb | notmorgan: another potential issue which we will just have to accept likely is that stderr is a shared file handle with the way we use testr | 17:13 |
| notmorgan | clarkb: i guess we don't rely on gentle now, i can pull out my conditional | 17:13 |
| clarkb | notmorgan: not sure we really want to lock around that to address it because that is uglyness | 17:14 |
| *** flepied has quit IRC | 17:14 | |
| notmorgan | clarkb: in this case,-- it is only ever hit because of a timeout and we want some level of output | 17:14 |
| *** PollyZ has joined #openstack-infra | 17:14 | |
| notmorgan | clarkb: if we're hitting a time out: a we have a real issue, and b: some output is better than none | 17:14 |
| *** gluytium has quit IRC | 17:15 | |
| notmorgan | clarkb: even if it's shared file handle | 17:15 |
| clarkb | notmorgan: right but you can have === Test timeout for foo ===\n=== Test timeout for bar ===\n log line for foo \n log line for bar and you won't be able to tell them apart | 17:15 |
| *** yamahata has joined #openstack-infra | 17:15 | |
| notmorgan | clarkb: oh i could just roll that into the single write. | 17:15 |
| jeblair | clarkb, notmorgan: didn't we have 2 test failures with this and found that they were distinguishable? | 17:15 |
| jeblair | oh, that was because of the single write | 17:15 |
| notmorgan | jeblair: the errors were, they also occured at different times | 17:16 |
| notmorgan | so the header/footer thing "worked" | 17:16 |
| jeblair | so you're saying add the header/footer into the single write | 17:16 |
| jeblair | gotcha | 17:16 |
| notmorgan | but it could cause inter-mixing | 17:16 |
| clarkb | jeblair: notmorgan probably the best solution here is a log prefix that ties to a header/footer | 17:16 |
| *** PollyZ has quit IRC | 17:16 | |
| AJaeger | jeblair: fine with me - if I notice a project-config change, I might review it nevertheless but no promises ;) | 17:16 |
| clarkb | since a single write won't completely fix it I don't think | 17:16 |
| jeblair | AJaeger: you're welcome to | 17:16 |
| jeblair | clarkb: why not? | 17:16 |
| notmorgan | clarkb: it seemed to not intermix data on a single write. | 17:17 |
| *** thiagop has joined #openstack-infra | 17:17 | |
| clarkb | jeblair: because two python processes with the same stderr aren't going to cooperate atomically around single writes | 17:17 |
| AJaeger | fungi: I'm happy with going forward with dguitarbite's request | 17:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Switch to async_status for zuul_runner task https://review.openstack.org/328649 | 17:17 |
| openstackgerrit | amrith proposed openstack-infra/project-config: Add trove period jobs with oslo master https://review.openstack.org/329086 | 17:17 |
| *** ayoung has joined #openstack-infra | 17:17 | |
| notmorgan | clarkb: my only concern with the prefix is there is a lot of extra data to add (func refs etc) -- but i'm happy to do that. | 17:17 |
| fungi | AJaeger: thanks for checking | 17:18 |
| clarkb | notmorgan: and it may be something that we don't bother with until it becomes a problem with overlapping timeouts | 17:18 |
| notmorgan | clarkb: wfm. i'll fix the private interface. and hopefullty can get testr fixed up somewhat | 17:18 |
| notmorgan | clarkb: so we can just drop this. it's def. a bit of a hack to work around the limitation of testr + timeout fixture | 17:19 |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Expose gearman_check_job_registration setting https://review.openstack.org/327351 | 17:19 |
| * notmorgan doesn't want this to linger around too long if at all avoidable | 17:19 | |
| *** ddieterly is now known as ddieterly[away] | 17:20 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/irc-meetings: Added meeting time for the JavaScript SDK & Tooling group. https://review.openstack.org/329120 | 17:21 |
| fungi | AJaeger: loquacities: unfortunately dguitarbite seems to have dropped off irc, and there's an open change on that branch which will need to be abandoned https://review.openstack.org/#/q/status:open+project:openstack/training-labs+branch:stable/mitaka | 17:21 |
| *** hdaniel has joined #openstack-infra | 17:21 | |
| *** infra-red has quit IRC | 17:22 | |
| *** sarob_ has joined #openstack-infra | 17:23 | |
| *** zhs has quit IRC | 17:25 | |
| *** zhs has joined #openstack-infra | 17:26 | |
| *** sarob has quit IRC | 17:26 | |
| AJaeger | fungi: and only dguitarbite and roger are cores there, so neither loquacities nor myself can abandon it ;( | 17:26 |
| AJaeger | fungi: could you abandon it, please? Otherwise we need to wait for tomorrow - and I would sent him an email... | 17:26 |
| pabelanger | crinkle: good to know | 17:28 |
| *** tphummel has joined #openstack-infra | 17:28 | |
| *** julim has quit IRC | 17:28 | |
| *** yamahata has quit IRC | 17:28 | |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run Heat functional tests in Identity v3-only https://review.openstack.org/314643 | 17:28 |
| *** salv-orl_ has joined #openstack-infra | 17:29 | |
| *** oanson has joined #openstack-infra | 17:29 | |
| jeblair | i'm approving https://review.openstack.org/327952 since it's the last thing holding up a zuul restart | 17:30 |
| fungi | AJaeger: not to question the governance of docs projects, but it seems weird that none of the normal docs core team has oversight control for a repo under that team. anyway, i'll grant myself access to abandon that change | 17:30 |
| *** PollyZ has joined #openstack-infra | 17:31 | |
| AJaeger | fungi: yeah, loquacities should have access, I'll talk with here... | 17:32 |
| *** salv-orl_ has quit IRC | 17:32 | |
| AJaeger | thanks, fungi | 17:32 |
| *** salv-orlando has quit IRC | 17:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add python3 wheel-builds https://review.openstack.org/329094 | 17:32 |
| *** salv-orlando has joined #openstack-infra | 17:32 | |
| *** PollyZ has quit IRC | 17:36 | |
| *** rbrndt has quit IRC | 17:36 | |
| *** vhosakot has quit IRC | 17:36 | |
| fungi | AJaeger: loquacities: i've deleted the stable/mitaka branch of openstack/training-labs (previously a469b7701d4ca98b5d4a142bce861cdc966e1257) at the request of dguitarbite | 17:36 |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/elastic-recheck: Add e-r query for 1590139 https://review.openstack.org/326739 | 17:37 |
| fungi | turns out as a gerrit admin i was already able to abandon it without changing my group membership | 17:37 |
| AJaeger | ;) | 17:37 |
| AJaeger | thanks, fungi! | 17:37 |
| *** pcaruana has joined #openstack-infra | 17:39 | |
| *** vhosakot has joined #openstack-infra | 17:41 | |
| *** annegentle has joined #openstack-infra | 17:42 | |
| *** sambetts is now known as sambetts|afk | 17:42 | |
| *** milan has left #openstack-infra | 17:43 | |
| *** bpokorny has quit IRC | 17:45 | |
| *** sarob_ has quit IRC | 17:45 | |
| *** tongli has joined #openstack-infra | 17:47 | |
| *** mtanino has quit IRC | 17:47 | |
| pabelanger | jeblair: Shrews: Seeing the following now: 2016-06-13 17:33:34,972 DEBUG zuul.NodeWorker.ubuntu-trusty-rax-dfw-1626092: Ansible output: fatal: [node]: FAILED! => {"changed": false, "failed": true, "msg": "implementation error: unknown type raw requested for msg"} | 17:47 |
| *** erlon has joined #openstack-infra | 17:49 | |
| *** piet has quit IRC | 17:49 | |
| pabelanger | it exists in 2.1.0.0 however https://github.com/ansible/ansible/blob/v2.1.0.0-1/lib/ansible/module_utils/basic.py#L657 | 17:51 |
| pabelanger | oh! | 17:52 |
| *** Sukhdev has quit IRC | 17:52 | |
| pabelanger | but not 2.0.0.2 | 17:52 |
| pabelanger | which zl01 is running | 17:52 |
| *** Sukhdev has joined #openstack-infra | 17:52 | |
| *** Sukhdev has quit IRC | 17:52 | |
| *** niska has joined #openstack-infra | 17:54 | |
| *** sarob has joined #openstack-infra | 17:54 | |
| *** yamahata has joined #openstack-infra | 17:55 | |
| *** PollyZ has joined #openstack-infra | 17:55 | |
| zaro | fungi: seems like online reindex is broken on latest gerrit https://groups.google.com/d/msg/repo-discuss/spphLajbVoQ/3gQJvcVIBgAJ | 17:55 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul: Ensure ansible 2.1.0.0 is installed https://review.openstack.org/329127 | 17:55 |
| *** hongbin has joined #openstack-infra | 17:56 | |
| pabelanger | jeblair: Shrews: ^ | 17:56 |
| zaro | which prevents me from adding the feature to master :( | 17:56 |
| pleia2 | zaro: sigh | 17:56 |
| zaro | *feature/ feature to force online reindex | 17:57 |
| pabelanger | s/2.0.0.2/2.0.2.0 | 17:57 |
| Shrews | pabelanger: ah, yeah. | 17:57 |
| Shrews | pabelanger: is there a way to set a min version? | 17:58 |
| openstackgerrit | Alex Kavanagh proposed openstack-infra/project-config: Add #openstack-charms and associated projects https://review.openstack.org/329129 | 17:59 |
| zaro | tried to add that feature to admin-console plugin but meeting resistance there as well https://gerrit-review.googlesource.com/#/c/77939/ | 17:59 |
| *** flepied has joined #openstack-infra | 17:59 | |
| pabelanger | Shrews: I don't think we can in puppet. Adding ansible into requirements.txt for zuul could do that, but we've opted to keep it out for the moment | 18:00 |
| *** rbrndt has joined #openstack-infra | 18:01 | |
| zaro | fungi: i guess 2 options left, gerrit groovy-plugin or roll our own plugin with change 77939. | 18:01 |
| *** trown|lunch is now known as trown | 18:01 | |
| *** csomerville has joined #openstack-infra | 18:01 | |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run Swift functional tests in Identity v3-only https://review.openstack.org/313659 | 18:02 |
| *** jamesmcarthur has quit IRC | 18:03 | |
| clarkb | pabelanger: Shrews also that brings up the queston of do we want to be running multiple versions of ansible to deal with multiple matrixes of version requirements for things | 18:03 |
| clarkb | on puppetmaster we can't use 2.1.0 because it broke ansible puppet | 18:03 |
| *** Apoorva_ has joined #openstack-infra | 18:04 | |
| *** jamesmcarthur has joined #openstack-infra | 18:04 | |
| *** cody-somerville has quit IRC | 18:04 | |
| fungi | zaro: if online reindex is broken upstream, are they interested in fixing it? | 18:04 |
| jogo | clarkb: random question do you still use bandersnatch for pypi mirroring? | 18:04 |
| jeblair | pabelanger: do we need to shut down zl01? | 18:04 |
| clarkb | jeblair: we do | 18:04 |
| jeblair | i will do so | 18:04 |
| clarkb | er jogo ^ | 18:04 |
| fungi | zaro: or are they wanting to remove it entirely? | 18:04 |
| clarkb | jeblair: but yes I also think if zl01 is not working we should turn it off | 18:05 |
| openstackgerrit | Thiago Paiva Brito proposed openstack-infra/devstack-gate: Removing 'horizon' from Ironic feature matrix https://review.openstack.org/329132 | 18:05 |
| jeblair | zl01 stopped | 18:05 |
| *** hdaniel has quit IRC | 18:05 | |
| jeblair | *theoretically* that should abort all jobs and zuul should reschedule them | 18:05 |
| *** mtanino has joined #openstack-infra | 18:05 | |
| jogo | clarkb: awesome thanks. Any known gotchas with it these days? | 18:06 |
| pabelanger | jeblair: ya, until we atleast decided to upgrade or support both versions of ansible per clarkb | 18:06 |
| fungi | pabelanger: does 329127 mean you solved the issues which caused us to pin it to <2.1? | 18:06 |
| clarkb | jogo: not really other than occasionally you have to rebuild your mirrors because of journal state problems | 18:06 |
| jeblair | clarkb, pabelanger: i think we should specify 2.1 in zuul for now, and not worry about it being the same as what we use for puppet | 18:06 |
| zaro | fungi: I’m sure there’s interest in fixing i’m guessing it’s just a time limitation. | 18:06 |
| *** Apoorva has quit IRC | 18:07 | |
| zaro | fungi: i have not heard of any news of wanting to remove it. | 18:07 |
| jeblair | fungi: that change is for zuul. to that (and clarkb's) point, i'm okay with them being different versions | 18:07 |
| jeblair | fungi: (zuul vs infra puppet) | 18:07 |
| jeblair | i don't think they should collide | 18:07 |
| fungi | ahh, yep, so not impacted by retaining the cap on puppetmaster.o.o | 18:07 |
| *** markusry has quit IRC | 18:07 | |
| pabelanger | jeblair: clarkb: agreed, for v25 we should stick to a single version. Then revisit for v3 | 18:07 |
| *** dimtruck is now known as zz_dimtruck | 18:08 | |
| pabelanger | fungi: 329127 is to address 328662. I can add a depends on | 18:08 |
| jogo | clarkb: awesome thanks! | 18:08 |
| pabelanger | fungi: it will also be limited to zuul-launcher nodes | 18:08 |
| zaro | fungi: I’m guessing that my post has raised awareness but there’s no indication that anybody is working on it yet. | 18:08 |
| *** hdaniel has joined #openstack-infra | 18:08 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul: Ensure ansible 2.1.0.0 is installed https://review.openstack.org/329127 | 18:09 |
| *** gluytium has joined #openstack-infra | 18:09 | |
| jeblair | pabelanger: i don't understand the depends-on. | 18:09 |
| *** acoles is now known as acoles_ | 18:09 | |
| pabelanger | jeblair: maybe related-to is better. I just link back to the patch that exposed the problem | 18:10 |
| AJaeger | dougwig, mestery, could either of you review some neutron infra changes, please? https://review.openstack.org/314977 https://review.openstack.org/326821 https://review.openstack.org/#/c/328445/ https://review.openstack.org/314604 https://review.openstack.org/324135 https://review.openstack.org/323961 https://review.openstack.org/326300 https://review.openstack.org/322812 - that's all I could find | 18:11 |
| jeblair | pabelanger: ah yeah. i think just a textual comment would have been more clear. because if both of those were actually in flight, i would say the dependency is reversed :) | 18:11 |
| AJaeger | quickly ;) | 18:11 |
| dougwig | AJaeger: on it | 18:11 |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run Swift functional tests in Identity v3-only https://review.openstack.org/313659 | 18:11 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul: Ensure ansible 2.1.0.0 is installed https://review.openstack.org/329127 | 18:11 |
| *** dtantsur is now known as dtantsur|afk | 18:11 | |
| jeblair | pabelanger: i take it this problem is what tanked http://logs.openstack.org/52/327952/2/gate/gate-puppet-openstackci-docs/3c45a85/console.html ? | 18:11 |
| jeblair | i have re-enqueued 952 | 18:12 |
| pabelanger | jeblair: ya, that made me go looking | 18:13 |
| AJaeger | thanks mestery and dougwig ! | 18:13 |
| jeblair | also 329109 | 18:13 |
| jeblair | which i will also re-enqueue | 18:13 |
| mestery | AJaeger: yw, and thanks for the pointer! | 18:13 |
| AJaeger | mestery, dougwig: Seems that the #openstack-firewall needs some more discussion, shouldn't have listed it: https://review.openstack.org/#/c/326821/ or what way do you see forward? | 18:14 |
| *** hdaniel has quit IRC | 18:14 | |
| dougwig | AJaeger: i think we should wait for jeblair to reclaim/register the original channel | 18:15 |
| mestery | dougwig AJaeger: I'm fine either way, I have no dog in that fight | 18:15 |
| jeblair | do i need to do something? | 18:15 |
| AJaeger | jeblair: see 326821 - can you claim #openstack-fwaas ? | 18:16 |
| jeblair | sure, i'll start on that | 18:16 |
| *** Jeffrey4l has quit IRC | 18:17 | |
| pabelanger | jeblair: I am eager to see if the abort process worked as expected. | 18:17 |
| AJaeger | jeblair: once you've done it, please comment in the change. | 18:17 |
| AJaeger | thanks, jeblair | 18:17 |
| *** xarses has quit IRC | 18:18 | |
| *** pvaneck has joined #openstack-infra | 18:20 | |
| *** ddieterly[away] has quit IRC | 18:20 | |
| * AJaeger waves good-bye | 18:20 | |
| *** sarob has quit IRC | 18:22 | |
| anteaya | AJaeger: enjoy your evening | 18:22 |
| *** chem` has joined #openstack-infra | 18:22 | |
| *** sdake_ has joined #openstack-infra | 18:23 | |
| *** jamesmcarthur has quit IRC | 18:23 | |
| jeblair | AJaeger, dougwig, mestery: #openstack-fwaas is registered now | 18:23 |
| mestery | thanks jeblair :) | 18:23 |
| *** chem has quit IRC | 18:23 | |
| dougwig | jeblair: did you set the acl's for infra, or can you op someone that can do that? | 18:24 |
| jeblair | dougwig: that's done | 18:24 |
| jeblair | dougwig: a change to project-config to add the bots to it should pass now | 18:24 |
| jeblair | (i did not write that change) | 18:24 |
| dougwig | jeblair: awesome, many thanks. | 18:24 |
| jeblair | np | 18:24 |
| *** csomerville has quit IRC | 18:25 | |
| *** csomerville has joined #openstack-infra | 18:25 | |
| *** HeOS has joined #openstack-infra | 18:26 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config: Add accessbot for #openstack-fwaas https://review.openstack.org/326821 | 18:26 |
| njohnston | ^^ hopefully that covers it. Thanks everyone! | 18:29 |
| *** hasharAway is now known as hashar | 18:31 | |
| *** sarob has joined #openstack-infra | 18:31 | |
| *** bhavik has joined #openstack-infra | 18:32 | |
| *** csomerville has quit IRC | 18:32 | |
| *** HeOS has quit IRC | 18:33 | |
| pleia2 | njohnston: doug has a valid comment, but access on the channel looks good otherwise | 18:33 |
| *** tqtran has joined #openstack-infra | 18:35 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config: Add accessbot for #openstack-fwaas https://review.openstack.org/326821 | 18:35 |
| pleia2 | njohnston: thumbs up | 18:36 |
| njohnston | pleia2: Thanks! | 18:36 |
| dougwig | njohnston: me too | 18:36 |
| *** zz_dimtruck is now known as dimtruck | 18:38 | |
| *** kushal has joined #openstack-infra | 18:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Add api-ref jobs for neutron https://review.openstack.org/314977 | 18:39 |
| *** annegentle has quit IRC | 18:39 | |
| *** tkelsey has quit IRC | 18:39 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable publishing of neutron-dynamic-routing docs https://review.openstack.org/323961 | 18:39 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Expose gearman_check_job_registration setting https://review.openstack.org/327952 | 18:39 |
| *** jamesmcarthur has joined #openstack-infra | 18:40 | |
| *** csomerville has joined #openstack-infra | 18:43 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: switch beaker/xenial jobs in gate https://review.openstack.org/329108 | 18:43 |
| *** sarob has quit IRC | 18:44 | |
| *** abregman has joined #openstack-infra | 18:44 | |
| *** pkoniszewski has quit IRC | 18:45 | |
| *** jesusaur has quit IRC | 18:45 | |
| *** jesusaur has joined #openstack-infra | 18:46 | |
| *** pkoniszewski has joined #openstack-infra | 18:48 | |
| jeblair | cool, i will restart zuul after returning from lunch | 18:48 |
| *** sarob has joined #openstack-infra | 18:49 | |
| *** mrmartin has quit IRC | 18:49 | |
| *** bhavik has quit IRC | 18:50 | |
| *** sputnik13 has joined #openstack-infra | 18:51 | |
| *** jamesmcarthur has quit IRC | 18:52 | |
| *** jamesmcarthur has joined #openstack-infra | 18:52 | |
| *** sarob has quit IRC | 18:54 | |
| *** annegentle has joined #openstack-infra | 18:54 | |
| *** ddieterly has joined #openstack-infra | 18:54 | |
| *** sputnik13 has quit IRC | 18:55 | |
| *** nadya has joined #openstack-infra | 18:56 | |
| *** jcoufal_ has joined #openstack-infra | 18:56 | |
| *** jcoufal has quit IRC | 18:59 | |
| EmilienM | jeblair: I noticed some jobs failing in Gerrit but exit 0 in actual logs | 18:59 |
| *** csomerville has quit IRC | 19:00 | |
| EmilienM | See https://review.openstack.org/#/c/329114/ and http://logs.openstack.org/14/329114/1/check/gate-puppet-keystone-puppet-lint/ab72baa/console.html#_2016-06-13_18_01_53_494126 | 19:00 |
| dmsimard | Seeing a couple DNS resolution errors (git.o.o, yum repositories and rubygems) is there any known issues ? | 19:00 |
| EmilienM | jeblair: and also http://logs.openstack.org/14/329114/1/check/gate-puppet-keystone-puppet-syntax-4-centos-7/6a94f93/console.html#_2016-06-13_17_26_57_834865 | 19:00 |
| fungi | my client was briefly reporting >250s lag to tepper.freenode.net | 19:01 |
| *** tphummel has quit IRC | 19:01 | |
| *** amoralej is now known as amoralej|off | 19:01 | |
| fungi | (from rackspace iad) | 19:01 |
| clarkb | dmsimard: no I don't think there is a known issue with DNS, do you have more specifics? is this in tests or locally? if in the CI pipeline do you have links to logs? | 19:01 |
| dmsimard | clarkb: rubygems: http://logs.openstack.org/35/329135/1/check/gate-puppet-openstack-integration-puppet-lint/af92f0b/console.html | 19:02 |
| dmsimard | git.o.o: http://logs.openstack.org/35/329135/1/check/gate-puppet-openstack-integration-3-scenario001-tempest-ubuntu-xenial/7443a85/console.html#_2016-06-13_18_30_13_426 | 19:02 |
| dmsimard | yum repo: http://logs.openstack.org/35/329135/1/check/gate-puppet-openstack-integration-3-scenario001-tempest-centos-7/5858c33/console.html#_2016-06-13_18_47_57_609 | 19:02 |
| clarkb | dmsimard: that doesn't look like dns issues | 19:02 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-infracloud: Add polling for image list tests https://review.openstack.org/329067 | 19:02 |
| clarkb | dmsimard: that is complaining of http connection resets | 19:03 |
| crinkle | pabelanger: something like that ^ what you had in mind? | 19:03 |
| fungi | dmsimard: that first one doesn't look dns-related at all | 19:03 |
| fungi | dmsimard: looks like fastly had issues serving the requested content? | 19:03 |
| dmsimard | clarkb: what do you mean unrelated to DNS resolution ? "git.openstack.org: Temporary failure in name resolution" "Could not resolve host: trunk.rdoproject.org; Name or service not known" | 19:03 |
| dmsimard | but I understand the rubygems isn't DNS related | 19:04 |
| *** salv-orlando has quit IRC | 19:04 | |
| clarkb | dmsimard: http://logs.openstack.org/35/329135/1/check/gate-puppet-openstack-integration-puppet-lint/af92f0b/console.html#_2016-06-13_18_53_16_544 | 19:04 |
| clarkb | that has nothing to do with DNS | 19:04 |
| dmsimard | clarkb: yes, rubygems isn't DNS related, the other two are, though | 19:04 |
| clarkb | ok lets not confuse them then | 19:04 |
| *** tphummel has joined #openstack-infra | 19:04 | |
| dmsimard | Aye. Network-related errors on three different jobs on three different destinations prompted me to believe there could be something funny going on. | 19:05 |
| *** salv-orlando has joined #openstack-infra | 19:05 | |
| fungi | yeah, i started with the first log you linked and am still looking at it, don't see dns issues there | 19:05 |
| clarkb | and the third is yum failing but not clear if that is also dns related | 19:05 |
| phschwartz | Has anyone seen the launch_node.py script throw an exception on reboot where the SSHException was thrown with no rc set? | 19:06 |
| clarkb | there is a rabbit with a collar on in my front yard | 19:06 |
| clarkb | ah I see further down in the yum error it says could not resolve host. | 19:07 |
| pleia2 | portland++ | 19:07 |
| fungi | dmsimard: the second one you linked also doesn't look like it's necessarily dns-specific. it's breaking the first time the node tries to do any outbound network activity (we've seen lots of network issues in rax ord, which is where that one ran) | 19:07 |
| openstackgerrit | Alex Kavanagh proposed openstack-infra/project-config: Add #openstack-charms and associated projects https://review.openstack.org/329129 | 19:07 |
| openstackgerrit | Merged openstack-infra/project-config: Add accessbot for #openstack-fwaas https://review.openstack.org/326821 | 19:07 |
| dmsimard | fungi, clarkb: okay, I guess I will do a recheck and see what happens | 19:08 |
| *** sarob has joined #openstack-infra | 19:08 | |
| clarkb | and trunk.rdoproject.org does resolve locally for me so it is a valid name at least currently | 19:09 |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 19:09 |
| clarkb | but its only that host that fails to resolve in that job looks like | 19:09 |
| *** oanson has quit IRC | 19:10 | |
| dmsimard | clarkb: what's your dns server ? trunk.rdoproject.org resolves for me and for 8.8.8.8 | 19:10 |
| clarkb | dmsimard: it should be google dns on those test instances | 19:10 |
| dmsimard | dig @8.8.8.8 wfm | 19:10 |
| *** e0ne has joined #openstack-infra | 19:11 | |
| *** annegentle has quit IRC | 19:11 | |
| clarkb | yay anycast dns | 19:11 |
| fungi | that third one does seem to have been able to perform dns resolution and perform other network tasks earlier in the job | 19:11 |
| fungi | but something might have happened to the instance partway through the job | 19:11 |
| anteaya | clarkb: someone's pet rabbit? | 19:11 |
| *** tqtran has quit IRC | 19:11 | |
| clarkb | anteaya: looks like but we don't know anyone in the neighborhood with rabbits | 19:12 |
| *** javeriak has joined #openstack-infra | 19:12 | |
| *** csomerville has joined #openstack-infra | 19:12 | |
| fungi | probably from out of town if it didn't also have skinny jeans and hipster glasses | 19:12 |
| clarkb | we probably should double check that our unbound setup is at least doing what we intend on our images | 19:14 |
| anteaya | hahaha | 19:14 |
| *** nadya has quit IRC | 19:14 | |
| *** javeriak_ has quit IRC | 19:15 | |
| openstackgerrit | Merged openstack-infra/puppet-infracloud: Copy hpuswest to vanilla https://review.openstack.org/327332 | 19:17 |
| dhellmann | has anyone seen the gate-project-config-gerrit job report as a failure even when the logs say it passed? https://review.openstack.org/#/c/328486/ | 19:17 |
| openstackgerrit | Merged openstack-infra/system-config: Upgrade groups-dev.o.o to trusty https://review.openstack.org/299238 | 19:17 |
| dhellmann | I can recheck, but that seemed weird enough to mention first, just in case. | 19:18 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: switch beaker/xenial jobs in gate https://review.openstack.org/329108 | 19:18 |
| *** bpokorny has joined #openstack-infra | 19:18 | |
| clarkb | dhellmann: that is likely a bug in the new zuul launcher. Jeblair and notmorgan and pabelanger are probably the best to look at that | 19:19 |
| dhellmann | clarkb : thanks! | 19:20 |
| anteaya | dhellmann: thanks for mentioning it | 19:20 |
| dhellmann | jeblair, notmorgan, pabelanger : do you want a chance to poke at that patch & logs in their current state, or is it safe to recheck? | 19:20 |
| dhellmann | anteaya : I don't find all the bugs, but when I do, I report them. ;-) | 19:21 |
| * dhellmann almost closed with "stay testy my friend" and realized that would have the wrong connotation | 19:21 | |
| anteaya | dhellmann: for your email do you want links to patches or is anecdotal evidence sufficient for your purposes? | 19:21 |
| anteaya | dhellmann: you and the dos equis man | 19:21 |
| dhellmann | anteaya : anecdotes are good enough | 19:21 |
| *** PollyZ has quit IRC | 19:22 | |
| anteaya | actually I see the resemblance | 19:22 |
| anteaya | dhellmann: anecdotes it is | 19:22 |
| anteaya | dhellmann: ha ha ha | 19:22 |
| *** baoli_ has quit IRC | 19:22 | |
| dhellmann | it doesn't have to be detailed at all, "I fix a bug about X in Y" is fine. I'm looking for a summary. | 19:22 |
| anteaya | I just sent a summary | 19:23 |
| anteaya | I fixed a whitespace in gerrit documentation, or a gerrit comment about 2 years ago | 19:23 |
| anteaya | woooo | 19:23 |
| *** HeOS has joined #openstack-infra | 19:24 | |
| anteaya | for those wondering if their contribution was significant enough to mention in reply to dhellmann's email | 19:24 |
| dhellmann | no contribution is too small | 19:24 |
| anteaya | agreed | 19:24 |
| *** mhickey has joined #openstack-infra | 19:26 | |
| openstackgerrit | Alex Kavanagh proposed openstack-infra/system-config: Add #openstack-charms to statusbot https://review.openstack.org/329159 | 19:27 |
| pabelanger | dhellmann: clarkb: a recheck should be fine. That ran on zl01 before we stopped it | 19:27 |
| notmorgan | pabelanger: ooh we have a zl01 thing? | 19:27 |
| notmorgan | happening | 19:27 |
| * notmorgan gets back to irc | 19:28 | |
| *** PollyZ has joined #openstack-infra | 19:28 | |
| *** maishsk has joined #openstack-infra | 19:28 | |
| *** csomerville has quit IRC | 19:29 | |
| dhellmann | pabelanger : great, thanks | 19:29 |
| pabelanger | notmorgan: zl01 currently stopped until we upgrade to ansible 2.1.0.0 | 19:30 |
| notmorgan | pabelanger: ahhh | 19:30 |
| notmorgan | pabelanger: makes sense | 19:30 |
| phschwartz | So thoughts on this: https://github.com/openstack-infra/system-config/blob/master/launch/launch-node.py#L169 | 19:31 |
| phschwartz | Should it be a reg exception we except or SSHException? | 19:31 |
| *** salv-orlando has quit IRC | 19:31 | |
| pabelanger | crinkle: sure, I'll review shortly | 19:31 |
| phschwartz | All the checks in the block are for the rc var that is only in SSHException | 19:31 |
| *** salv-orlando has joined #openstack-infra | 19:31 | |
| notmorgan | phschwartz: good question | 19:31 |
| *** kushal has quit IRC | 19:32 | |
| phschwartz | notmorgan: it is stepping me up a bit on the IBM beta cloud. We launch an instance and then reboot it. We are getting an exception thrown that lands us in that block that is not an SSHException so there is no rc set. It is an exception related to ssh timeout | 19:32 |
| fungi | need to step out and run some errands while things aren't too busy, but should be back in a couple hours | 19:32 |
| *** PollyZ has quit IRC | 19:33 | |
| *** ihrachys has joined #openstack-infra | 19:35 | |
| pabelanger | fungi: ack | 19:35 |
| *** Sukhdev has joined #openstack-infra | 19:36 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: switch beaker/xenial jobs in gate https://review.openstack.org/329108 | 19:36 |
| anteaya | fungi: happy errands | 19:36 |
| *** maishsk has quit IRC | 19:37 | |
| *** xarses has joined #openstack-infra | 19:38 | |
| *** rkukura has quit IRC | 19:38 | |
| *** rkukura has joined #openstack-infra | 19:39 | |
| *** gluytium has quit IRC | 19:40 | |
| *** PollyZ has joined #openstack-infra | 19:41 | |
| *** baoli has joined #openstack-infra | 19:42 | |
| *** baoli has quit IRC | 19:43 | |
| *** baoli has joined #openstack-infra | 19:43 | |
| clarkb | phschwartz: check for the existence of rc first then? | 19:44 |
| clarkb | phschwartz: I don't think we have ever run into timeouts if the process gets that far along | 19:44 |
| clarkb | phschwartz: if networking doesn't work it will break much earlier | 19:44 |
| *** javeriak has quit IRC | 19:44 | |
| jeblair | back; i'm going to begin work on restarting zuul now | 19:45 |
| jeblair | wow, zuul is not slammed. it's restarting. | 19:47 |
| *** e0ne has quit IRC | 19:48 | |
| dhellmann | fungi : another borked release, this time python-ironicclient 0.8.2 (a liberty release from last week) https://review.openstack.org/#/c/324883/ -- can you queue that up when you have some time, please? | 19:49 |
| jeblair | #status log zuul has been restarted with registration checks disabled -- we should no longer see NOT_REGISTERED errors after zuul restarts. | 19:50 |
| openstackstatus | jeblair: finished logging | 19:50 |
| jeblair | mriedem: zuul is running with the 10s delay patch now | 19:51 |
| rcarrillocruz | nibalizer: re ansible-puppet, agreed on the one logfile per host, i'll put it on /var/log/ansible-puppet. Re: color, i would have sworn puppet --help didn't show me the nocolor thing, and for the logdest not giving back stdout i believe it gave to me? | 19:51 |
| jeblair | and we should start to see benefit from the MASS_DO change as we restart zuul workers that use it. | 19:51 |
| rcarrillocruz | byt anyway, not really in front of my computer, will check and get back to you | 19:51 |
| EmilienM | jeblair: I'm seeing weird things | 19:52 |
| EmilienM | look http://logs.openstack.org/06/328806/4/check/gate-puppet-aodh-releasenotes/7badb8d/console.html#_2016-06-13_18_06_02_575945 | 19:52 |
| dhellmann | oh, I just noticed fungi said he was stepping out. can someone else queue up that python-ironicclient release ^^ please? | 19:52 |
| EmilienM | jeblair: but in Gerrit, it's red: https://review.openstack.org/#/c/328806/ | 19:52 |
| jeblair | EmilienM: yeah, we broke and fixed that one this morning, sorry. | 19:53 |
| EmilienM | ah no problem | 19:53 |
| *** bhavik has joined #openstack-infra | 19:55 | |
| clarkb | phschwartz: having an explicit except block for the specific type of exception would likely be best there | 19:55 |
| clarkb | dhellmann: I am about to eat lunch but can do that after if no one else does it first | 19:55 |
| dhellmann | clarkb : thanks! | 19:55 |
| mriedem | jeblair: cool. i'll keep that in mind the next time i need to rebase that big series. | 19:56 |
| jeblair | pabelanger: i'm going to start zl01 now and graceful zl02 | 19:56 |
| jeblair | pabelanger: (i manually upgraded ansible on zl01) | 19:56 |
| *** dtardivel has quit IRC | 19:57 | |
| pabelanger | jeblair: ack | 19:57 |
| openstackgerrit | Jef Mallal proposed openstack-infra/jenkins-job-builder: Add support for retry in multijob plugin https://review.openstack.org/329169 | 19:58 |
| *** cody-somerville has joined #openstack-infra | 20:00 | |
| *** amitgandhinz has quit IRC | 20:01 | |
| *** amitgandhinz has joined #openstack-infra | 20:02 | |
| openstackgerrit | Merged openstack-infra/project-config: add email notification of release job failures https://review.openstack.org/328486 | 20:03 |
| *** baoli has quit IRC | 20:04 | |
| *** baoli has joined #openstack-infra | 20:05 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Ensure ansible 2.1.0.0 is installed https://review.openstack.org/329127 | 20:05 |
| *** baoli has quit IRC | 20:05 | |
| jeblair | pabelanger: did we have a change to detect ansible exit code 3? | 20:06 |
| *** baoli has joined #openstack-infra | 20:06 | |
| *** sdake_ is now known as sdake | 20:06 | |
| *** zul has quit IRC | 20:06 | |
| jeblair | Shrews, notmorgan: ^ ? | 20:07 |
| notmorgan | jeblair: not that i am aware of | 20:07 |
| jeblair | oh! | 20:08 |
| Shrews | jeblair: that sounds familiar to me, but i didn't make the change | 20:08 |
| notmorgan | jeblair: or more to the point, not that i've seen yet. | 20:08 |
| jeblair | it landed but i think it may have a bug | 20:08 |
| notmorgan | ah | 20:08 |
| jeblair | I8c4d92fa7156c7d8591ca5b54be8cb30c249eacf | 20:08 |
| *** jcoufal_ has quit IRC | 20:09 | |
| pabelanger | jeblair: yes | 20:10 |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck: Add query for os-brick privsep sudo bug 1592043 https://review.openstack.org/329084 | 20:10 |
| openstack | bug 1592043 in os-brick "os-brick 1.4.0 increases volume setup failure rates" [Undecided,Confirmed] https://launchpad.net/bugs/1592043 | 20:10 |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck: Fix e-r with fixtures 3.0.0 https://review.openstack.org/329170 | 20:10 |
| nibalizer | rcarrillocruz: yea luppet --help cant be trusted | 20:11 |
| *** tonytan4ever has quit IRC | 20:12 | |
| *** chlong has quit IRC | 20:12 | |
| nibalizer | also the override for syslog is in system-config | 20:12 |
| pabelanger | nice: http://logs.openstack.org/48/318948/23/check/gate-nova-pep8/37d893f/console.html#_2016-06-13_20_03_22_137597 | 20:12 |
| nibalizer | we are overriding the sefualts in own own software lol | 20:12 |
| pabelanger | should maybe turn that into fqdn, but looks good | 20:13 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: fix check of ansible exit code https://review.openstack.org/329171 | 20:14 |
| jeblair | pabelanger, notmorgan, Shrews: ^ | 20:14 |
| *** hongbin has quit IRC | 20:14 | |
| pabelanger | jeblair: looking | 20:15 |
| openstackgerrit | Philip Marc Schwartz proposed openstack-infra/system-config: Move from generic exception to SSHException https://review.openstack.org/329172 | 20:15 |
| phschwartz | clarkb: notmorgan: ^ | 20:15 |
| notmorgan | oh it's not a status 3 explicitly? | 20:15 |
| jeblair | notmorgan: ansible exits with (3), we were just checking it too late (after we had coerced it to bool) | 20:16 |
| *** tongli has quit IRC | 20:16 | |
| notmorgan | ahhh | 20:16 |
| notmorgan | got it | 20:16 |
| Shrews | jeblair: would be nice if we had some docs on those methods for return values and whatnot, but that change lgtm | 20:17 |
| notmorgan | Shrews: ++ | 20:17 |
| sdague | jeblair: doh, the zuul restart was like 1 minute before the os-brick pin actually finished in check, so had to start over from scratch | 20:17 |
| pabelanger | jeblair: +2 | 20:18 |
| jeblair | sdague: it's rarely a good time for restarts. i'm aiming for rolling restarts of multiple zuul schedulers in v3.5. :) | 20:18 |
| jeblair | sdague: do you need to put that one in gate or anything? | 20:19 |
| sdague | jeblair: yeh, no worries. It was just personally painful this time, as that patch started life at 9am this morning | 20:19 |
| *** baoli has quit IRC | 20:19 | |
| openstackgerrit | Merged openstack-infra/zuul: Move a debug logging line by one https://review.openstack.org/323044 | 20:19 |
| sdague | and was approved by 10am | 20:19 |
| sdague | jeblair: yeh, if you wouldn't mind sticking it in gate - https://review.openstack.org/#/c/329069/ | 20:19 |
| sdague | it's probably fine to not promote it | 20:20 |
| sdague | oh, man, not until noon? | 20:20 |
| sdague | my sense of time today is so far off | 20:20 |
| sdague | so, sorry, my timeline is wrong, it's only been since noon. | 20:20 |
| jeblair | sdague: ah well, engatefied anyway :) | 20:21 |
| sdague | jeblair: thank you | 20:21 |
| jeblair | np | 20:21 |
| *** kgiusti has quit IRC | 20:22 | |
| jeblair | pabelanger: i'm going to try to land 171 before continuing with the zl worker start | 20:23 |
| jeblair | well, i'll stop 02, but won't do anything further | 20:24 |
| pabelanger | jeblair: ack | 20:24 |
| *** hongbin has joined #openstack-infra | 20:29 | |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: fix check of ansible exit code https://review.openstack.org/329171 | 20:29 |
| openstackgerrit | Merged openstack-infra/system-config: Add zl03 https://review.openstack.org/329109 | 20:30 |
| *** asselin has joined #openstack-infra | 20:31 | |
| openstackgerrit | C.J. Collier proposed openstack-infra/gerritbot: Improve gerritbot error handling https://review.openstack.org/287056 | 20:32 |
| lifeless | notmorgan: o/ | 20:33 |
| *** tqtran has joined #openstack-infra | 20:34 | |
| *** baoli has joined #openstack-infra | 20:35 | |
| openstackgerrit | Scott DAngelo proposed openstack-infra/devstack-gate: Allow Cinder multi-backend tests https://review.openstack.org/329178 | 20:35 |
| openstackgerrit | Scott DAngelo proposed openstack-infra/project-config: Add Cinder multi-backend for Tempest tests https://review.openstack.org/329180 | 20:38 |
| sdague | are dib images being used on rax yet? | 20:39 |
| pabelanger | yes | 20:39 |
| clarkb | sdague: they are used everywhere now | 20:39 |
| *** tlian2 has joined #openstack-infra | 20:39 | |
| pabelanger | sdague: https://review.openstack.org/#/c/329080/ also | 20:39 |
| sdague | hmmmm | 20:39 |
| sdague | ok, well that theory is out the window then | 20:39 |
| jeblair | yay? :) | 20:39 |
| sdague | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22privsep%20log%3A%20sudo%3A%20no%20tty%20present%20and%20no%20askpass%20program%20specified%5C%22%20AND%20%20%20message%3A%5C%22FailedToDropPrivileges%3A%20privsep%20helper%20command%20exited%20non-zero%20(1)%5C%22%20AND%20%20%20message%3A%5C%22get_connector_properties%5C%22%20AND%20%20%20tags%3A%5C%22screen-n-cpu.txt%5C%22 | 20:40 |
| sdague | still trying to figure out why privsep only seems to fail during grenade multinode (on new) and not on single node | 20:40 |
| clarkb | dhellmann: starting to get myself updated on what needs to happen for ironicclient, there is nothing on pypi for 0.8.2 so Ishould be able to just trigger those jobs | 20:41 |
| dhellmann | clarkb : I think the tarball job was marked as a failure. I didn't actually check if we had a valid tarball, yet, I can do that | 20:41 |
| dhellmann | clarkb : https://tarballs.openstack.org/python-ironicclient/python-ironicclient-0.8.2.tar.gz reports 404 | 20:42 |
| clarkb | dhellmann: I don't see the tarball either | 20:42 |
| openstackgerrit | Alex Kavanagh proposed openstack-infra/project-config: Add #openstack-charms and associated projects https://review.openstack.org/329129 | 20:42 |
| dhellmann | so yeah, I think start it from scratch | 20:42 |
| clarkb | so I think I need to renqueue the entire tag event | 20:42 |
| *** tonytan4ever has joined #openstack-infra | 20:42 | |
| clarkb | ya | 20:42 |
| *** tlian has quit IRC | 20:42 | |
| openstackgerrit | Jef Mallal proposed openstack-infra/jenkins-job-builder: Add support for retry in multijob plugin https://review.openstack.org/329169 | 20:43 |
| clarkb | jeblair: if I queue up this tag's jobs is now a good time to do that and avoid zuul restarts? | 20:46 |
| samueldmq | hi folks | 20:46 |
| *** ihrachys has quit IRC | 20:47 | |
| openstackgerrit | Merged openstack-infra/project-config: Add zl03 to nodepool https://review.openstack.org/329111 | 20:47 |
| samueldmq | in OpenStack deployments, are there tools that run on the log files to find information? | 20:47 |
| samueldmq | do we do this ? | 20:47 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Fix e-r with fixtures 3.0.0 https://review.openstack.org/329170 | 20:47 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for os-brick privsep sudo bug 1592043 https://review.openstack.org/329084 | 20:48 |
| openstack | bug 1592043 in os-brick "os-brick 1.4.0 increases volume setup failure rates" [Undecided,Confirmed] https://launchpad.net/bugs/1592043 | 20:48 |
| *** matrohon has quit IRC | 20:48 | |
| *** vgridnev has joined #openstack-infra | 20:48 | |
| clarkb | samueldmq: many of the CI test logs are indexed by elasticserach and we have tools that look for bug fingerprints in them to identify bug occurences. We also parse and store the subunit logs in a database so that we can query large aggregate info about test results | 20:48 |
| samueldmq | clarkb: nice; I am wondering if changing log levels could affect such tools | 20:49 |
| clarkb | we parse the log level if that is what you mean | 20:49 |
| *** julim has joined #openstack-infra | 20:50 | |
| *** ihrachys has joined #openstack-infra | 20:50 | |
| *** ddieterly is now known as ddieterly[away] | 20:50 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: switch beaker/xenial jobs in gate https://review.openstack.org/329108 | 20:50 |
| jeblair | clarkb: i'm not anticipating further zuul restarts | 20:50 |
| jeblair | clarkb: (i'm rolling launchers, but that shouldn't impact) | 20:51 |
| samueldmq | clarkb: what happens if the log level of lots of log meesages change from INFO to DEBUG? | 20:51 |
| samueldmq | clarkb: may that break such tools ? | 20:51 |
| clarkb | jeblair: thanks | 20:52 |
| pleia2 | debug wouldn't be included in what we index | 20:52 |
| clarkb | dhellmann: the ref is enqueued, and shows up in the zuul status page | 20:52 |
| samueldmq | pleia2: hmm, I am wondering if this keystone change https://review.openstack.org/#/c/329028 would impact on those tools | 20:53 |
| samueldmq | pleia2: maybe it would, since debug isn't included | 20:53 |
| *** cfarquhar has quit IRC | 20:53 | |
| clarkb | samueldmq: no the tools actively parse the data and would just write DEBUG or INFO instead of the old level when they change | 20:53 |
| samueldmq | there are several messages that are in INFO level now (and someone thinks it's too verbose), so proposing to change log level | 20:53 |
| *** psilvad has quit IRC | 20:54 | |
| samueldmq | k, just wanted to check :) | 20:54 |
| clarkb | the biggest thing you need to worry about when changing log levels like that is to not force ops or users to use DEBUG during normal operations | 20:54 |
| samueldmq | looks like I was talking about log level backward compatibility here ;) | 20:54 |
| samueldmq | clarkb: ++ | 20:54 |
| *** vgridnev has quit IRC | 20:55 | |
| *** thorst has quit IRC | 20:56 | |
| jeblair | restarting nodepool | 20:57 |
| *** Goneri has quit IRC | 20:58 | |
| *** jamesmcarthur has quit IRC | 20:58 | |
| samueldmq | clarkb: pleia2: thanks | 20:58 |
| *** gouthamr has quit IRC | 20:59 | |
| *** ericksonsantos has quit IRC | 20:59 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Add undercloud idempotency test https://review.openstack.org/279218 | 21:00 |
| *** shamail has joined #openstack-infra | 21:00 | |
| clarkb | dhellmann: ok the pypi bits claim to be done now | 21:00 |
| dhellmann | clarkb : thanks! | 21:00 |
| *** hichihara has joined #openstack-infra | 21:01 | |
| *** jamesmcarthur has joined #openstack-infra | 21:02 | |
| *** raildo is now known as raildo-afk | 21:02 | |
| *** jamesmcarthur has quit IRC | 21:02 | |
| *** thorst has joined #openstack-infra | 21:03 | |
| *** eharney has quit IRC | 21:04 | |
| *** rockyg has joined #openstack-infra | 21:04 | |
| *** trown is now known as trown|outtypewww | 21:06 | |
| *** xarses has quit IRC | 21:07 | |
| *** moravec1 has quit IRC | 21:07 | |
| *** thorst has quit IRC | 21:07 | |
| pabelanger | notmorgan: jeblair: not the time, but maybe once we replace jenkins with zuul-launcher we can talk about node states in nodepool. Specifically it is possible for a node to be READY both when zuul-launcher has been (not)assigned the node. | 21:08 |
| openstackgerrit | C.J. Collier proposed openstack-infra/gerritbot: Improve gerritbot error handling https://review.openstack.org/287056 | 21:08 |
| pabelanger | if that make sense | 21:08 |
| mriedem | clarkb: thanks | 21:09 |
| clarkb | pabelanger: we don't make the build to ready state transition when assignment happens like we do with jenkins targets? | 21:10 |
| clarkb | I thought that code was shared and its just the deails of the transition that differ | 21:11 |
| *** aysyd has quit IRC | 21:11 | |
| pabelanger | clarkb: I think that is the issue, we no longer assign the node to zuul-launcher, we assign it to zuul (target) then wait for a zuul-launcher to be online to then grab it | 21:11 |
| pabelanger | clarkb: http://grafana.openstack.org/dashboard/db/nodepool is a good example of what I currently mean | 21:11 |
| pabelanger | clarkb: we have 70+ nodes ready in nodepool but no zuul-launcher online. When it does come online they will say ready or go used | 21:12 |
| jeblair | pabelanger: oh, that's my bad, i should start zl03 | 21:13 |
| jeblair | pabelanger: no zuul launchers online is, once we finish the transition, effectively "we are stopped" so this is not a problem we need to solve | 21:13 |
| pabelanger | clarkb: I was thinking maybe we need something between building and ready for zuul-launcher, like an available state. Nodes are available to be used but not assigned to zuul-launcher | 21:13 |
| jeblair | please no | 21:13 |
| *** ihrachys has quit IRC | 21:14 | |
| pabelanger | jeblair: right, maybe we don't need to worry about it once jenkins is gone | 21:14 |
| *** ihrachys has joined #openstack-infra | 21:14 | |
| clarkb | pabelanger: gotcha, nodepool isn't aware of the launchers just of the larger "zuul" | 21:14 |
| jeblair | (unfortunately i didn't manually accept the ssh key when i launched it on the puppetmaster, so zl03 is not up to date -- that is why it's not running now) | 21:14 |
| jeblair | it would be neat if that were automated | 21:14 |
| pabelanger | clarkb: ya | 21:15 |
| jeblair | the ansible pulse is starting now, so it shouldn't be much longer | 21:15 |
| *** ldnunes has quit IRC | 21:16 | |
| openstackgerrit | Scott DAngelo proposed openstack-infra/devstack-gate: Allow Cinder multi-backend tests https://review.openstack.org/329178 | 21:16 |
| *** sputnik13 has joined #openstack-infra | 21:17 | |
| *** ddieterly[away] is now known as ddieterly | 21:18 | |
| *** xarses has joined #openstack-infra | 21:21 | |
| *** inc0 has quit IRC | 21:21 | |
| *** ffrank has joined #openstack-infra | 21:23 | |
| *** sputnik13 has quit IRC | 21:27 | |
| *** vgridnev has joined #openstack-infra | 21:28 | |
| *** gouthamr has joined #openstack-infra | 21:28 | |
| jeblair | starting zl03 now | 21:29 |
| *** bpokorny has quit IRC | 21:29 | |
| *** abregman has quit IRC | 21:30 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add zl04 to nodepool https://review.openstack.org/329194 | 21:32 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Add undercloud idempotency test https://review.openstack.org/279218 | 21:33 |
| *** julim has quit IRC | 21:33 | |
| *** sputnik13 has joined #openstack-infra | 21:33 | |
| *** sarob has quit IRC | 21:33 | |
| *** sputnik13 has quit IRC | 21:33 | |
| *** shamail has quit IRC | 21:34 | |
| *** _ari_ has quit IRC | 21:34 | |
| *** thorst has joined #openstack-infra | 21:35 | |
| *** matt-borland has quit IRC | 21:35 | |
| *** cody-somerville has quit IRC | 21:35 | |
| *** cody-somerville has joined #openstack-infra | 21:36 | |
| openstackgerrit | Merged openstack-infra/system-config: Add zl04 https://review.openstack.org/329110 | 21:37 |
| *** rhallisey has quit IRC | 21:39 | |
| *** thorst_ has joined #openstack-infra | 21:39 | |
| *** sarob has joined #openstack-infra | 21:39 | |
| jeblair | restarted zlstatic01 | 21:39 |
| *** bpokorny has joined #openstack-infra | 21:39 | |
| *** thorst has quit IRC | 21:39 | |
| *** cfarquhar has joined #openstack-infra | 21:40 | |
| *** cfarquhar has quit IRC | 21:40 | |
| *** cfarquhar has joined #openstack-infra | 21:40 | |
| *** amitgandhinz has quit IRC | 21:41 | |
| *** amitgandhinz has joined #openstack-infra | 21:42 | |
| *** sputnik13 has joined #openstack-infra | 21:42 | |
| *** thorst_ has quit IRC | 21:43 | |
| *** hashar has quit IRC | 21:43 | |
| *** csomerville has joined #openstack-infra | 21:43 | |
| *** cody-somerville has quit IRC | 21:44 | |
| *** Guest47159 has quit IRC | 21:44 | |
| *** Hal has joined #openstack-infra | 21:45 | |
| *** Hal is now known as Guest85438 | 21:45 | |
| *** rockyg has quit IRC | 21:47 | |
| *** dmk0202 has joined #openstack-infra | 21:47 | |
| *** rcernin has quit IRC | 21:48 | |
| *** sputnik13 has quit IRC | 21:49 | |
| openstackgerrit | C.J. Collier proposed openstack-infra/gerritbot: Improve gerritbot error handling https://review.openstack.org/287056 | 21:50 |
| *** bpokorny has quit IRC | 21:50 | |
| *** xarses has quit IRC | 21:51 | |
| *** mhickey has quit IRC | 21:52 | |
| *** njohnston is now known as njohnston|afk | 21:55 | |
| *** hichihara has quit IRC | 21:57 | |
| jeblair | restarted zl01 and zl02 | 21:58 |
| jeblair | pabelanger: all the launchers are up to date now | 21:58 |
| *** mdrabe has quit IRC | 21:58 | |
| *** Julien-zte has joined #openstack-infra | 21:58 | |
| pabelanger | jeblair: excellent | 21:58 |
| *** ihrachys has quit IRC | 22:00 | |
| pabelanger | jeblair: any idea why the 45~ ready nodes for zuul target are not launching? | 22:00 |
| *** bpokorny has joined #openstack-infra | 22:03 | |
| *** moravec has joined #openstack-infra | 22:05 | |
| *** ihrachys has joined #openstack-infra | 22:05 | |
| *** thiagop has quit IRC | 22:05 | |
| *** edmondsw has quit IRC | 22:07 | |
| *** xarses has joined #openstack-infra | 22:08 | |
| pabelanger | clarkb: if you have a moment, I wouldn't mind the review on 327314. Trying to bring baremetal00.vanilla online | 22:08 |
| *** ddieterly has quit IRC | 22:10 | |
| clarkb | pabelanger: +2, but not approved. I will let you do that when you can make sure puppet and ansible are all happy | 22:10 |
| clarkb | what is the plan though | 22:10 |
| clarkb | are we just going to deploy then redeploy after the next move or? | 22:10 |
| pabelanger | clarkb: I wouldn't mind seeing if upgrading openstack works as expected. I think we have bits not running liberty still | 22:12 |
| pabelanger | clarkb: but ya, redeploy see if we have bugs, to prepare for the move again | 22:12 |
| crinkle | pabelanger: if you want to see it upgraded want to look at 304946 312318 and 312319 for me? | 22:14 |
| crinkle | or do we want to just stay on liberty for a while | 22:14 |
| *** rhallisey has joined #openstack-infra | 22:14 | |
| *** amotoki has quit IRC | 22:17 | |
| *** ihrachys_ has joined #openstack-infra | 22:18 | |
| *** sputnik13 has joined #openstack-infra | 22:18 | |
| *** ihrachys has quit IRC | 22:20 | |
| *** chlong has joined #openstack-infra | 22:21 | |
| *** tonytan4ever has quit IRC | 22:23 | |
| jeblair | pabelanger: yep, those are nodes that were pending assignment when nodepool restarted. we could probaby have nodepool delete those on startup. | 22:23 |
| *** Guest85438 has quit IRC | 22:23 | |
| jeblair | pabelanger: that might make a compelling argument for your extra state :) [though it's not a pre-requisite, just makes it a little more clear] | 22:24 |
| *** ajmiller_ has quit IRC | 22:24 | |
| *** ajmiller has joined #openstack-infra | 22:24 | |
| jeblair | (though, in practice, this number should be quite small) | 22:24 |
| *** gordc has quit IRC | 22:24 | |
| jeblair | i will delete those manually now | 22:24 |
| *** ihrachys_ has quit IRC | 22:25 | |
| *** ihrachys has joined #openstack-infra | 22:27 | |
| *** ayoung has quit IRC | 22:27 | |
| *** mriedem is now known as mriedem_away | 22:29 | |
| *** bhavik has quit IRC | 22:30 | |
| *** burgerk has quit IRC | 22:30 | |
| *** EricGonczer_ has quit IRC | 22:31 | |
| *** jamesmcarthur has joined #openstack-infra | 22:31 | |
| *** sputnik13 has quit IRC | 22:31 | |
| *** ihrachys has quit IRC | 22:33 | |
| *** akshai has joined #openstack-infra | 22:36 | |
| *** sputnik13 has joined #openstack-infra | 22:36 | |
| *** bhavik has joined #openstack-infra | 22:36 | |
| clarkb | does assignment require enough of a handshake that we could just reassign them on startup? | 22:36 |
| *** jamesmcarthur has quit IRC | 22:37 | |
| *** bhavik1 has joined #openstack-infra | 22:37 | |
| *** dimtruck is now known as zz_dimtruck | 22:37 | |
| *** tkelsey has joined #openstack-infra | 22:37 | |
| *** sputnik13 has quit IRC | 22:38 | |
| *** ihrachys has joined #openstack-infra | 22:39 | |
| *** bhavik has quit IRC | 22:40 | |
| *** bhavik1 is now known as bhavik | 22:40 | |
| *** tkelsey has quit IRC | 22:42 | |
| jeblair | clarkb: we could probably do that | 22:42 |
| *** akshai has quit IRC | 22:45 | |
| *** ihrachys has quit IRC | 22:48 | |
| Nakato | jeblair: Could I get you got follow up on https://review.openstack.org/#/c/293194/ this week? | 22:48 |
| *** rbrndt has quit IRC | 22:50 | |
| *** rbrndt has joined #openstack-infra | 22:51 | |
| *** rbrndt has quit IRC | 22:51 | |
| *** salv-orlando has quit IRC | 22:51 | |
| *** ihrachys has joined #openstack-infra | 22:52 | |
| *** dmk0202 has quit IRC | 22:52 | |
| *** salv-orlando has joined #openstack-infra | 22:52 | |
| pabelanger | crinkle: Sure, I'll poke at them shortly | 22:54 |
| *** jamesmcarthur has joined #openstack-infra | 22:54 | |
| pabelanger | jeblair: sure, we can figure it out once jenkins is replaced | 22:55 |
| openstackgerrit | Merged openstack-infra/project-config: Add zl04 to nodepool https://review.openstack.org/329194 | 22:55 |
| pabelanger | I believe our ftp publish is now working too | 22:57 |
| pabelanger | 2016-06-13 22:40:48,574 DEBUG zuul.NodeWorker.ubuntu-trusty-ovh-bhs1-1636098: Ansible post output: changed: [node -> 127.0.0.1] => {"changed": true, "cmd": "lftp -f /tmp/tmp2rANyq/ansible/tmp2loe0C/script", "delta": "0:00:04.117569", "end": "2016-06-13 22:40:48.552464", "rc": 0, "start": "2016-06-13 22:40:44.434895", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []} | 22:57 |
| *** PollyZ_ has joined #openstack-infra | 22:58 | |
| jhesketh | Morning | 23:00 |
| *** ihrachys has quit IRC | 23:01 | |
| *** hongbin has quit IRC | 23:01 | |
| *** PollyZ has quit IRC | 23:01 | |
| *** ihrachys has joined #openstack-infra | 23:01 | |
| *** PollyZ_ has quit IRC | 23:02 | |
| *** sheeprine has quit IRC | 23:03 | |
| pabelanger | crinkle: all look good. Lets confirm with infra-root we want to move to mitaka, then we can land the 2 patches. Maybe had quick topic for tomorrows meeting? | 23:05 |
| dstufft | fungi: happy to hear Warehouse has been working well :3 | 23:05 |
| crinkle | pabelanger: sure | 23:05 |
| *** xyang1 has quit IRC | 23:05 | |
| pleia2 | mitaka++ | 23:06 |
| pleia2 | the puppet modules have worked well for everything I've done with them so far | 23:06 |
| pleia2 | (using 14.04) | 23:06 |
| *** sheeprine has joined #openstack-infra | 23:06 | |
| *** jamesmcarthur has quit IRC | 23:07 | |
| *** Daisy has joined #openstack-infra | 23:07 | |
| *** pradk has quit IRC | 23:07 | |
| craige | o/ | 23:08 |
| openstackgerrit | Merged openstack-infra/system-config: Bring baremetal00.vanilla.ic.openstack.org online https://review.openstack.org/327314 | 23:09 |
| *** amitgandhinz has quit IRC | 23:09 | |
| *** jamesmcarthur has joined #openstack-infra | 23:10 | |
| *** amitgandhinz has joined #openstack-infra | 23:11 | |
| *** jamesmcarthur has quit IRC | 23:11 | |
| *** yamahata has quit IRC | 23:11 | |
| *** Daisy has quit IRC | 23:12 | |
| *** akshai has joined #openstack-infra | 23:13 | |
| openstackgerrit | C.J. Collier proposed openstack-infra/gerritbot: Improve gerritbot error handling https://review.openstack.org/287056 | 23:14 |
| jeblair | Nakato: cmd.cloner shouldn't have any logic in it other than that involved in translating environment and command line arguments into argument for lib.cloner. otherwise we create a situation where the command as run from the shell differs in behavior from what is tested by the unit tests (and any other potential users of the cloner) | 23:15 |
| openstackgerrit | Merged openstack-infra/puppet-infracloud: Fix ssl_key relationships https://review.openstack.org/312318 | 23:15 |
| *** EricGonczer_ has joined #openstack-infra | 23:17 | |
| *** ihrachys has quit IRC | 23:17 | |
| *** tlian2 has quit IRC | 23:20 | |
| *** Julien-zte has quit IRC | 23:21 | |
| *** ihrachys has joined #openstack-infra | 23:22 | |
| lifeless | notmorgan: do you want to arrange a hangout or other high bw thing? | 23:23 |
| *** akshai_ has joined #openstack-infra | 23:24 | |
| *** akshai has quit IRC | 23:24 | |
| *** EricGonczer_ has quit IRC | 23:25 | |
| *** EricGonczer_ has joined #openstack-infra | 23:25 | |
| *** tpsilva has quit IRC | 23:27 | |
| *** salv-orl_ has joined #openstack-infra | 23:28 | |
| *** dingyichen has joined #openstack-infra | 23:30 | |
| *** Swami has quit IRC | 23:31 | |
| *** salv-orlando has quit IRC | 23:31 | |
| pabelanger | jeblair: nodepoold appears to be stopped, should it be? | 23:31 |
| *** ihrachys has quit IRC | 23:33 | |
| *** r-daneel has quit IRC | 23:33 | |
| *** ihrachys has joined #openstack-infra | 23:33 | |
| *** ayoung has joined #openstack-infra | 23:33 | |
| *** EricGonczer_ has quit IRC | 23:34 | |
| jeblair | pabelanger: expected. i just restarted it. | 23:34 |
| jeblair | (added zl04) | 23:34 |
| pabelanger | jeblair: cool! | 23:34 |
| pabelanger | also \o/ for zl04 | 23:35 |
| anteaya | morning jhesketh | 23:37 |
| anteaya | pleia2: yay | 23:37 |
| anteaya | jeblair: yay | 23:37 |
| anteaya | if someone is bored and feeling like taking a peek at the storyboard-dev logs, I would be interested in hearing if my local gerrit instance hit the api at all in the last 2 hours or so | 23:38 |
| *** gouthamr_ has joined #openstack-infra | 23:38 | |
| *** sdague has quit IRC | 23:38 | |
| anteaya | I'm not seeing what I expect yet, but I don't even know if it is hitting the storyboard-dev api | 23:39 |
| pleia2 | I can have a look | 23:40 |
| anteaya | pleia2: thanks | 23:41 |
| pleia2 | anteaya: do you know your ip address? | 23:41 |
| anteaya | 192.168.122.94 | 23:42 |
| *** gouthamr has quit IRC | 23:42 | |
| pleia2 | anteaya: that's the address on your LAN | 23:42 |
| anteaya | that is the ip of the vm running locally on my laptop | 23:42 |
| anteaya | how do I get you what you need, the ip of my laptop? | 23:42 |
| pleia2 | that one won't be in the logs, I need the one that you're connected to the internet with | 23:42 |
| pleia2 | are you sshed into a server for IRC? | 23:42 |
| *** akshai_ has quit IRC | 23:42 | |
| anteaya | pleia2: 76.75.113.51 | 23:43 |
| pleia2 | on the server you're on, you can type "who" into a shell to see what address you're connected with | 23:43 |
| pleia2 | ah ok | 23:43 |
| anteaya | pleia2: no I'm not, I'm running xchat on my laptop | 23:43 |
| pleia2 | oh alright, I could have just whoised you then :) | 23:44 |
| anteaya | who doesn't get me anything useful | 23:44 |
| pabelanger | I recently found ifconfig.co helpful | 23:44 |
| pleia2 | anteaya: yep, I'm seeing it hit the api | 23:44 |
| pabelanger | you could do: curl ifconfig.co from a CLI and should get back your IP address | 23:44 |
| anteaya | pabelanger: yup, ifconfig was what i used | 23:44 |
| anteaya | pleia2: yay, can you share what it did? | 23:45 |
| anteaya | progress at least | 23:45 |
| pleia2 | grepped for your address in /var/log/apache2/storyboard-ssl-access.log | 23:45 |
| anteaya | pleia2: cool | 23:45 |
| anteaya | and it was there! | 23:45 |
| pleia2 | ifconfig.co is a website, there are several like whatismyip.com and such | 23:45 |
| pabelanger | anteaya: Great, I find myself using it more and more now. | 23:45 |
| anteaya | ah I see | 23:45 |
| anteaya | oh I just ran ifconfig at the command line | 23:46 |
| *** amotoki has joined #openstack-infra | 23:46 | |
| pleia2 | that only works if your laptop is connected directly to the internet, wouldn't work on my LAN where all machines have local addresses | 23:46 |
| anteaya | pleia2: ah | 23:46 |
| jeblair | https://www.google.com/search?q=ip%20address also works for me | 23:46 |
| pleia2 | anyway, when I looked at the logs I saw your address hitting the /api urls | 23:46 |
| anteaya | thanks | 23:47 |
| *** ihrachys has quit IRC | 23:47 | |
| anteaya | pleia2: what was I doing or trying to do? and when did I hit it last? | 23:47 |
| pleia2 | anteaya: want me to pastebin some lines? | 23:47 |
| anteaya | pleia2: that would be ideal, thank you | 23:47 |
| *** ihrachys has joined #openstack-infra | 23:47 | |
| *** johnny_ has quit IRC | 23:48 | |
| pleia2 | anteaya: actually, now I'm not sure, it's showing useragent as mozilla, are you accessing the API with mozilla too? | 23:48 |
| pleia2 | s/mozilla/firefox | 23:48 |
| pleia2 | http://paste.openstack.org/show/515857/ | 23:49 |
| anteaya | pleia2: no, I did on Friday or Thursday last week via the browser | 23:49 |
| anteaya | but tried about 90 minutes ago with my local instance | 23:49 |
| anteaya | well that is from today, so yay! | 23:50 |
| pleia2 | this is my first time looking at storyboard apache logs, so I'm not sure what's normal for browsing it vs using the api | 23:50 |
| jeblair | pabelanger: (i only added it as a listener to nodepool, i have not started zl04 yet | 23:50 |
| *** amotoki has quit IRC | 23:50 | |
| pabelanger | jeblair: ack | 23:50 |
| anteaya | pleia2: thanks for looking, so only GETs, no POSTs | 23:50 |
| *** yamahata has joined #openstack-infra | 23:51 | |
| anteaya | GET is a start but I am hoping for some POST activity at some point | 23:51 |
| pleia2 | anteaya: just checked, grep isn't returning any POSTs | 23:51 |
| anteaya | pleia2: okay thanks | 23:51 |
| *** bhavik has quit IRC | 23:51 | |
| Nakato | jeblair: At this point that's all it does. I can't only pass NewRev without the Project environment variable, which currently is not passed as the user is required to pass it on the command line and is a list, which means I'm passing multiple shell values into the lib, which is fine, but only doing so to immediatly turn them into something usable, and appears to contradict the current lib and cmd seperation(project_branches). It also means if | 23:51 |
| Nakato | in the future we wanted to support multiple checkouts we would end up passing the argparse value directly into the lib to parse there, or bringing it back out to where it is now. As well, potential consumers couldn't easily do multiple rev based checkouts as we've made that singular by hiding that behind shell logic in the lib, which contradicts how lib.cloner does multiple checkouts in a single call. I'll change it if you still think it should | 23:51 |
| Nakato | be moved, but I do think these issues should brought up before I do so. | 23:51 |
| anteaya | well at least my local instance can find storyboard-dev | 23:51 |
| anteaya | so yay | 23:51 |
| pleia2 | seems so :) | 23:52 |
| *** EricGonczer_ has joined #openstack-infra | 23:53 | |
| anteaya | and that all seems to be one incident based on timestamps | 23:53 |
| *** bhavik has joined #openstack-infra | 23:53 | |
| anteaya | which corresponds with my last comment on my patchset in my local instance | 23:53 |
| anteaya | which means that it seems to be triggering on comments added to a patch that has Story: <story-id> in the commit message | 23:54 |
| *** weshay has quit IRC | 23:54 | |
| anteaya | so yay, this is awesome | 23:54 |
| *** ihrachys has quit IRC | 23:55 | |
| anteaya | zaro: so right now the plugin seems to be finding the correct story and GETing useful things but I don't seem to be PUTing or POSTing yet | 23:55 |
| jeblair | Nakato: yeah, i think both project and newrev should be passed in as they would now be required inputs to the cloner. and that's why i think they belong there -- because they are both necessary for the cloner to decide what to do. | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!