| *** zhurong has quit IRC | 00:00 | |
| *** gouthamr has joined #openstack-infra | 00:01 | |
| *** nwkarsten has joined #openstack-infra | 00:03 | |
| *** thorst has joined #openstack-infra | 00:05 | |
| *** thorst has quit IRC | 00:06 | |
| *** thorst has joined #openstack-infra | 00:10 | |
| *** camunoz has joined #openstack-infra | 00:15 | |
| *** vhosakot has joined #openstack-infra | 00:15 | |
| *** thorst has quit IRC | 00:15 | |
| *** fguillot has joined #openstack-infra | 00:18 | |
| *** Jeffrey4l_ has joined #openstack-infra | 00:20 | |
| *** Jeffrey4l__ has quit IRC | 00:20 | |
| *** baoli has joined #openstack-infra | 00:22 | |
| *** gnuoy has quit IRC | 00:23 | |
| *** vhosakot has quit IRC | 00:25 | |
| *** gnuoy has joined #openstack-infra | 00:26 | |
| *** zxiiro-away is now known as zxiiro | 00:33 | |
| *** fguillot has quit IRC | 00:33 | |
| *** yamamoto_ has joined #openstack-infra | 00:40 | |
| *** morgabra has quit IRC | 00:41 | |
| *** morgabra has joined #openstack-infra | 00:41 | |
| *** fguillot has joined #openstack-infra | 00:46 | |
| *** adrian_otto has joined #openstack-infra | 00:47 | |
| *** yamamoto_ has quit IRC | 00:48 | |
| *** shashank_hegde has quit IRC | 00:48 | |
| *** thorst has joined #openstack-infra | 00:50 | |
| *** adrian_otto has quit IRC | 00:50 | |
| *** thorst has quit IRC | 00:50 | |
| *** thorst has joined #openstack-infra | 00:51 | |
| *** tphummel has joined #openstack-infra | 00:57 | |
| *** thorst has quit IRC | 00:59 | |
| *** Goneri has joined #openstack-infra | 01:00 | |
| *** simonmcc has quit IRC | 01:09 | |
| *** sridhar_ram_afk has quit IRC | 01:09 | |
| *** vdrok has quit IRC | 01:09 | |
| *** tphummel has quit IRC | 01:10 | |
| *** zhurong has joined #openstack-infra | 01:10 | |
| *** serverascode has quit IRC | 01:10 | |
| *** ameade has quit IRC | 01:10 | |
| *** tphummel has joined #openstack-infra | 01:10 | |
| *** skath has quit IRC | 01:10 | |
| *** aimeeu has quit IRC | 01:10 | |
| *** jraim has quit IRC | 01:10 | |
| *** TheJulia has quit IRC | 01:10 | |
| *** maximov has quit IRC | 01:10 | |
| *** tdurakov has quit IRC | 01:10 | |
| *** ashtokolov has quit IRC | 01:11 | |
| *** eeiden has quit IRC | 01:11 | |
| *** flaviof has quit IRC | 01:11 | |
| *** fdegir has quit IRC | 01:11 | |
| *** rpodolyaka has quit IRC | 01:11 | |
| *** scottda has quit IRC | 01:11 | |
| *** betherly has quit IRC | 01:11 | |
| *** zhiyan has quit IRC | 01:11 | |
| *** vinaypotluri has joined #openstack-infra | 01:13 | |
| *** mtanino has joined #openstack-infra | 01:16 | |
| *** yamamoto_ has joined #openstack-infra | 01:17 | |
| *** salv-orlando has joined #openstack-infra | 01:17 | |
| *** yanyanhu has joined #openstack-infra | 01:18 | |
| *** TheJulia has joined #openstack-infra | 01:19 | |
| *** ashtokolov has joined #openstack-infra | 01:20 | |
| *** yamamoto_ has quit IRC | 01:22 | |
| *** salv-orlando has quit IRC | 01:22 | |
| *** scottda has joined #openstack-infra | 01:23 | |
| *** rpodolyaka has joined #openstack-infra | 01:23 | |
| *** maximov has joined #openstack-infra | 01:23 | |
| *** fdegir has joined #openstack-infra | 01:24 | |
| *** vdrok has joined #openstack-infra | 01:25 | |
| *** aimeeu has joined #openstack-infra | 01:25 | |
| *** simonmcc has joined #openstack-infra | 01:25 | |
| *** ameade has joined #openstack-infra | 01:25 | |
| *** sridhar_ram_afk has joined #openstack-infra | 01:25 | |
| *** sdake_ has quit IRC | 01:25 | |
| *** tdurakov has joined #openstack-infra | 01:26 | |
| *** flaviof has joined #openstack-infra | 01:26 | |
| *** skath has joined #openstack-infra | 01:26 | |
| *** kzaitsev_mb has quit IRC | 01:26 | |
| *** serverascode has joined #openstack-infra | 01:26 | |
| *** tphummel has quit IRC | 01:27 | |
| *** betherly has joined #openstack-infra | 01:27 | |
| *** zhiyan has joined #openstack-infra | 01:27 | |
| *** jraim has joined #openstack-infra | 01:28 | |
| *** jamielennox is now known as jamielennox|away | 01:30 | |
| *** sdake has joined #openstack-infra | 01:30 | |
| *** thorst has joined #openstack-infra | 01:32 | |
| *** thorst has quit IRC | 01:32 | |
| *** baoli has quit IRC | 01:36 | |
| *** kaisers_ has joined #openstack-infra | 01:36 | |
| *** thorst has joined #openstack-infra | 01:37 | |
| *** caowei has joined #openstack-infra | 01:38 | |
| *** kaisers_ has quit IRC | 01:41 | |
| *** thorst has quit IRC | 01:42 | |
| *** jamielennox|away is now known as jamielennox | 01:43 | |
| *** baoli has joined #openstack-infra | 01:44 | |
| *** Goneri has quit IRC | 01:46 | |
| *** rkukura has quit IRC | 01:46 | |
| *** gildub has joined #openstack-infra | 01:47 | |
| *** baoli has quit IRC | 01:49 | |
| *** Goneri has joined #openstack-infra | 01:52 | |
| *** thorst has joined #openstack-infra | 01:52 | |
| *** thorst has quit IRC | 01:54 | |
| *** baoli has joined #openstack-infra | 01:56 | |
| *** zhurong_ has joined #openstack-infra | 02:02 | |
| *** zhurong has quit IRC | 02:06 | |
| *** tphummel has joined #openstack-infra | 02:13 | |
| *** rkukura has joined #openstack-infra | 02:14 | |
| *** yamahata has quit IRC | 02:14 | |
| *** yamamoto_ has joined #openstack-infra | 02:22 | |
| *** amotoki has joined #openstack-infra | 02:24 | |
| *** adrian_otto has joined #openstack-infra | 02:24 | |
| *** salv-orlando has joined #openstack-infra | 02:24 | |
| *** salv-orlando has quit IRC | 02:27 | |
| *** adrian_otto has quit IRC | 02:28 | |
| *** yamamoto_ has quit IRC | 02:29 | |
| *** Goneri has quit IRC | 02:30 | |
| *** sarob has joined #openstack-infra | 02:32 | |
| *** yamamoto_ has joined #openstack-infra | 02:33 | |
| *** fguillot has quit IRC | 02:33 | |
| *** apetrich has quit IRC | 02:38 | |
| *** tphummel has quit IRC | 02:39 | |
| *** senk has joined #openstack-infra | 02:39 | |
| *** apetrich has joined #openstack-infra | 02:39 | |
| *** yamamoto_ has quit IRC | 02:42 | |
| *** tphummel has joined #openstack-infra | 02:47 | |
| *** yuanying has quit IRC | 02:51 | |
| *** yuanying has joined #openstack-infra | 02:52 | |
| *** thorst has joined #openstack-infra | 02:55 | |
| *** thorst has quit IRC | 03:03 | |
| *** tphummel has quit IRC | 03:06 | |
| *** jerryz has joined #openstack-infra | 03:07 | |
| *** sarob has quit IRC | 03:17 | |
| *** jamielennox is now known as jamielennox|away | 03:21 | |
| *** amotoki has quit IRC | 03:21 | |
| *** kaisers_ has joined #openstack-infra | 03:26 | |
| *** markvoelker has joined #openstack-infra | 03:27 | |
| *** markvoelker_ has joined #openstack-infra | 03:29 | |
| *** kaisers_ has quit IRC | 03:30 | |
| *** markvoelker has quit IRC | 03:33 | |
| *** binbincong has joined #openstack-infra | 03:33 | |
| *** changzhi has joined #openstack-infra | 03:35 | |
| *** adrian_otto has joined #openstack-infra | 03:38 | |
| *** adrian_otto has quit IRC | 03:39 | |
| *** amotoki has joined #openstack-infra | 03:40 | |
| *** vikrant has joined #openstack-infra | 03:41 | |
| *** yamamoto_ has joined #openstack-infra | 03:43 | |
| *** yuanying has quit IRC | 03:44 | |
| *** adrian_otto has joined #openstack-infra | 03:46 | |
| *** yamamoto_ has quit IRC | 03:47 | |
| *** adrian_otto has quit IRC | 03:51 | |
| *** adrian_otto has joined #openstack-infra | 03:52 | |
| *** markvoelker_ has quit IRC | 03:53 | |
| *** markvoelker has joined #openstack-infra | 03:54 | |
| *** adrian_otto has quit IRC | 03:56 | |
| *** jamielennox|away is now known as jamielennox | 03:56 | |
| *** roxanaghe has joined #openstack-infra | 03:57 | |
| *** vikrant is now known as vikrant|brb | 03:58 | |
| *** gouthamr has quit IRC | 04:01 | |
| *** thorst has joined #openstack-infra | 04:01 | |
| *** shashank_hegde has joined #openstack-infra | 04:05 | |
| *** roxanaghe has quit IRC | 04:06 | |
| *** thorst has quit IRC | 04:09 | |
| *** chlong has quit IRC | 04:12 | |
| *** tonytan4ever has quit IRC | 04:13 | |
| *** vikrant|brb is now known as vikrant | 04:14 | |
| *** yamamoto_ has joined #openstack-infra | 04:17 | |
| *** salv-orlando has joined #openstack-infra | 04:18 | |
| *** roxanaghe has joined #openstack-infra | 04:19 | |
| *** senk has quit IRC | 04:19 | |
| *** yamamoto_ has quit IRC | 04:20 | |
| *** yamamoto_ has joined #openstack-infra | 04:20 | |
| *** amotoki has quit IRC | 04:21 | |
| *** yamamoto_ has quit IRC | 04:22 | |
| *** salv-orlando has quit IRC | 04:23 | |
| *** chlong has joined #openstack-infra | 04:26 | |
| *** coolsvap_ has joined #openstack-infra | 04:28 | |
| *** coolsvap_ is now known as coolsvap_\ | 04:28 | |
| *** coolsvap_\ is now known as coolsvap | 04:28 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 04:30 |
|---|---|---|
| *** links has joined #openstack-infra | 04:33 | |
| *** csomerville has joined #openstack-infra | 04:38 | |
| *** cody-somerville has joined #openstack-infra | 04:40 | |
| *** amotoki has joined #openstack-infra | 04:42 | |
| *** csomerville has quit IRC | 04:43 | |
| *** amotoki has quit IRC | 04:48 | |
| *** jaosorior has joined #openstack-infra | 04:52 | |
| *** senk has joined #openstack-infra | 04:57 | |
| *** amotoki has joined #openstack-infra | 05:03 | |
| *** javeriak has joined #openstack-infra | 05:03 | |
| *** yuanying has joined #openstack-infra | 05:03 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:04 | |
| *** thorst has joined #openstack-infra | 05:07 | |
| *** yamamoto_ has joined #openstack-infra | 05:07 | |
| *** sdake_ has joined #openstack-infra | 05:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Add timestamp.txt for rsync mirrors https://review.openstack.org/360748 | 05:12 |
| *** cody-somerville has quit IRC | 05:12 | |
| *** sdake_ is now known as dake | 05:13 | |
| *** sdake has quit IRC | 05:13 | |
| *** dake is now known as sdake | 05:13 | |
| *** kzaitsev_mb has quit IRC | 05:13 | |
| *** tonytan4ever has joined #openstack-infra | 05:13 | |
| *** thorst has quit IRC | 05:14 | |
| *** kaisers_ has joined #openstack-infra | 05:14 | |
| *** jerryz has quit IRC | 05:15 | |
| *** mtanino has quit IRC | 05:15 | |
| *** yamamoto_ has quit IRC | 05:16 | |
| *** shardy has joined #openstack-infra | 05:17 | |
| *** adriant has quit IRC | 05:17 | |
| openstackgerrit | Merged openstack-infra/system-config: Fix URL in gerrit tuning comments https://review.openstack.org/360726 | 05:18 |
| *** tonytan4ever has quit IRC | 05:18 | |
| *** kaisers_ has quit IRC | 05:19 | |
| *** jerryz has joined #openstack-infra | 05:19 | |
| *** yamamoto_ has joined #openstack-infra | 05:20 | |
| *** changzhi has quit IRC | 05:23 | |
| *** pgadiya has joined #openstack-infra | 05:24 | |
| *** javeriak has quit IRC | 05:24 | |
| *** binbincong has quit IRC | 05:25 | |
| *** jianghuaww has joined #openstack-infra | 05:26 | |
| *** sdake has quit IRC | 05:26 | |
| *** zshuo has joined #openstack-infra | 05:30 | |
| *** jianghuaww has quit IRC | 05:30 | |
| *** jianghuaw_ has joined #openstack-infra | 05:31 | |
| *** ilyashakhat has joined #openstack-infra | 05:31 | |
| *** psachin has joined #openstack-infra | 05:33 | |
| *** markvoelker has quit IRC | 05:36 | |
| *** salv-orlando has joined #openstack-infra | 05:36 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 05:37 |
| *** roxanaghe has quit IRC | 05:39 | |
| *** jianghuaw has joined #openstack-infra | 05:39 | |
| *** florianf has joined #openstack-infra | 05:40 | |
| *** yamamoto_ has quit IRC | 05:44 | |
| *** yamamoto_ has joined #openstack-infra | 05:45 | |
| *** jianghuaw has quit IRC | 05:45 | |
| *** jianghuaw_ is now known as jianghuaw | 05:45 | |
| *** sandanar has joined #openstack-infra | 05:47 | |
| *** yamamoto_ has quit IRC | 05:49 | |
| *** yamamoto_ has joined #openstack-infra | 05:49 | |
| *** sridhar_ram_afk is now known as sridhar_ram | 05:51 | |
| *** bexelbie has quit IRC | 05:52 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 05:55 |
| *** ilyashakhat has quit IRC | 05:55 | |
| *** bexelbie has joined #openstack-infra | 05:55 | |
| *** adrian_otto has joined #openstack-infra | 05:56 | |
| *** yamamoto_ has quit IRC | 05:57 | |
| *** adrian_otto has quit IRC | 06:04 | |
| *** salv-orlando has quit IRC | 06:05 | |
| *** watanabe_isao has joined #openstack-infra | 06:06 | |
| *** markvoelker has joined #openstack-infra | 06:07 | |
| *** salv-orlando has joined #openstack-infra | 06:07 | |
| *** zshuo has quit IRC | 06:07 | |
| *** esikachev has joined #openstack-infra | 06:09 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 06:10 |
| *** jerryz has quit IRC | 06:11 | |
| *** markvoelker has quit IRC | 06:11 | |
| *** thorst has joined #openstack-infra | 06:12 | |
| *** sandanar has quit IRC | 06:12 | |
| *** matrohon has quit IRC | 06:17 | |
| *** thorst has quit IRC | 06:19 | |
| *** changzhi has joined #openstack-infra | 06:21 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 06:21 |
| *** yuanying has quit IRC | 06:29 | |
| *** amotoki has quit IRC | 06:29 | |
| *** ilyashakhat has joined #openstack-infra | 06:29 | |
| *** esikachev has quit IRC | 06:30 | |
| *** ggnel_t has joined #openstack-infra | 06:34 | |
| *** ilyashakhat has quit IRC | 06:36 | |
| *** roxanaghe has joined #openstack-infra | 06:36 | |
| *** markvoelker has joined #openstack-infra | 06:36 | |
| *** roxanaghe has quit IRC | 06:40 | |
| *** markvoelker has quit IRC | 06:41 | |
| *** javeriak has joined #openstack-infra | 06:45 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 06:45 |
| *** kaisers_ has joined #openstack-infra | 06:45 | |
| *** rcernin has joined #openstack-infra | 06:47 | |
| *** andreas_s has joined #openstack-infra | 06:49 | |
| *** amotoki has joined #openstack-infra | 06:50 | |
| penguinolog | Hello | 06:53 |
| *** camunoz has quit IRC | 06:54 | |
| *** Hal1 has joined #openstack-infra | 06:57 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 06:57 |
| *** amotoki has quit IRC | 06:58 | |
| *** yamamoto_ has joined #openstack-infra | 06:58 | |
| *** pcaruana has joined #openstack-infra | 07:00 | |
| *** tesseract- has joined #openstack-infra | 07:01 | |
| *** yamamoto_ has quit IRC | 07:02 | |
| *** zshuo has joined #openstack-infra | 07:03 | |
| *** yamamoto_ has joined #openstack-infra | 07:04 | |
| *** markvoelker has joined #openstack-infra | 07:06 | |
| *** markvoelker has quit IRC | 07:12 | |
| *** yamamoto_ has quit IRC | 07:13 | |
| *** shashank_hegde has quit IRC | 07:13 | |
| *** shashank_hegde has joined #openstack-infra | 07:15 | |
| *** ifarkas_afk is now known as ifarkas | 07:16 | |
| *** dizquierdo has joined #openstack-infra | 07:17 | |
| *** thorst has joined #openstack-infra | 07:17 | |
| *** martinkopec has joined #openstack-infra | 07:17 | |
| penguinolog | review, please https://review.openstack.org/#/c/361843/ - we started fuel-qa refactoring and need coverage job to prevent changes without unit tests | 07:17 |
| *** nmagnezi has joined #openstack-infra | 07:17 | |
| *** yaume has joined #openstack-infra | 07:17 | |
| *** dtardivel has joined #openstack-infra | 07:17 | |
| *** yamamoto_ has joined #openstack-infra | 07:19 | |
| *** amotoki has joined #openstack-infra | 07:19 | |
| *** _nadya_ has joined #openstack-infra | 07:21 | |
| *** _nadya_ has quit IRC | 07:21 | |
| *** _nadya_ has joined #openstack-infra | 07:21 | |
| *** shashank_hegde has quit IRC | 07:23 | |
| *** thorst has quit IRC | 07:24 | |
| *** Na3iL has joined #openstack-infra | 07:26 | |
| *** shardy has quit IRC | 07:26 | |
| *** r-mibu has quit IRC | 07:27 | |
| *** acabot has joined #openstack-infra | 07:28 | |
| *** martinkopec has quit IRC | 07:28 | |
| *** acabot has quit IRC | 07:29 | |
| *** martinkopec has joined #openstack-infra | 07:29 | |
| *** acabot has joined #openstack-infra | 07:29 | |
| *** yamamoto_ has quit IRC | 07:32 | |
| *** matrohon has joined #openstack-infra | 07:34 | |
| *** yamamoto_ has joined #openstack-infra | 07:34 | |
| *** yamamoto_ has quit IRC | 07:34 | |
| *** drifterza has joined #openstack-infra | 07:35 | |
| *** amotoki has quit IRC | 07:36 | |
| *** markvoelker has joined #openstack-infra | 07:36 | |
| *** nmagnezi has quit IRC | 07:37 | |
| *** jpich has joined #openstack-infra | 07:37 | |
| *** esikachev has joined #openstack-infra | 07:40 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 07:41 |
| *** markvoelker has quit IRC | 07:41 | |
| *** hashar has joined #openstack-infra | 07:42 | |
| *** esikachev has quit IRC | 07:43 | |
| *** esikachev has joined #openstack-infra | 07:43 | |
| *** amotoki has joined #openstack-infra | 07:45 | |
| *** Sukhdev has quit IRC | 07:46 | |
| AJaeger | rcarrillocruz, ianw, jhesketh : If either of you is around and has some time, please review https://review.openstack.org/#/c/361475/ | 07:47 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Increase timeout for ironic power status report https://review.openstack.org/361915 | 07:48 |
| *** nmagnezi has joined #openstack-infra | 07:48 | |
| *** pilgrimstack has joined #openstack-infra | 07:49 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Increase timeout for ironic power status report https://review.openstack.org/361915 | 07:51 |
| *** yuanying has joined #openstack-infra | 07:51 | |
| *** bexelbie has quit IRC | 07:55 | |
| *** bexelbie has joined #openstack-infra | 07:55 | |
| *** dingyichen has quit IRC | 07:58 | |
| *** drifterza_ has joined #openstack-infra | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** yamamoto_ has joined #openstack-infra | 08:01 | |
| *** zzzeek has joined #openstack-infra | 08:01 | |
| *** martinkopec_ has joined #openstack-infra | 08:01 | |
| *** mikelk has joined #openstack-infra | 08:01 | |
| *** mkopec__ has joined #openstack-infra | 08:02 | |
| *** martinkopec has quit IRC | 08:02 | |
| *** mikelk has quit IRC | 08:02 | |
| *** mikelk has joined #openstack-infra | 08:02 | |
| *** martinkopec_ has quit IRC | 08:02 | |
| *** drifterza has quit IRC | 08:02 | |
| *** drifterza__ has joined #openstack-infra | 08:02 | |
| *** r-mibu has joined #openstack-infra | 08:03 | |
| *** drifterza__ is now known as drifterza | 08:03 | |
| *** yamamoto_ has quit IRC | 08:03 | |
| *** mkopec__ has quit IRC | 08:04 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Increase timeout for ironic power status report https://review.openstack.org/361915 | 08:05 |
| *** e0ne has joined #openstack-infra | 08:05 | |
| jhesketh | AJaeger: sure | 08:05 |
| *** markvoelker has joined #openstack-infra | 08:05 | |
| *** links has quit IRC | 08:06 | |
| *** martinkopec has joined #openstack-infra | 08:06 | |
| jhesketh | AJaeger: +w | 08:07 |
| *** drifterza_ has quit IRC | 08:07 | |
| *** admcleod_ has joined #openstack-infra | 08:07 | |
| *** martinkopec has quit IRC | 08:07 | |
| AJaeger | thanks, jhesketh | 08:07 |
| *** martinkopec has joined #openstack-infra | 08:08 | |
| *** admcleod has quit IRC | 08:08 | |
| *** markvoelker has quit IRC | 08:10 | |
| *** derekh has joined #openstack-infra | 08:12 | |
| *** greghaynes has quit IRC | 08:13 | |
| *** javeriak has quit IRC | 08:16 | |
| *** gildub has quit IRC | 08:16 | |
| openstackgerrit | Merged openstack-infra/project-config: Adds an api-ref publishing job for cinder - Block Storage API https://review.openstack.org/361475 | 08:16 |
| *** caowei has quit IRC | 08:17 | |
| *** martinkopec has quit IRC | 08:18 | |
| *** martinkopec has joined #openstack-infra | 08:20 | |
| *** javeriak has joined #openstack-infra | 08:20 | |
| *** lucas-afk is now known as lucasagomes | 08:21 | |
| *** martinkopec has quit IRC | 08:21 | |
| *** coolsvap is now known as coolsvap_ | 08:21 | |
| *** pblaho has joined #openstack-infra | 08:21 | |
| *** martinkopec has joined #openstack-infra | 08:21 | |
| *** vinaypotluri has quit IRC | 08:21 | |
| *** thorst has joined #openstack-infra | 08:22 | |
| *** roxanaghe has joined #openstack-infra | 08:24 | |
| *** roxanaghe has quit IRC | 08:29 | |
| *** thorst has quit IRC | 08:29 | |
| *** markvoelker has joined #openstack-infra | 08:33 | |
| *** martinkopec has quit IRC | 08:33 | |
| *** martinkopec has joined #openstack-infra | 08:34 | |
| *** martinkopec has quit IRC | 08:35 | |
| *** martinkopec has joined #openstack-infra | 08:35 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Disable sahara tests in tempest https://review.openstack.org/361936 | 08:38 |
| *** pgadiya has quit IRC | 08:38 | |
| *** markvoelker has quit IRC | 08:38 | |
| *** martinkopec has quit IRC | 08:38 | |
| *** martinkopec has joined #openstack-infra | 08:39 | |
| *** greghaynes has joined #openstack-infra | 08:40 | |
| *** pgadiya has joined #openstack-infra | 08:41 | |
| *** martinkopec has quit IRC | 08:41 | |
| *** martinkopec has joined #openstack-infra | 08:42 | |
| *** tonytan4ever has joined #openstack-infra | 08:44 | |
| *** rossella_s has joined #openstack-infra | 08:45 | |
| *** martinkopec has quit IRC | 08:46 | |
| *** martinkopec has joined #openstack-infra | 08:46 | |
| *** abregman has joined #openstack-infra | 08:47 | |
| *** dizquierdo has quit IRC | 08:48 | |
| *** tonytan4ever has quit IRC | 08:49 | |
| *** salv-orlando has quit IRC | 08:49 | |
| *** yamamoto has joined #openstack-infra | 08:50 | |
| *** martinkopec has quit IRC | 08:52 | |
| *** watanabe_isao has quit IRC | 08:53 | |
| *** martinkopec has joined #openstack-infra | 08:53 | |
| *** yamamoto has quit IRC | 08:56 | |
| *** electrofelix has joined #openstack-infra | 09:00 | |
| *** markvoelker has joined #openstack-infra | 09:03 | |
| *** javeriak has quit IRC | 09:03 | |
| *** markvoelker has quit IRC | 09:07 | |
| wznoinsk_ | fungi: around? | 09:09 |
| *** wznoinsk_ is now known as wznoinsk | 09:09 | |
| AJaeger | wznoinsk: it's night time in the US. If you need anything, best ask here and somebody will later read backscroll and try to help | 09:10 |
| wznoinsk | AJaeger: sure, it was rather admin type of question, in email with my ATC got hit by retention policy and I can't access it anymore | 09:11 |
| wznoinsk | s/in/an | 09:11 |
| AJaeger | wznoinsk: ah, then best talk to fungi, he should be around in 4 hours. | 09:12 |
| wznoinsk | AJaeger: will do, thanks | 09:12 |
| *** sshnaidm is now known as sshnaidm|afk | 09:13 | |
| *** javeriak has joined #openstack-infra | 09:14 | |
| *** martinkopec has quit IRC | 09:16 | |
| *** martinkopec has joined #openstack-infra | 09:18 | |
| *** martinkopec has quit IRC | 09:18 | |
| *** martinkopec has joined #openstack-infra | 09:20 | |
| *** martinkopec has quit IRC | 09:20 | |
| *** salv-orlando has joined #openstack-infra | 09:20 | |
| *** martinkopec has joined #openstack-infra | 09:21 | |
| *** thorst has joined #openstack-infra | 09:27 | |
| *** pgadiya has quit IRC | 09:30 | |
| *** markvoelker has joined #openstack-infra | 09:31 | |
| openstackgerrit | Mykola Stolyarenko proposed openstack-infra/project-config: Setup gate for openstack/python-muranoclient with glare https://review.openstack.org/361965 | 09:31 |
| *** thorst has quit IRC | 09:34 | |
| *** flaper87 has quit IRC | 09:35 | |
| *** markvoelker has quit IRC | 09:35 | |
| electrofelix | zxiiro waynr: just realised I'm not going to be around this Friday for the v2.0 sprint timeslot, can still review anything already up and anything that is there by Wednesday | 09:36 |
| *** tosky has joined #openstack-infra | 09:37 | |
| *** dtantsur|afk is now known as dtantsur | 09:44 | |
| *** caowei has joined #openstack-infra | 09:48 | |
| *** kzaitsev_mb has joined #openstack-infra | 09:50 | |
| *** yanyanhu has quit IRC | 09:52 | |
| *** flaper87 has joined #openstack-infra | 09:53 | |
| *** flaper87 has quit IRC | 09:54 | |
| *** flaper87 has joined #openstack-infra | 09:54 | |
| zigo_ | AJaeger: Can you confirm that the correct path for the AFS volume really is BASE=/afs/.openstack.org/mirror/debian-openstack ? | 09:55 |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Fix BASE=/afs/.openstack.org/mirror/debian-openstack https://review.openstack.org/361977 | 09:55 |
| zigo_ | AJaeger: Last patch from Paul broke something: http://logs.openstack.org/6a/6a1eea75d05732b8932995267384653a3a309bb5/post/reprepro-import-debian-openstack-jessie-backports/c5f23aa/console.html | 09:56 |
| *** zigo_ is now known as zigo | 09:56 | |
| *** yamamoto has joined #openstack-infra | 09:57 | |
| AJaeger | zigo: No, cannot verify that - please work with pabelanger on this. | 09:57 |
| *** zshuo has quit IRC | 09:57 | |
| zigo | AJaeger: 2 files have different path, so I'm guessing the right one isn't the one who is having an error in the POST job ... | 09:58 |
| zigo | AJaeger: Besides this, I can remember he did the mistake once. | 09:58 |
| openstackgerrit | Mykola Stolyarenko proposed openstack-infra/project-config: Setup gate for openstack/python-muranoclient with glare https://review.openstack.org/361965 | 09:58 |
| *** dizquierdo has joined #openstack-infra | 10:00 | |
| *** markvoelker has joined #openstack-infra | 10:00 | |
| *** ihrachys has joined #openstack-infra | 10:03 | |
| *** yamamoto has quit IRC | 10:03 | |
| *** zhurong_ has quit IRC | 10:04 | |
| *** nmagnezi has quit IRC | 10:04 | |
| *** markvoelker has quit IRC | 10:05 | |
| *** coolsvap_ is now known as coolsvap | 10:06 | |
| *** JerryOpenix has joined #openstack-infra | 10:12 | |
| JerryOpenix | Core-developers: pls review https://review.openstack.org/#/c/355139/ | 10:12 |
| *** roxanaghe has joined #openstack-infra | 10:12 | |
| *** _degorenko|afk is now known as degorenko | 10:14 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Inject CA certificate via service profile https://review.openstack.org/357791 | 10:14 |
| openstackgerrit | Mykola Stolyarenko proposed openstack-infra/project-config: Setup gate for openstack/python-muranoclient with glare https://review.openstack.org/361965 | 10:15 |
| *** pgadiya has joined #openstack-infra | 10:16 | |
| *** nmagnezi has joined #openstack-infra | 10:16 | |
| *** roxanaghe has quit IRC | 10:17 | |
| *** kzaitsev_mb has quit IRC | 10:23 | |
| *** markvoelker has joined #openstack-infra | 10:28 | |
| *** oomichi has quit IRC | 10:28 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config: [Rally] Make py35 job voting https://review.openstack.org/361994 | 10:28 |
| *** JerryOpenix has quit IRC | 10:29 | |
| *** oomichi has joined #openstack-infra | 10:30 | |
| *** javeriak has quit IRC | 10:31 | |
| *** markvoelker has quit IRC | 10:32 | |
| *** javeriak has joined #openstack-infra | 10:32 | |
| *** e0ne has quit IRC | 10:32 | |
| *** martinkopec has quit IRC | 10:35 | |
| *** ptacemic has joined #openstack-infra | 10:35 | |
| *** caowei has quit IRC | 10:36 | |
| openstackgerrit | Merged openstack-infra/release-tools: fix announce.sh for projects with setup_requires https://review.openstack.org/359351 | 10:37 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/project-config: Add one new Fuel's plugin https://review.openstack.org/362002 | 10:40 |
| *** HeOS has joined #openstack-infra | 10:41 | |
| *** amotoki has quit IRC | 10:41 | |
| *** martinkopec has joined #openstack-infra | 10:45 | |
| *** tonytan4ever has joined #openstack-infra | 10:45 | |
| *** amotoki has joined #openstack-infra | 10:46 | |
| *** amotoki has quit IRC | 10:46 | |
| *** _sigmavirus24 is now known as sigmavirus | 10:47 | |
| *** sigmavirus has joined #openstack-infra | 10:47 | |
| *** javeriak has quit IRC | 10:49 | |
| *** tonytan4ever has quit IRC | 10:49 | |
| *** amotoki has joined #openstack-infra | 10:50 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute041.vanilla to Ansible static inventory https://review.openstack.org/362010 | 10:52 |
| *** thorst has joined #openstack-infra | 10:54 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute042.vanilla to Ansible static inventory https://review.openstack.org/362012 | 10:56 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute043.vanilla to Ansible static inventory https://review.openstack.org/362018 | 10:59 |
| *** markvoelker has joined #openstack-infra | 10:59 | |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/project-config: Add one new Fuel's plugin https://review.openstack.org/362002 | 10:59 |
| *** yamamoto has joined #openstack-infra | 11:00 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute044.vanilla to Ansible static inventory https://review.openstack.org/362019 | 11:00 |
| *** gildub has joined #openstack-infra | 11:01 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute045.vanilla to Ansible static inventory https://review.openstack.org/362020 | 11:02 |
| *** amotoki has quit IRC | 11:02 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:03 | |
| *** markvoelker has quit IRC | 11:03 | |
| *** martinkopec has quit IRC | 11:08 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute041.vanilla to Ansible static inventory https://review.openstack.org/362010 | 11:09 |
| *** martinkopec has joined #openstack-infra | 11:09 | |
| *** rodrigods has quit IRC | 11:13 | |
| *** rodrigods has joined #openstack-infra | 11:13 | |
| *** sdague has joined #openstack-infra | 11:14 | |
| openstackgerrit | Igor Yozhikov proposed openstack-infra/irc-meetings: Add deb-packaging team meeting https://review.openstack.org/362025 | 11:15 |
| *** kzaitsev_mb has joined #openstack-infra | 11:20 | |
| *** ramishra has quit IRC | 11:21 | |
| *** ramishra has joined #openstack-infra | 11:22 | |
| *** yamamoto has quit IRC | 11:22 | |
| *** Na3iL has quit IRC | 11:27 | |
| *** yamamoto has joined #openstack-infra | 11:27 | |
| *** markvoelker has joined #openstack-infra | 11:28 | |
| *** javeriak has joined #openstack-infra | 11:28 | |
| *** jkilpatr has joined #openstack-infra | 11:31 | |
| *** markvoelker has quit IRC | 11:32 | |
| *** Wei_Liu has quit IRC | 11:35 | |
| *** ldnunes has joined #openstack-infra | 11:35 | |
| *** pradk has joined #openstack-infra | 11:42 | |
| *** yamamoto has quit IRC | 11:42 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Fix iLO IP address for compute009.vanilla https://review.openstack.org/362040 | 11:42 |
| *** Guest85979 is now known as beagles | 11:44 | |
| *** pradk has quit IRC | 11:46 | |
| *** rtheis has joined #openstack-infra | 11:47 | |
| *** shardy has joined #openstack-infra | 11:50 | |
| *** shardy_ has joined #openstack-infra | 11:52 | |
| *** shardy_ has quit IRC | 11:52 | |
| *** lucasagomes is now known as lucas-hungry | 11:55 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute042.vanilla to Ansible static inventory https://review.openstack.org/362012 | 11:56 |
| openstackgerrit | Arie Bregman proposed openstack-infra/zuul: Add 'reset_branch' option to Merger https://review.openstack.org/362049 | 11:56 |
| *** markvoelker has joined #openstack-infra | 11:57 | |
| *** raildo has joined #openstack-infra | 11:58 | |
| *** dprince has joined #openstack-infra | 12:00 | |
| *** eset- has quit IRC | 12:01 | |
| *** markvoelker has quit IRC | 12:01 | |
| dlahn | good afternoon. | 12:01 |
| *** jaosorior has quit IRC | 12:02 | |
| *** jaosorior has joined #openstack-infra | 12:03 | |
| *** rfolco has joined #openstack-infra | 12:03 | |
| *** eset has joined #openstack-infra | 12:03 | |
| openstackgerrit | Arie Bregman proposed openstack-infra/project-config: Add zuul channel to gerritbot channels https://review.openstack.org/362052 | 12:06 |
| *** gouthamr has joined #openstack-infra | 12:06 | |
| *** eset has quit IRC | 12:06 | |
| *** eset has joined #openstack-infra | 12:06 | |
| *** rlandy has joined #openstack-infra | 12:07 | |
| *** salv-orl_ has joined #openstack-infra | 12:08 | |
| *** pradk has joined #openstack-infra | 12:09 | |
| *** salv-orlando has quit IRC | 12:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute043.vanilla to Ansible static inventory https://review.openstack.org/362018 | 12:13 |
| *** tpsilva has joined #openstack-infra | 12:14 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute044.vanilla to Ansible static inventory https://review.openstack.org/362019 | 12:14 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute045.vanilla to Ansible static inventory https://review.openstack.org/362020 | 12:15 |
| penguinolog | +W please on https://review.openstack.org/#/c/361843/ , if possible | 12:16 |
| *** salv-orl_ has quit IRC | 12:16 | |
| openstackgerrit | Merged openstack-infra/system-config: Fix iLO IP address for compute009.vanilla https://review.openstack.org/362040 | 12:17 |
| *** amotoki has joined #openstack-infra | 12:17 | |
| *** gordc has joined #openstack-infra | 12:17 | |
| *** amotoki has quit IRC | 12:17 | |
| *** zhurong has joined #openstack-infra | 12:18 | |
| *** baoli_ has joined #openstack-infra | 12:18 | |
| *** rhallisey has joined #openstack-infra | 12:19 | |
| *** sarob has joined #openstack-infra | 12:20 | |
| *** baoli has quit IRC | 12:21 | |
| *** sarob has quit IRC | 12:25 | |
| *** Ravikiran_K has joined #openstack-infra | 12:25 | |
| *** markvoelker has joined #openstack-infra | 12:26 | |
| *** changzhi has quit IRC | 12:26 | |
| *** chem has joined #openstack-infra | 12:27 | |
| *** vikrant has quit IRC | 12:28 | |
| openstackgerrit | Merged openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/361843 | 12:28 |
| *** salv-orlando has joined #openstack-infra | 12:29 | |
| *** markvoelker has quit IRC | 12:30 | |
| pabelanger | zigo: +3 | 12:37 |
| zigo | pabelanger: Cheers! | 12:38 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Increase timeout for ironic power status report https://review.openstack.org/361915 | 12:38 |
| zigo | pabelanger: Some packages were published, but Sources.gz and Packages.gz were not changed, therefore, deb-auto-backports rebuild them, and reprepro refuses to import the new packages. We'll see what happens on the next package import, but possibly, we'll need some deletion... | 12:39 |
| *** javeriak has quit IRC | 12:40 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Increase timeout for ironic power status report https://review.openstack.org/361915 | 12:41 |
| openstackgerrit | Mykola Stolyarenko proposed openstack-infra/project-config: Setup gate for openstack/python-muranoclient with glare https://review.openstack.org/361965 | 12:42 |
| pabelanger | zigo: is there a log file? | 12:42 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute009.vanilla to Ansible static inventory https://review.openstack.org/362082 | 12:42 |
| *** yamamoto has joined #openstack-infra | 12:42 | |
| *** kgiusti has joined #openstack-infra | 12:42 | |
| HeOS | Hello, infra-team! Please review: https://review.openstack.org/#/c/362002/. I'll appreciate your help there. | 12:43 |
| pabelanger | mordred: jeblair: looks like infracloud is filtered by spamhaus: http://www.spamhaus.org/query/bl?ip=15.184.64.16 | 12:45 |
| pabelanger | mordred: jeblair: I am getting Mail Delivery System emails for both your email addresses | 12:45 |
| pabelanger | from our ic.openstack.org nodes | 12:45 |
| *** pradk has quit IRC | 12:46 | |
| abregman | hey, how we can add anther gerritbot instance? we reached the 120 limit | 12:49 |
| *** yamamoto has quit IRC | 12:49 | |
| zigo | pabelanger: Looks like it imported correctly: http://logs.openstack.org/6a/6a1eea75d05732b8932995267384653a3a309bb5/post/reprepro-import-debian-openstack-jessie-backports/c5f23aa/console.html | 12:49 |
| *** coreyob has quit IRC | 12:49 | |
| *** psachin has quit IRC | 12:49 | |
| *** Goneri has joined #openstack-infra | 12:50 | |
| pabelanger | zigo: yes, because of the timestamp failure it was not released properly. The next builds will do that step, and you should see the correct info in Sources.gz | 12:50 |
| zigo | ok | 12:50 |
| openstackgerrit | Merged openstack-infra/project-config: Fix BASE=/afs/.openstack.org/mirror/debian-openstack https://review.openstack.org/361977 | 12:50 |
| *** pilgrimstack1 has joined #openstack-infra | 12:50 | |
| *** javeriak has joined #openstack-infra | 12:51 | |
| *** amotoki has joined #openstack-infra | 12:51 | |
| *** amitgandhinz has quit IRC | 12:52 | |
| *** pilgrimstack has quit IRC | 12:53 | |
| *** amitgandhinz has joined #openstack-infra | 12:53 | |
| *** slaweq_ has quit IRC | 12:54 | |
| *** Na3iL has joined #openstack-infra | 12:55 | |
| *** slaweq_ has joined #openstack-infra | 12:55 | |
| *** markvoelker has joined #openstack-infra | 12:56 | |
| *** lucas-hungry is now known as lucasagomes | 12:56 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute009.vanilla to Ansible static inventory https://review.openstack.org/362082 | 12:58 |
| *** esikachev has quit IRC | 12:58 | |
| *** gildub has quit IRC | 12:58 | |
| *** markvoelker has quit IRC | 13:00 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute020.vanilla to Ansible static inventory https://review.openstack.org/362095 | 13:01 |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add deb-python-hacking repo https://review.openstack.org/362096 | 13:01 |
| *** pcaruana has quit IRC | 13:02 | |
| cloudnull | mornings | 13:02 |
| *** jordanP has joined #openstack-infra | 13:02 | |
| *** sdake has joined #openstack-infra | 13:03 | |
| *** roxanaghe has joined #openstack-infra | 13:03 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-nodepool https://review.openstack.org/362098 | 13:03 |
| jroll | AJaeger: hi, you were planning on each project moving to bindep themselves, yes? I thought I read something about you proposing patches, but I can't find it now | 13:03 |
| jroll | and didn't want to assume :) | 13:03 |
| openstackgerrit | Merged openstack-infra/yaml2ical: Update hacking test-requirement https://review.openstack.org/359478 | 13:03 |
| AJaeger | jroll: no, not at all. | 13:04 |
| openstackgerrit | Merged openstack-infra/yaml2ical: Manual sync to global-requirements https://review.openstack.org/359480 | 13:04 |
| AJaeger | jroll: I just did a very small number to check that everything worked nicely - and figure out what we need to fix or document. | 13:04 |
| AJaeger | jroll: So, go ahead and do you rown, please ;) | 13:05 |
| jroll | AJaeger: okay, thought so, will do :) | 13:05 |
| jroll | thanks | 13:05 |
| openstackgerrit | Mykola Stolyarenko proposed openstack-infra/project-config: Setup gate for openstack/python-muranoclient with glare https://review.openstack.org/361965 | 13:05 |
| openstackgerrit | Merged openstack-infra/yaml2ical: Remove discover from test-requirements https://review.openstack.org/345809 | 13:05 |
| *** coreyob has joined #openstack-infra | 13:05 | |
| AJaeger | jroll: if you have questions or bindep, fungi or myself can probably help... | 13:05 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config: [sahara] scenario jobs refactoring https://review.openstack.org/361185 | 13:05 |
| jroll | AJaeger: noted | 13:05 |
| *** strigazi_AFK is now known as strigazi | 13:05 | |
| *** tonytan4ever has joined #openstack-infra | 13:06 | |
| *** daemontool has joined #openstack-infra | 13:06 | |
| *** Goneri has quit IRC | 13:06 | |
| *** markvoelker has joined #openstack-infra | 13:06 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-nginx https://review.openstack.org/362102 | 13:06 |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/362103 | 13:06 |
| *** markvoelker_ has joined #openstack-infra | 13:07 | |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add deb-python-hacking repo https://review.openstack.org/362096 | 13:07 |
| *** vgridnev has joined #openstack-infra | 13:07 | |
| *** roxanaghe has quit IRC | 13:08 | |
| *** raildo has quit IRC | 13:08 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-jenkins-plugins https://review.openstack.org/362105 | 13:09 |
| *** markvoelker_ has quit IRC | 13:10 | |
| *** markvoelker has quit IRC | 13:10 | |
| *** markvoelker_ has joined #openstack-infra | 13:10 | |
| *** markvoelker has joined #openstack-infra | 13:11 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-jenkins-job-builder https://review.openstack.org/361772 | 13:11 |
| abregman | which infra project runs gerritbot instance? | 13:12 |
| pabelanger | puppet-gerritbot | 13:13 |
| *** markvoelker_ has quit IRC | 13:13 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute020.vanilla to Ansible static inventory https://review.openstack.org/362095 | 13:14 |
| abregman | thanks | 13:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Support dual-stack neutron networks https://review.openstack.org/357517 | 13:16 |
| *** shardy has quit IRC | 13:17 | |
| *** pcaruana has joined #openstack-infra | 13:17 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-jenkins-job-builder https://review.openstack.org/361772 | 13:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-jenkins https://review.openstack.org/361771 | 13:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-diskimage-builder repo https://review.openstack.org/361770 | 13:18 |
| *** pgadiya has quit IRC | 13:19 | |
| *** xyang1 has joined #openstack-infra | 13:20 | |
| abregman | pabelanger: which project pass the channel_file variable?: https://github.com/openstack-infra/puppet-gerritbot/blob/028315cf9180822c23279efc9f125a3032f4cac7/manifests/init.pp#L4 | 13:20 |
| AJaeger | abregman: system-config, file modules/openstack_project/manifests/review.pp | 13:21 |
| zigo | pabelanger: https://review.openstack.org/#/c/362096/ <--- Adding python-hacking which we forgot. | 13:22 |
| abregman | AJaeger: ack. not sure what would be a good way to run anther gerritbot instace. should we split the channels file (in project-config) and pass each file to different instance of gerritbot? | 13:23 |
| zigo | AJaeger: I believe I did it right this time (with Needed-By and Depends-On tags) | 13:23 |
| *** esikachev has joined #openstack-infra | 13:23 | |
| *** sdake_ has joined #openstack-infra | 13:23 | |
| *** matt-borland has joined #openstack-infra | 13:24 | |
| tosky | infra cores, here I am again with another change for Sahara jobs: https://review.openstack.org/#/c/361185/ | 13:24 |
| *** mriedem has joined #openstack-infra | 13:25 | |
| *** sdake has quit IRC | 13:25 | |
| *** _ari_ has joined #openstack-infra | 13:25 | |
| penguinolog | Look please https://review.openstack.org/#/c/362103/ - coverage jobs is impossible for fuel-qa just now, due to it's not packaged for now :(. And pep8/py2.7 is still required for some old branches. | 13:26 |
| *** abregman_ has joined #openstack-infra | 13:27 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add deb-packaging team meeting https://review.openstack.org/362025 | 13:27 |
| *** abregman has quit IRC | 13:30 | |
| *** markvoelker has quit IRC | 13:30 | |
| *** mikelk has quit IRC | 13:32 | |
| *** bhagyashris has joined #openstack-infra | 13:34 | |
| *** _nadya_ has quit IRC | 13:35 | |
| bhagyashris | Hi all, please any one let me known which micro version of python2.7 is used by jenkins job? | 13:36 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-nginx https://review.openstack.org/362102 | 13:37 |
| *** eharney has joined #openstack-infra | 13:37 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-nodepool https://review.openstack.org/362098 | 13:37 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-jenkins-job-builder https://review.openstack.org/361772 | 13:37 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-jenkins https://review.openstack.org/361771 | 13:37 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-diskimage-builder https://review.openstack.org/361770 | 13:37 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Rename ansible-role-jenkins-plugins https://review.openstack.org/362105 | 13:37 |
| mordred | pabelanger: what's windmill? | 13:39 |
| penguinolog | AJaeger, please look https://review.openstack.org/#/c/362103/ - previous change has an issue: cover gate requires packaging (install.py), which is not used in fuel-qa just now :( | 13:39 |
| *** chem has quit IRC | 13:40 | |
| *** chem has joined #openstack-infra | 13:40 | |
| penguinolog | part of previous commit is required for proper work, but we had blocked CI for any new changes | 13:40 |
| pabelanger | mordred: something I've been hacking on for a while. Bascially, bunch of ansible roles to deploy infra services downstream. I basically use it as a testing platform for our CI | 13:40 |
| *** raildo has joined #openstack-infra | 13:40 | |
| *** sc68cal_ is now known as sc68cal | 13:44 | |
| *** nmagnezi has quit IRC | 13:49 | |
| *** elacheche has joined #openstack-infra | 13:49 | |
| *** esberglu has joined #openstack-infra | 13:50 | |
| *** sdake_ has quit IRC | 13:50 | |
| *** M-docaedo_vector has quit IRC | 13:51 | |
| *** sdake has joined #openstack-infra | 13:51 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 13:52 |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config: Introduce functional/fullstack Neutron Xenial jobs https://review.openstack.org/362127 | 13:53 |
| *** esikachev has quit IRC | 13:54 | |
| *** esikachev has joined #openstack-infra | 13:54 | |
| *** woodster_ has joined #openstack-infra | 13:54 | |
| *** mriedem1 has joined #openstack-infra | 13:54 | |
| *** mriedem has quit IRC | 13:55 | |
| *** pcaruana has quit IRC | 13:55 | |
| openstackgerrit | Mykola Stolyarenko proposed openstack-infra/project-config: Setup gate for openstack/python-muranoclient with glare https://review.openstack.org/361965 | 13:56 |
| *** markvoelker has joined #openstack-infra | 13:57 | |
| *** jamesdenton has joined #openstack-infra | 13:57 | |
| *** Shrews has quit IRC | 13:58 | |
| *** asselin_ has joined #openstack-infra | 13:58 | |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config: Introduce functional/fullstack Neutron Xenial jobs https://review.openstack.org/359843 | 13:59 |
| *** javeriak has quit IRC | 14:01 | |
| *** drawsmcgraw has joined #openstack-infra | 14:01 | |
| *** Shrews has joined #openstack-infra | 14:01 | |
| *** markvoelker has quit IRC | 14:01 | |
| *** pradk has joined #openstack-infra | 14:02 | |
| *** fguillot has joined #openstack-infra | 14:02 | |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config: Introduce functional/fullstack Neutron Xenial jobs https://review.openstack.org/359843 | 14:03 |
| *** senk has quit IRC | 14:04 | |
| *** marcusvrn_ has quit IRC | 14:04 | |
| *** mkoderer has quit IRC | 14:06 | |
| *** lmiccini has quit IRC | 14:07 | |
| *** mtanino has joined #openstack-infra | 14:07 | |
| *** cgross has quit IRC | 14:08 | |
| *** caowei has joined #openstack-infra | 14:08 | |
| *** mkoderer has joined #openstack-infra | 14:08 | |
| *** pcaruana has joined #openstack-infra | 14:09 | |
| zigo | pabelanger: Is it possible to forbid having 2 deb-auto-backports POST jobs at the same time? | 14:09 |
| zigo | It's potentially an issue. | 14:09 |
| *** cgross has joined #openstack-infra | 14:09 | |
| zigo | ie: 2 POST jobs building the same backport, the 2nd one will fail to import with reprepro if the package isn't reproducible, and probably this will prevent another backport to be published. | 14:10 |
| pabelanger | zigo: not today, it is something we've discussed before however | 14:10 |
| zigo | Ah... | 14:10 |
| zigo | Zuul 3 ? :) | 14:10 |
| pabelanger | yes | 14:10 |
| zigo | Ok. | 14:10 |
| zigo | Good to know. | 14:11 |
| *** abregman_ is now known as abregman_|mtg | 14:11 | |
| pabelanger | other projects have this issue too | 14:12 |
| *** kushal has joined #openstack-infra | 14:12 | |
| pabelanger | IIRC, the docs jobs will fail in post, if too many are trying to upload contents to the FTP server | 14:12 |
| *** thiagop has joined #openstack-infra | 14:13 | |
| *** annegentle has joined #openstack-infra | 14:13 | |
| *** Goneri has joined #openstack-infra | 14:14 | |
| *** kencjohnston_ has quit IRC | 14:14 | |
| *** lmiccini has joined #openstack-infra | 14:14 | |
| *** kencjohnston has joined #openstack-infra | 14:15 | |
| *** rbrndt has joined #openstack-infra | 14:16 | |
| pabelanger | zigo: actually, do you have a log of the failure? | 14:16 |
| pabelanger | zigo: is this something you have seen? | 14:16 |
| *** _nadya_ has joined #openstack-infra | 14:17 | |
| *** adrian_otto has joined #openstack-infra | 14:18 | |
| *** spzala has joined #openstack-infra | 14:18 | |
| zigo | pabelanger: I've seen it yes, but it's useless to see logs, it basically just says "deb has this checksum in the repo, I can't import this package with a different checksum" | 14:19 |
| zigo | pabelanger: Re-running the script is then fine, because as the package is in the repo, it wont rebuild and republish. | 14:19 |
| zigo | It's just a concurrency issue. | 14:19 |
| *** ptacemic has quit IRC | 14:20 | |
| *** hongbin has joined #openstack-infra | 14:20 | |
| pabelanger | zigo: the issue is, pkgdeb-publish-auto-backports could build the same package twice, right? If it doesn't exist in the repo yet? | 14:20 |
| fungi | wznoinsk: can you elaborate on your issue? when you say you lost your access to your "atc" due to an e-mail retention policy, are you talking about your summit registration discount code? | 14:21 |
| *** annegent_ has joined #openstack-infra | 14:21 | |
| *** sarob has joined #openstack-infra | 14:21 | |
| zigo | pabelanger: deb-auto-backports uses a list of packages in a file called "packages-list". With that, it checks what's in the Debian repos and what's in our repository. If there's no package in our repo, or if the version is higher in Debian, the it attempts to rebuild. | 14:21 |
| zxiiro | electrofelix: waynr ok cool. Did we ever get the 2 patches still in discussion decoupled? maybe a good goal for this week is to try to merge the trivial patches that come after the 2 patches once they are decoupled | 14:22 |
| zigo | Checks are done with madison-lite. | 14:22 |
| cloudnull | hello all, looks like there have been a few more "error node launch attempts" http://grafana.openstack.org/dashboard/db/nodepool-osic?panelId=11&fullscreen | 14:22 |
| *** ilyashakhat has joined #openstack-infra | 14:22 | |
| zigo | (to fetch version, then compare is done with dpkg --version-compare) | 14:22 |
| cloudnull | we we know what the errors are? | 14:22 |
| cloudnull | anything I should/need to go look into ? | 14:22 |
| *** zz_dimtruck is now known as dimtruck | 14:22 | |
| pabelanger | zigo: okay | 14:24 |
| *** psachin has joined #openstack-infra | 14:24 | |
| *** annegentle has quit IRC | 14:24 | |
| *** jaosorior is now known as jaosorior_away | 14:25 | |
| *** M-docaedo_vector has joined #openstack-infra | 14:25 | |
| *** sarob has quit IRC | 14:26 | |
| zigo | Everything worked on our last addition of something in jessie-backports. | 14:26 |
| zigo | (ie: we added python-mock from Debian experimental) | 14:26 |
| *** bhagyashris has quit IRC | 14:28 | |
| *** coreyob has quit IRC | 14:29 | |
| *** jtomasek|afk has quit IRC | 14:30 | |
| pabelanger | zigo: http://mirror.dfw.rax.openstack.org/debian-openstack/timestamp.txt | 14:30 |
| pabelanger | know you know when it was last released | 14:30 |
| anteaya | cloudnull: pabelanger would probably be able to access that information | 14:32 |
| anteaya | I don't think I am able, can I pabelanger? | 14:32 |
| anteaya | it is in logs only -root have access to, correct? | 14:33 |
| AJaeger | fungi, jeblair: I've just wrote up two problems with zuul-cloner that I run into and put them in an etherpad: https://etherpad.openstack.org/p/oy7A2OLJfD . The second was reported by pc_m to me - he asked why I removed some code... | 14:33 |
| sdague | fungi / pabelanger / mordred - can I get a quick review here - https://review.openstack.org/#/c/360565/ - it drops horizon from grenade service list, as grenade doesn't really test horizon | 14:33 |
| mordred | sdague: wfm | 14:34 |
| *** martinkopec has quit IRC | 14:34 | |
| *** clenimar has joined #openstack-infra | 14:34 | |
| pabelanger | cloudnull: anteaya: looking, but likely: https://review.openstack.org/#/c/361449/ We need to land that today so we can get new images and fix the firewall rules on centos-7 | 14:34 |
| AJaeger | fungi, could you look at proposal slave and confirm that my changes are fine - and perhaps also, explain why some jobs failed and then worked again? The in-between failure really puzzles me, it broke my whole theory - but I saw it too late ;) | 14:35 |
| *** rhallisey has quit IRC | 14:35 | |
| *** coreyob has joined #openstack-infra | 14:35 | |
| anteaya | pabelanger: you need a +A on that, think mordred would be willing to review? | 14:35 |
| cloudnull | ah. ok. thanks pabelanger | 14:35 |
| anteaya | pabelanger: do you think the error is on centos node launch attempts? | 14:36 |
| *** ilyashakhat has quit IRC | 14:36 | |
| AJaeger | fungi, jeblair, I guess I'll ask later to discuss these - just wanted to mention that I need some help here. | 14:36 |
| pabelanger | anteaya: Yes, a quick look at nodepool.log show the failures that 361449 will fix. | 14:38 |
| anteaya | pabelanger: awesome thank you | 14:38 |
| anteaya | so clarkb's review implies that approving that is a timing issue, when is the right time to approve? | 14:39 |
| *** mriedem1 is now known as mriedem | 14:39 | |
| pabelanger | I think we can do it today | 14:39 |
| fungi | AJaeger: in situation #2 i strongly suspect some neutron driver being installed is declaring a dependency on neutron's master branch. i'll see if i can track it down in the tox/pip logs | 14:39 |
| anteaya | great | 14:39 |
| AJaeger | Thanks, fungi. What confuses me is that the fwaas functional test works with the depends-on - but not in the experimental case | 14:41 |
| fungi | AJaeger: probably neutron-vpnaas from the looks of it | 14:41 |
| *** asettle has joined #openstack-infra | 14:41 | |
| *** rhallisey has joined #openstack-infra | 14:41 | |
| fungi | from zuul | 14:41 |
| fungi | gah, stray newline | 14:41 |
| anteaya | the bane of monday mornings | 14:41 |
| fungi | from zuul's perspective, experimental and check should work the same in that regard | 14:41 |
| anteaya | random newlines | 14:41 |
| *** tonytan_brb has joined #openstack-infra | 14:42 | |
| fungi | no, monday's are what caused me to try to spell it "newtron" | 14:42 |
| anteaya | it is a new one | 14:43 |
| anteaya | to be sure | 14:43 |
| fungi | newer than quantum | 14:43 |
| anteaya | ha ha ha | 14:43 |
| AJaeger | fungi, looking at the log files, I expected them to work the same. But it didn't ;( | 14:43 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/tripleo-ci: Do not reuse dnsmasq-ironic.conf for forcing MTU 1400 https://review.openstack.org/362162 | 14:43 |
| AJaeger | So much to my MOnday theories ;) | 14:43 |
| *** senk has joined #openstack-infra | 14:44 | |
| anteaya | well at least you had a theory to greet Monday with | 14:44 |
| anteaya | I just have a purring cat | 14:44 |
| anteaya | take that Monday | 14:45 |
| * anteaya brandishes her purring cat | 14:45 | |
| AJaeger | ;) | 14:45 |
| *** tonytan4ever has quit IRC | 14:45 | |
| *** tonytan_brb is now known as tonytan4ever | 14:45 | |
| anteaya | :) | 14:45 |
| *** kaisers_ has quit IRC | 14:46 | |
| jeblair | AJaeger: what's the problem description for #1? | 14:47 |
| *** thcipriani|afk is now known as thcipriani | 14:48 | |
| AJaeger | jeblair: I enhanced etherpad | 14:49 |
| jeblair | AJaeger: ah. that follows: | 14:50 |
| jeblair | 2016-08-28 08:51:11.875717 | InvalidGitRepositoryError: /tmp/openstack/neutron | 14:50 |
| AJaeger | jeblair: the git error - cloning failed - is the problem. And then my theory broke when it worked. | 14:50 |
| AJaeger | ... worked the next day. | 14:51 |
| *** kzaitsev_mb has quit IRC | 14:51 | |
| *** e0ne has joined #openstack-infra | 14:51 | |
| jeblair | AJaeger: perhaps tmpcleaner ran | 14:51 |
| AJaeger | jeblair: in parallel to zuul-clone? | 14:52 |
| AJaeger | mmh, interesting... | 14:52 |
| AJaeger | In that case, my moving it to a a subdir of the checked out tree - that we can git clean - will avoid it. | 14:53 |
| fungi | AJaeger: on scenario #2, i'm running into a dead end when it invokes tox since https://git.openstack.org/cgit/openstack/neutron-vpnaas/tree/tools/tox_install.sh isn't set -x so it's hard to see where the logic is falling down there | 14:53 |
| openstackgerrit | Ryan Brandt proposed openstack-infra/project-config: Enable java test as voting on monasca-api https://review.openstack.org/361511 | 14:53 |
| *** jtomasek has joined #openstack-infra | 14:53 | |
| fungi | but i'm pretty sure something in that script is resulting in a checkout of master rather than the zuul ref | 14:53 |
| AJaeger | fungi, but not with depends-on. | 14:54 |
| AJaeger | fungi, let me push a patch to add -x for vpnaas... | 14:54 |
| openstackgerrit | Ryan Brandt proposed openstack-infra/project-config: Enable java test as voting on monasca-api https://review.openstack.org/361511 | 14:54 |
| *** asettle has quit IRC | 14:55 | |
| fungi | AJaeger: well, the job is using a devstack plugin with a custom post-test hook that invokes tox which in turn uses that tox_install.sh wrapper to install packages, and it's debug output of that script i think we need | 14:55 |
| AJaeger | yes - but that means we need to upload the tox log files for that change as well... | 14:55 |
| jeblair | AJaeger: on a recent successful run of neutron-vpnaas-propose-translation-update, i don't see zuul-cloner running | 14:56 |
| wznoinsk | fungi: yes, I got an email with barcelonoa ATC code couple of weeks back but I no longer have it | 14:56 |
| fungi | AJaeger: oh, right, tox will swallow the stdout of the install_command | 14:56 |
| AJaeger | jeblair: you don't see it failing. | 14:56 |
| *** caowei has quit IRC | 14:56 | |
| jeblair | AJaeger: i don't see it running | 14:56 |
| AJaeger | jeblair: it'S in the tox log files | 14:57 |
| jeblair | AJaeger: i don't see it in http://logs.openstack.org/periodic/neutron-vpnaas-propose-translation-update/c6de953/console.html | 14:57 |
| AJaeger | tox.ini has install_command that calls zuul-cloner - https://git.openstack.org/cgit/openstack/neutron-vpnaas/tree/tools/tox_install.sh | 14:57 |
| jeblair | AJaeger: you pointed me to http://logs.openstack.org/periodic/neutron-vpnaas-propose-translation-update/899b8bb/console.html#_2016-08-28_08_51_11_876188 | 14:58 |
| *** chem has quit IRC | 14:58 | |
| *** kzaitsev_mb has joined #openstack-infra | 14:58 | |
| fungi | yeah, we need to collect tox logs for these jobs to have any hope of troubleshooting zuul-cloner behavior when called within a tox install_command wrapper script | 14:58 |
| jeblair | oh, it's only outputting that on success? | 14:58 |
| jeblair | er failure | 14:58 |
| *** eharney has quit IRC | 14:59 | |
| AJaeger | I | 14:59 |
| fungi | since tox will redirect the output from zuul-cloner to its own logfile | 14:59 |
| jeblair | yuck | 14:59 |
| AJaeger | I'll need a break now - and will then add some more debugging output... | 14:59 |
| *** abregman_|mtg has quit IRC | 14:59 | |
| AJaeger | jeblair: Yeah ;( | 15:01 |
| AJaeger | jeblair: perhaps we need to design this differently: neutron-vpnaas wants to isntall neutron for their tests. | 15:01 |
| AJaeger | jeblair, fungi: They might be doing it wrong - and 20+ repos use the same approach ;( | 15:01 |
| AJaeger | bbl | 15:02 |
| fungi | AJaeger: i expect so, yes | 15:02 |
| *** bin_ has joined #openstack-infra | 15:02 | |
| *** hockeynut has joined #openstack-infra | 15:03 | |
| *** rhallisey has quit IRC | 15:03 | |
| fungi | this goes back to the original assertion with upper constraints that having lots of projects use a file centrally located in one common git repo would be "simpler" than syncing that to every repo that needs it. maybe better, but definitely not simpler | 15:03 |
| *** rhallisey has joined #openstack-infra | 15:04 | |
| fungi | the subtle complexities of turning every job (and local test runs on developer machines) into integration tests just keeps going deeper and deeper | 15:04 |
| jeblair | well, i don't necessarily think they're doing it wrong -- i helped design it. | 15:05 |
| jeblair | i'm not sure that some error in the git repo in tmp means that there is a design flaw. | 15:06 |
| fungi | in this case "wrong" may just be a logic error in that wrapper script none of us are spotting | 15:06 |
| jeblair | fungi: oh, are you talking about #2? | 15:06 |
| fungi | jeblair: yes | 15:06 |
| fungi | but i expect there are lots of projects (at least lots of neutron drivers and advanced services) who have copied that typo around if so | 15:07 |
| pabelanger | stepping out to run an errand, should be back shortly | 15:07 |
| jeblair | fungi: ah. i don't have an opinion on #2 yet other than "it should have logs" :) | 15:07 |
| fungi | need the tox logs to know for sure whether it's actually calling zuul-cloner in that job at all, for that matter | 15:07 |
| *** daemontool has quit IRC | 15:07 | |
| jeblair | which is not that different than where we are on #1 :) | 15:08 |
| fungi | indeed | 15:08 |
| *** daemontool has joined #openstack-infra | 15:09 | |
| *** jaosorior_away is now known as jaosorior | 15:10 | |
| fungi | the transparent fallback for not using zuul-cloner in the wrapper script (for the benefit of local developer test runs) means that the job can skip using zuul-cloner entirely and still return success | 15:10 |
| anteaya | if anyone feels like assisting with a gearman plugin failing to register jobs issue, we are discussing one in the third-party meeting over in -meeting right now | 15:10 |
| *** dprince has quit IRC | 15:11 | |
| jeblair | AJaeger: that host is configured to delete files from /tmp older than 12 hours | 15:12 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Fix provisionining_ipv4_address param name for compute045.vanilla https://review.openstack.org/362183 | 15:12 |
| *** abregman_|mtg has joined #openstack-infra | 15:12 | |
| fungi | today's graphs at the bottom of http://status.openstack.org/zuul/ are a thing of beauty. activity is ramping up and up and zuul is keeping up pace perfectly | 15:12 |
| *** signed8bit has joined #openstack-infra | 15:12 | |
| anteaya | fungi: I boggled at that myself | 15:13 |
| *** persia has quit IRC | 15:13 | |
| *** gongysh has joined #openstack-infra | 15:14 | |
| *** vstoiko has joined #openstack-infra | 15:15 | |
| jeblair | fungi, AJaeger: i made a copy of the current /tmp/openstack/neutron directory and ran zuul-cloner on the copy; it worked without error | 15:15 |
| *** armax has joined #openstack-infra | 15:15 | |
| *** psilvad has joined #openstack-infra | 15:15 | |
| *** beagles is now known as beagles_brb | 15:15 | |
| jeblair | fungi, AJaeger: this might be more productive if the most recent run had been a failure, not a success. maybe we check again tomorrow? | 15:16 |
| *** persia has joined #openstack-infra | 15:16 | |
| vstoiko | hi guys, can you pls check this request for new repo - https://review.openstack.org/#/c/359076/5 | 15:16 |
| fungi | that might be necessary. or else we spend time trying to recreate the failure by guessing what might have triggered it and repeating that scenario | 15:16 |
| *** penguinolog__ has joined #openstack-infra | 15:17 | |
| penguinolog__ | Hello again | 15:17 |
| *** markvoelker has joined #openstack-infra | 15:17 | |
| *** ilyashakhat has joined #openstack-infra | 15:17 | |
| penguinolog__ | please, +w somebody - it's BLOCKER for fuel-qa | 15:17 |
| *** sdake_ has joined #openstack-infra | 15:18 | |
| fungi | wznoinsk: looks like i sent it to you on 2016-06-20. want me to resend a copy to the same e-mail address now? | 15:19 |
| *** sdake has quit IRC | 15:20 | |
| wznoinsk | fungi: yes please | 15:20 |
| *** andreas_s has quit IRC | 15:20 | |
| *** mriedem1 has joined #openstack-infra | 15:20 | |
| *** vinaypotluri has joined #openstack-infra | 15:20 | |
| *** esikachev has quit IRC | 15:21 | |
| fungi | wznoinsk: sent just now | 15:21 |
| mordred | fungi: WOW. that's like the healthiest we've ever seen those graphs I think | 15:21 |
| wznoinsk | fungi: see it, thanks | 15:22 |
| fungi | wznoinsk: you're welcome! | 15:22 |
| fungi | sorry your e-mail provider doesn't let you keep two months of messages | 15:22 |
| *** mriedem has quit IRC | 15:22 | |
| anteaya | mordred: yes, I have never seen better | 15:24 |
| *** sdake has joined #openstack-infra | 15:24 | |
| wznoinsk | fungi: we can keep it for longer we just need to setup archiving properly, I just fixed the way I pick the important emails I want to keep for long time | 15:25 |
| *** sdake_ has quit IRC | 15:26 | |
| *** vhosakot has joined #openstack-infra | 15:27 | |
| wznoinsk | lennyb: here? | 15:27 |
| *** hashar is now known as hasharAway | 15:28 | |
| *** adrian_otto has quit IRC | 15:29 | |
| *** eharney has joined #openstack-infra | 15:30 | |
| lennyb | wznoinsk: yeap | 15:31 |
| *** Sukhdev has joined #openstack-infra | 15:31 | |
| *** harlowja_at_home has joined #openstack-infra | 15:32 | |
| *** adrian_otto has joined #openstack-infra | 15:32 | |
| *** markvoelker has quit IRC | 15:32 | |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config: Introduce functional/fullstack Neutron Xenial jobs https://review.openstack.org/359843 | 15:33 |
| *** hockeynut has quit IRC | 15:33 | |
| *** mriedem1 is now known as mriedem | 15:33 | |
| *** penguinolog__ is now known as penguinolog_wfh | 15:34 | |
| *** gyee has joined #openstack-infra | 15:34 | |
| lennyb | wznoinsk, regarding scripts to check your CI against the others, did you had time to check if you can share it ? | 15:34 |
| *** Ravikiran_K has quit IRC | 15:34 | |
| *** hieulq_ has joined #openstack-infra | 15:35 | |
| *** adrian_otto has quit IRC | 15:35 | |
| anteaya | lennyb: wznoinsk these are scripts for the ironic ci work you mentioned in the third-party meeting? | 15:36 |
| wznoinsk | anteaya: so far I've managed to deploy ironic based lcoud using kolla mitaka, CI using that provider is the next step I haven't started yet | 15:37 |
| *** yamahata has joined #openstack-infra | 15:37 | |
| lennyb | anteaya: no, this is something else | 15:37 |
| anteaya | wznoinsk: well done | 15:37 |
| anteaya | lennyb: ah okay, sorry to interrupt | 15:37 |
| wznoinsk | lennyb: I had my colleague fixing this check for me, I need to check with him tomorrow is it in a good shape now | 15:37 |
| lennyb | wznoinsk, thanks, they will be used internally, so quick and dirty is good enough | 15:38 |
| wznoinsk | aim is to submit it to third-party-tools anyways | 15:38 |
| anteaya | wznoinsk: thank you | 15:39 |
| lennyb | wznoinsk, this will be wanderful | 15:39 |
| zaro | morning | 15:40 |
| *** tesseract- has quit IRC | 15:42 | |
| AJaeger | jeblair: since I changed paths, /tmp/openstack/neutron is not used anymore. Let's assume it's tmpcleaner and ignore that one problem for now. | 15:43 |
| openstackgerrit | Merged openstack-infra/devstack-gate: don't start horizon in grenade https://review.openstack.org/360565 | 15:43 |
| *** pcaruana has quit IRC | 15:43 | |
| AJaeger | fungi, jeblair: If it happens again, I'll tell you directly - but hten it would be a different path. | 15:43 |
| *** rcernin has quit IRC | 15:44 | |
| anteaya | morning zaro | 15:44 |
| *** tongli has joined #openstack-infra | 15:47 | |
| *** chem has joined #openstack-infra | 15:47 | |
| *** yamahata has quit IRC | 15:48 | |
| *** matrohon has quit IRC | 15:48 | |
| *** xarses_ has quit IRC | 15:48 | |
| *** gordc has quit IRC | 15:48 | |
| *** gordc has joined #openstack-infra | 15:48 | |
| *** chem has quit IRC | 15:48 | |
| *** chem has joined #openstack-infra | 15:49 | |
| *** chem has quit IRC | 15:49 | |
| *** chem has joined #openstack-infra | 15:49 | |
| AJaeger | fungi, looking at http://logs.openstack.org/42/362042/1/check/gate-neutron-vpnaas-dsvm-functional/b977452/tox/ , the directory is empty - so I wonder, whether that's because vpnaas uses sudo here: http://git.openstack.org/cgit/openstack/neutron-vpnaas/tree/neutron_vpnaas/tests/contrib/post_test_hook.sh#n35 | 15:50 |
| *** chem has quit IRC | 15:50 | |
| AJaeger | Do we need to collect other log files or change this to get logs? | 15:51 |
| *** chem has joined #openstack-infra | 15:54 | |
| *** amotoki has quit IRC | 15:55 | |
| jeblair | AJaeger: do we need to make any changes to the afs doc spec (276482)? do we need more time to discuss, or should we vote on it as written? | 15:55 |
| *** zhurong has quit IRC | 15:55 | |
| AJaeger | jeblair: best ask loquacities as PTL as well. | 15:56 |
| *** apuimedo is now known as apuimedo|away | 15:56 | |
| AJaeger | jeblair: IMHO we're fine to move forward - but need to discuss migration and retention policies. | 15:56 |
| AJaeger | But those can be done independent IMHO. | 15:57 |
| openstackgerrit | Corentin Ardeois proposed openstack-infra/project-config: NPM Projects: sudo needed for DSVM jobs https://review.openstack.org/362222 | 15:57 |
| jeblair | AJaeger: i think those are fundamental -- they affect how we would perform the work. | 15:57 |
| fungi | AJaeger: i'm pretty sure the envvars zuul-cloner relies on won't persist through a sudo call without extra effort | 15:57 |
| *** edtubill has joined #openstack-infra | 15:57 | |
| AJaeger | fungi, could well be... | 15:57 |
| *** gongysh has quit IRC | 15:58 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Cleanup old internal/external network handling https://review.openstack.org/362223 | 15:58 |
| AJaeger | fungi, still, why does it work then by accident with depends-on but not in the experimental case? | 15:58 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Print services statuses of nodes in CI https://review.openstack.org/352916 | 15:59 |
| *** yamahata has joined #openstack-infra | 16:00 | |
| *** chem has quit IRC | 16:00 | |
| AJaeger | fungi, looks like the functional test has workspace as /home/jenkins/workspace/gate-neutron-vpnaas-dsvm-functional/ but runs tox in /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional - and thus does not collect tox log files. Is that correct? | 16:01 |
| *** david-lyle_ is now known as david-lyle | 16:01 | |
| *** chem has joined #openstack-infra | 16:02 | |
| *** shashank_hegde has joined #openstack-infra | 16:03 | |
| zigo | mordred: Could you allow adding deb-python-hacking please? https://review.openstack.org/362096 | 16:03 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add E-R query for bug 1617447 https://review.openstack.org/361458 | 16:03 |
| openstack | bug 1617447 in os-vif "InterfaceAttachFailed: Failed to attach network adapter device" [Undecided,In progress] https://launchpad.net/bugs/1617447 - Assigned to Kevin Benton (kevinbenton) | 16:03 |
| AJaeger | fungi, so is this a scenario to support? Use case here seems: change for neutron/neutron-lib is done, then neutron-vpnaas tests should run on it with that change checked out. Do we need to add the NEUTRON_DIR hack back in from https://review.openstack.org/#/c/361756/1/tools/tox_install.sh ? | 16:03 |
| *** hieulq_ has quit IRC | 16:03 | |
| clarkb | pabelanger: anteaya ya I just didnt want to merge a change that cpuld affect the firewall on all our hosts on friday evening | 16:04 |
| notmyname | whoever updated the wiki to use recaptcha "I'm not a robot" instead of the "type the N {word|letter} of this question"... THANK YOU! | 16:04 |
| *** devananda|OPS is now known as devananda | 16:05 | |
| anteaya | clarkb: I salute your decision | 16:05 |
| clarkb | pabelanger: anteaya If we test the puppet I am happy to approve now or deferto others willing to babysit | 16:05 |
| anteaya | notmyname: I believe that is the work of jpmaxman and Krenair | 16:05 |
| anteaya | notmyname: thanks for the feedback | 16:05 |
| waynr | zxiiro: by "the 2 questions still in discussion" are you referring to the patch that merges builder.Builder and builder.Jenkins and the patch that causes the resulting builder.Jenkins class to inherit from jenkins.Jenkins? | 16:06 |
| anteaya | notmyname: it seems to be a change that is trimming the wiki spam somewhat | 16:06 |
| jeblair | AJaeger, loquacities: whether we migrate is key to the plan (work related to migrating will probably be more than half the effort, much of which is tedious), so if we're not going to migrate that's a substantial difference. if we haven't decided on that, then we need to pull the spec from voting this week until that is answered with some confidence. | 16:06 |
| waynr | if so, i am +1 on not inheriting from jenkins.Jenkins and continuing discussion on details of the builder.Builder and builder.Jenkins merger | 16:07 |
| zxiiro | waynr: oh yeah those ones | 16:07 |
| *** penguinolog_wfh has quit IRC | 16:07 | |
| zxiiro | waynr: the 2 on etherpad | 16:07 |
| waynr | i did decouple them and went with electrofelix's suggestion of naming the merged class builder.JenkinsManager | 16:08 |
| waynr | I was thinking of renaming the module to "manager" rather than "builder" | 16:08 |
| waynr | or something like that | 16:08 |
| AJaeger | jeblair: it's not wheter - it's what content needs to be moved over and when can we throw old content away | 16:08 |
| zxiiro | waynr: electrofelix: cool sounds good I'll take a look at the rest of the patches later today and see if we can get another set for this week to start merging | 16:08 |
| AJaeger | s/wheter/whether/ | 16:08 |
| fungi | AJaeger: oh, if this is running as the stack user, then that NEUTRON_DIR fallback of $HOME/neutron will be the one devstack-gate prepares at /opt/stack/new/neutron | 16:08 |
| waynr | zxiiro: cool | 16:09 |
| fungi | AJaeger: which will have the correct ZUUL_REF for openstack/neutron already applied | 16:09 |
| AJaeger | fungi, yeah - and then it seems to work (haven't tested). | 16:09 |
| AJaeger | But that looks so hacky... | 16:09 |
| fungi | AJaeger: yes, at a minimum it could use some very visible code comments explaining that's why it uses that as a default | 16:10 |
| waynr | zxiiro electrofelix by the way, I was thinking of adding one more API change prior to the 2.0.0 release...specifically I was thinking that our customer yaml loader class should return a custom dictionary with extra methods that return metadata about where the dictionary came from (file, and maybe beginning line number) | 16:10 |
| AJaeger | I would have expected it to just work - and I'm confused why the functional test works when run as part of neutron-fwaas with depends-on but fails when run as experimental job on neutron. | 16:10 |
| *** ifarkas is now known as ifarkas_afk | 16:11 | |
| AJaeger | I can just add the NEUTRON_DIR change back and be happy - but I would prefer if this wouldn't be needed. | 16:11 |
| waynr | (that change will allow us to greatly improve the usefulness of error output) | 16:12 |
| *** beagles_brb is now known as beagles | 16:12 | |
| zxiiro | waynr: sounds useful to me | 16:12 |
| *** pilgrimstack1 has quit IRC | 16:12 | |
| waynr | cool, i'll see if i can throw up a patch before the sprint on friday so we can talk about it more (if we have time) | 16:13 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Extract /var/log from all nodes https://review.openstack.org/361164 | 16:13 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade: Add ability to list all servers https://review.openstack.org/362241 | 16:13 |
| jeblair | AJaeger: i wrote the spec assuming that everything needed to be moved because you were deleting the things you wanted to already. if you want to change that, it's fine. but if so, i'm going to put it on hold until someone from the docs team writes up what should be moved, and we can add that to the spec. | 16:13 |
| jlvillal | ilyashakhat: Not sure if you are around. I am seeing some strangeness today using the Stackalytics REST API. Not sure if that is a known issue. Has worked fine previous weeks for me. | 16:13 |
| *** xarses_ has joined #openstack-infra | 16:13 | |
| zigo | How come I'm not seeing someone else's CR on packaging-deb in "Incoming reviews" ? | 16:14 |
| zigo | I'm supposed to see them all, as a core reviewer, no? | 16:14 |
| jlvillal | ilyashakhat: Like here I am trying to find my commits: http://stackalytics.com/api/1.0/activity?user_id=happycamp&end_date=1472169600&metric=commits&page_size=5000&start_date=1471651200&project_type=openstack&release=all&company=Intel | 16:14 |
| jlvillal | ilyashakhat: And a lot of other people show up... | 16:14 |
| *** e0ne has quit IRC | 16:14 | |
| clarkb | zigo: no you have to configure your gerrit settings for that.has nothing to do with being a core reviewer | 16:15 |
| AJaeger | jeblair: seems then I'm confused - last infra meeting I got the action item at http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-08-23-19.01.html - "AJaeger start a docs ml thread to refresh constraints around retention of old/unmaintained documents". Isn't that related? | 16:15 |
| zigo | clarkb: Ok, thanks. | 16:15 |
| clarkb | zigo: anyone with an account cando it | 16:15 |
| jeblair | AJaeger: yes, unfortunately, it doesn't look like that has come to a conclusion | 16:16 |
| AJaeger | jeblair: we can go forward and migrate everything for ever. Right now looks like it's 12 GB. | 16:16 |
| AJaeger | jeblair: I'm currently mirroring the complete docs.o.o content locally to look a bit at content. | 16:16 |
| AJaeger | jeblair: I'm at 179861 files and 3.5 GB - so, if we really have 500000 files... | 16:17 |
| pabelanger | clarkb: sure, I can help babysit too | 16:17 |
| zigo | clarkb: Should I manually add each and every project to my watch list? | 16:17 |
| ilyashakhat | jlvillal: hi! this request returns 6 records, all of them are your, while the last one is co-auhtored with you | 16:17 |
| *** tphummel has joined #openstack-infra | 16:17 | |
| zigo | (ie: a way too many...) | 16:17 |
| jeblair | AJaeger: okay. someone (not me) is going to spend some time putting dotfiles into strategic places in the tree to make sure that old data aren't deleted. they won't be happy if we move all the data, they do all that work, then we delete it. :) | 16:17 |
| *** abregman_|mtg has quit IRC | 16:17 | |
| *** drifterza has quit IRC | 16:17 | |
| clarkb | zigo: yes I think thats the way to do it. | 16:17 |
| *** sandanar has joined #openstack-infra | 16:17 | |
| jlvillal | ilyashakhat: Not for me. Strange | 16:18 |
| jeblair | AJaeger: (this is work item #6) | 16:18 |
| openstackgerrit | Merged openstack-infra/project-config: Fuel-qa & fuel-devops: gates changes https://review.openstack.org/362103 | 16:18 |
| ilyashakhat | jlvillal: http://paste.openstack.org/show/564601/ | 16:18 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute009.vanilla to Cacti https://review.openstack.org/362247 | 16:19 |
| *** shashank_hegde has quit IRC | 16:19 | |
| *** sdake has quit IRC | 16:19 | |
| jlvillal | ilyashakhat: http://paste.openstack.org/show/564602/ | 16:19 |
| jlvillal | dueling pastebins ;) | 16:19 |
| jlvillal | ilyashakhat: yours does show me. Odd. | 16:19 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute020.vanilla to Cacti https://review.openstack.org/362250 | 16:19 |
| jlvillal | ilyashakhat: Well odd that mine does not | 16:19 |
| AJaeger | jeblair: I propose an alternative to that: Rename docs.o.o to old-docs.o.o, make it read-only and have new content ;) | 16:20 |
| *** armax has quit IRC | 16:20 | |
| jeblair | AJaeger: that's fine too. i just want someone from the docs team to tell me what they want, and i want it to happen *before* we vote on the spec. | 16:21 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute025.vanilla to Cacti https://review.openstack.org/362251 | 16:21 |
| *** dprince has joined #openstack-infra | 16:21 | |
| jlvillal | ilyashakhat: Now, I am behind an HTTP proxy server at our company. Not sure if that could be an issue. It might cache data | 16:21 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute005.vanilla to Cacti https://review.openstack.org/362252 | 16:22 |
| jlvillal | ilyashakhat: I did SSH to a box outside of work, and I get the correct data. | 16:22 |
| AJaeger | jeblair: I'll send a followup email - let's see whether we can push it forward. I doubt it ;( | 16:22 |
| *** armax has joined #openstack-infra | 16:22 | |
| jlvillal | ilyashakhat: I'll copy my script to my system outside of work | 16:22 |
| *** e0ne has joined #openstack-infra | 16:23 | |
| *** inc0 has joined #openstack-infra | 16:23 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute008.vanilla to Cacti https://review.openstack.org/362254 | 16:24 |
| jlvillal | ilyashakhat: Thank you! | 16:24 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute031.vanilla to Cacti https://review.openstack.org/362255 | 16:24 |
| jlvillal | ilyashakhat: Spasibo. Dos ve dos :) | 16:24 |
| *** rossella_s has quit IRC | 16:25 | |
| *** jpich has quit IRC | 16:25 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute032.vanilla to Cacti https://review.openstack.org/362256 | 16:25 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute033.vanilla to Cacti https://review.openstack.org/362258 | 16:26 |
| jeblair | AJaeger, loquacities: okay, i'm going to wait for loquacities to give me a definitive answer on what to migrate before i put that spec up for approval. i don't have time to drive that kind of requirements gathering myself -- that's up to you. i'm happy to answer any questions about the implications of various options. | 16:26 |
| jeblair | fungi: fyi ^ | 16:26 |
| *** salv-orlando has quit IRC | 16:27 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute034.vanilla to Cacti https://review.openstack.org/362259 | 16:27 |
| *** salv-orlando has joined #openstack-infra | 16:27 | |
| AJaeger | jeblair: since loquacities has now asked the legal list, we have to wait a few days I fear ;( | 16:27 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute035.vanilla to Cacti https://review.openstack.org/362260 | 16:27 |
| jeblair | AJaeger: or not. i mean, if the legal list comes back with an answer, i will probably lose interest in the project entirely. | 16:28 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute036.vanilla to Cacti https://review.openstack.org/362261 | 16:28 |
| *** yamahata has quit IRC | 16:28 | |
| jeblair | this is decidedly a volunteer effort for me. | 16:29 |
| jlvillal | ilyashakhat: An an FYI. Even on my VM (outside my company) I am seeing strangeness. Where data for users, which are not the searched user, is showing up. | 16:29 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute037.vanilla to Cacti https://review.openstack.org/362262 | 16:29 |
| AJaeger | jeblair: yeah ;) | 16:29 |
| *** sdake has joined #openstack-infra | 16:30 | |
| *** abregman has joined #openstack-infra | 16:30 | |
| *** Odd_Bloke has quit IRC | 16:30 | |
| *** salv-orlando has quit IRC | 16:30 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute038.vanilla to Cacti https://review.openstack.org/362264 | 16:30 |
| openstackgerrit | Merged openstack-infra/shade: Support dual-stack neutron networks https://review.openstack.org/357517 | 16:31 |
| fungi | jeblair: yep, i seriously doubt the foundation's legal representation has any concern at all for how long (or even whether) we keep software documentation, but there seemed to be some concern that they might | 16:31 |
| *** mhickey has joined #openstack-infra | 16:31 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute039.vanilla to Cacti https://review.openstack.org/362265 | 16:31 |
| fungi | i'm not going to tell people not to take questions to the legal-discuss ml, but i really hope we're not setting a precedent for asking lawyers every time we want to change community process | 16:31 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute040.vanilla to Cacti https://review.openstack.org/362266 | 16:32 |
| jlvillal | ilyashakhat: I'm wondering if there might be an issue with a load balancer / reverse proxy server | 16:32 |
| *** sarob has joined #openstack-infra | 16:32 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute009.vanilla to Cacti https://review.openstack.org/362247 | 16:32 |
| jlvillal | ilyashakhat: On the Stackalytics side | 16:32 |
| jeblair | fungi: i'm afraid it's already cost us at least a week. also, i'm taking next week off. so 2 weeks. | 16:32 |
| *** jordanP has quit IRC | 16:33 | |
| *** _sarob has joined #openstack-infra | 16:33 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute020.vanilla to Cacti https://review.openstack.org/362250 | 16:34 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute041.vanilla to Cacti https://review.openstack.org/362268 | 16:34 |
| *** abregman has quit IRC | 16:34 | |
| *** Odd_Bloke has joined #openstack-infra | 16:35 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Poll for image to be ready for PUT protocol https://review.openstack.org/361602 | 16:35 |
| *** kaisers_ has joined #openstack-infra | 16:35 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute042.vanilla to Cacti https://review.openstack.org/362269 | 16:36 |
| jeblair | rcarrillocruz: all of hpuswest is removed from cacti. that actually wasn't hard. | 16:36 |
| rcarrillocruz | thx a bunch jeblair | 16:36 |
| rcarrillocruz | i'm closed to wrap up the entire vanilla region | 16:36 |
| *** sarob has quit IRC | 16:36 | |
| openstackgerrit | Merged openstack-dev/pbr: Add more words to a confusing error message https://review.openstack.org/361317 | 16:37 |
| jeblair | rcarrillocruz: when you're done, i'll set up a tree for it | 16:37 |
| *** dizquierdo has quit IRC | 16:37 | |
| rcarrillocruz | ++ | 16:37 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute043.vanilla to Cacti https://review.openstack.org/362271 | 16:38 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute044.vanilla to Cacti https://review.openstack.org/362272 | 16:38 |
| *** Guest35918 is now known as mgagne | 16:39 | |
| *** mgagne has quit IRC | 16:39 | |
| *** mgagne has joined #openstack-infra | 16:39 | |
| *** ddieterly has joined #openstack-infra | 16:39 | |
| *** harlowja_at_home has quit IRC | 16:39 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute045.vanilla to Cacti https://review.openstack.org/362273 | 16:39 |
| *** kaisers_ has quit IRC | 16:40 | |
| *** Apoorva has joined #openstack-infra | 16:40 | |
| zigo | clarkb: https://review.openstack.org/#/c/362096/ <--- Nobody is reviewing it, and it's a blocker. Could you please accept deb-python-hacking as new repo? | 16:41 |
| *** Apoorva has quit IRC | 16:41 | |
| *** Apoorva has joined #openstack-infra | 16:42 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute025.vanilla to Cacti https://review.openstack.org/362251 | 16:43 |
| AJaeger | fungi, here's a neutron-vpnaas change to readd the NEUTRON_DIR: https://review.openstack.org/362274 - is that comment accurate? Could you review, please? | 16:44 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute005.vanilla to Cacti https://review.openstack.org/362252 | 16:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Refactor TaskManager to be more generic https://review.openstack.org/358646 | 16:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add submit_function method to TaskManager https://review.openstack.org/358647 | 16:45 |
| *** lucasagomes is now known as lucas-afk | 16:45 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Change naming style of submitTask https://review.openstack.org/358684 | 16:45 |
| clarkb | pabelanger: we still have a few regions with old/stale xenial/trusty images. I am going to start some foreground image uploads to those regions for those images then we can work on the iptables thing | 16:45 |
| clarkb | pabelanger: have you tested the iptables change? I would just want to make sure if doesn't break existing iptables somehow | 16:45 |
| tlbr | infra-core please review/merge https://review.openstack.org/#/c/362096/ | 16:46 |
| *** jcoufal has joined #openstack-infra | 16:46 | |
| *** Swami has joined #openstack-infra | 16:47 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute008.vanilla to Cacti https://review.openstack.org/362254 | 16:48 |
| pabelanger | clarkb: okay | 16:48 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute031.vanilla to Cacti https://review.openstack.org/362255 | 16:48 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute032.vanilla to Cacti https://review.openstack.org/362256 | 16:48 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute033.vanilla to Cacti https://review.openstack.org/362258 | 16:48 |
| pabelanger | clarkb: I didn't test puppet, but I did confirm enabling ip6tables server was the fix | 16:48 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute034.vanilla to Cacti https://review.openstack.org/362259 | 16:48 |
| clarkb | pabelanger: ya I think we should quickly test the puppetsince that affects every single host we run | 16:48 |
| *** jtomasek has quit IRC | 16:50 | |
| *** Ahharu has quit IRC | 16:50 | |
| mordred | clarkb: btw - the shade fix for image uploads is making its way through the gate | 16:50 |
| clarkb | mordred: cool this was for the thing you found against ovh right? | 16:51 |
| clarkb | if I can get these last few image uploads done then I can finally merge the ntpdate change in d-g | 16:51 |
| clarkb | which should make our average job runtime fall a few minutes which will be nice | 16:51 |
| clarkb | pabelanger: I have an ubuntu-xenial instance somewhere I acn quickly run puppet against that module change with | 16:52 |
| clarkb | pabelanger: if you want to double check the puppet is happy on centos | 16:52 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute035.vanilla to Cacti https://review.openstack.org/362260 | 16:53 |
| pabelanger | clarkb: sure | 16:53 |
| mordred | clarkb: yah. it's possible it's also a problem elsewhere, but we just haven't seen it | 16:55 |
| *** ddieterly is now known as ddieterly[away] | 16:55 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute036.vanilla to Cacti https://review.openstack.org/362261 | 16:55 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute037.vanilla to Cacti https://review.openstack.org/362262 | 16:55 |
| *** _sarob has quit IRC | 16:55 | |
| *** shashank_hegde has joined #openstack-infra | 16:57 | |
| *** tonytan4ever has quit IRC | 16:57 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute038.vanilla to Cacti https://review.openstack.org/362264 | 16:57 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute039.vanilla to Cacti https://review.openstack.org/362265 | 16:58 |
| *** tonytan4ever has joined #openstack-infra | 16:58 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 16:58 |
| ilyashakhat | jlvillal: it appears that one of Stackalytics workers hangs... have to restart it | 16:58 |
| *** mriedem has quit IRC | 16:58 | |
| jlvillal | ilyashakhat: Thanks. I was worried I was going crazy over here ;) | 16:58 |
| *** lucas-afk is now known as lucasagomes | 17:00 | |
| *** jaosorior has quit IRC | 17:00 | |
| *** mpaolino has joined #openstack-infra | 17:01 | |
| *** tonytan4ever has quit IRC | 17:01 | |
| *** ilyashakhat has quit IRC | 17:01 | |
| *** ilyashakhat has joined #openstack-infra | 17:02 | |
| *** chem has quit IRC | 17:03 | |
| *** ddieterly[away] is now known as ddieterly | 17:04 | |
| *** shashank_hegde has quit IRC | 17:04 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute040.vanilla to Cacti https://review.openstack.org/362266 | 17:05 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute041.vanilla to Cacti https://review.openstack.org/362268 | 17:05 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute042.vanilla to Cacti https://review.openstack.org/362269 | 17:05 |
| *** sarob has joined #openstack-infra | 17:06 | |
| *** yamahata has joined #openstack-infra | 17:07 | |
| *** ilyashakhat has quit IRC | 17:07 | |
| clarkb | pabelanger: just waiting for our install puppet script to finish running | 17:08 |
| pabelanger | clarkb: I think I see an issue too | 17:08 |
| *** sarob has quit IRC | 17:08 | |
| *** tonytan4ever has joined #openstack-infra | 17:08 | |
| *** sandanar has quit IRC | 17:09 | |
| *** jtomasek has joined #openstack-infra | 17:10 | |
| *** ddieterly is now known as ddieterly[away] | 17:11 | |
| *** sarob has joined #openstack-infra | 17:11 | |
| pabelanger | clarkb: updating patch | 17:12 |
| clarkb | pabelanger: ok | 17:13 |
| openstackgerrit | greghaynes proposed openstack-infra/irc-meetings: diskimage-builder meeting is bi-weekly https://review.openstack.org/362286 | 17:13 |
| *** psachin has quit IRC | 17:13 | |
| clarkb | pabelanger: I have unpatched puppet-iptables working lcoally on xenial willtest your latest patchset as soon as I can get it | 17:13 |
| *** javeriak has joined #openstack-infra | 17:15 | |
| clarkb | mgagne: any update on networkign in mtl01? are you ready for us to test things there again? | 17:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-iptables: Add ip6tables service support for centos-7 https://review.openstack.org/361449 | 17:16 |
| pabelanger | clarkb: updated patch ^ | 17:16 |
| *** mdrabe has joined #openstack-infra | 17:16 | |
| mgagne | clarkb: yes (and no), we removed the problematic nodes from the pool. This means you should be able to use the others just fine. | 17:16 |
| pabelanger | clarkb: I incorrectly added the service enable outside the chroot | 17:16 |
| *** bin_ has quit IRC | 17:16 | |
| mgagne | clarkb: we are still investigating the other nodes | 17:17 |
| pabelanger | meaning it would fail during DIB build | 17:17 |
| pabelanger | err, not be enabled | 17:17 |
| mgagne | clarkb: lets use double-check before | 17:17 |
| clarkb | mgagne: ok | 17:17 |
| mgagne | clarkb: we found 3 compute nodes showing the problem. We will recheck the others to make sure they aren't affected | 17:17 |
| *** jerryz has joined #openstack-infra | 17:18 | |
| clarkb | pabelanger: also does fedora match the condition of redhat and release >= 7? | 17:19 |
| clarkb | pabelanger: probably what we want if fedora also has ip6tables service but not if it doesn't | 17:20 |
| pabelanger | Ya | 17:20 |
| pabelanger | I didn't check fedora, let me do that now | 17:20 |
| pabelanger | infact, we can just drop the release | 17:20 |
| *** _sarob has joined #openstack-infra | 17:20 | |
| pabelanger | but will confirm fedora needs it | 17:20 |
| clarkb | kk | 17:21 |
| *** mpaolino has quit IRC | 17:21 | |
| *** javeriak has quit IRC | 17:21 | |
| *** javeriak has joined #openstack-infra | 17:21 | |
| pabelanger | ya, same issue with fedora | 17:22 |
| pabelanger | updating patch | 17:22 |
| *** nmagnezi has joined #openstack-infra | 17:22 | |
| clarkb | pabelanger: looks happy on ubuntu (xenial) | 17:22 |
| *** nwkarsten has quit IRC | 17:23 | |
| *** nwkarsten has joined #openstack-infra | 17:24 | |
| *** AnarchyAo has joined #openstack-infra | 17:24 | |
| *** sarob has quit IRC | 17:25 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-iptables: Add ip6tables service support for Red Hat https://review.openstack.org/361449 | 17:25 |
| pabelanger | slowly getting bigger and bigger | 17:25 |
| *** _ari_ has quit IRC | 17:27 | |
| pabelanger | testing fedora now | 17:27 |
| *** _ari_ has joined #openstack-infra | 17:27 | |
| *** nwkarsten has quit IRC | 17:28 | |
| pabelanger | clarkb: works both on centos / fedora now | 17:29 |
| clarkb | pabelanger: ok let me just double check latest patchset on xenial | 17:29 |
| mgagne | clarkb: ok, on hold for now. we found a lot more nodes with the problem. | 17:31 |
| clarkb | pabelanger: +2 lgtm | 17:31 |
| * mgagne takes torches and pitchforks to netadmin's office | 17:31 | |
| anteaya | mgagne: isn't the closet outside that office where they are stored? | 17:32 |
| *** trown is now known as trown|lunch | 17:32 | |
| *** yamahata__ has quit IRC | 17:33 | |
| mgagne | anteaya: we use them daily tbh, I really like to blame our netadmin team for our problems =) | 17:33 |
| anteaya | saves time I guess | 17:33 |
| mgagne | coworker blamed them once because his keyboard stopped working. | 17:33 |
| anteaya | ha ha ha | 17:33 |
| clarkb | as a former network ops person I am familiar with this behavior | 17:33 |
| anteaya | mgagne: how did the work in terms of finding a solution? | 17:34 |
| *** shashank_hegde has joined #openstack-infra | 17:34 | |
| clarkb | pabelanger: so we just need one more reviewer :) | 17:34 |
| mgagne | "When you unplugged the fiber, my keyboard stopped working." | 17:34 |
| anteaya | clarkb: did you have a torches and pitchforks defense? | 17:34 |
| anteaya | did you/do you? | 17:34 |
| mgagne | anteaya: working on it =) | 17:34 |
| anteaya | mgagne: ha ha ha, good luck in that direction | 17:34 |
| clarkb | anteaya: our greatest defense was root causuing things to put the blame back on the accusers | 17:35 |
| anteaya | nothing like facts to clear the air | 17:35 |
| clarkb | once we discovered that ftp transfers were super slow because microsoft windows ftp server set its tcp window to like 512 byes | 17:35 |
| clarkb | that was a fun one | 17:35 |
| fungi | oh, yeah, it's terrible | 17:36 |
| *** electrofelix has quit IRC | 17:37 | |
| clarkb | woot one image upload completed successfully | 17:37 |
| clarkb | now I need about 4 more | 17:37 |
| anteaya | yay successful image upload | 17:37 |
| *** mriedem has joined #openstack-infra | 17:38 | |
| *** zul has quit IRC | 17:39 | |
| pabelanger | fungi: mind reviewing 361449 when you have the bandwidth | 17:39 |
| *** ihrachys has quit IRC | 17:40 | |
| *** _nadya_ has quit IRC | 17:41 | |
| jeblair | fungi, clarkb, mgagne: when every application is a network application, every problem is a network problem :) | 17:41 |
| mgagne | :D | 17:41 |
| *** tonytan4ever has quit IRC | 17:42 | |
| *** zul has joined #openstack-infra | 17:42 | |
| *** AnarchyAo has quit IRC | 17:42 | |
| pabelanger | #status log email sent to rackspace about rax-iad networking issue. The region is still disabled in nodepool | 17:42 |
| openstackstatus | pabelanger: finished logging | 17:43 |
| *** _ari_ has quit IRC | 17:43 | |
| *** _ari_ has joined #openstack-infra | 17:43 | |
| *** tosky has quit IRC | 17:43 | |
| *** Sukhdev has quit IRC | 17:45 | |
| *** AnarchyAo has joined #openstack-infra | 17:47 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: [sahara] scenario jobs refactoring https://review.openstack.org/361185 | 17:47 |
| clarkb | woot 2 uploads down. Slowly but surely this is working | 17:47 |
| *** _ari_ has quit IRC | 17:48 | |
| * clarkb hopes that the d-g ntpdate change can finally get in today | 17:48 | |
| *** _ari_ has joined #openstack-infra | 17:48 | |
| anteaya | clarkb: do it in the style of the count | 17:49 |
| *** sputnik13 has joined #openstack-infra | 17:49 | |
| *** eharney_ has joined #openstack-infra | 17:49 | |
| * anteaya pictures clarkb with lightening flashes and bats | 17:49 | |
| clarkb | one image ahahaha! two images ahahaha! ? | 17:49 |
| anteaya | there we go | 17:49 |
| anteaya | thank you | 17:49 |
| anteaya | :) | 17:50 |
| *** _ari_ has quit IRC | 17:50 | |
| anteaya | why do I now crave a chocolate chip cookie? | 17:50 |
| mordred | mmm. cookie | 17:50 |
| *** eharney has quit IRC | 17:51 | |
| *** markvoelker has joined #openstack-infra | 17:53 | |
| amrith | mtreinish, would you please re-review https://review.openstack.org/#/c/362203/. I believe that it now reflects all of your comments. thanks! | 17:55 |
| *** eharney_ is now known as eharney | 17:55 | |
| *** rajinir has joined #openstack-infra | 17:55 | |
| *** _ari_ has joined #openstack-infra | 17:56 | |
| *** cody-somerville has joined #openstack-infra | 17:57 | |
| mtreinish | amrith: ok, I'll take another look after I find some food | 17:57 |
| *** asettle has joined #openstack-infra | 17:58 | |
| *** esikachev has joined #openstack-infra | 17:58 | |
| *** _nadya_ has joined #openstack-infra | 17:58 | |
| *** _ari_ has quit IRC | 17:59 | |
| *** isq has joined #openstack-infra | 18:01 | |
| *** degorenko is now known as _degorenko|afk | 18:01 | |
| *** adrian_otto has joined #openstack-infra | 18:01 | |
| *** Sukhdev has joined #openstack-infra | 18:01 | |
| *** tqtran has joined #openstack-infra | 18:01 | |
| *** _ari_ has joined #openstack-infra | 18:02 | |
| * anteaya guesses mtreinish will also be looking for a cookie | 18:02 | |
| *** esikachev has quit IRC | 18:02 | |
| *** dtantsur is now known as dtantsur|afk | 18:02 | |
| *** asettle has quit IRC | 18:03 | |
| *** csomerville has joined #openstack-infra | 18:04 | |
| *** mugsie_ is now known as mugsie | 18:05 | |
| openstackgerrit | Mike Fedosin proposed openstack-infra/project-config: Create python-glareclient https://review.openstack.org/362307 | 18:05 |
| openstackgerrit | Merged openstack-infra/puppet-iptables: Add ip6tables service support for Red Hat https://review.openstack.org/361449 | 18:07 |
| *** cody-somerville has quit IRC | 18:07 | |
| *** Na3iL has quit IRC | 18:09 | |
| *** markvoelker has quit IRC | 18:09 | |
| *** _ari_ has quit IRC | 18:10 | |
| *** trown|lunch is now known as trown | 18:11 | |
| *** jcoufal has quit IRC | 18:11 | |
| *** _ari__ has joined #openstack-infra | 18:12 | |
| *** _ari__ has quit IRC | 18:13 | |
| openstackgerrit | Merged openstack-infra/shade: Poll for image to be ready for PUT protocol https://review.openstack.org/361602 | 18:17 |
| clarkb | down to one image upload remaining | 18:17 |
| anteaya | yay | 18:18 |
| openstackgerrit | Merged openstack-infra/project-config: Enable java test as voting on monasca-api https://review.openstack.org/361511 | 18:18 |
| openstackgerrit | Merged openstack-infra/project-config: Added new neutronclient gate to test against stable server release https://review.openstack.org/361603 | 18:19 |
| *** isq has quit IRC | 18:19 | |
| *** ddieterly[away] is now known as ddieterly | 18:19 | |
| *** annegent_ has quit IRC | 18:19 | |
| *** isq has joined #openstack-infra | 18:20 | |
| *** _ari_ has joined #openstack-infra | 18:21 | |
| *** ihrachys has joined #openstack-infra | 18:22 | |
| *** kaisers_ has joined #openstack-infra | 18:24 | |
| *** cardeois has joined #openstack-infra | 18:24 | |
| *** jcoufal has joined #openstack-infra | 18:25 | |
| *** isq has quit IRC | 18:26 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute043.vanilla to Cacti https://review.openstack.org/362271 | 18:26 |
| *** nwkarsten has joined #openstack-infra | 18:26 | |
| *** chem has joined #openstack-infra | 18:28 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config: Make neutron-fwaas functional job not experimental https://review.openstack.org/359320 | 18:29 |
| *** kaisers_ has quit IRC | 18:29 | |
| openstackgerrit | Dean Troyer proposed openstack/os-client-config: Add prompting for KSA options https://review.openstack.org/362314 | 18:30 |
| *** kzaitsev_mb has quit IRC | 18:30 | |
| amrith | thx mtreinish | 18:32 |
| dougwig | this review is adding tests to a job that is already at 75 minutes (neutron dvr). what's the current job length target? are longer jobs ok? i'm hesitant to add a +1 to this and make a long job longer without knowing that it's something that's allowed/desired/supported. https://review.openstack.org/#/c/355344/ | 18:32 |
| fungi | dougwig: ideally, job runtimes should try to target one hour or less | 18:33 |
| AJaeger | dougwig: IMHO especially in the integrated gate | 18:33 |
| amrith | dougwig, I'm trying to take a machette to some trove jobs that go past 120m on a bright sunny day with a cool breeze blowing. On a day like to day when the CI is being hammered, they go longer. 75 m is long but not outrageous. | 18:33 |
| *** pvaneck has joined #openstack-infra | 18:33 | |
| *** lucasagomes is now known as lucas-dinner | 18:33 | |
| fungi | a couple hours and we stop caring much if we have to abort it while doing maintenance or troubleshooting a problem with the ci system | 18:33 |
| anteaya | amrith: the ci system has never been in better shape than today | 18:34 |
| *** isq has joined #openstack-infra | 18:34 | |
| dougwig | well, i don't want to say "yes" to this review and make the crunch worse, but nor do i want to block it if it's accepted. my last feeling from reviewing p-c in general is that this job is too long. | 18:34 |
| anteaya | today is a bright sunny day with a cool breeze blowing if ever there was one | 18:34 |
| amrith | anteaya, great to know. | 18:34 |
| *** salv-orlando has joined #openstack-infra | 18:34 | |
| dougwig | amrith: 62/37 is an empty 8 lane superhighway. :) | 18:35 |
| amrith | anteaya, hello! | 18:35 |
| fungi | we've cranked up to 1750jph and zuul is still not backlogged on resources | 18:35 |
| amrith | oh great, I just need to drag my jobs onto this supercomputer ... | 18:35 |
| AJaeger | everybody on vacation? | 18:35 |
| anteaya | dougwig: follow your heart | 18:35 |
| fungi | AJaeger: this week may be slightly slower, but i think it's mostly all the extra quota we got from osic | 18:36 |
| anteaya | amrith: if your jobs are running in the check and gate queues they already | 18:36 |
| fungi | s/slower/lower activity/ | 18:36 |
| anteaya | amrith: already are | 18:36 |
| *** jcoufal has quit IRC | 18:36 | |
| clarkb | yes osic is really helping keep things move quickly | 18:36 |
| mordred | clarkb: ++ | 18:36 |
| cloudnull | :) | 18:36 |
| AJaeger | fungi: I think both together - and no real blocking bugs. So, thanks osic folks! | 18:36 |
| *** jcoufal has joined #openstack-infra | 18:37 | |
| amrith | well, I'm seeing jobs queued for quite a while | 18:37 |
| amrith | on occasions today | 18:37 |
| dougwig | if everyone is getting bored, i could break ipv6 again. | 18:37 |
| anteaya | amrith: which jobs? | 18:37 |
| amrith | like now, I have one sitting enqueued for 11m | 18:37 |
| fungi | yeah, oldest changes in check and gate pipelines are around 2hrs (not waiting for nodes, but waiting for their jobs to finish running). biggest indicator though is that there are basically no changes in the post pipeline, which is the lowest-priority pipeline | 18:37 |
| anteaya | dougwig: no | 18:37 |
| AJaeger | dougwig: NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO | 18:37 |
| AJaeger | please ;) | 18:37 |
| dougwig | :) | 18:37 |
| amrith | 326311 in the check gate | 18:37 |
| cloudnull | dougwig: please no | 18:37 |
| amrith | check | 18:37 |
| clarkb | dougwig: please now I am almost about to catch up on my backlog list of work :) | 18:37 |
| amrith | check gate would be meaningless ... check queue | 18:38 |
| * fungi envies clarkb | 18:38 | |
| dougwig | you all heard it, he said "please now". | 18:38 |
| anteaya | amrith: check queue, or check pipeline | 18:38 |
| fungi | surely you mean your priority backlog, and not your backlogged backlog ;) | 18:38 |
| *** tonytan4ever has joined #openstack-infra | 18:38 | |
| clarkb | fungi: yes that | 18:38 |
| anteaya | amrith: gate queue, or gate pipeline | 18:38 |
| sdague | clarkb: it also helps that we have 0 uncategorized gate issues for the last 48 hours - http://status.openstack.org/elastic-recheck/data/integrated_gate.html | 18:38 |
| anteaya | check and gate are mutually exclusive | 18:38 |
| *** stewie925 has quit IRC | 18:38 | |
| clarkb | its get d-g thing merged, finish cleaning up image leaks, work on mtl01 when mgagne says go for it, then work on xenialing | 18:39 |
| *** eeiden has joined #openstack-infra | 18:39 | |
| dougwig | anteaya: fungi: AJaeger: i'm leaving a -1 on the 75m job getting longer, until I at least hear why it's so long, and that we have a plan to address it. armax can overrule me if he wishes. | 18:39 |
| *** ihrachys has quit IRC | 18:39 | |
| amrith | anteaya, I know. hence I said check queue | 18:39 |
| armax | dougwig: where? | 18:39 |
| anteaya | amrith: I have no results in status.openstack.org/zuul for 326311 | 18:39 |
| amrith | anteaya, I'm looking at zuul.openstack.org | 18:40 |
| anteaya | dougwig: thank you for following your heart, happy to bear witness | 18:40 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Collect more logs for translation jobs https://review.openstack.org/362315 | 18:40 |
| amrith | anteaya, typo: 362311 | 18:40 |
| AJaeger | dougwig: I commented as well, will put it at the bottom of my backlog for now ;) | 18:40 |
| *** hockeynut has joined #openstack-infra | 18:40 | |
| AJaeger | 362315 should give us on translation jobs more logs - in case we have again to investigate zuul-cloner problems... | 18:41 |
| pabelanger | Ya, I'm enjoying a relaxing day today over here. | 18:41 |
| sdague | amrith: it's still acquiring nodes for all the jobs | 18:41 |
| sdague | amrith: 362313,1 ? | 18:41 |
| amrith | sdague, yes | 18:41 |
| clarkb | arg the last image upload failed | 18:41 |
| *** salv-orlando has quit IRC | 18:42 | |
| clarkb | things were going so well | 18:42 |
| anteaya | amrith: patch has been in check 1 hour 25 minutes: https://review.openstack.org/#/c/362311/ | 18:42 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Use priority queues for worker threads https://review.openstack.org/362318 | 18:42 |
| anteaya | amrith: what is your definition of 'quite a while'? | 18:42 |
| amrith | anteaya, the change has been in existance only 20 minutes. I assume that 1h25m is ETA. I was just curious about the queue time of ~15m (for this one). I've seen a couple queue for 30m today. | 18:44 |
| clarkb | the priorities on the gearman queues will still take effect so gate is serviced before check and instance boot isn't instantaneous | 18:44 |
| pabelanger | anteaya: isn't 362311 only 18mins in check queue? | 18:44 |
| amrith | pabelanger, yes | 18:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Change naming style of submitTask https://review.openstack.org/358684 | 18:44 |
| fungi | anteaya: i think the status is actually showing that it's been in the check pipeline for 15 minutes and has another hour and 25 minutes to go based on the estimated runtimes of certain jobs (particularly gate-trove-functional-dsvm-mysql) | 18:44 |
| amrith | 1h25 or 1h41 is the ETA | 18:44 |
| amrith | but not a problem | 18:45 |
| anteaya | fungi: ah yes you are correct, my mistake | 18:45 |
| amrith | this is about average as I've seen at this time of the release cycle | 18:45 |
| anteaya | amrith: yeah, it doesn't get any better than this | 18:45 |
| *** ilyashakhat has joined #openstack-infra | 18:45 | |
| pabelanger | amrith: http://grafana.openstack.org/dashboard/db/nodepool?panelId=17&fullscreen show the time to ready for our regions | 18:45 |
| *** Sukhdev has quit IRC | 18:46 | |
| anteaya | amrith: express your gratitude to your recepient of choice | 18:46 |
| anteaya | I've never seen check gate and post in better shape | 18:46 |
| *** ddieterly is now known as ddieterly[away] | 18:46 | |
| pabelanger | and since gate has a higher priority, as new ubuntu-xenial nodes came online, nodepool allocated them to the gate queue | 18:46 |
| sdague | pabelanger: that's a neat data set | 18:47 |
| fungi | amrith: actually, at this point (immediately prior to feature freeze) in previous cycles we were dealing with backlogs multiple days in length. this is way faster | 18:47 |
| sdague | I guess that speaks to the fact that we're getting spread all over the rax region, but on the others they have us in a more dedicated set of nodes so we get the glance image caching | 18:47 |
| jeblair | pabelanger: did we decide to do a straight-up comparison benchmark of our providers? | 18:47 |
| fungi | amrith: the jobs take a similar amount of time to run, but changes are spending almost no time waiting for nodes to run them on | 18:47 |
| clarkb | sdague: the images in rax are also like 3-4x the size of the other clouds | 18:48 |
| cloudnull | yikes ^ | 18:48 |
| clarkb | so the affects are more pronounced when you don't get them cached | 18:48 |
| jeblair | i ask because we spent a few years avoiding creating graphs that measured performance differences between providers, since we didn't want to be in the benchmarking buisiness. | 18:48 |
| amrith | fungi, this is awesome compared to last cycle!!! | 18:49 |
| tlbr | mordred, could you please review as our PTL https://review.openstack.org/362096 ? | 18:49 |
| fungi | yeah, we've in fact tried very hard to _not_ compare performance of our donor providers because we don't want to make any of them look bad | 18:49 |
| pabelanger | jeblair: Right, I only use it to add in debugging failures rather then benchmark clouds. | 18:49 |
| clarkb | pabelanger: the ip6tables change has merged, we need to rebuild centos7 images ya? | 18:49 |
| sdague | clarkb: what makes them so much bigger? | 18:49 |
| amrith | I'm still trying to get people to hold off on the nitpicky changes like remove #noqa on one line, or correct a typo in a comment in a file that is no longer used | 18:49 |
| clarkb | sdague: vhd vs qcow2 | 18:50 |
| amrith | but I can't stop all of those ... | 18:50 |
| anteaya | we are grateful to all our providers | 18:50 |
| mat128 | while you guys are not trying to compare providers, I must say I'm happy to see those graphs | 18:50 |
| *** ddieterly[away] is now known as ddieterly | 18:50 | |
| pabelanger | clarkb: ya, for osic-cloud1 | 18:50 |
| sdague | clarkb: ah, and there is no way to compress that? | 18:50 |
| clarkb | sdague: not that I have been able to determine | 18:50 |
| clarkb | sdague: and at least with raw you can store them sparesely on disk, I don't see aw ay to do that with vhd | 18:50 |
| clarkb | sdague: so its worse than raw and qcow2 as a format imo | 18:50 |
| sdague | ok, np. still an interesting dataset. Also good to know that the average ready time is 1 - 5 mins | 18:50 |
| fungi | mat128: granted, knowing those things can be useful, but we also don't want to scare off providers who might be willing to donate resources | 18:51 |
| jeblair | well, it looks like when i went around and got permission, i said we may end up making graphs like that public, so i guess we're okay :) | 18:51 |
| *** jcoufal has quit IRC | 18:51 | |
| anteaya | jeblair: I remember you saying that | 18:51 |
| sdague | maybe we can get one of the xen folks to comment on any optimization there | 18:51 |
| openstackgerrit | Merged openstack-infra/shade: Cleanup old internal/external network handling https://review.openstack.org/362223 | 18:51 |
| jeblair | so carry on :) | 18:52 |
| jeblair | pabelanger: ^ :) | 18:52 |
| pabelanger | great, thanks | 18:52 |
| *** jcoufal has joined #openstack-infra | 18:52 | |
| jeblair | pabelanger: i wonder if we could move the whisper files on graphite for old clouds into a new hierarchy... | 18:53 |
| jeblair | pabelanger: maybe an archive hierarchy | 18:53 |
| jeblair | or just delete them if we don't care | 18:53 |
| jeblair | pabelanger: so that we don't have hpcloud on there | 18:53 |
| pabelanger | Right, I was trying to think of a way of dealing with that. I'd hate to delete them, if moving them is an option, I'd like to see that | 18:54 |
| jeblair | sdague, clarkb: i thought we benefited from an on-host image cache in rax? | 18:54 |
| jeblair | pabelanger: should be as simple as a 'mv' command. we may need to stop carbon-cache while we do it. | 18:54 |
| clarkb | jeblair: I think we do. What sdague is saying though is that since we don't have dedicated hosts we can and likely do oftenish land on hypervisors withut the image cached | 18:54 |
| clarkb | jeblair: whereas in the other providers by having dedicated hosts we will hit the cache more often | 18:55 |
| jeblair | clarkb: yeah, are we seeing variation in rax? | 18:55 |
| clarkb | jeblair: we are seeing 5-12 minuets based on that graph | 18:55 |
| fungi | the only suggestions i can find for making vhd images smaller is how to resize their filesystems, totally not useful :/ | 18:55 |
| clarkb | in rax dfw | 18:55 |
| *** tongli has quit IRC | 18:56 | |
| jeblair | clarkb: i'd be careful looking at spikes though -- that number gets less reliable the busier nodepool is. it can spend a long time getting around to servicing a thread. | 18:56 |
| pabelanger | jeblair: sure, if it is 'mv' I think we should do that | 18:57 |
| *** annegentle has joined #openstack-infra | 18:57 | |
| *** dtardivel has quit IRC | 18:57 | |
| sdague | jeblair: the actual cpu launch should be pretty quick | 18:57 |
| clarkb | jeblair: you would expect that to be consistent across the clouds though if it was a nodepool starvation issue though right? | 18:57 |
| *** annegentle has quit IRC | 18:57 | |
| sdague | most of the launch time is typically because of image transfer costs | 18:57 |
| sdague | and the fact that there is a 5m floor on rax, I wouldn't expect that if we had a locally cached image | 18:58 |
| jeblair | clarkb: yes, and when i look at some rax spikes, i see spikes from other providers | 18:59 |
| jeblair | sdague: i agree about the floor calling the image cache into question | 18:59 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute044.vanilla to Cacti https://review.openstack.org/362272 | 18:59 |
| fungi | however there may be more than one influence on boot time | 19:00 |
| sdague | jeblair: and, with a custom image, it does not surprise me we'd have very bad caching if we are spread widely in a region | 19:00 |
| fungi | remember, we also have seen that rackspace's ip address assignment mechanism holds up completion of the boot process | 19:01 |
| sdague | anyway, it was mostly a hypothesis | 19:01 |
| openstackgerrit | Merged openstack-infra/shade: Refactor TaskManager to be more generic https://review.openstack.org/358646 | 19:01 |
| sdague | fungi: ah, sure, that could also be a part | 19:01 |
| *** edtubill has quit IRC | 19:01 | |
| fungi | so maybe sometimes boot waits a long time for an image cache update, but other times we could just be seeing how long it takes rackspace to complete address assignment through their custom network stack | 19:01 |
| *** ihrachys has joined #openstack-infra | 19:02 | |
| *** annegentle has joined #openstack-infra | 19:02 | |
| fungi | hard to know as it's a big black box to us | 19:02 |
| pabelanger | It would be interesting to follow up on it, just in rax-iad we've gone from 3.8min time to ready to 6min when I disabled it last week. | 19:03 |
| openstackgerrit | Merged openstack-infra/shade: Add submit_function method to TaskManager https://review.openstack.org/358647 | 19:03 |
| *** xarses_ is now known as xarses | 19:04 | |
| cardeois | AJaeger I've added some details on my review about NPM projects using DSVM gates https://review.openstack.org/#/c/362222/ let me know if you have any other question | 19:06 |
| AJaeger | cardeois: thanks. Still, I wonder why we only notice this now | 19:06 |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: [sahara] scenario jobs refactoring https://review.openstack.org/361185 | 19:06 |
| *** jcoufal has quit IRC | 19:06 | |
| cardeois | AJaeger Because js-openstack-lib is the first project to use any dsvm gate. We introduce this gates few weeks ago as non voting | 19:07 |
| *** esikachev has joined #openstack-infra | 19:07 | |
| *** jcoufal has joined #openstack-infra | 19:07 | |
| AJaeger | cardeois: and there it didn't fail? | 19:07 |
| pabelanger | clarkb: going to start our centos-7 build | 19:07 |
| clarkb | pabelanger: ok | 19:07 |
| anteaya | cardeois: jobs, you introduced jobs | 19:07 |
| *** pt_15 has joined #openstack-infra | 19:09 | |
| cardeois | anteaya Yes right sorry. And AJaeger I can't tell if it won't fail, it depends on another review: https://review.openstack.org/#/c/361525/ This one setup sets up a devstack local.conf with CORS headers, and it needs sudo to do so | 19:09 |
| AJaeger | cardeois: +2 for the job | 19:12 |
| * AJaeger waves good night | 19:12 | |
| cardeois | AJaeger awesome thanks ! | 19:12 |
| anteaya | AJaeger: thanks for the day, g'night | 19:12 |
| *** Na3iL has joined #openstack-infra | 19:12 | |
| anteaya | cardeois: do you need https://review.openstack.org/#/c/361525/5 to merge prior to 362222? | 19:12 |
| cardeois | anteaya no that's alright, I'll manage this one. Thanks | 19:13 |
| anteaya | so if I approve 362222 with the job pass or fail right now? | 19:13 |
| *** kushal has quit IRC | 19:14 | |
| cardeois | If you merge 362222, 361525 review will pass | 19:14 |
| anteaya | okay thank you, so 361525 is dependant on 362222 | 19:15 |
| * anteaya just likes typing all those 2s | 19:15 | |
| cardeois | anteaya yes indeed, sorry for the confusion | 19:15 |
| anteaya | ah you put the depends on in a comment, instead of the commit message, I see it now | 19:16 |
| anteaya | thank you | 19:16 |
| cardeois | btw is there any way to declare cross project dependencies in gerrit? | 19:16 |
| anteaya | in future if you use Depends-on: and Needed-By: in commit messages it helps | 19:16 |
| anteaya | let me get you the docs | 19:16 |
| *** sputnik13 has quit IRC | 19:16 | |
| cardeois | anteaya awesome thanks, I didn't know that | 19:16 |
| anteaya | cardeois: http://docs.openstack.org/infra/manual/developers.html#cross-repository-dependencies | 19:17 |
| anteaya | cardeois: thanks for listening | 19:17 |
| *** javeriak has quit IRC | 19:18 | |
| clarkb | ok with the backlog of image uploads ubuntu-trusty rax-ord likely won't get uploaded for a while. I can continue to wait for it before approving the d-g change to use ntpdate or we can accept that ntpdate will fail there (jobs won't fail unless the time later jumps and causes a timeout) | 19:19 |
| clarkb | infra-root ^ thoughts on whether that is an acceptable risk or should we just wait? | 19:20 |
| clarkb | its a trade off between no ntp updates and waiting for ntp-wait | 19:20 |
| openstackgerrit | Andy Smith proposed openstack-infra/project-config: Move puppet-qdr to Puppet OpenStack https://review.openstack.org/362333 | 19:22 |
| fungi | clarkb: the risk is probably relatively small... i think worst case is that if we're booting nodes with skewed system time there, the timestamps in our job logs will be off a little? | 19:24 |
| fungi | (or maybe off a lot, but still...) | 19:24 |
| clarkb | fungi: ya which they would probably be anyways with ntp-wait since it won't step | 19:24 |
| fungi | indeed. i'd be fine with going ahead | 19:25 |
| openstackgerrit | Merged openstack-infra/project-config: NPM Projects: sudo needed for DSVM jobs https://review.openstack.org/362222 | 19:25 |
| *** tkelsey has joined #openstack-infra | 19:28 | |
| fungi | cardeois: ^ now that it's merged, probably in the neighborhood of 15 minutes before it takes effect | 19:28 |
| openstackgerrit | Merged openstack-infra/system-config: Add compute045.vanilla to Cacti https://review.openstack.org/362273 | 19:29 |
| *** ddieterly has quit IRC | 19:31 | |
| *** tkelsey has quit IRC | 19:33 | |
| cardeois | fungi ok thanks, I was indeed wondering how long I should wait | 19:33 |
| *** adrian_otto has quit IRC | 19:33 | |
| anteaya | cardeois: nice work on the commit message for 361525 | 19:33 |
| cardeois | anteaya Thanks, the code doesn't speak by itself so I've thought it was important to be clear about why this review is needed | 19:35 |
| *** wajdi has joined #openstack-infra | 19:35 | |
| anteaya | cardeois: can you spend more time in channel please? | 19:35 |
| * anteaya signs up for cardeois' newsletter | 19:35 | |
| cardeois | What do you mean? | 19:35 |
| anteaya | I mean I agree with you | 19:36 |
| anteaya | and preach that message far and wide | 19:36 |
| cardeois | ahah oh I see | 19:36 |
| anteaya | I'm grateful you are of that opinion | 19:36 |
| anteaya | thank you | 19:36 |
| cardeois | sure no pb! | 19:36 |
| pabelanger | jeblair: clarkb: So, a follow up to http://paste.openstack.org/show/564641/ from last week. I don't see the data in gearman missing (I have tcpdumps). I am starting to wonder if we are just not waiting long enough in nodepool for the job.waitForCompletion(). And since we don't do any validation of job.data, we raise the exception | 19:36 |
| anteaya | :) | 19:36 |
| *** SotK_ is now known as SotK | 19:36 | |
| *** Benj_ has joined #openstack-infra | 19:37 | |
| *** sputnik13 has joined #openstack-infra | 19:37 | |
| pabelanger | clarkb: fungi: As long as we note the possible issue, I'm fine moving forward | 19:38 |
| openstackgerrit | Ken Giusti proposed openstack-infra/project-config: Update the oslo.messaging-dsvm-functional test https://review.openstack.org/358745 | 19:38 |
| clarkb | pabelanger: oh so its no longer blocking on the wait, continuing before it gets the gearman message then failing? | 19:38 |
| clarkb | pabelanger: at the very least we can probably make that failure condition a bit more clear | 19:38 |
| *** yaume has quit IRC | 19:38 | |
| clarkb | like error timed out waiting instead of contents are [] | 19:38 |
| pabelanger | clarkb: I think so, but would need jeblair to confirm | 19:39 |
| *** godp1301_ has joined #openstack-infra | 19:39 | |
| *** xhku has joined #openstack-infra | 19:39 | |
| *** esikachev has quit IRC | 19:39 | |
| *** GG has joined #openstack-infra | 19:39 | |
| pabelanger | clarkb: self.nodepool.gearman_client.submitJob(job, timeout=300) is the code in question | 19:39 |
| pabelanger | we then wait | 19:39 |
| *** kzaitsev_mb has joined #openstack-infra | 19:40 | |
| pabelanger | Actually | 19:40 |
| pabelanger | Hmm | 19:40 |
| pabelanger | we job.waitForCompletion() with no timeout | 19:40 |
| clarkb | there is probably a default though | 19:40 |
| clarkb | *default timeout | 19:40 |
| rajinir | https://www.irccloud.com/pastebin/joXLdB2c/dracclient | 19:41 |
| *** xhku has left #openstack-infra | 19:41 | |
| *** salv-orlando has joined #openstack-infra | 19:42 | |
| rajinir | ifarkas_afk: Need help with python-dracclient installation | 19:42 |
| clarkb | pabelanger: fungi re ntp the more I think about this the les I am concerned as the current setup isn't really fixing the issue as well | 19:43 |
| openstackgerrit | Varun Gadiraju proposed openstack-infra/project-config: works without %% agent https://review.openstack.org/354344 | 19:43 |
| clarkb | so at least in the case with no ntpdate we will move along quicker | 19:43 |
| clarkb | I will recheck my change and look at more recent results to double check | 19:44 |
| *** Na3iL has quit IRC | 19:44 | |
| *** salv-orlando has quit IRC | 19:44 | |
| *** edtubill has joined #openstack-infra | 19:45 | |
| clarkb | the daily image uploads have 64 images going | 19:45 |
| *** pradk has quit IRC | 19:46 | |
| *** thorst_ has joined #openstack-infra | 19:46 | |
| openstackgerrit | Varun Gadiraju proposed openstack-infra/project-config: Migrating vars that devstack_gate uses to vars in DEVSTACK_LOCAL_CONFIG https://review.openstack.org/354344 | 19:47 |
| *** GG has left #openstack-infra | 19:47 | |
| *** sdake has quit IRC | 19:48 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: use utf-8 for reading the readme files https://review.openstack.org/362345 | 19:48 |
| *** thorst has quit IRC | 19:48 | |
| *** esikachev has joined #openstack-infra | 19:49 | |
| openstackgerrit | Varun Gadiraju proposed openstack-infra/project-config: Migrating vars that devstack_gate uses to vars in DEVSTACK_LOCAL_CONFIG https://review.openstack.org/354344 | 19:49 |
| *** annegentle has quit IRC | 19:50 | |
| *** zul has quit IRC | 19:51 | |
| *** annegentle has joined #openstack-infra | 19:51 | |
| clarkb | I am going to restart my short oneliner to delete leaked images in rax-iad | 19:52 |
| clarkb | it will likely continue to throw 500s but I am around topay attention to it now | 19:52 |
| *** markvoelker has joined #openstack-infra | 19:52 | |
| clarkb | jroll: JayF ^ you don't happen to know if glance image deletes throwing 500s but showing the image is gone would interest rax support/ | 19:53 |
| JayF | clarkb: I can't speak for support; but I'd imagine it's something our technical folks would wanna know about | 19:53 |
| jroll | clarkb: as in, a delete request always returns 500, but appears to always work? | 19:53 |
| clarkb | jroll: ya exactly | 19:53 |
| jroll | lol | 19:53 |
| JayF | jroll: clarkb; not always, right? just sometimes? | 19:54 |
| clarkb | JayF: always in my small sample last friday | 19:54 |
| jroll | clarkb: I'll file a bug, if you want to give any more info that'd be awesome | 19:54 |
| clarkb | jroll: ya let me go ahead and run one and get you the output | 19:54 |
| jroll | thanks | 19:54 |
| clarkb | well thats interesting it just worked now :) | 19:55 |
| clarkb | so not always just on fridays maybe | 19:56 |
| jroll | lovely! | 19:56 |
| clarkb | let me get the script running and if it catches any I will share | 19:56 |
| jroll | cool | 19:56 |
| *** sshnaidm is now known as sshnaidm|afk | 19:57 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Add a wiki-dev.o.o server to test newer mediawiki https://review.openstack.org/358246 | 19:59 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Use a host group for wiki.o.o https://review.openstack.org/362351 | 19:59 |
| *** pradk has joined #openstack-infra | 19:59 | |
| *** AnarchyAo has quit IRC | 20:01 | |
| *** inc0 has quit IRC | 20:02 | |
| fungi | i saw gerrit httpd threads reach 32 at one point earlier, but they're back down to the new minimum of 20 again for now | 20:03 |
| clarkb | fungi: I checked melody when I first got to computer this morning and the indicators there all looked fine | 20:03 |
| *** kaisers_ has joined #openstack-infra | 20:03 | |
| *** derekh has quit IRC | 20:03 | |
| fungi | so at least for that few minutes or whatever we would have previously been hitting the old max and probably returning errors | 20:03 |
| clarkb | I also haven't seen any reports of gerrit being extra slow or 500ing so maybe its happy now | 20:03 |
| *** inc0 has joined #openstack-infra | 20:04 | |
| fungi | anyway, i need to step out to run some errands/get early dinner, but will return in a while | 20:04 |
| *** mcarden has quit IRC | 20:05 | |
| *** gus has quit IRC | 20:05 | |
| *** mattoliverau has quit IRC | 20:05 | |
| *** darrenc has quit IRC | 20:05 | |
| *** jhesketh has quit IRC | 20:05 | |
| *** e0ne has quit IRC | 20:06 | |
| *** _sarob has quit IRC | 20:06 | |
| *** darrenc has joined #openstack-infra | 20:07 | |
| clarkb | images are deleting in rax-iad happily so far | 20:08 |
| *** markvoelker has quit IRC | 20:08 | |
| *** Gg has joined #openstack-infra | 20:09 | |
| *** sarob has joined #openstack-infra | 20:10 | |
| *** mcarden has joined #openstack-infra | 20:11 | |
| *** matt6434 has joined #openstack-infra | 20:11 | |
| *** jkilpatr has quit IRC | 20:12 | |
| *** jhesketh has joined #openstack-infra | 20:12 | |
| *** esikachev has quit IRC | 20:14 | |
| *** gus has joined #openstack-infra | 20:15 | |
| *** trown is now known as trown|outtypewww | 20:16 | |
| pabelanger | 2016-08-29 20:16:06,106 INFO nodepool.image.build.centos-7: Notice: /Stage[main]/Iptables/Service[ip6tables]/enable: enable changed 'false' to 'true' | 20:16 |
| pabelanger | that looks right | 20:16 |
| stevemar | fungi: oh, i just remembered | 20:16 |
| stevemar | fungi: andreas suggested i reach out to someone on the infra team to clean up stuff | 20:17 |
| *** markvoelker has joined #openstack-infra | 20:17 | |
| stevemar | fungi: https://review.openstack.org/#/c/361401/ some details are here | 20:17 |
| *** srobert has joined #openstack-infra | 20:17 | |
| stevemar | fungi: but basically, this stuff >> http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3.html was generated after some commit to keystone-specs, but i want it to go away | 20:18 |
| stevemar | fungi: we no longer build that either (AFAIR) | 20:18 |
| *** tonytan_brb has joined #openstack-infra | 20:18 | |
| *** Sukhdev has joined #openstack-infra | 20:20 | |
| mtreinish | fungi: I've been working on as a side project: https://github.com/mtreinish/lpmqtt | 20:21 |
| mtreinish | fungi: when it's ready I'm wondering the best way we could run that | 20:21 |
| *** tonytan4ever has quit IRC | 20:21 | |
| mtreinish | since it requires an imap account subscribed to lp to get the event stream to publish in mqtt | 20:21 |
| openstackgerrit | Merged openstack-infra/shade: Change naming style of submitTask https://review.openstack.org/358684 | 20:23 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute046.vanilla to baremetal.json https://review.openstack.org/362409 | 20:24 |
| *** ilyashakhat has quit IRC | 20:25 | |
| clarkb | ok deleting rax-ord images now | 20:25 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Add new promote jobs for TripleO https://review.openstack.org/361427 | 20:28 |
| jeblair | pabelanger, clarkb: no default timeout. pabelanger can i peek at the tcpdumps? | 20:28 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: import the dependencies needed for creating stable branches https://review.openstack.org/362418 | 20:31 |
| mordred | clarkb: ok. I believe we've finally landed all of the shade patches we need to land | 20:32 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: move stable branch tools to project-config https://review.openstack.org/362419 | 20:32 |
| clarkb | mordred: yay | 20:32 |
| jeblair | mordred: i'll push up a change to make the shade repo read-only, sound good? ;) | 20:32 |
| mordred | jeblair: yes! | 20:32 |
| mordred | clarkb: I'm thinking that perhaps stopping puppet, updating shade on nodepool to be tip of master and doing the restart and watching everything before we cut an actual release might be nice ... there are a LOT of changes in this release | 20:33 |
| clarkb | mordred: if we do the nodepoold restart today we need to reupload centos-7 to osic and ubuntu-trusty to rax-ord | 20:33 |
| mordred | clarkb: I was thinking maybe first thing in the morning when I wake up | 20:33 |
| mordred | as the load hasn't really started then yet | 20:33 |
| clarkb | mordred: that seems fine to me, we should just do our best to get the state back to managed before too long if we do that | 20:33 |
| mordred | clarkb: ++ | 20:33 |
| pabelanger | jeblair: sure, they are located in the /tmp folder on nodepool.o.o | 20:33 |
| mordred | as soon as we're happy with it, cutting a release and installing that would be immediate next step | 20:34 |
| pabelanger | jeblair: I was looking at pcap5 | 20:34 |
| mordred | I'm actually pretty confident in this one ... the nodepool dsvm test has been really helpful this cycle | 20:34 |
| openstackgerrit | Peter Stachowski proposed openstack-infra/project-config: [trove] Set py35 to voting https://review.openstack.org/362420 | 20:34 |
| clarkb | ya it does a good job of catching the cloud interactions | 20:34 |
| clarkb | especially since we floating IP it | 20:34 |
| clarkb | which seems like half of the problems ever are releated to floating ips | 20:34 |
| mordred | yup. also, it does dual-stack ipv4/ipv6 - so we catch that too | 20:35 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Reassign server to compute037.vanilla https://review.openstack.org/362409 | 20:35 |
| *** Jeffrey4l_ has quit IRC | 20:35 | |
| clarkb | mordred: btw we can probably further tweak that job by removing the snapshot build and instead adding a couple "new" providers that talk to the existing cloud with different settings | 20:35 |
| jeblair | pabelanger: so pcap5 should include http://paste.openstack.org/show/564641/ ? | 20:35 |
| clarkb | mordred: and since they can all share a single dib buidl that shouldn't increase job time much | 20:36 |
| mordred | clarkb: ++ | 20:36 |
| *** Jeffrey4l_ has joined #openstack-infra | 20:36 | |
| mordred | clarkb: I did that with one config already | 20:36 |
| jeblair | clarkb: nice | 20:36 |
| clarkb | mordred: but then we can have floating ip cloud, ipv4 only cloud, and dual stack cloud or something | 20:36 |
| clarkb | I don't quite know which scenarios make the most sense but we have this option in that test | 20:36 |
| pabelanger | jeblair: Yes sir, if I got my timestamp correct | 20:36 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Reassign server to compute037.vanilla https://review.openstack.org/362409 | 20:37 |
| mordred | yup | 20:37 |
| jeblair | pabelanger: i don't suppose you've wroked out a gearman job id or anything for that? | 20:37 |
| mordred | jeblair: oh - speaking of: https://review.openstack.org/#/c/360445/ and https://review.openstack.org/#/c/349697/ are both nodepool changes that might be nice to get in while we're on this | 20:37 |
| *** spzala has quit IRC | 20:37 | |
| jlvillal | Anyone else get Error 525 for: https://www.openstack.org/summit/barcelona-2016/ ? | 20:37 |
| *** spzala has joined #openstack-infra | 20:38 | |
| clarkb | rax-ord is now leaked image clean | 20:38 |
| *** zzelle has joined #openstack-infra | 20:38 | |
| jlvillal | Error 525 Ray ID: 2da2cd339c3d2846 • 2016-08-29 20:37:34 UTC SSL handshake failed | 20:38 |
| clarkb | jlvillal: yup I can pass that along to foundation people | 20:38 |
| jlvillal | clarkb: Thank you! | 20:38 |
| tlbr | mordred, could you please review https://review.openstack.org/#/c/362096/ ? | 20:38 |
| clarkb | on to rax dfw | 20:38 |
| pabelanger | jeblair: sadly no. I tried to line up timestamps the best I could. | 20:38 |
| mordred | tlbr: done | 20:39 |
| *** Gg has quit IRC | 20:39 | |
| tlbr | mordred, thank you :) | 20:39 |
| *** Sukhdev_ has joined #openstack-infra | 20:40 | |
| loquacities | jeblair: my ears are burning ;) | 20:40 |
| *** Sukhdev has quit IRC | 20:40 | |
| jeblair | loquacities: good morning! | 20:40 |
| loquacities | hi :) | 20:40 |
| tlbr | infra-core, please merge https://review.openstack.org/#/c/362096/ :) | 20:40 |
| jeblair | loquacities: i think the short version is AJaeger's latest message to the -docs list | 20:40 |
| loquacities | yeah | 20:41 |
| loquacities | so, what i would *really* like is to have a static set of pages somewhere | 20:41 |
| loquacities | then we can go ahead and delete everything non-current | 20:41 |
| *** jkilpatr has joined #openstack-infra | 20:41 | |
| *** spzala has quit IRC | 20:42 | |
| clarkb | dfw image deletes are running | 20:42 |
| loquacities | if we do that, then it doesn't really matter what legal say | 20:42 |
| loquacities | i actually think legal won't give a proverbial anyway | 20:42 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Reassign server to compute008.vanilla https://review.openstack.org/362432 | 20:42 |
| clarkb | jlvillal: it should load now. | 20:42 |
| jlvillal | clarkb: not yet for me. | 20:43 |
| clarkb | jlvillal: try a hard refresh? that worked for me | 20:43 |
| jlvillal | clarkb: Now it works. I guess it took a few more seconds for me. I did a 'shift' and click the refresh | 20:44 |
| jlvillal | But it works | 20:44 |
| jeblair | loquacities: so maybe copy all the existing data to some location so it's still accessible for a while (until we decide to delete it, or copy it back into 'current'), but otherwise, do new builds only? | 20:44 |
| *** spzala has joined #openstack-infra | 20:44 | |
| loquacities | jeblair: yeah, is that sane? | 20:45 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/devstack-gate: set up stable/newton https://review.openstack.org/362435 | 20:45 |
| jeblair | loquacities: yes, i think that will make the process easier, and also low risk | 20:45 |
| *** senk has quit IRC | 20:46 | |
| dhellmann | fungi : we've started branching library projects for stable/newton and will be doing the others later this week. My process document missed a step, so ^^ is coming a bit late. | 20:46 |
| jeblair | loquacities: i'll modify the spec in a few minutes, when i'm done with this tcpdump thing, and ping you on it, and it should be done in time for the infra meeting tomorrow and we can vote on it | 20:46 |
| loquacities | jeblair: i think it ticks the boxes, and it means we have a way of winding things back if the proverbial hits the fan | 20:46 |
| clarkb | dhellmann: reviewing | 20:46 |
| loquacities | jeblair: great! thanks a bunch :D | 20:46 |
| dhellmann | clarkb : thanks | 20:46 |
| jeblair | loquacities: thank you :) | 20:46 |
| loquacities | np, sorry i wasn't around earlier | 20:47 |
| clarkb | dhellmann: ooh I will have to get the grenade jobs working on xenial now too (since it will be newton to master) | 20:47 |
| jeblair | loquacities: we can't all be in all timezones like fungi :) | 20:47 |
| *** markvoelker has quit IRC | 20:48 | |
| jeblair | or, is it just that fungi is in one timezone -- utc -- that doesn't correspond to a place on earth? | 20:48 |
| loquacities | lol | 20:48 |
| clarkb | dhellmann: one issue is that I don't know if we can merge that until everything in the core bits of devstack have a newton branch? do nova neutron glance cinder swift et all have the branch now too? | 20:48 |
| loquacities | sometimes i travel so much i feel like i'm in all timezones simultaneously ;) | 20:48 |
| dhellmann | clarkb : no, not until friday | 20:48 |
| dhellmann | clarkb : I thought it was the other way around, that this was a pre-req for being able to land anything in the new branches as we made them? | 20:49 |
| jeblair | pabelanger: packet number 229984 is the node assignment job submission | 20:49 |
| *** Sukhdev_ has quit IRC | 20:49 | |
| *** dprince has quit IRC | 20:49 | |
| clarkb | dhellmann: half of that change is, the features.yaml portion. But the grenade portion I think needs to wait unitl everything else is done | 20:49 |
| clarkb | mtreinish: ^ does that match your memory | 20:49 |
| dhellmann | oh, ok, I can split it | 20:49 |
| *** florianf has quit IRC | 20:50 | |
| clarkb | dhellmann: the reason is as is we will test mitaka to ocata temporarily which is fine but if we merge this early then we can do master to master due to checkout branch default rules | 20:50 |
| clarkb | dhellmann: eg no newton so checkout master then we upgrade to master | 20:50 |
| mtreinish | clarkb: I think so, but it's been ~6 months :) | 20:50 |
| jeblair | pabelanger: i *think* 229994 is the response, and the handle is H:23.253.73.160:488326 (i do not like that the gearman protocol doesn't let us positively confirm that) | 20:51 |
| *** salv-orlando has joined #openstack-infra | 20:51 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/devstack-gate: set up stable/newton https://review.openstack.org/362435 | 20:52 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/devstack-gate: update grenade settings for stable/newton https://review.openstack.org/362438 | 20:52 |
| dhellmann | clarkb, mtreinish : ^^ | 20:52 |
| *** sdake has joined #openstack-infra | 20:52 | |
| jeblair | pabelanger: packet 230008 is a work_data packet for :488326 that includes {"manager": "zl04"} | 20:53 |
| jeblair | pabelanger: packet 230010 is work_complete | 20:53 |
| *** ilyashakhat has joined #openstack-infra | 20:55 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/devstack-gate: DNM: test placement-api https://review.openstack.org/362441 | 20:56 |
| *** sarob has quit IRC | 20:56 | |
| clarkb | rax dfw has had its images cleaned up. That should be all of our cloud regions | 20:57 |
| pabelanger | jeblair: okay, I think had that too. I was able to use zl04 to find the correct launcher-debug.log | 20:58 |
| *** dstufft has quit IRC | 20:58 | |
| *** dstufft has joined #openstack-infra | 20:58 | |
| clarkb | dhellmann: I +2'd both chagnes. Maybe we can WIP the second one? | 20:58 |
| clarkb | dhellmann: then you can remove the WIP when you have completed the branch addition to all the managed projects | 20:59 |
| dhellmann | clarkb : yeah, I'll do that | 20:59 |
| *** salv-orlando has quit IRC | 20:59 | |
| *** raildo has quit IRC | 20:59 | |
| *** Kaiyan has joined #openstack-infra | 21:00 | |
| *** ilyashakhat has quit IRC | 21:00 | |
| *** sarob has joined #openstack-infra | 21:01 | |
| Kaiyan | Hi Im looking for some documentation for how to move a repo from github into openstack. Can I get help please? Thanks! | 21:01 |
| pleia2 | Kaiyan: you want http://docs.openstack.org/infra/manual/creators.html | 21:02 |
| jeblair | pabelanger: i believe there may be a code path through nodepool where job.failure is not set to True but onCompleted is called -- if nodepool disconnects from the gearman server. however, i can't see any indication in the capture that happened. | 21:02 |
| pleia2 | Kaiyan: that includes instructions if you have an upstream to import (like a project in github) | 21:02 |
| jeblair | pabelanger: we may need to enable gear debug logging in nodepoold. | 21:02 |
| *** gouthamr has quit IRC | 21:02 | |
| jeblair | pabelanger: which is huge. | 21:02 |
| *** hockeynut has quit IRC | 21:04 | |
| openstackgerrit | Merged openstack-infra/system-config: Reassign server to compute037.vanilla https://review.openstack.org/362409 | 21:04 |
| Kaiyan | pleia2: Thank you! | 21:04 |
| pabelanger | jeblair: okay. Do we need to worry about performance with debug enabled? | 21:04 |
| jeblair | pabelanger: but i think the tcpdumps have served well to show us that it looks like the packets are being sent | 21:04 |
| jeblair | pabelanger: not really, only disk space | 21:05 |
| openstackgerrit | Mathieu Mitchell proposed openstack/diskimage-builder: Generate and use upper-constraints for ironic-agent https://review.openstack.org/360547 | 21:05 |
| jeblair | pabelanger: continuous gearman traffic in nodepool is small (unlike zuul) -- but it runs 'status' a lot, which is something like 1.5MB each time. | 21:06 |
| *** AnarchyAo has joined #openstack-infra | 21:06 | |
| jeblair | (at most, every 10 seconds, but usually much less often -- weirdly, the busier the system, the less often this will be. so it's the fully idle system that will eat up our disk space in this case :) | 21:07 |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: Simplify delete by removing unnecessary loop https://review.openstack.org/357990 | 21:08 |
| pabelanger | jeblair: okay, let me see if there is a pattern for the IndexError to help narrow down the window | 21:08 |
| *** Goneri has quit IRC | 21:09 | |
| openstackgerrit | Merged openstack/os-client-config: Add prompting for KSA options https://review.openstack.org/362314 | 21:10 |
| openstackgerrit | Ken Giusti proposed openstack-infra/project-config: Update the oslo.messaging-dsvm-functional test https://review.openstack.org/358745 | 21:10 |
| jeblair | pabelanger, clarkb, mordred: we may want to incorporate a logging config change into the forthcoming nodepool restart | 21:10 |
| *** _sarob has joined #openstack-infra | 21:11 | |
| openstackgerrit | Merged openstack-infra/system-config: Reassign server to compute008.vanilla https://review.openstack.org/362432 | 21:11 |
| jeblair | pabelanger, clarkb, mordred: (this communication is serving to connect two efforts that both need a restart :) | 21:11 |
| *** daemontool has quit IRC | 21:11 | |
| pabelanger | I'll need to step away here shortly, so I'll have to pass the torch for the logging change | 21:12 |
| clarkb | pabelanger: mordred was talking doing it tomorrow morning so we have time | 21:13 |
| jeblair | (i'm holding too many torches atm) | 21:13 |
| pabelanger | clarkb: cool, that works | 21:13 |
| *** thorst_ has quit IRC | 21:13 | |
| jeblair | just need to get the change ready before then | 21:13 |
| pabelanger | yup | 21:13 |
| *** daemontool has joined #openstack-infra | 21:13 | |
| *** thorst has joined #openstack-infra | 21:14 | |
| *** ldnunes has quit IRC | 21:14 | |
| *** sarob has quit IRC | 21:15 | |
| *** inc0 has quit IRC | 21:15 | |
| *** godp1301_ has quit IRC | 21:15 | |
| *** salv-orlando has joined #openstack-infra | 21:16 | |
| *** wajdi has quit IRC | 21:17 | |
| *** mhickey has quit IRC | 21:17 | |
| clarkb | we want to log gear at debug level? | 21:18 |
| *** thorst has quit IRC | 21:18 | |
| * clarkb digs up the config file | 21:19 | |
| pabelanger | centos-7 image uploading to osic-cloud1 | 21:19 |
| pabelanger | clarkb: yes | 21:19 |
| *** srobert has quit IRC | 21:19 | |
| openstackgerrit | James E. Blair proposed openstack-infra/infra-specs: Update doc publishing spec for AFS https://review.openstack.org/276482 | 21:22 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Log gear at debug level on nodepoold https://review.openstack.org/362455 | 21:23 |
| clarkb | pabelanger: jeblair I think ^ is all we need | 21:23 |
| jeblair | loquacities, AJaeger, annegentle, fungi: i updated https://review.openstack.org/276482 | 21:23 |
| clarkb | well that and a service restart | 21:23 |
| *** jcoufal has quit IRC | 21:24 | |
| annegentle | jeblair where did you and loquacities chat? | 21:24 |
| jeblair | annegentle: here about 45 mins ago | 21:24 |
| openstackgerrit | Peter Stachowski proposed openstack-infra/project-config: [trove] Set py35 to voting https://review.openstack.org/362420 | 21:24 |
| annegentle | jeblair thanks, I'll read | 21:24 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/devstack-gate: update grenade settings for stable/newton https://review.openstack.org/362438 | 21:24 |
| *** jcoufal has joined #openstack-infra | 21:24 | |
| loquacities | annegentle: heya :) | 21:26 |
| clarkb | jeblair: pabelanger its possible we want a new handler for that too to avoid filling the normal logs but wasn't sure if that would hinder debugging if you couldn't see it interleaved | 21:26 |
| *** nmagnezi has quit IRC | 21:26 | |
| jeblair | clarkb: i think a new handler would be okay; maybe we could give it a max size rotating handler or something too | 21:26 |
| annegentle | hey loquacities sounds good to me thanks jeblair for carrying the torch | 21:27 |
| clarkb | jeblair: would you prefer I didn't log it to the existing handlers then? | 21:27 |
| jeblair | clarkb: no preference | 21:29 |
| clarkb | I will leave it as is then, I find it easier to debug things with a single file so I don't have to merge a bunch of stuff | 21:30 |
| jeblair | k | 21:30 |
| EmilienM | hi there, can someone review this project-config patch please? :-) https://review.openstack.org/#/c/361433/ | 21:30 |
| *** Marx314 has joined #openstack-infra | 21:32 | |
| *** kgiusti has left #openstack-infra | 21:32 | |
| *** sdake has quit IRC | 21:33 | |
| pleia2 | EmilienM: thumbs up | 21:33 |
| clarkb | looking into why melody records so many http errors against our gerrit server it looks like every single avatar GET is a 404 | 21:33 |
| clarkb | and that is really skewing our data | 21:33 |
| EmilienM | pleia2: thank you :-) /me dancing | 21:33 |
| clarkb | zaro: ^ do you know if it is possible to turn off the avatars entirely? | 21:33 |
| *** eharney has quit IRC | 21:34 | |
| *** rtheis_ has joined #openstack-infra | 21:38 | |
| clarkb | my googling says its not possible to turn off the lookups | 21:39 |
| *** edtubill has quit IRC | 21:40 | |
| openstackgerrit | Merged openstack-infra/project-config: tripleo: align CI scenarios with Puppet CI https://review.openstack.org/361433 | 21:40 |
| *** rtheis has quit IRC | 21:40 | |
| zigo | pabelanger: Hi there! Could you validate this? https://review.openstack.org/362096 | 21:41 |
| clarkb | we haev almost 350k 404s over the last day according to melody the vast majority of which are due to avatars not being a thing for us | 21:41 |
| clarkb | then we have maybe 1k other errors | 21:42 |
| clarkb | and the other 1k errors are potentially interesting just drowned out by the 404s | 21:42 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add cancel-builds-on-update to GHPRB trigger https://review.openstack.org/360407 | 21:43 |
| *** mriedem is now known as mriedem_afk | 21:44 | |
| *** spzala has quit IRC | 21:45 | |
| mgagne | clarkb: I think it got fixed in 2.12: https://gerrit-review.googlesource.com/#/c/73592/ | 21:47 |
| *** hasharAway has quit IRC | 21:47 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: docker-build-publish: Add support for buildContext https://review.openstack.org/354224 | 21:47 |
| openstackgerrit | Kaiyan Sheng proposed openstack-infra/project-config: Add monasca-events-api repo to openstack https://review.openstack.org/362462 | 21:49 |
| clarkb | mgagne: I think that will still hit the api and 404 just not show you the empty square in the UI? | 21:49 |
| clarkb | mgagne: I want gerrit to not even ask for the avatars | 21:49 |
| mgagne | clarkb: I think they added server-side validation? will have to load 2.12 to check for sure | 21:50 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Correct compute037.vanilla iLO address https://review.openstack.org/362463 | 21:50 |
| *** dfflanders has joined #openstack-infra | 21:51 | |
| *** zzelle has quit IRC | 21:53 | |
| *** lucas-dinner has quit IRC | 21:56 | |
| *** wfoster has quit IRC | 21:56 | |
| *** esberglu has quit IRC | 21:57 | |
| zaro | clarkb: no, i don't see a path to disable avatar. i don't think later gerrit versions have an option for it either. | 21:57 |
| *** matt-borland has quit IRC | 21:57 | |
| *** hockeynut has joined #openstack-infra | 21:59 | |
| clarkb | zaro it creates hundreds of thousands of useless requests per day fwiw | 21:59 |
| *** devananda has quit IRC | 21:59 | |
| *** wfoster has joined #openstack-infra | 21:59 | |
| *** lucasagomes has joined #openstack-infra | 21:59 | |
| mgagne | could mod_rewrite be used to server an empty image instead which could then be cached forever? | 22:00 |
| *** devananda has joined #openstack-infra | 22:00 | |
| clarkb | mgagne: possibly | 22:00 |
| *** rhallisey has quit IRC | 22:01 | |
| *** devananda has quit IRC | 22:01 | |
| zaro | looks like this is a known issue, but never fixed. https://bugs.chromium.org/p/gerrit/issues/detail?id=1987 | 22:02 |
| *** devananda has joined #openstack-infra | 22:02 | |
| *** aeng has quit IRC | 22:03 | |
| *** annegentle has quit IRC | 22:05 | |
| zaro | change https://gerrit-review.googlesource.com/c/47600/ seems to indicate that the fix is to cached on client side. | 22:05 |
| *** _sarob has quit IRC | 22:07 | |
| *** jamesdenton has quit IRC | 22:08 | |
| clarkb | ugh | 22:09 |
| clarkb | clearly the approach they took does not work... | 22:09 |
| openstackgerrit | Kaiyan Sheng proposed openstack-infra/project-config: Add monasca-events-api repo to openstack https://review.openstack.org/362462 | 22:10 |
| mordred | clarkb: I like the rewrite rule to serve empty image with super strong cache headers | 22:11 |
| mordred | but also- it would be really neat if they could be turned off | 22:11 |
| dhellmann | fungi, clarkb, jeblair : it looks like the launchpad script is hung on the signing node, does anyone want to poke at it to see why before the job times out? telnet://signing01.ci.openstack.org:19885 | 22:11 |
| *** thiagop has quit IRC | 22:11 | |
| zaro | clarkb: i think we can propose a change to have it optionally disabled though. i don't think that would have a problem landing upstream but won't help for 2.11 though. | 22:12 |
| clarkb | mordred: I don't think that will fix anything if gerrit is properly caching the 404s now | 22:12 |
| clarkb | mordred: we will still have hundreds of thousands of requests for uncached data | 22:12 |
| clarkb | mordred: the assertion gerrit is making is they cache the 404s so not ap roblem | 22:12 |
| clarkb | except clearly it is | 22:12 |
| mordred | clarkb: right. but if all of them return the same 1-byte image .. then we'll only have as many requests as we have users | 22:13 |
| mordred | since gerrit will cache them, right? | 22:13 |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add deb-bandit and deb-python-hacking repo https://review.openstack.org/362096 | 22:13 |
| clarkb | mordred: yes my point is supposedly gerrit is already doign taht for the 404 responses | 22:13 |
| dhellmann | fungi , clarkb , jeblair : even if the "poking" is just to give it a sigint so we can get a traceback that might be useful | 22:13 |
| clarkb | mordred: and we still end up with hundred of thousands of requests implying that caching the responses is not enough | 22:13 |
| jeblair | dhellmann: i'll take a look | 22:14 |
| dhellmann | jeblair : thanks | 22:14 |
| clarkb | mordred: since caching a 1byte image and caching a 404 should be roughly equivalent | 22:14 |
| * dhellmann guesses the problem has to do with the hated keyring library | 22:14 | |
| mordred | clarkb: yah. unless they only cache on success? | 22:14 |
| jeblair | dhellmann: it is doing work | 22:14 |
| mordred | zaro: ++ to proposing a change | 22:14 |
| dhellmann | it has one bug to update, and it has been doing that for 10 minutes | 22:15 |
| clarkb | mordred: the change above strongly implies that its remembering the 404s to not rerequest things that will 404 | 22:15 |
| dhellmann | jeblair : are you looking at strace output? | 22:15 |
| jeblair | dhellmann: yeah, it looks like it's continually making new https requests to lp | 22:16 |
| dhellmann | jeblair : I wonder if it's in a redirect loop | 22:16 |
| *** mdrabe has quit IRC | 22:16 | |
| jeblair | dhellmann: it looks periodic and similar enough that it seems like some kind of a loop | 22:16 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Revert "Use ntp-wait instead of ntpdate to set time" https://review.openstack.org/358105 | 22:17 |
| clarkb | yay I am crossing things off my list | 22:17 |
| jeblair | dhellmann: i can't really tell anything further since it's ssl :/ | 22:17 |
| clarkb | mgagne: do you think the mtl01 things will have an answer today? or should I go ahead and push worrying about that back? | 22:18 |
| jeblair | dhellmann: any other ideas, or want me to kill it? | 22:18 |
| dhellmann | jeblair : the script loops over a list of bugs it gets on the command line. In this case it has 1 argument. | 22:18 |
| dhellmann | jeblair : go ahead and kill it gently so we get a traceback, please | 22:18 |
| *** gordc has quit IRC | 22:18 | |
| jeblair | dhellmann: sigint? (signal 2)? | 22:18 |
| dhellmann | jeblair : yes, I think sigint would do it | 22:19 |
| dhellmann | I've confirmed that it has not updated the bug it's supposed to update | 22:19 |
| dhellmann | so whatever it's doing, it's failing | 22:19 |
| jeblair | dhellmann: done (but you know that i'm guessing :) | 22:19 |
| jeblair | dhellmann: oh, was it doing it's authorization poll? | 22:20 |
| *** aeng has joined #openstack-infra | 22:20 | |
| *** ihrachys has quit IRC | 22:20 | |
| dhellmann | jeblair : yeah, I wonder if it's looking at the wrong auth file | 22:20 |
| dhellmann | fungi told me the name of the file it should read, and I assumed that was the default | 22:21 |
| clarkb | zaro: to be clear this doesn't have a major impact on perforamcne from what I can see, but it does make it harder to use tools like melody to determine why real errors are occuring | 22:21 |
| dhellmann | jeblair : thanks, this gives me something to debug | 22:21 |
| *** cardeois has quit IRC | 22:21 | |
| jeblair | dhellmann: np | 22:21 |
| openstackgerrit | Merged openstack-infra/system-config: Correct compute037.vanilla iLO address https://review.openstack.org/362463 | 22:22 |
| *** jkilpatr has quit IRC | 22:27 | |
| *** rtheis_ has quit IRC | 22:28 | |
| *** signed8b_ has joined #openstack-infra | 22:28 | |
| *** signed8bit has quit IRC | 22:30 | |
| *** aeng has quit IRC | 22:32 | |
| *** thorst has joined #openstack-infra | 22:34 | |
| *** yamahata has quit IRC | 22:36 | |
| zaro | clarkb: that seems like something that can be filtered? | 22:40 |
| clarkb | zaro: melody doesn't relly provide a mechanism for that since its really just giving you the last 100 occurences and counters for the rest | 22:41 |
| *** xyang1 has quit IRC | 22:41 | |
| *** rlandy has quit IRC | 22:43 | |
| *** Goneri has joined #openstack-infra | 22:43 | |
| zaro | clarkb: is how about newer version of melody? maybe roll newer version into the the gerrit plugin? | 22:44 |
| clarkb | zaro: that I don't know. Maybe it can provide better ways to see the data | 22:44 |
| *** aeng has joined #openstack-infra | 22:45 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: tell the release job where the LP credentials are https://review.openstack.org/362481 | 22:46 |
| dhellmann | jeblair : I think ^^ will fix the issue | 22:46 |
| *** spzala has joined #openstack-infra | 22:46 | |
| *** sarob has joined #openstack-infra | 22:47 | |
| openstackgerrit | Kaiyan Sheng proposed openstack-infra/project-config: Add monasca-events-api repo to openstack https://review.openstack.org/362462 | 22:47 |
| openstackgerrit | Kaiyan Sheng proposed openstack-infra/project-config: Add monasca-events-api repo to openstack https://review.openstack.org/362462 | 22:48 |
| clarkb | zaro: its not super urgent either i was just trying to figure out why our http errors graph looked so terrible | 22:49 |
| *** daemontool has quit IRC | 22:49 | |
| clarkb | zaro: and basically found out its because we have a ton of 404s due to avatars | 22:49 |
| *** daemontool has joined #openstack-infra | 22:50 | |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config: Go ahead and handle YAML list in region_name https://review.openstack.org/362483 | 22:50 |
| *** moravec has quit IRC | 22:51 | |
| *** moravec has joined #openstack-infra | 22:51 | |
| *** spzala has quit IRC | 22:53 | |
| *** rbrndt has quit IRC | 22:54 | |
| *** sdague has quit IRC | 22:55 | |
| *** yamahata has joined #openstack-infra | 22:55 | |
| *** nwkarsten has quit IRC | 22:56 | |
| * fungi is back and catching up now | 22:56 | |
| *** nwkarsten has joined #openstack-infra | 22:56 | |
| *** Kaiyan has quit IRC | 22:56 | |
| fungi | the nice thing about living in utc is that it's _always_ beer-thirty here | 22:56 |
| fungi | wow, i missed so many pings | 22:57 |
| mordred | fungi: welcome back! | 22:58 |
| *** csomerville has quit IRC | 22:59 | |
| *** cody-somerville has joined #openstack-infra | 23:00 | |
| *** cody-somerville has joined #openstack-infra | 23:00 | |
| *** nwkarsten has quit IRC | 23:00 | |
| *** markvoelker has joined #openstack-infra | 23:02 | |
| *** dimtruck is now known as zz_dimtruck | 23:02 | |
| *** mmedvede has quit IRC | 23:02 | |
| *** hongbin has quit IRC | 23:03 | |
| *** annegentle has joined #openstack-infra | 23:04 | |
| *** rajinir has quit IRC | 23:05 | |
| *** matt6434 is now known as mattoliverau | 23:06 | |
| *** mmedvede has joined #openstack-infra | 23:06 | |
| *** gouthamr has joined #openstack-infra | 23:07 | |
| *** annegentle has quit IRC | 23:07 | |
| *** AnarchyAo has quit IRC | 23:09 | |
| fungi | stevemar: i can delete openstack/keystone-specs/api/v3/identity-api-v3.html from the site... do you have a list of what you want removed? just the url patterns mentioned in 361401 or is there more? | 23:10 |
| stevemar | fungi: you could rm -rf keystone-specs/api/ | 23:11 |
| clarkb | mordred: fungi canyou quickly reivew https://review.openstack.org/#/c/362435/2 to help dhellmann and the release process? | 23:11 |
| *** Benj_ has quit IRC | 23:11 | |
| fungi | mtreinish: for lpmqtt we don't really have a mail provider we use that would be well suited to that... would probably need to work out something | 23:11 |
| mtreinish | fungi: yeah, that's what I figured would be the case | 23:12 |
| *** xarses has quit IRC | 23:12 | |
| mtreinish | fungi: I was planning to make a throwaway gmail to test it. I guess we don't really want to use that for anything in prod though | 23:13 |
| clarkb | there is the infra root alias thing | 23:13 |
| *** vhosakot has quit IRC | 23:14 | |
| clarkb | which can be accessed via imap but might be too noisy to start throwing lp at it too | 23:14 |
| clarkb | er not an alias its a proper account | 23:14 |
| *** _nadya_ has quit IRC | 23:14 | |
| mtreinish | clarkb: from a flood perspective? because we can always just filter things from lp into a separate mailbox | 23:16 |
| mtreinish | assuming there is sieve or something on it | 23:16 |
| clarkb | mtreinish: just thinking about how we use that account to hopefulyl get important email and that extra volume might make it more difficult to see the important stuff but you are right we could filter into another location and just ignore it | 23:17 |
| *** pradk has quit IRC | 23:17 | |
| jeblair | clarkb, mtreinish: yes, filtering is an option; you could make lpmqtt just check a folder too | 23:18 |
| jeblair | that way it isn't parsing the rest of the root mail | 23:18 |
| jeblair | (assuming it doesn't already, i didn't look) | 23:18 |
| mtreinish | jeblair: yeah that was a config option I was gonna add soon. Right after ssl support :) | 23:18 |
| mtreinish | jeblair: it's not there yet | 23:19 |
| mtreinish | I haven't even tried to run it yet :) | 23:19 |
| jeblair | fungi, clarkb: i'm assuming our $mailprovider has imap access? | 23:19 |
| clarkb | jeblair: ya I am pretty sure I checked that when we were trying to figure out how to get that info to us humans more easily and my other account with the same provider is currently fetched via imap | 23:20 |
| jeblair | clarkb: cool | 23:21 |
| *** reed_ has joined #openstack-infra | 23:22 | |
| *** camunoz has joined #openstack-infra | 23:24 | |
| *** spzala has joined #openstack-infra | 23:24 | |
| fungi | stevemar: i've moved keystone-specs/api to my homedir on the webserver for the moment. if all looks good i'll delete the backup copy out of my homedir later | 23:25 |
| *** spzala has quit IRC | 23:25 | |
| *** spzala has joined #openstack-infra | 23:25 | |
| stevemar | fungi: ++ | 23:25 |
| *** salv-orlando has quit IRC | 23:25 | |
| stevemar | fungi: well they aren't there any longer, i suspect we won't need them again, since it's all been moved to http://developer.openstack.org/api-ref/identity/v3/index.html | 23:26 |
| stevemar | fungi: feel free to delete whenever you end up doing a clean up or something | 23:26 |
| stevemar | fungi: it's also worth noting that i moved the source files to https://github.com/openstack/keystone-specs/tree/master/attic - what you moved was the built html | 23:27 |
| fungi | jeblair: mtreinish: clarkb: yep, i thought about the rackspace apps site where we have our @openstack.org role aliases... not 100% convinced one of our existing aliases makes sense for this but also we might want to be very careful with the credentials for that account... yet the foundation is kinda stingy about creating new accounts there so... i can be convinced but it's possible we want to just | 23:28 |
| fungi | set up a mailbox on a server we already run and use that? | 23:28 |
| *** zul has joined #openstack-infra | 23:29 | |
| *** yamamoto has joined #openstack-infra | 23:30 | |
| openstackgerrit | Kaiyan Sheng proposed openstack-infra/project-config: Add monasca-events-api repo to openstack https://review.openstack.org/362462 | 23:30 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/puppet-iptables: Parameterize SNMP source hosts https://review.openstack.org/362490 | 23:32 |
| jeblair | fungi: i think either option (folder on infra-root, or new imap server) would work technically and is tractable. | 23:32 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 23:35 |
| mtreinish | jeblair: https://github.com/mtreinish/lpmqtt/commit/b278fa00088a2b6b4b7cc21759a0203c8fc21a0b still untested. But it passes pep8 :) | 23:36 |
| mordred | mtreinish: well, we all know that means it works :) | 23:37 |
| fungi | heck, mtreinish wrote it... what more evidence do you need? | 23:38 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/system-config: Specify cacti ip addresses for snmp access https://review.openstack.org/362491 | 23:38 |
| *** hockeynut has quit IRC | 23:39 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/puppet-iptables: Use site-agnostic default parameter values https://review.openstack.org/362492 | 23:40 |
| mtreinish | fungi: hah, past experience disagrees with that :) | 23:40 |
| mtreinish | mordred: it's definitely higher than the normal testing threshold of a random github repo | 23:41 |
| *** xarses has joined #openstack-infra | 23:44 | |
| *** zhurong has joined #openstack-infra | 23:44 | |
| *** sputnik13 has quit IRC | 23:53 | |
| *** tpsilva has quit IRC | 23:54 | |
| *** jerryz has quit IRC | 23:56 | |
| *** _nadya_ has joined #openstack-infra | 23:57 | |
| *** markvoelker has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!