| *** gyee has quit IRC | 00:21 | |
| *** hamalq has quit IRC | 00:30 | |
| *** rfolco has quit IRC | 00:42 | |
| *** Goneri has quit IRC | 00:59 | |
| *** mugsie has quit IRC | 01:00 | |
| *** mugsie has joined #openstack-infra | 01:04 | |
| *** dave-mccowan has quit IRC | 01:20 | |
| *** armax has quit IRC | 01:25 | |
| *** rcernin has quit IRC | 02:38 | |
| *** yamamoto has joined #openstack-infra | 02:38 | |
| *** zbr has quit IRC | 02:51 | |
| *** rcernin has joined #openstack-infra | 03:06 | |
| *** psachin has joined #openstack-infra | 03:10 | |
| *** zbr has joined #openstack-infra | 03:12 | |
| *** ramishra has joined #openstack-infra | 03:17 | |
| *** rcernin has quit IRC | 03:26 | |
| *** auristor has quit IRC | 03:43 | |
| *** rcernin has joined #openstack-infra | 03:47 | |
| *** rcernin has quit IRC | 03:47 | |
| *** rcernin has joined #openstack-infra | 03:47 | |
| *** auristor has joined #openstack-infra | 03:47 | |
| *** yamamoto_ has joined #openstack-infra | 03:57 | |
| *** yamamoto has quit IRC | 03:59 | |
| *** ykarel has joined #openstack-infra | 04:31 | |
| *** ykarel has quit IRC | 04:35 | |
| *** ykarel has joined #openstack-infra | 04:36 | |
| *** pcrews has joined #openstack-infra | 04:49 | |
| *** ykarel has quit IRC | 05:10 | |
| *** ykarel has joined #openstack-infra | 05:16 | |
| *** ykarel_ has joined #openstack-infra | 05:18 | |
| *** ykarel has quit IRC | 05:21 | |
| *** vishalmanchanda has joined #openstack-infra | 05:28 | |
| *** armax has joined #openstack-infra | 05:32 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-infra | 05:33 | |
| *** ysandeep|holiday is now known as ysandeep|ruck | 05:42 | |
| *** matt_kosut has joined #openstack-infra | 05:44 | |
| *** zbr1 has joined #openstack-infra | 06:03 | |
| *** zbr has quit IRC | 06:06 | |
| *** zbr1 is now known as zbr | 06:06 | |
| *** lmiccini has joined #openstack-infra | 06:46 | |
| *** dklyle has quit IRC | 07:02 | |
| *** ralonsoh has joined #openstack-infra | 07:07 | |
| *** sboyron has joined #openstack-infra | 07:08 | |
| *** dtantsur|afk is now known as dtantsur | 07:20 | |
| *** eolivare has joined #openstack-infra | 07:28 | |
| *** kmarc has quit IRC | 07:30 | |
| *** kmarc has joined #openstack-infra | 07:31 | |
| *** rpittau|afk is now known as rpittau | 07:42 | |
| *** ysandeep|ruck is now known as ysandeep|lunch | 07:46 | |
| *** piotrowskim has joined #openstack-infra | 07:52 | |
| *** ykarel_ is now known as ykarel | 07:56 | |
| *** jcapitao has joined #openstack-infra | 08:00 | |
| *** andrewbonney has joined #openstack-infra | 08:10 | |
| *** slaweq has joined #openstack-infra | 08:11 | |
| *** lpetrut has joined #openstack-infra | 08:11 | |
| *** ralonsoh has quit IRC | 08:13 | |
| *** ykarel has quit IRC | 08:13 | |
| *** ykarel has joined #openstack-infra | 08:16 | |
| *** hberaud has joined #openstack-infra | 08:19 | |
| *** rcernin has quit IRC | 08:23 | |
| *** tosky has joined #openstack-infra | 08:36 | |
| *** ralonsoh has joined #openstack-infra | 08:37 | |
| *** gfidente has joined #openstack-infra | 08:53 | |
| *** jpena|off is now known as jpena | 08:58 | |
| *** ysandeep|lunch is now known as ysandeep|ruck | 09:01 | |
| *** yamamoto_ has quit IRC | 09:09 | |
| *** lucasagomes has joined #openstack-infra | 09:11 | |
| *** ociuhandu has joined #openstack-infra | 09:19 | |
| *** yamamoto has joined #openstack-infra | 09:21 | |
| *** yamamoto has quit IRC | 09:21 | |
| *** yamamoto has joined #openstack-infra | 09:25 | |
| *** yamamoto has quit IRC | 09:27 | |
| *** hashar has joined #openstack-infra | 09:31 | |
| *** yamamoto has joined #openstack-infra | 09:50 | |
| *** yamamoto has quit IRC | 09:54 | |
| *** ykarel_ has joined #openstack-infra | 10:06 | |
| *** ykarel has quit IRC | 10:09 | |
| *** hberaud has quit IRC | 10:20 | |
| *** hberaud has joined #openstack-infra | 10:21 | |
| *** yamamoto has joined #openstack-infra | 10:23 | |
| *** yamamoto has quit IRC | 10:26 | |
| *** rfolco has joined #openstack-infra | 10:31 | |
| *** takamatsu is now known as mauro|call | 10:37 | |
| *** mauro|call is now known as takamatsu | 10:37 | |
| *** yamamoto has joined #openstack-infra | 10:46 | |
| *** yamamoto has quit IRC | 10:47 | |
| *** dchen has quit IRC | 10:52 | |
| *** yamamoto has joined #openstack-infra | 11:00 | |
| *** yamamoto has quit IRC | 11:08 | |
| *** yamamoto has joined #openstack-infra | 11:10 | |
| *** ykarel_ is now known as ykarel | 11:17 | |
| *** yamamoto has quit IRC | 11:19 | |
| *** yamamoto has joined #openstack-infra | 11:22 | |
| *** sboyron has quit IRC | 11:45 | |
| *** yamamoto has quit IRC | 11:54 | |
| *** rlandy has joined #openstack-infra | 11:59 | |
| *** lpetrut has quit IRC | 12:01 | |
| *** jpena is now known as jpena|lunch | 12:02 | |
| *** jcapitao is now known as jcapitao_lunch | 12:02 | |
| *** lpetrut has joined #openstack-infra | 12:03 | |
| *** yamamoto has joined #openstack-infra | 12:33 | |
| *** dtantsur is now known as dtantsur|brb | 12:34 | |
| *** yamamoto has quit IRC | 12:38 | |
| *** Goneri has joined #openstack-infra | 12:47 | |
| *** grandfso has joined #openstack-infra | 12:48 | |
| *** jpena|lunch is now known as jpena | 13:01 | |
| *** dave-mccowan has joined #openstack-infra | 13:07 | |
| *** thiagop has joined #openstack-infra | 13:10 | |
| *** zxiiro has joined #openstack-infra | 13:12 | |
| *** tdasilva_ has quit IRC | 13:15 | |
| *** thiago__ has joined #openstack-infra | 13:15 | |
| *** grandfso has quit IRC | 13:22 | |
| *** jcapitao_lunch is now known as jcapitao | 13:22 | |
| *** mmethot has joined #openstack-infra | 13:23 | |
| *** eharney has quit IRC | 13:45 | |
| *** dave-mccowan has quit IRC | 13:46 | |
| *** dave-mccowan has joined #openstack-infra | 13:49 | |
| *** eharney has joined #openstack-infra | 14:03 | |
| *** dtantsur|brb is now known as dtantsur | 14:03 | |
| *** sshnaidm|rover has quit IRC | 14:05 | |
| *** sshnaidm has joined #openstack-infra | 14:05 | |
| *** sshnaidm is now known as sshnaidm|rover | 14:06 | |
| *** slaweq is now known as slaweq|ptg | 14:07 | |
| *** lpetrut has quit IRC | 14:32 | |
| *** dklyle has joined #openstack-infra | 14:32 | |
| *** lpetrut has joined #openstack-infra | 14:33 | |
| JayF | http://ptg.openstack.org/ptg.html is loading to a blank page | 14:34 |
|---|---|---|
| *** tdasilva_ has joined #openstack-infra | 14:34 | |
| JayF | Well, not exactly blank, but everything after "See what is discussed currently at the PTG" is blank. The table of meeting/topic slots is missing. | 14:35 |
| *** thiago__ has quit IRC | 14:36 | |
| *** thiago__ has joined #openstack-infra | 14:37 | |
| *** tdasilva_ has quit IRC | 14:40 | |
| clarkb | it was discussed in the ptg channel, I think ttx debugged and fixed it | 14:40 |
| JayF | ack; thanks! | 14:41 |
| *** david-lyle has joined #openstack-infra | 15:09 | |
| *** dklyle has quit IRC | 15:09 | |
| openstackgerrit | Thierry Carrez proposed openstack/ptgbot master: Fix issue in case unknown location is provided https://review.opendev.org/759885 | 15:19 |
| *** lpetrut has quit IRC | 15:23 | |
| openstackgerrit | Thierry Carrez proposed openstack/ptgbot master: Fix issue in case unknown location is provided https://review.opendev.org/759885 | 15:24 |
| *** pmannidi has quit IRC | 15:26 | |
| *** pmannidi has joined #openstack-infra | 15:26 | |
| *** david-lyle has quit IRC | 15:30 | |
| *** dklyle has joined #openstack-infra | 15:30 | |
| *** pmannidi has quit IRC | 15:32 | |
| *** gyee has joined #openstack-infra | 15:34 | |
| *** ykarel has quit IRC | 15:40 | |
| *** pmannidi has joined #openstack-infra | 15:40 | |
| ttx | FTR -- those usually boil down to something weird making its way to the ptg.json, and the client-side JavaScript having trouble interpreting what it sees there. So looking at recent commands usually helps finding the cause, and once identified it can generally be fixed by undoing the change or uploading (using ~fetchdb) a fixed ptg.json | 15:42 |
| ttx | and then plugging the hole so that it does not happen again | 15:42 |
| ttx | (by looking at the JavaScript console on that failed page and seeing what the error is) | 15:43 |
| openstackgerrit | Merged openstack/ptgbot master: Fix issue in case unknown location is provided https://review.opendev.org/759885 | 15:45 |
| *** sboyron has joined #openstack-infra | 15:46 | |
| *** dklyle has quit IRC | 15:59 | |
| *** __ministry1 has joined #openstack-infra | 16:04 | |
| *** tosky_ has joined #openstack-infra | 16:07 | |
| *** tosky has quit IRC | 16:08 | |
| *** ysandeep|ruck is now known as ysandeep|away | 16:08 | |
| *** slaweq|ptg is now known as slaweq | 16:08 | |
| openstackgerrit | Merged openstack/project-config master: Mirror charm-neutron-api-plugin-ironic to GitHub https://review.opendev.org/758429 | 16:25 |
| *** lmiccini has quit IRC | 16:28 | |
| *** dklyle has joined #openstack-infra | 16:32 | |
| *** lucasagomes has quit IRC | 16:48 | |
| *** jpena is now known as jpena|brb | 16:56 | |
| *** tosky_ is now known as tosky | 16:56 | |
| *** __ministry1 has quit IRC | 16:56 | |
| *** odyssey4me is now known as odyssey4me|PTO | 17:04 | |
| *** ricolin has quit IRC | 17:10 | |
| *** aedc_ has joined #openstack-infra | 17:10 | |
| *** aedc has quit IRC | 17:11 | |
| *** yamamoto has joined #openstack-infra | 17:14 | |
| *** yamamoto has quit IRC | 17:19 | |
| *** ralonsoh has quit IRC | 17:25 | |
| *** jpena|brb is now known as jpena | 17:29 | |
| *** vishalmanchanda has quit IRC | 17:30 | |
| *** jcapitao has quit IRC | 17:34 | |
| *** rpittau is now known as rpittau|afk | 17:34 | |
| *** ociuhandu_ has joined #openstack-infra | 17:35 | |
| *** ociuhandu has quit IRC | 17:38 | |
| *** ociuhandu_ has quit IRC | 17:40 | |
| *** eolivare has quit IRC | 17:49 | |
| *** hashar has quit IRC | 17:59 | |
| *** hashar has joined #openstack-infra | 18:02 | |
| *** dtantsur is now known as dtantsur|afk | 18:13 | |
| *** andrewbonney has quit IRC | 18:21 | |
| *** hberaud has quit IRC | 18:25 | |
| *** hashar has quit IRC | 18:29 | |
| *** psachin has quit IRC | 18:39 | |
| *** jpena is now known as jpena|off | 18:40 | |
| *** gfidente is now known as gfidente|ack | 18:40 | |
| *** ociuhandu has joined #openstack-infra | 19:13 | |
| *** mmethot has quit IRC | 19:33 | |
| *** mmethot has joined #openstack-infra | 19:37 | |
| *** hberaud has joined #openstack-infra | 19:55 | |
| *** yamamoto has joined #openstack-infra | 19:56 | |
| *** yamamoto has quit IRC | 20:01 | |
| *** hberaud has quit IRC | 20:05 | |
| *** melwitt has joined #openstack-infra | 20:07 | |
| *** rlandy is now known as rlandy|brb | 20:11 | |
| *** ociuhandu has quit IRC | 20:15 | |
| *** ociuhandu has joined #openstack-infra | 20:16 | |
| *** slaweq has quit IRC | 20:16 | |
| *** rpittau|afk is now known as rpittau | 20:18 | |
| *** slaweq has joined #openstack-infra | 20:19 | |
| *** rlandy|brb is now known as rlandy | 20:22 | |
| *** hberaud has joined #openstack-infra | 20:23 | |
| otherwiseguy_ | So I notice a bunch of things sitting in the check queue for 6 to 11.5 hours... | 20:33 |
| *** otherwiseguy_ is now known as otherwiseguy | 20:33 | |
| fungi | otherwiseguy: long time no see! | 20:37 |
| * otherwiseguy waves | 20:37 | |
| fungi | i'll take a look at what's blocking them | 20:37 |
| otherwiseguy | fungi: everytime I see your handle, I get really hungry for stroganoff... | 20:38 |
| fungi | otherwiseguy: not jägerschnitzel? | 20:40 |
| otherwiseguy | fungi: well now I want both. Thanks. | 20:41 |
| fungi | so... most of them are just lower priority node allocations because we balance those by queue, and tripleo has a disproportionate number of builds currently | 20:41 |
| otherwiseguy | fungi: Ok. Had just noticed with https://review.opendev.org/#/c/759758/ that was for fixing our gate that 3 of the tests ran long ago, and the rest just were sitting for several hours. so started looking around. | 20:43 |
| otherwiseguy | since it was a bit unusual :) | 20:44 |
| fungi | looking at http://grafana.openstack.org/d/T6vSHcSik/zuul-status?orgId=1 we're backlogged on node requests, though i suspect having taken inap out of the pool recently has made that worse | 20:44 |
| fungi | we're floating around 550 nodes in use with a roughly 1k node request backlog | 20:45 |
| fungi | gerrit event frequency is dropping steadily though so i expect that to come down any time now | 20:45 |
| otherwiseguy | fungi: sweet. thanks for checking. so the solution as always is more hardware, less and faster running tests. :) | 20:46 |
| fungi | tripleo is the only one with any appreciable gate pipeline queue and it's not long, so i don't think thrash from gate resets is a problem right this moment at least | 20:47 |
| clarkb | the other reason you can have unstarted jobs when others have run is if the job is being retried but I think zuul notes that in the web ui dashboard now | 20:58 |
| fungi | melwitt: is https://review.opendev.org/759831 something we should enqueue into the gate and promote? seems like at least neutron and maybe others are trying to get changes in to disable neutron jobs as a workaround | 20:58 |
| *** gouthamr has quit IRC | 20:58 | |
| *** logan- has quit IRC | 20:58 | |
| fungi | er, to disable grenade jobs | 20:58 |
| *** gouthamr has joined #openstack-infra | 20:59 | |
| *** gouthamr has quit IRC | 20:59 | |
| melwitt | fungi: yes, many projects are blocked behind that fix as they are running python 3.6 in the affected jobs. so promoting would be a good thing I think | 20:59 |
| *** gouthamr has joined #openstack-infra | 20:59 | |
| fungi | okay, i'll do that now | 20:59 |
| melwitt | any job with python 3.6 + libvirt-python 6.8.0 that is running nova will fail until that fix lands | 21:00 |
| melwitt | great thanks | 21:00 |
| *** logan- has joined #openstack-infra | 21:01 | |
| *** rpittau is now known as rpittau|afk | 21:03 | |
| *** slaweq has quit IRC | 21:04 | |
| *** hamalq has joined #openstack-infra | 21:20 | |
| *** slaweq has joined #openstack-infra | 21:22 | |
| *** hberaud has quit IRC | 21:28 | |
| *** aedc_ has quit IRC | 21:42 | |
| *** Goneri has quit IRC | 21:48 | |
| *** sshnaidm|rover is now known as sshnaidm|afk | 21:50 | |
| *** thiagop has quit IRC | 22:01 | |
| *** sboyron has quit IRC | 22:06 | |
| *** yamamoto has joined #openstack-infra | 22:09 | |
| *** rfolco has quit IRC | 22:12 | |
| *** matt_kosut has quit IRC | 22:12 | |
| *** yamamoto has quit IRC | 22:22 | |
| *** yamamoto has joined #openstack-infra | 22:27 | |
| *** yamamoto has quit IRC | 22:27 | |
| *** gfidente|ack has quit IRC | 22:28 | |
| *** yamamoto has joined #openstack-infra | 22:28 | |
| *** yamamoto has quit IRC | 22:33 | |
| *** ociuhandu has quit IRC | 22:39 | |
| *** yamamoto has joined #openstack-infra | 22:42 | |
| fungi | i guess that one failed on different errors this time? | 22:44 |
| *** yamamoto has quit IRC | 22:46 | |
| melwitt | fungi: yeah, I noted them in a comment on the review. nova-grenade-multinode failed for bug https://bugs.launchpad.net/nova/+bug/1901739 again (we have no idea what this is about yet ... we've been seeing it in the gate for awhile) and then nova-next failed in cinder when trying to delete a volume, a cinder-rootwrap command unexpectedly failed | 23:01 |
| openstack | Launchpad bug 1901739 in OpenStack Compute (nova) " libvirt.libvirtError: internal error: missing block job data for disk 'vda'" [Undecided,New] | 23:01 |
| *** rcernin has joined #openstack-infra | 23:02 | |
| *** rlandy has quit IRC | 23:06 | |
| *** slaweq has quit IRC | 23:10 | |
| *** yamamoto has joined #openstack-infra | 23:13 | |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: Use internal address for RAX ORD https://review.opendev.org/759972 | 23:14 |
| *** yamamoto has quit IRC | 23:19 | |
| *** yamamoto has joined #openstack-infra | 23:20 | |
| *** dchen has joined #openstack-infra | 23:37 | |
| *** Goneri has joined #openstack-infra | 23:38 | |
| *** tosky has quit IRC | 23:52 | |
| *** armax has quit IRC | 23:55 | |
| *** zxiiro has quit IRC | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!