| *** undefined_ is now known as Guest5241 | 00:01 | |
| *** undefined is now known as Guest5244 | 00:35 | |
| *** undefined_ is now known as Guest5251 | 01:13 | |
| *** undefined is now known as Guest5256 | 01:36 | |
| *** undefined_ is now known as Guest5257 | 01:46 | |
| *** undefined is now known as Guest5260 | 02:43 | |
| *** undefined_ is now known as Guest5261 | 03:05 | |
| *** undefined is now known as Guest5266 | 03:46 | |
| ianw | Clark[m]: i think merge 850112 to go back to 2.12 ... if we're seeing the issues; a) have something to look for in backtraces from the referenced bugs and b) can merge 849795 and test; that way we will know it fixes it | 04:18 |
|---|---|---|
| *** undefined_ is now known as Guest5269 | 04:24 | |
| opendevreview | Merged opendev/system-config master: Revert "Force ansible 2.9 on infra-prod jobs" https://review.opendev.org/c/opendev/system-config/+/850112 | 04:28 |
| *** undefined is now known as Guest5272 | 05:17 | |
| *** undefined_ is now known as Guest5275 | 05:57 | |
| *** undefined is now known as Guest5276 | 06:30 | |
| *** undefined_ is now known as Guest5277 | 07:00 | |
| *** undefined is now known as Guest5278 | 07:43 | |
| *** undefined_ is now known as Guest5281 | 08:02 | |
| *** undefined is now known as Guest5324 | 19:40 | |
| ianw | ... and ... jobs are timing out again | 22:07 |
| ianw | https://zuul.opendev.org/t/openstack/builds?project=opendev%2Fsystem-config&skip=0 | 22:14 |
| ianw | some hourly jobs are running now, i'll see if i can catch one that looks stuck and see any backtraces | 22:14 |
| fungi | at least we'll have ruled out a few potential causes | 22:21 |
| *** rlandy__ is now known as rlandy|out | 22:26 | |
| ianw | infra-prod-service-eavesdrop is about to run, and seems to pretty reliably timeout; i'll see if i can catch anything. but next step would be to roll out the zuul container update i'd say | 22:30 |
| fungi | makes sense, yeah | 22:31 |
| ianw | i just have to do school run, if i come back and service-eavesdrop is stuck that will be good and i can see if any backtraces line up. either way, i can then do the updated zuul container | 22:32 |
| ianw | https://zuul.openstack.org/stream/1631671f43ea4a388ec5b87f905f9ebe?logfile=console.log is stuck | 23:27 |
| ianw | https://paste.opendev.org/show/bcY3cpvwrWqeMhktLxY0/ | 23:30 |
| ianw | 1205512 would be the stuck one i guess | 23:30 |
| ianw | https://paste.opendev.org/show/bJrkQjbKbUmSx4jGHC7i/ | 23:38 |
| ianw | so ... something to do with grantpt()/opentty, etc etc | 23:39 |
| Clark[m] | Which is the same path that Albin found | 23:40 |
| ianw | https://sourceware.org/bugzilla/show_bug.cgi?id=24941 | 23:40 |
| ianw | "Make grantpt usable after multi-threaded fork in more cases" | 23:40 |
| ianw | i think that's about as close to a smoking gun as we can get... | 23:40 |
| ianw | i've approved 849795 and will update executors | 23:44 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!