| openstackgerrit | Paul Belanger proposed zuul/zuul master: Switch to review.opendev.org for README.rst https://review.opendev.org/676529 | 00:01 |
|---|---|---|
| *** mattw4 has quit IRC | 00:09 | |
| *** ociuhandu has joined #openstack-infra | 00:16 | |
| *** dave-mccowan has quit IRC | 00:22 | |
| *** markvoelker has joined #openstack-infra | 00:24 | |
| *** exsdev has quit IRC | 00:24 | |
| *** exsdev has joined #openstack-infra | 00:26 | |
| *** diga has joined #openstack-infra | 00:33 | |
| *** rfolco has quit IRC | 00:35 | |
| *** jcoufal has joined #openstack-infra | 00:35 | |
| *** gyee has quit IRC | 00:40 | |
| *** jcoufal has quit IRC | 00:48 | |
| *** markvoelker has quit IRC | 00:51 | |
| *** markvoelker has joined #openstack-infra | 00:52 | |
| *** aaronsheffield has quit IRC | 00:52 | |
| *** hongbin has joined #openstack-infra | 00:58 | |
| *** markvoelker has quit IRC | 01:01 | |
| *** ociuhandu has quit IRC | 01:02 | |
| *** ricolin has joined #openstack-infra | 01:03 | |
| *** markvoelker has joined #openstack-infra | 01:05 | |
| *** markvoelker has quit IRC | 01:11 | |
| *** markvoelker has joined #openstack-infra | 01:16 | |
| *** spsurya has joined #openstack-infra | 01:28 | |
| clarkb | corvus: the whole swift unrevert chain lgtm. Tomorrow will be even more afk likely. back friday | 01:30 |
| fungi | yeah, i'm happy to see it land, but will also be basically entirely afk myself until sometime friday once i get home (and hopefully some sleep) | 01:38 |
| *** yamamoto has quit IRC | 01:40 | |
| *** yamamoto has joined #openstack-infra | 01:40 | |
| *** apetrich has quit IRC | 02:09 | |
| *** gregoryo has joined #openstack-infra | 02:12 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:42 | |
| *** noorul has joined #openstack-infra | 02:45 | |
| *** noorul has quit IRC | 02:49 | |
| *** bobh has joined #openstack-infra | 02:55 | |
| *** bhavikdbavishi1 has joined #openstack-infra | 02:57 | |
| *** bhavikdbavishi has quit IRC | 02:58 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:58 | |
| *** armax has quit IRC | 03:09 | |
| *** markvoelker has quit IRC | 03:10 | |
| *** altlogbot_0 has quit IRC | 03:16 | |
| *** altlogbot_2 has joined #openstack-infra | 03:17 | |
| *** odicha has joined #openstack-infra | 03:25 | |
| *** hongbin has quit IRC | 03:27 | |
| *** noorul has joined #openstack-infra | 03:35 | |
| *** yamamoto has quit IRC | 03:37 | |
| *** ykarel|away has joined #openstack-infra | 03:41 | |
| *** yamamoto has joined #openstack-infra | 03:45 | |
| *** noorul has quit IRC | 03:45 | |
| *** bobh has quit IRC | 03:47 | |
| *** yamamoto has quit IRC | 03:53 | |
| *** markvoelker has joined #openstack-infra | 03:55 | |
| *** markvoelker has quit IRC | 04:00 | |
| *** ykarel|away has quit IRC | 04:01 | |
| *** iokiwi has quit IRC | 04:06 | |
| *** iokiwi has joined #openstack-infra | 04:07 | |
| *** dave-mccowan has joined #openstack-infra | 04:23 | |
| *** bobh has joined #openstack-infra | 04:27 | |
| *** ociuhandu has joined #openstack-infra | 04:30 | |
| *** dave-mccowan has quit IRC | 04:31 | |
| *** ociuhandu has quit IRC | 04:35 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote-openstack-manuals jobs https://review.opendev.org/676489 | 04:41 |
| *** markvoelker has joined #openstack-infra | 04:50 | |
| *** tkajinam has quit IRC | 04:53 | |
| *** rh-jelabarre has quit IRC | 04:54 | |
| *** tkajinam has joined #openstack-infra | 04:54 | |
| *** markvoelker has quit IRC | 04:54 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote-openstack-manuals jobs https://review.opendev.org/676489 | 05:00 |
| *** tkajinam has quit IRC | 05:00 | |
| *** bobh has quit IRC | 05:00 | |
| *** bobh has joined #openstack-infra | 05:02 | |
| *** tkajinam has joined #openstack-infra | 05:05 | |
| *** bobh has quit IRC | 05:07 | |
| *** jaosorior has joined #openstack-infra | 05:16 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote-openstack-manuals jobs https://review.opendev.org/676489 | 05:19 |
| AJaeger | corvus, I'm ready ^ - reviewed and updated everything, and prepared the docs repos... | 05:20 |
| *** tkajinam has quit IRC | 05:31 | |
| *** markvoelker has joined #openstack-infra | 05:35 | |
| *** markvoelker has quit IRC | 05:41 | |
| *** mgoddard has quit IRC | 05:54 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: yum-minimal: install fedora-release-cloud https://review.opendev.org/676581 | 05:58 |
| *** bhavikdbavishi has quit IRC | 05:59 | |
| *** mgoddard has joined #openstack-infra | 06:01 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] f30 boot test https://review.opendev.org/676582 | 06:01 |
| *** tkajinam has joined #openstack-infra | 06:04 | |
| *** xenos76 has joined #openstack-infra | 06:10 | |
| *** yamamoto has joined #openstack-infra | 06:16 | |
| *** hongbin has joined #openstack-infra | 06:28 | |
| *** hongbin has quit IRC | 06:32 | |
| *** yamamoto has quit IRC | 06:36 | |
| *** yamamoto has joined #openstack-infra | 06:37 | |
| *** yamamoto has quit IRC | 06:42 | |
| *** bhavikdbavishi has joined #openstack-infra | 06:45 | |
| *** tdasilva has quit IRC | 06:47 | |
| *** tdasilva has joined #openstack-infra | 06:47 | |
| *** tkajinam has quit IRC | 06:49 | |
| *** yamamoto has joined #openstack-infra | 06:54 | |
| *** yamamoto has quit IRC | 06:54 | |
| *** tkajinam has joined #openstack-infra | 07:00 | |
| *** ricolin_ has joined #openstack-infra | 07:03 | |
| *** ricolin has quit IRC | 07:05 | |
| *** d34dh0r53 has quit IRC | 07:07 | |
| *** cloudnull has quit IRC | 07:08 | |
| *** cloudnull has joined #openstack-infra | 07:08 | |
| *** d34dh0r53 has joined #openstack-infra | 07:10 | |
| *** jtomasek has joined #openstack-infra | 07:15 | |
| *** kopecmartin|off is now known as kopecmartin | 07:16 | |
| *** zhangfei has joined #openstack-infra | 07:18 | |
| *** kjackal has joined #openstack-infra | 07:19 | |
| *** alemgeta has joined #openstack-infra | 07:19 | |
| *** jaosorior has quit IRC | 07:23 | |
| *** bhavikdbavishi has quit IRC | 07:28 | |
| *** larainema has joined #openstack-infra | 07:29 | |
| *** gregoryo has quit IRC | 07:32 | |
| *** ricolin_ is now known as ricolin | 07:33 | |
| *** noama has joined #openstack-infra | 07:34 | |
| *** jaicaa has quit IRC | 07:36 | |
| *** kjackal has quit IRC | 07:55 | |
| *** lucasagomes has joined #openstack-infra | 07:57 | |
| *** jaicaa has joined #openstack-infra | 07:58 | |
| *** dchen has quit IRC | 08:00 | |
| *** apetrich has joined #openstack-infra | 08:01 | |
| *** jaicaa has quit IRC | 08:05 | |
| openstackgerrit | Ghanshyam Mann proposed opendev/irc-meetings master: Modify the QA office hour time https://review.opendev.org/676623 | 08:05 |
| *** jaicaa has joined #openstack-infra | 08:06 | |
| openstackgerrit | Andreas Jaeger proposed openstack/openstack-zuul-jobs master: Use opendev-tox-docs for api jobs https://review.opendev.org/676624 | 08:06 |
| *** e0ne has joined #openstack-infra | 08:15 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote api-ref/guide jobs https://review.opendev.org/676630 | 08:16 |
| *** jaosorior has joined #openstack-infra | 08:17 | |
| openstackgerrit | Andreas Jaeger proposed openstack/openstack-zuul-jobs master: Switch to promote jobs for api-ref/-guide https://review.opendev.org/676631 | 08:18 |
| *** markvoelker has joined #openstack-infra | 08:20 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote api-ref/guide jobs https://review.opendev.org/676630 | 08:21 |
| *** markvoelker has quit IRC | 08:25 | |
| *** kjackal has joined #openstack-infra | 08:25 | |
| *** yamamoto has joined #openstack-infra | 08:26 | |
| *** yamamoto has quit IRC | 08:28 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote-openstack-manuals jobs https://review.opendev.org/676489 | 08:34 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: WIP: Add promote api-ref/guide jobs https://review.opendev.org/676630 | 08:34 |
| *** yamamoto has joined #openstack-infra | 08:34 | |
| AJaeger | corvus, mordred, I need help with 676630 on how to compute the publish path ^, please | 08:35 |
| *** yamamoto has quit IRC | 08:38 | |
| *** derekh has joined #openstack-infra | 08:40 | |
| *** tkajinam has quit IRC | 08:47 | |
| *** yamamoto has joined #openstack-infra | 08:47 | |
| *** tdasilva has quit IRC | 08:48 | |
| *** tdasilva has joined #openstack-infra | 08:48 | |
| *** bobh has joined #openstack-infra | 09:06 | |
| *** ricolin has quit IRC | 09:07 | |
| *** bobh has quit IRC | 09:10 | |
| *** sshnaidm|afk is now known as sshnaidm | 09:26 | |
| *** zbr|flu is now known as zbr | 09:33 | |
| zbr | AJaeger: can you please look at https://review.opendev.org/#/c/676464/ -- you already know it (ensure-tox changes). | 09:34 |
| *** alemgeta has quit IRC | 09:36 | |
| AJaeger | zbr: why is software factory failing? That is a red-herring for me. In general, I'd like a few others to chime in on usefullness of that change | 09:37 |
| zbr | AJaeger: i already contacted them, i can explain you. | 09:38 |
| zbr | mainly it fails because they run the role in a container without root access. role fails to fix broken system tox launcher in that case. | 09:38 |
| zbr | afaik that is not the role fault, but if you have ideas I am open to suggestions. | 09:39 |
| zbr | there is nowhere documented that ensure-tox would not be allowed to sudo | 09:42 |
| AJaeger | zbr: add that explanation as comment, please | 09:42 |
| AJaeger | zbr: left a couple of nits... This is a change that I like mordred and corvus to review further | 09:43 |
| AJaeger | sorry, I'm a bit conservative for zuul-jobs ;) | 09:44 |
| zbr | AJaeger: sure. I was not expecting quick merge on this sensitive area, just wanted to prepare it. | 09:44 |
| zbr | i really want to address the outdated tox version issue. | 09:44 |
| AJaeger | understood - I suggest to ask later today again... | 09:45 |
| zbr | AJaeger: the only task failing is the last one, the one that attempts to repair system tox if is broken. I could make it silently pass because tox is still working correctly if called as module. | 09:46 |
| zbr | so mainly the become part is desirable but not mandatory for the functioning. | 09:47 |
| zbr | in fact I could do something else: attempt to fix broken system tox only if tox_executable is just tox instead of python module calling, because that is what it will be used later. | 09:48 |
| zbr | this would avoid failure on SF | 09:48 |
| *** onovy has quit IRC | 09:48 | |
| *** dpawlik has joined #openstack-infra | 09:48 | |
| zbr | i guess the contract of ensure-tox is that tox will be working if called using "tox_executable", right? | 09:49 |
| zbr | so it would not be a breach of contract | 09:49 |
| *** hamzy_ has joined #openstack-infra | 09:51 | |
| AJaeger | yes, I think that's that contract | 09:52 |
| zbr | AJaeger: I could change the tox_executable from current tox to "{{...}} -m tox" before doing this change. does it worth? | 09:53 |
| *** hamzy has quit IRC | 09:54 | |
| *** onovy has joined #openstack-infra | 09:56 | |
| * AJaeger is not sure - better discuss with rest of team | 09:56 | |
| zbr | ok, and thanks | 09:58 |
| *** yamamoto has quit IRC | 10:00 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Allow ensure-tox to upgrade tox version https://review.opendev.org/676464 | 10:13 |
| *** Lucas_Gray has joined #openstack-infra | 10:20 | |
| *** hongbin has joined #openstack-infra | 10:22 | |
| *** markvoelker has joined #openstack-infra | 10:36 | |
| *** apetrich has quit IRC | 10:37 | |
| *** markvoelker has quit IRC | 10:41 | |
| *** yamamoto has joined #openstack-infra | 10:53 | |
| *** yamamoto has quit IRC | 10:59 | |
| zbr | AJaeger: sorry to bother again, but now one of the SF jobs (f27) is failing because they decided to forcerly use python2 on it instead of official python (py36). | 11:05 |
| zbr | who's to blame the role or the test? | 11:06 |
| *** yamamoto has joined #openstack-infra | 11:06 | |
| *** yamamoto has quit IRC | 11:10 | |
| *** hongbin has quit IRC | 11:16 | |
| *** jaosorior has quit IRC | 11:18 | |
| mordred | zbr: the patch seems fine - but wouldn't an easy fix to the issue you're describing just simply be for a deployer to not include tox pre-installed from distro packages in their base images? | 11:19 |
| mordred | because the current behavior is "if there is a tox pre-installed, we trust that you know what you're doing, otherwise we'll install it for you, which will install the latest version" | 11:20 |
| *** zhangfei has quit IRC | 11:23 | |
| *** ijw has joined #openstack-infra | 11:27 | |
| *** kjackal has quit IRC | 11:28 | |
| *** kjackal has joined #openstack-infra | 11:30 | |
| *** yamamoto has joined #openstack-infra | 11:31 | |
| zbr | mordred: that role is used in various ways, multiple zuul instances, multiple images, totally outside out control. i am sure we cannot cover for all cases, but that does not mean we should not make the role easier to reuse. | 11:31 |
| *** ijw has quit IRC | 11:31 | |
| mordred | oh sure - like I said, the change seems mostly fine. it's just that at some point maybe people shouldn't pre-install old and broken software on images | 11:33 |
| zbr | mordred++ | 11:34 |
| mordred | that said - the behavior change thing is important - and I think we should rethink that bit | 11:35 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Set failed, unreachable, skipped statuses in json plugin https://review.opendev.org/676516 | 11:39 |
| *** markvoelker has joined #openstack-infra | 11:40 | |
| *** yamamoto_ has joined #openstack-infra | 11:41 | |
| *** yamamoto has quit IRC | 11:45 | |
| *** Lucas_Gray has quit IRC | 11:45 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Add appending yaml log plugin https://review.opendev.org/623256 | 11:46 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Process yaml log files if they exist https://review.opendev.org/676246 | 11:46 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write errors from ansible execution into yaml log https://review.opendev.org/676250 | 11:46 |
| *** rlandy has joined #openstack-infra | 11:46 | |
| *** rlandy is now known as rlandy|rover | 11:46 | |
| *** tdasilva has quit IRC | 11:51 | |
| *** jaosorior has joined #openstack-infra | 11:58 | |
| *** bhavikdbavishi has joined #openstack-infra | 12:00 | |
| *** electrofelix has joined #openstack-infra | 12:01 | |
| *** rh-jelabarre has joined #openstack-infra | 12:11 | |
| *** bhavikdbavishi1 has joined #openstack-infra | 12:13 | |
| *** rfolco has joined #openstack-infra | 12:14 | |
| *** bhavikdbavishi has quit IRC | 12:14 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 12:15 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: WIP: Allow ensure-tox to upgrade tox version https://review.opendev.org/676464 | 12:18 |
| *** yamamoto_ has quit IRC | 12:20 | |
| *** bobh has joined #openstack-infra | 12:21 | |
| *** tdasilva has joined #openstack-infra | 12:27 | |
| *** yamamoto has joined #openstack-infra | 12:28 | |
| openstackgerrit | Mohammed Naser proposed opendev/system-config master: ircbot: add #openstack-ansible-sig to bots https://review.opendev.org/676687 | 12:34 |
| *** smcginnis is now known as smcginnis|pto | 12:34 | |
| *** ricolin has joined #openstack-infra | 12:37 | |
| *** dpawlik has quit IRC | 12:38 | |
| *** tdasilva has quit IRC | 12:42 | |
| *** hongbin has joined #openstack-infra | 12:43 | |
| *** hongbin has quit IRC | 12:43 | |
| *** guoqiao has quit IRC | 12:47 | |
| *** guoqiao has joined #openstack-infra | 12:47 | |
| *** dpawlik has joined #openstack-infra | 12:50 | |
| *** bobh has quit IRC | 12:50 | |
| AJaeger | mnaser: you first need to add to accessbot in project-config ^ | 12:53 |
| *** dpawlik has quit IRC | 12:55 | |
| AJaeger | mordred: could you review https://review.opendev.org/676488 and https://review.opendev.org/676489 , please? | 12:55 |
| *** eharney has joined #openstack-infra | 12:55 | |
| *** apetrich has joined #openstack-infra | 13:00 | |
| AJaeger | thanks, mordred ! | 13:04 |
| mordred | sure thing! | 13:05 |
| *** yamamoto has quit IRC | 13:06 | |
| *** aaronsheffield has joined #openstack-infra | 13:06 | |
| openstackgerrit | Merged opendev/base-jobs master: Allow not writing root-marker file for docs publishing https://review.opendev.org/676488 | 13:11 |
| *** yamamoto has joined #openstack-infra | 13:17 | |
| *** mriedem has joined #openstack-infra | 13:18 | |
| *** tdasilva has joined #openstack-infra | 13:19 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: Remove support for ansible 2.5 https://review.opendev.org/650431 | 13:20 |
| *** jeliu_ has joined #openstack-infra | 13:22 | |
| *** bobh has joined #openstack-infra | 13:25 | |
| *** yamamoto has quit IRC | 13:29 | |
| *** bobh has quit IRC | 13:30 | |
| *** yamamoto has joined #openstack-infra | 13:35 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: Remove support for ansible 2.5 https://review.opendev.org/650431 | 13:38 |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: Switch ansible_default to 2.8 https://review.opendev.org/676695 | 13:38 |
| *** priteau has joined #openstack-infra | 13:38 | |
| *** yamamoto has quit IRC | 13:39 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: WIP: Support Ansible 2.9 https://review.opendev.org/674854 | 13:42 |
| openstackgerrit | Paul Belanger proposed openstack/project-config master: Set default-ansible-version to 2.7 for all tenant (minus zuul) https://review.opendev.org/676697 | 13:45 |
| *** yamamoto has joined #openstack-infra | 13:53 | |
| *** guoqiao has quit IRC | 13:59 | |
| *** kjackal has quit IRC | 14:09 | |
| *** kjackal has joined #openstack-infra | 14:11 | |
| openstackgerrit | Mohammed Naser proposed openstack/project-config master: ircbot: add #openstack-ansible-sig to bots https://review.opendev.org/676705 | 14:11 |
| *** xenos76 has quit IRC | 14:15 | |
| *** noorul has joined #openstack-infra | 14:16 | |
| *** xenos76 has joined #openstack-infra | 14:17 | |
| *** diga has quit IRC | 14:20 | |
| *** kjackal has quit IRC | 14:20 | |
| openstackgerrit | Carlos Goncalves proposed openstack/diskimage-builder master: Allow extra repositories to be added to images https://review.opendev.org/676708 | 14:24 |
| openstackgerrit | Mohammed Naser proposed opendev/system-config master: ircbot: add #openstack-ansible-sig to bots https://review.opendev.org/676687 | 14:29 |
| *** yamamoto has quit IRC | 14:29 | |
| *** yamamoto has joined #openstack-infra | 14:30 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Add appending yaml log plugin https://review.opendev.org/623256 | 14:33 |
| openstackgerrit | Merged zuul/zuul-jobs master: Log errors better in case of unknown REST errors https://review.opendev.org/676476 | 14:39 |
| *** noorul has quit IRC | 14:42 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Retry more operations https://review.opendev.org/676518 | 14:44 |
| *** bobh has joined #openstack-infra | 14:44 | |
| *** noorul has joined #openstack-infra | 14:47 | |
| *** bobh has quit IRC | 14:51 | |
| *** noorul has quit IRC | 14:52 | |
| *** armax has joined #openstack-infra | 14:53 | |
| *** noorul has joined #openstack-infra | 14:57 | |
| *** gyee has joined #openstack-infra | 14:57 | |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create zookeeper operator and zuul CR to k8s test https://review.opendev.org/676458 | 14:58 |
| openstackgerrit | Merged opendev/base-jobs master: Revert "Revert "Store logs in swift"" https://review.opendev.org/676523 | 15:00 |
| openstackgerrit | Merged opendev/base-jobs master: Add FN to swift log storage https://review.opendev.org/676527 | 15:00 |
| corvus | infra-root: we have switched to storing logs in swift again, this time with the addition of fortnebula ^ | 15:00 |
| fungi | thanks for the heads-up corvus! | 15:01 |
| fungi | this is very exciting | 15:01 |
| mnaser | could i get a +2 on this goofup -- https://review.opendev.org/#/c/676422/ | 15:01 |
| AJaeger | \o/ | 15:01 |
| AJaeger | mnaser: rebase it first on master, it depends on a failing change | 15:02 |
| mnaser | i already rebased it on a merged change AJaeger | 15:02 |
| *** noorul has quit IRC | 15:02 | |
| mnaser | the other one is sitting seperately afaik | 15:02 |
| AJaeger | mnaser: ah, missed the rebase | 15:03 |
| *** kaiokmo has quit IRC | 15:06 | |
| *** noorul has joined #openstack-infra | 15:07 | |
| AJaeger | corvus: could you put https://review.opendev.org/676489 on your review list, please? And I need help with https://review.opendev.org/676630 at some point (not urgent) | 15:09 |
| *** noorul has quit IRC | 15:11 | |
| *** noorul has joined #openstack-infra | 15:11 | |
| openstackgerrit | Merged openstack/project-config master: projects: add openstack/ansible-role-uwsgi to zuul https://review.opendev.org/676422 | 15:14 |
| Shrews | infra-root: we seem to have more than a few long-running (in days) dib processes on nb01 and nb02. I'm not sure what that's about or how to clean it up. I'd like to restart nodepool-builder to get the new version of openstacksdk | 15:20 |
| *** dave-mccowan has joined #openstack-infra | 15:22 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write json log file by seeking and appending https://review.opendev.org/676717 | 15:23 |
| mordred | Shrews: ++ | 15:24 |
| Shrews | At least one from the 12th: http://paste.openstack.org/show/757338/ | 15:24 |
| corvus | i see one post_failure, and it failed on "include upload logs role" | 15:24 |
| corvus | there's no other data because of the no_log | 15:25 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Create zookeeper operator and zuul CR to k8s test https://review.opendev.org/676458 | 15:27 |
| *** jamesdenton has quit IRC | 15:29 | |
| *** dave-mccowan has quit IRC | 15:30 | |
| corvus | donnyd: found this one at random: https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_50/675750/4/check/openstack-tox-pep8/89f4fd2/ :) | 15:36 |
| *** sthussey has joined #openstack-infra | 15:38 | |
| donnyd | So it looks like its working | 15:39 |
| mordred | \o/ | 15:39 |
| donnyd | I also was able to tune up my air handler a little last night to push just a bit more air and it looks like I will be able to run a few more swift servers. So they are up and running and heat seems to be looking good | 15:40 |
| *** efried has joined #openstack-infra | 15:40 | |
| Shrews | #status log Restarted all nodepool builders to pick up openstacksdk 0.34.0 | 15:40 |
| donnyd | What it means is capacity went from 15TB to 30 | 15:40 |
| openstackstatus | Shrews: finished logging | 15:40 |
| efried | o/ | 15:41 |
| efried | qq: If I have a series, patch1=>patch2, and patch1 has a Depends-On, will patch2 run zuul with the same dep loaded in? | 15:41 |
| donnyd | https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_14/492114/12/ | 15:41 |
| donnyd | It would be awesome if we could have a listing at the root of the container | 15:42 |
| donnyd | It would at least make it so someone could connect and automagically find links | 15:43 |
| corvus | donnyd: i agree, but i think we found that the radosgw couldn't handle having both index.html and auto-generated indexes. so we went with index.html files for maximum compatibility. | 15:43 |
| corvus | (i'm not 100% sure i've remembered that bug correctly, it was like a year ago, but it was some nonsense like that :) | 15:44 |
| *** markvoelker has quit IRC | 15:44 | |
| donnyd | Can we drop on of those in each level of the container? | 15:44 |
| donnyd | makes sense as to why, but if we can have one in each then we could connect something that is unaware of the rest of the system to get listings | 15:45 |
| donnyd | but of course I have no idea how complex it would be to do what I am asking | 15:45 |
| pabelanger | efried: yes, it should | 15:45 |
| efried | thanks | 15:45 |
| *** efried has quit IRC | 15:46 | |
| corvus | donnyd: no, the upload job only knows about the subtree it's working on -- there is no part of zuul that knows what the siblings might be | 15:46 |
| donnyd | can we drop an index.html * | 15:46 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write json log file by seeking and appending https://review.opendev.org/676717 | 15:46 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write errors from ansible execution into json log https://review.opendev.org/676723 | 15:46 |
| donnyd | Ok, then that makes sense to me | 15:46 |
| donnyd | Here is how many containers are there at present https://usercontent.irccloud-cdn.com/file/ibn5Oh8r/image.png | 15:47 |
| *** markvoelker has joined #openstack-infra | 15:48 | |
| corvus | we'll end up with a lot of those eventually. about like this: http://logs.openstack.org/ | 15:52 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: WIP: install-openshift: bump version to 3.11.0 https://review.opendev.org/672785 | 15:54 |
| *** tdasilva has quit IRC | 15:55 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/nodepool master: DNM: test openshift version bump https://review.opendev.org/672788 | 15:55 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: link the buildset page from the build https://review.opendev.org/675493 | 15:57 |
| *** lucasagomes has quit IRC | 15:58 | |
| donnyd | I am going to work at some form of backup solution for the logs. Anyone know of a lightweight way to backup swift? | 15:58 |
| clarkb | I think a common method is to replicate to another swift? for our purposes the swift built in redundancy is probably fine for logs | 15:59 |
| *** slaweq has joined #openstack-infra | 16:00 | |
| *** noorul has quit IRC | 16:01 | |
| donnyd | so swift download from primary / swift upload to backup | 16:01 |
| *** markvoelker has quit IRC | 16:02 | |
| *** noorul has joined #openstack-infra | 16:03 | |
| *** yamamoto has quit IRC | 16:05 | |
| *** yamamoto has joined #openstack-infra | 16:05 | |
| *** yamamoto has quit IRC | 16:07 | |
| *** kopecmartin is now known as kopecmartin|off | 16:09 | |
| openstackgerrit | Merged opendev/irc-meetings master: Modify the QA office hour time https://review.opendev.org/676623 | 16:09 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: manager: specify report failure in logs https://review.opendev.org/671760 | 16:09 |
| *** yamamoto has joined #openstack-infra | 16:11 | |
| *** yamamoto has quit IRC | 16:11 | |
| *** markvoelker has joined #openstack-infra | 16:11 | |
| *** yamamoto has joined #openstack-infra | 16:11 | |
| openstackgerrit | Tristan Cacqueray proposed opendev/gerritbot master: Support missing daemon lib https://review.opendev.org/664472 | 16:13 |
| *** slaweq has quit IRC | 16:14 | |
| *** yamamoto has quit IRC | 16:16 | |
| *** noorul has quit IRC | 16:16 | |
| *** ricolin has quit IRC | 16:17 | |
| *** mattw4 has joined #openstack-infra | 16:18 | |
| *** eharney has quit IRC | 16:20 | |
| corvus | AJaeger: on 676489, is it correct for sphinx_output_source to not have a trailing / ? | 16:25 |
| corvus | it looks like that's a change, and since it's rsync, could make a difference | 16:26 |
| *** e0ne has quit IRC | 16:28 | |
| *** slaweq has joined #openstack-infra | 16:31 | |
| AJaeger | corvus: better be safe... thanks | 16:33 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote-openstack-manuals jobs https://review.opendev.org/676489 | 16:33 |
| AJaeger | mordred: could you review again, please? ^ | 16:35 |
| AJaeger | thanks, corvus | 16:35 |
| mordred | corvus: good catch | 16:35 |
| AJaeger | corvus, mordred, do you have any advise on https://review.opendev.org/#/c/676630 - how can I calculate the path? | 16:37 |
| AJaeger | can I just put in a playbook? | 16:37 |
| *** slaweq has quit IRC | 16:38 | |
| corvus | AJaeger: i'm not 100% sure i understand the question | 16:39 |
| mnaser | hmm | 16:40 |
| Shrews | nb01 seems to be having problems building images. I see errors near the point dib creates a block device. | 16:40 |
| timburke | donnyd, depending on your tolerance for pain, you could also look at setting up container sync for swift-to-swift replication: https://docs.openstack.org/swift/latest/overview_container_sync.html | 16:40 |
| mnaser | is it expected behaviour that clicking job names in zuul dashboard (that are in status page) which are completed goes to the log page, but not the build page? | 16:40 |
| AJaeger | corvus: we need to map from zuul.project to service_type_data.service_type, so from nova to compute or from swift to object-store | 16:40 |
| timburke | (disclaimer: i've never set it up personally, and my understanding is that it isn't well-maintained) | 16:41 |
| corvus | mnaser: yes; the build page doesn't exist until the change is reported. we'd like to change that, but it's a bigger architectural change to zuul, so probably won't happen for a while | 16:41 |
| Shrews | corvus: mordred: do you recall what needs to be done when dib encounters block device issues? ^^^ | 16:41 |
| AJaeger | corvus: can I just do the calculation ? | 16:41 |
| mnaser | corvus: ok cool, makes sense, thank you :) | 16:41 |
| corvus | AJaeger: okay, i think i understand now -- gimme a sec | 16:41 |
| AJaeger | sure, thanks | 16:42 |
| corvus | Shrews: unsure, let's also ping clarkb and fungi | 16:42 |
| corvus | (they are busy, but just in case they're in like a boring meeting or something) | 16:42 |
| Shrews | a boring meeting? do such things exist??? | 16:43 |
| *** slaweq has joined #openstack-infra | 16:48 | |
| corvus | AJaeger: i think the problem is going to be getting that service_type variable to the docs promote playbook https://opendev.org/opendev/base-jobs/src/branch/master/playbooks/docs/promote.yaml | 16:49 |
| clarkb | uh in the past I have disabled the service in systemd and rebooted | 16:49 |
| clarkb | then checked that things are happy disk wise (available space etc) and started nodepool again | 16:49 |
| clarkb | that clears out mounts and devices and all that that dib can leak | 16:49 |
| corvus | AJaeger: anything we do in any previous playbooks won't affect that one. so even if we run the set-service-type-data-fact role in a pre-playbook, it'll be cleared when we run that playbook. i *think* this job will need its own playbook. | 16:50 |
| AJaeger | corvus: how sad ;( Should I place that playbook in project-config and copy over? | 16:50 |
| corvus | AJaeger: yeah, let's do that. i don't really think this warrants a role. | 16:51 |
| corvus | AJaeger: this was designed to cover 95% of the cases. i think this is the other 5% :) | 16:51 |
| AJaeger | ;) Thanks for looking into it! | 16:51 |
| * AJaeger will update... | 16:52 | |
| corvus | (for the record, i have thought of some terrible ideas about saving variables into files and loading them in other playbooks based on paths encoded into secrets, but that sounds way more complicated and hard to maintain than just making a custom playbook) | 16:52 |
| *** sshnaidm is now known as sshnaidm|afk | 16:54 | |
| AJaeger | corvus: btw. do I need to duplicate the secret if we change data in it like I do in https://review.opendev.org/#/c/676630/3/zuul.d/secrets.yaml ? Or is there a way to update docs_master_path? | 16:54 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: install-openshift: bump version to 3.11.0 https://review.opendev.org/672785 | 16:54 |
| corvus | AJaeger: maybe a yaml anchor will work, but i've never tried that with a type marker (the !encrypted/pkcs1-oaep bit) so i'm not sure where to put it. | 16:55 |
| AJaeger | fun ;/ | 16:56 |
| *** ramishra has joined #openstack-infra | 16:57 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: WIP: Allow ensure-tox to upgrade tox version https://review.opendev.org/676464 | 16:58 |
| corvus | AJaeger: if you want to try, i think it would still go before the tag | 16:58 |
| AJaeger | Ok, might be a separate change ;) | 16:58 |
| corvus | AJaeger: so "keytab: &afskey !encrypted/pkcs1-oaep\n - ..." and then "keytab: *afskey" | 16:58 |
| corvus | that's my guess from skimming the spec | 16:59 |
| AJaeger | thanks, corvus | 16:59 |
| donnyd | timburke: My tolerance is pretty high :) | 17:00 |
| corvus | AJaeger: also, for this specific case, if you're "hardcoding" the paths into the playbook, you might be able to just use the same secret and ignore the paths variables on it | 17:00 |
| openstackgerrit | Merged openstack/project-config master: Add promote-openstack-manuals jobs https://review.opendev.org/676489 | 17:00 |
| AJaeger | yeah... | 17:00 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write json log file by seeking and appending https://review.opendev.org/676717 | 17:01 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write errors from ansible execution into json log https://review.opendev.org/676723 | 17:01 |
| *** derekh has quit IRC | 17:05 | |
| corvus | i sent a followup email to the infra and openstack-discuss lists letting folks know we've made the switch | 17:06 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: WIP: Allow ensure-tox to upgrade tox version https://review.opendev.org/676464 | 17:09 |
| *** electrofelix has quit IRC | 17:10 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote api-ref/guide jobs https://review.opendev.org/676630 | 17:11 |
| AJaeger | corvus: here's a try ^ | 17:11 |
| corvus | AJaeger: that looks promising! | 17:13 |
| AJaeger | \o/ | 17:13 |
| AJaeger | then we need https://review.opendev.org/676624 as well | 17:14 |
| AJaeger | Now merging and testing first openstack-manuals | 17:14 |
| AJaeger | infra-root, I'll merge some changes to openstack-manuals, if the docs site becomes screwed up, you know whom to blame... | 17:15 |
| mordred | AJaeger: ourselves? | 17:19 |
| AJaeger | mordred: I wrote the changes, you just reviewed ;) | 17:19 |
| AJaeger | and didn't stop me :) | 17:19 |
| AJaeger | so, answer is: Me | 17:20 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote api-ref/guide jobs https://review.opendev.org/676630 | 17:23 |
| *** ramishra has quit IRC | 17:26 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: manager: specify report failure in logs https://review.opendev.org/671760 | 17:27 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Fix download_artifact_job names for manuals jobs https://review.opendev.org/676740 | 17:27 |
| *** eharney has joined #openstack-infra | 17:32 | |
| *** rosmaita has quit IRC | 17:33 | |
| *** rosmaita has joined #openstack-infra | 17:37 | |
| *** priteau has quit IRC | 17:39 | |
| *** bobh has joined #openstack-infra | 17:39 | |
| AJaeger | corvus, mordred, I mistyped the job names, could you review https://review.opendev.org/676740 , please? | 17:41 |
| *** armax has quit IRC | 17:49 | |
| Shrews | infra-root: I'm going to reboot nb01 | 17:50 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Build wheels with javascript and tarballs without https://review.opendev.org/676470 | 17:51 |
| *** e0ne has joined #openstack-infra | 17:52 | |
| mordred | Shrews: kk | 17:52 |
| zbr | AJaeger: mordred: can you have another look at https://review.opendev.org/#/c/676464/ ? (ensure-tox) | 17:52 |
| Shrews | #status log Rebooted nb01.o.o to clear out stale dib mounts | 17:54 |
| openstackstatus | Shrews: finished logging | 17:54 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote api-ref/guide jobs https://review.opendev.org/676630 | 17:54 |
| *** bobh has quit IRC | 17:55 | |
| AJaeger | corvus: our jobs output in the logs something like "Log URL (when completed): https://logs.opendev.org..." but this URL is not correct anymore, see https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_50/675750/4/check/tripleo-ci-centos-7-undercloud-containers/6bf12ad/job-output.txt | 17:59 |
| AJaeger | should we remove it from job-header? | 18:00 |
| corvus | AJaeger: i'm thinking remove; i'm not sure it's that useful anymore | 18:00 |
| AJaeger | on it... | 18:00 |
| pabelanger | maybe a toggle, we still use it in zuul.a.c | 18:00 |
| corvus | (we can't point it to the swift location because we don't know it; we could point it at the build page, but that may not exist until well after the job completes | 18:01 |
| AJaeger | corvus: we hae "if zuul_log_url is defined and zuul_log_path is define" - is it correct that both are defined in our case? | 18:01 |
| AJaeger | the message says "LOG URL (when completed)", so we can use the build page... | 18:02 |
| corvus | AJaeger: no i mean the build page isn't there until the *buildset* is completed | 18:02 |
| corvus | could be hours in openstack's case | 18:02 |
| corvus | (ie, if a pep8 job finishes in 5 minutes and a devstack job taks 90m, there will be a window of 85 minutes where the pep8 job will have completed but the build page for it won't exist) | 18:03 |
| AJaeger | corvus: indeed. Still, better than nothing. | 18:03 |
| corvus | is it? | 18:03 |
| corvus | i think nothing is better than a lie :) | 18:03 |
| corvus | but i don't really know the use case | 18:03 |
| openstackgerrit | Merged openstack/project-config master: Fix download_artifact_job names for manuals jobs https://review.opendev.org/676740 | 18:03 |
| AJaeger | for me in a post job: Yes. Quickest way to get the logs ;) | 18:03 |
| corvus | i mean, this thing dates to when we were running with jenkins | 18:03 |
| AJaeger | oh, historic | 18:04 |
| corvus | AJaeger: great, can you elaborate? like, what's the workflow that involves this? | 18:04 |
| AJaeger | corvus: I can get it differently via buildsets nowadays, so I can change my habit ;) | 18:05 |
| corvus | biab | 18:05 |
| AJaeger | I was looking at stream run and wanted to look at something in the final logs but had no reference to it. | 18:05 |
| AJaeger | so, questoin remains: should we remove it completely - or is the error that zuul_log_url is defined and shouuln't be dfeined? | 18:06 |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Remove logging of URL in emit-job-header https://review.opendev.org/676746 | 18:07 |
| AJaeger | this just removes it - pabelanger will you -1? ^ | 18:08 |
| pabelanger | yah, I'd prefer if we did a toggle | 18:08 |
| pabelanger | as we have users that still look for it | 18:08 |
| pabelanger | not everybody is going to use swift too | 18:09 |
| AJaeger | how best to toggle and where to define the toggle? | 18:09 |
| AJaeger | pabelanger: but do users really need it ? Please read my commit message... | 18:09 |
| openstackgerrit | James E. Blair proposed opendev/base-jobs master: WIP Remove log header https://review.opendev.org/676747 | 18:09 |
| corvus | AJaeger, pabelanger: ^ we have a toggle | 18:09 |
| corvus | that's wip because it needs to go through the test cycle | 18:09 |
| pabelanger | wfm | 18:09 |
| pabelanger | +2 on corvus | 18:10 |
| pabelanger | that should allow users that still define it to print if wanted | 18:10 |
| AJaeger | corvus: remove the WIP ;) That looks like it should justdo the right thing! thanks | 18:10 |
| corvus | AJaeger: i'm not going to approve a change like that without going through base-test first | 18:11 |
| corvus | and i'm supposed to be in a meeting right now, so i can't type it up | 18:11 |
| AJaeger | sure - thanks. | 18:11 |
| *** rlandy|rover is now known as rlandy|rover|dra | 18:25 | |
| *** rlandy|rover|dra is now known as rlandy|drappt | 18:25 | |
| *** rfolco is now known as rfolco|rucker|te | 18:25 | |
| *** rfolco|rucker|te is now known as rfolco|rucker | 18:25 | |
| *** armax has joined #openstack-infra | 18:27 | |
| *** e0ne has quit IRC | 18:36 | |
| *** efried has joined #openstack-infra | 18:42 | |
| efried | o/ | 18:43 |
| efried | I'm noticing logs (sometimes?) no longer have linkable timestamps and level filters. | 18:43 |
| efried | Am I clicking through wrong? | 18:43 |
| efried | E.g. https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_99/642899/40/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/402a73a/controller/logs/screen-n-cpu.txt.gz | 18:43 |
| efried | https://3a529b9c3a74f1543cc3-752ab92f1df702672d910e68345a54f5.ssl.cf2.rackcdn.com/656027/23/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/fcf71db/controller/logs/screen-n-cpu.txt.gz | 18:43 |
| TheJulia | efried: do you find the back button doesn't actually take you to a rendered page? | 18:44 |
| efried | TheJulia: The nav buttons seem to be working okay for me | 18:44 |
| TheJulia | efried: I guess is a result of moving to swift since I think the other things were webserver magic | 18:44 |
| efried | right; but I thought I remembered the emails saying the timestamp and filter links would still be there. | 18:45 |
| TheJulia | so there is a timestamp link on the rendered webpage.... but it doesn't actually line up in my browser :\ | 18:45 |
| efried | http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008313.html | 18:45 |
| efried | Next to the "Summary" tab, you'll | 18:46 |
| efried | find the "Logs" tab. This contains an expandable index of all the log | 18:46 |
| efried | files uploaded for the build. If they are text files, they can be | 18:46 |
| efried | rendered in-app with line-number hyperlinks and severity filtering. | 18:46 |
| efried | There are also direct links to the files on the log server. | 18:46 |
| efried | ohhhhh | 18:46 |
| * TheJulia suspects it clicked | 18:46 | |
| efried | I wasn't using the twisties | 18:47 |
| efried | but ugh, when I do, I get bumped to https://zuul.opendev.org/tenants | 18:47 |
| efried | the hover has a double-slash in it https://zuul.opendev.org/t/openstack/build/402a73a9238643c2b893d53b37a6ce27/log/controller//logs/screen-n-cpu.txt.gz -- if I try to put that directly into a browser it does the same thing (redirects to /tenants). If I try to squash the slashes I get an error. | 18:49 |
| TheJulia | okay, I figured out why back was not working. It is funkiness if I loaded the page from zuul.openstack.org instead of zuul.opendev.org | 18:51 |
| *** spsurya has quit IRC | 18:53 | |
| *** ociuhandu has joined #openstack-infra | 18:56 | |
| corvus | efried: that's weird https://zuul.opendev.org/t/openstack/build/402a73a9238643c2b893d53b37a6ce27/log/controller/logs/screen-n-cpu.txt.gz works for me | 18:58 |
| corvus | (squashing the slashes) | 18:58 |
| *** yamamoto has joined #openstack-infra | 18:58 | |
| efried | corvus: let me get you the error... | 18:58 |
| *** ociuhandu has quit IRC | 18:58 | |
| efried | oh, now it's working of course | 18:59 |
| corvus | obviously the double slash should be something we can fix -- but if that link doesn't work in some cases, that's a weirder thing | 18:59 |
| efried | corvus: Yeah, now I can't remember the error, but it was an app-ish one, not like a 4xx or whatever. | 19:00 |
| efried | like a red and white popover in the top right. | 19:00 |
| corvus | TheJulia: and yeah, i was just noticing that sometimes the scroll-to-line thing scrolls up too much -- i think it's probably a viewport issue (like the browser thinks more lines are visible but they aren't due to the header) | 19:01 |
| corvus | but https://zuul.opendev.org/t/openstack/build/402a73a9238643c2b893d53b37a6ce27/log/controller/logs/screen-n-cpu.txt.gz#946 should take you to directly to a line | 19:01 |
| TheJulia | corvus: line only ? not date? | 19:02 |
| TheJulia | err datetime code | 19:02 |
| corvus | TheJulia: yes | 19:02 |
| *** yamamoto has quit IRC | 19:03 | |
| corvus | the old system had to use dates because line numbers weren't compatible with severity filtering; this can do both, so we get line number links in every file (even ones without dates) | 19:04 |
| efried | corvus: I'm composing a list of issues I've noticed with the swift-based logs. I noticed the "anchor too far up" thing as well. (FWIW the same thing happens when you try to look at latest docs and you get that annoying banner warning you that stein is the latest release) | 19:05 |
| corvus | i'll start working on the double slash issue right after i get back from lunch; that seems the most critical | 19:06 |
| corvus | (and should be easiest to fix) | 19:06 |
| corvus | (i think you have to get to a certain depth before it kicks in, which is why we didn't notice before) | 19:06 |
| corvus | efried: a popup in the top right *could* be a transient network error | 19:07 |
| efried | okay | 19:08 |
| corvus | (it could also be a systemic problem, but also a simple "the remote server barfed" would show up like that too) | 19:08 |
| corvus | i'm off to forage, biab | 19:08 |
| efried | thanks corvus. FYI http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008518.html | 19:11 |
| *** dmsimard has quit IRC | 19:15 | |
| frickler | hmm, if I go two levels deep on https://zuul.opendev.org/t/openstack/build/402a73a9238643c2b893d53b37a6ce27/logs , I get links like https://zuul.opendev.org/t/openstack/build/402a73a9238643c2b893d53b37a6ce27/log/controller//logs/_.localrc_auto.txt.gz that lead to broken pages (note the "//" in the middle) | 19:21 |
| *** dmsimard has joined #openstack-infra | 19:23 | |
| efried | frickler: See above | 19:24 |
| frickler | efried: ah, thx | 19:26 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remove log_url from emit-job-header https://review.opendev.org/676755 | 19:28 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remove log_url from emit-job-header https://review.opendev.org/676756 | 19:28 |
| AJaeger | corvus: I think that's what you wanted to do ^ | 19:28 |
| AJaeger | argh, updating... | 19:28 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remove log_url from emit-job-header (base-test) https://review.opendev.org/676755 | 19:29 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remove log_url from emit-job-header https://review.opendev.org/676756 | 19:29 |
| AJaeger | now with different subject lines ^ | 19:29 |
| AJaeger | pabelanger: ^ | 19:30 |
| *** vesper11 has quit IRC | 19:30 | |
| *** kjackal has joined #openstack-infra | 19:30 | |
| frickler | it would also be nice to have the current file name that is being displayed on the page, since the url is too long and cut off | 19:31 |
| frickler | maybe even have this as clickable links to allow easy navigating back into the tree, similar to what gitea does | 19:32 |
| pabelanger | AJaeger: left comment | 19:32 |
| AJaeger | frickler: do you have time to review https://review.opendev.org/#/c/676624/ , please? | 19:33 |
| AJaeger | pabelanger: will change - thanks | 19:33 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remove log_url from emit-job-header (base-test) https://review.opendev.org/676755 | 19:33 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remove log_url from emit-job-header https://review.opendev.org/676756 | 19:33 |
| pabelanger | +2 | 19:34 |
| mriedem | clarkb: fungi: i've noticed that with the log / zuul change stuff lately, console log lines are no longer linkable, e.g. https://object-storage-ca-ymq-1.vexxhost.net/v1/86bbbcfa8ad043109d2d7af530225c72/logs_39/676539/4/check/nova-grenade-live-migration/bd182b5/logs/grenade.sh.txt.gz | 19:34 |
| efried | mriedem: use the twisties | 19:34 |
| mriedem | huh? | 19:35 |
| efried | mriedem: see http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008518.html | 19:35 |
| mriedem | i'm looking at raw | 19:35 |
| efried | right | 19:35 |
| AJaeger | pabelanger: I need +2 on https://review.opendev.org/#/c/676755/3 - we need to merge that first for testing... | 19:35 |
| mriedem | so https://zuul.opendev.org/t/openstack/build/bd182b5bef2143e09ba6737b4f893d2e/log/logs/grenade.sh.txt.gz#45562 | 19:35 |
| mriedem | ok | 19:35 |
| efried | mriedem: looking at raw you can't link timestamps or filter by log level. You have to be looking in app mode, for which you need to use the twisties on the left (tree navigation dealy) | 19:35 |
| mriedem | thanks | 19:36 |
| pabelanger | done | 19:36 |
| mriedem | sob, | 19:36 |
| mriedem | https://review.opendev.org/#/c/673923/ | 19:36 |
| mriedem | still not merged | 19:36 |
| mriedem | clarkb: slaweq: we might want to consider backporting the devstack change to run n-api-meta with caching | 19:37 |
| mriedem | since we're still seeing ssh failures in grenade jobs on master (and the normal ssh fails in stable) | 19:38 |
| slaweq | mriedem: that could be good idea | 19:38 |
| slaweq | should I propose this backport? | 19:38 |
| mriedem | slaweq: sure | 19:38 |
| mriedem | i'm not devstack stable core though b/c they don't trust me | 19:39 |
| mriedem | by they i mean mtreinish | 19:39 |
| mriedem | :P | 19:39 |
| slaweq | mriedem: sure :D | 19:40 |
| slaweq | done https://review.opendev.org/#/q/I9082be077b59acd3a39910fa64e29147cb5c2dd7 | 19:40 |
| *** noama has quit IRC | 19:40 | |
| *** jcoufal has joined #openstack-infra | 19:41 | |
| mriedem | slaweq: do we have any data to backup that random tempest ssh fails have gone done on master since your change merged? i *feel* like they have, but don't have anything tracking it to back it up | 19:41 |
| slaweq | mriedem: do You think about some logstash query? | 19:43 |
| slaweq | or something like that | 19:43 |
| mriedem | http://status.openstack.org/elastic-recheck/#1808010 ? | 19:44 |
| mriedem | http://status.openstack.org/elastic-recheck/#1463631 | 19:44 |
| slaweq | mriedem: yes, I was always using query similar to http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22failed%20to%20get%20http:%2F%2F169.254.169.254%2F2009-04-04%2Fmeta-data%5C%22%20AND%20tags:%5C%22console%5C%22%20AND%20voting:1&from=864000s | 19:47 |
| slaweq | to find them | 19:47 |
| slaweq | basically it was always this issue when You could see this "failed to get http://169.254.169.254/2009-04-04/meta-data/public-keys" in instance's console log | 19:48 |
| *** jcoufal has quit IRC | 19:49 | |
| mtreinish | mriedem: you don't have stable devstack core? I can fix that | 19:50 |
| mriedem | mtreinish: nope | 19:50 |
| mtreinish | mriedem: you should be careful what you wish for... | 19:50 |
| mriedem | slaweq: this is the grenade one: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%2270_cinder%5C%22%20AND%20message%3A%5C%22SSH%20to%20the%20client%20did%20not%20work%2C%20something%20very%20wrong%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d | 19:50 |
| mriedem | mtreinish: i wish i had a monkey's paw | 19:51 |
| slaweq | mriedem: hard to say without console log if this was the same issue | 19:53 |
| mriedem | yeah i'll put up a separate bug 1840355 and e-r for this grenade one | 19:55 |
| openstack | bug 1840355 in grenade "cinder server ping check intermittently fails in the gate with "SSH to the client did not work, something very wrong"" [Undecided,New] https://launchpad.net/bugs/1840355 | 19:55 |
| openstackgerrit | Matt Riedemann proposed opendev/elastic-recheck master: Add query for grenade ping check fail bug 1840355 https://review.opendev.org/676811 | 19:58 |
| openstack | bug 1840355 in grenade "cinder server ping check intermittently fails in the gate with "SSH to the client did not work, something very wrong"" [Undecided,Confirmed] https://launchpad.net/bugs/1840355 | 19:58 |
| *** gouthamr is now known as gouthamr|brb | 20:02 | |
| *** eharney has quit IRC | 20:08 | |
| *** dougsz has joined #openstack-infra | 20:10 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Fix double slash in log browsing https://review.opendev.org/676813 | 20:15 |
| openstackgerrit | Merged opendev/elastic-recheck master: Add query for grenade ping check fail bug 1840355 https://review.opendev.org/676811 | 20:16 |
| openstack | bug 1840355 in grenade "cinder server ping check intermittently fails in the gate with "SSH to the client did not work, something very wrong"" [Undecided,Confirmed] https://launchpad.net/bugs/1840355 | 20:16 |
| *** dougsz has quit IRC | 20:18 | |
| *** kjackal has quit IRC | 20:24 | |
| *** jtomasek has quit IRC | 20:27 | |
| *** kjackal has joined #openstack-infra | 20:28 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: JS: account for header when scrolling to line https://review.opendev.org/676818 | 20:30 |
| *** gouthamr|brb is now known as gouthamr | 20:30 | |
| *** mattw4 has quit IRC | 20:31 | |
| *** mattw4 has joined #openstack-infra | 20:31 | |
| *** mattw4 has quit IRC | 20:32 | |
| *** dougsz has joined #openstack-infra | 20:33 | |
| *** guoqiao has joined #openstack-infra | 20:41 | |
| *** jcoufal has joined #openstack-infra | 20:42 | |
| *** rlandy|drappt is now known as rlandy|rover | 20:47 | |
| *** rfolco|rucker is now known as rfolco | 20:48 | |
| *** dougsz has quit IRC | 20:48 | |
| *** jcoufal has quit IRC | 20:49 | |
| *** slaweq has quit IRC | 20:50 | |
| *** vesper11 has joined #openstack-infra | 20:51 | |
| efried | corvus: fyi, got a repro on that popover error (I think it's the same one). Text is: | 20:54 |
| efried | Network Error (Unable to fetch URL, check your network connectivity, browser plugins, and ad-blockers) https://object-storage-ca-ymq-1.vexxhost.net/v1/86bbbcfa8ad043109d2d7af530225c72/logs_42/664842/19/check/ironic-standalone/7e91a4f/zuul-manifest.json | 20:54 |
| efried | Happening here right now: https://zuul.opendev.org/t/openstack/build/7e91a4fd087a4d6dae8039c57661c944 | 20:54 |
| corvus | neat, we somehow got a cors error | 20:54 |
| *** bdodd has quit IRC | 20:56 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: extract pure functions from the TaskOutput component https://review.opendev.org/675460 | 20:59 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: test trailing slash are removed from renderTree https://review.opendev.org/676824 | 20:59 |
| *** mriedem has quit IRC | 21:00 | |
| corvus | efried: somehow (?) that container was created without the cors header set... it's possible that it was created before we had the retries around operations like that... anyway, i have set the cors header for that container now, so it should be good | 21:03 |
| corvus | efried: i was able to see some improvement with a hard refresh of that, but one error still lingered until i restarted my browser (it may be that FF caches some cors responses) | 21:03 |
| corvus | efried: thanks very much for the report :) | 21:04 |
| corvus | efried: (btw, we have fixes for the double / and line-link-scrolling in flight) | 21:04 |
| corvus | TheJulia: ^ | 21:04 |
| TheJulia | corvus: awesome! | 21:05 |
| efried | Thanks corvus. The next biggest bugbear for me is the horizontal scrolling. | 21:07 |
| corvus | efried: yeah, i was just looking into that | 21:07 |
| efried | It also seems like the entire line is hyperlinked to anchor, so I can't select (for copy) inside of a line. | 21:08 |
| *** jeliu_ has quit IRC | 21:09 | |
| efried | Is there a storyboard for this or something, so I don't just shotgun you every time I find something? | 21:09 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Ensure tenant web_root url has a trailing slash https://review.opendev.org/676826 | 21:10 |
| corvus | efried: you could use the zuul storyboard for backlog-level stuff, but if it's really hurting productivity, better to ping me or keep an etherpad or something | 21:10 |
| efried | okay, cool. | 21:11 |
| *** slaweq has joined #openstack-infra | 21:11 | |
| corvus | i'm on firefighting duty :) | 21:11 |
| efried | lucky you | 21:11 |
| efried | (lucky me, really) | 21:11 |
| * mordred hands corvus a helmet and hose | 21:12 | |
| mordred | corvus: I feel like you need either a hallberd or a halyard with your helmet and hose but I can't decide which is more helpful | 21:13 |
| corvus | efried: as a workaround now, i found i can click in the whitespace in the log and use arrow keys to scroll (i also don't have a horizontal scrolling pointer) | 21:13 |
| efried | okay, gtk | 21:14 |
| efried | mordred: what kind of hose? | 21:14 |
| mordred | efried: nylon clearly | 21:15 |
| efried | that goes better with the halberd | 21:15 |
| efried | and the helmet, depending. | 21:15 |
| mordred | ++ | 21:15 |
| *** slaweq has quit IRC | 21:17 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: JS: Break log viewer out of the panel https://review.opendev.org/676827 | 21:23 |
| corvus | efried: ^ it may not be beautiful, but it's functional :) | 21:23 |
| efried | corvus: -2 for a typo in your commit message! | 21:24 |
| efried | it's greek to me, unfortunately; how can I test it? | 21:24 |
| efried | And | 21:25 |
| efried | I *always* *always* prefer functional to beautiful. | 21:25 |
| efried | personal preference. | 21:25 |
| mordred | efried++ | 21:26 |
| corvus | efried: when zuul gets around to returning results, there will be a link to a preview build for the 'zuul-build-dashboard' job. but with any luck, it'll already be merged by the time that happens :) | 21:26 |
| efried | cool | 21:26 |
| mordred | corvus: it's got the +2 from me | 21:26 |
| efried | thank you very much for getting on this stuff so quickly. | 21:26 |
| corvus | np -- reducing productivity was not a project goal :) | 21:26 |
| mordred | corvus: AND - I'd like to add that I actually do fully understand the whole patch and none of that +2 is just taking your word for it :) | 21:26 |
| corvus | mordred: \o/ | 21:27 |
| *** tdasilva has joined #openstack-infra | 21:27 | |
| *** tdasilva has quit IRC | 21:27 | |
| *** markvoelker has quit IRC | 21:29 | |
| *** xenos76 has quit IRC | 21:36 | |
| *** pkopec has joined #openstack-infra | 21:38 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: JS: add line numbers to log file https://review.opendev.org/676830 | 21:54 |
| corvus | efried: that's the simplest solution i can think of for the hyperlink issue. it also happens to match just about every other app out there, so seems pretty reasonable :) | 21:54 |
| efried | cool | 21:55 |
| efried | corvus: I can't tell, does that also make the line numbers (and not the rest) link to the anchor? | 21:55 |
| *** yamamoto has joined #openstack-infra | 21:55 | |
| efried | okay, yeah, it looks like it does. | 21:56 |
| * efried resurrects html/js knowledge from 20y ago. | 21:56 | |
| corvus | efried: https://imgur.com/a/E0a3tjP | 21:56 |
| corvus | that's what the stack of changes together looks like | 21:56 |
| efried | it sho is pretty | 21:56 |
| corvus | and yeah, just the line no's | 21:56 |
| efried | I thought you said you were going to make it ugly | 21:56 |
| corvus | oh, the way i did the screenshot didn't capture the scrollbars, but the normal browser scrollbars are both there | 21:57 |
| openstackgerrit | Merged zuul/zuul master: Fix double slash in log browsing https://review.opendev.org/676813 | 21:58 |
| *** trident has quit IRC | 22:03 | |
| *** pkopec has quit IRC | 22:05 | |
| *** ijw has joined #openstack-infra | 22:05 | |
| *** markvoelker has joined #openstack-infra | 22:05 | |
| *** markvoelker has quit IRC | 22:09 | |
| *** trident has joined #openstack-infra | 22:10 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: JS: add line numbers to log file https://review.opendev.org/676830 | 22:11 |
| *** slaweq has joined #openstack-infra | 22:11 | |
| *** markvoelker has joined #openstack-infra | 22:11 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: JS: account for header when scrolling to line https://review.opendev.org/676818 | 22:12 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: JS: Break log viewer out of the panel https://review.opendev.org/676827 | 22:12 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: JS: add line numbers to log file https://review.opendev.org/676830 | 22:12 |
| *** slaweq has quit IRC | 22:16 | |
| *** kjackal has quit IRC | 22:17 | |
| openstackgerrit | Tristan Cacqueray proposed opendev/base-jobs master: base: generate the zuul-manifest before ara-report https://review.opendev.org/676836 | 22:22 |
| *** goldyfruit has quit IRC | 22:35 | |
| *** goldyfruit has joined #openstack-infra | 22:38 | |
| *** markvoelker has quit IRC | 22:40 | |
| *** prometheanfire has quit IRC | 22:42 | |
| *** armax has quit IRC | 22:44 | |
| zigo | Hi there! Your generous help to get in touch with OpenStack cloud providers is very much welcome. | 22:49 |
| zigo | I would love to see some OpenStack cloud provider during the talks and weight in more, because I always feel alone there. | 22:50 |
| zigo | The meetups have so far happened in Seattle, at Google, Amazon or Microsoft. | 22:50 |
| zigo | fungi: ^ | 22:50 |
| zigo | fungi: So having OpenStack public cloud providers showing up during a Debian cloud image sprint is definitively a good thing. | 22:51 |
| zigo | mnaser: Would you come to Boston for the Debian cloud image meet-up? | 22:52 |
| zigo | That'd be in mid-october... | 22:52 |
| * mnaser looks at schedule | 22:52 | |
| mnaser | i know i'm at ansiblefest in september, and openinfra at november so october could be ok | 22:53 |
| mnaser | i think it's not too far that i can just do the drive | 22:53 |
| mnaser | 5 hour drive, not bad | 22:53 |
| *** yamamoto has quit IRC | 22:53 | |
| corvus | i've done that drive, it's very enjoyable :) | 22:54 |
| mnaser | i could probably pop down there if there's more info on the event :> | 22:55 |
| *** goldyfruit has quit IRC | 22:56 | |
| *** goldyfruit has joined #openstack-infra | 22:56 | |
| *** yamamoto has joined #openstack-infra | 22:57 | |
| *** tkajinam has joined #openstack-infra | 23:00 | |
| *** yamamoto has quit IRC | 23:02 | |
| *** aaronsheffield has quit IRC | 23:06 | |
| *** slaweq has joined #openstack-infra | 23:11 | |
| *** slaweq has quit IRC | 23:16 | |
| ianw | pabelanger: finally getting back to f30 -- this is not what you want to see : https://i.imgur.com/17z5UCH.png . somehow we have to relabel them during image build whilst they're mounted :/ | 23:23 |
| pabelanger | ouch | 23:24 |
| pabelanger | ianw: is that why dbus didn't start? | 23:24 |
| pabelanger | I couldn't figure it out last I looked | 23:25 |
| ianw | yep, it can't setup it's mount namespaces | 23:25 |
| *** dchen has joined #openstack-infra | 23:26 | |
| ianw | and everything goes south from there :) | 23:26 |
| *** dchen has quit IRC | 23:26 | |
| pabelanger | ianw: does autorelabel on boot work? | 23:26 |
| pabelanger | touch /.autorelable | 23:26 |
| pabelanger | touch /.autorelabel* | 23:26 |
| pabelanger | IIRC | 23:26 |
| *** dchen has joined #openstack-infra | 23:27 | |
| ianw | yeah, maybe ... always avoided that because it's slow | 23:27 |
| pabelanger | yah, | 23:27 |
| *** ijw has quit IRC | 23:29 | |
| ianw | i wonder if we have enough tools to do it in yum-minimal root.d ... before dib mounts /dev /proc to the host versions ... | 23:31 |
| *** sthussey has quit IRC | 23:34 | |
| *** markvoelker has joined #openstack-infra | 23:45 | |
| *** igordc has joined #openstack-infra | 23:46 | |
| *** rlandy|rover has quit IRC | 23:49 | |
| *** markvoelker has quit IRC | 23:50 | |
| *** armax has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
