| openstackgerrit | Ian Wienand proposed opendev/system-config master: Cleanup grafana.openstack.org https://review.opendev.org/739625 | 00:04 |
|---|---|---|
| *** ryohayakawa has joined #opendev | 00:07 | |
| *** hrw has quit IRC | 00:16 | |
| *** hrw has joined #opendev | 00:21 | |
| *** hrw has quit IRC | 00:36 | |
| *** hrw has joined #opendev | 00:36 | |
| mnaser | can i get a quick peek at https://review.opendev.org/#/c/739602/ ? | 00:38 |
| openstackgerrit | Merged opendev/system-config master: Graphite container deployment https://review.opendev.org/738125 | 00:41 |
| openstackgerrit | Mohammed Naser proposed openstack/project-config master: add vexxhost/openstack-tools https://review.opendev.org/739627 | 00:45 |
| *** hashar has joined #opendev | 00:46 | |
| openstackgerrit | Merged openstack/project-config master: Add openstack/devstack-plugin-ceph in vexxhost tenant https://review.opendev.org/739602 | 00:48 |
| *** DSpider has quit IRC | 00:59 | |
| *** hashar has quit IRC | 01:11 | |
| openstackgerrit | Merged openstack/project-config master: Add "Review-Priority" label to qa projects https://review.opendev.org/739550 | 01:11 |
| ianw | ok, deploy has now failed because the openedge mirror is down | 01:12 |
| fungi | oh, we should add it to the emergency disable list | 01:17 |
| ianw | yeah, have done, not sure if it's a longer term thing and we want to remove from the inventory from prior conversation | 01:17 |
| fungi | unless we still have a disable list in system-config group vars | 01:17 |
| donnyd | Worst case it will be back in Oct | 02:18 |
| *** ysandeep|away is now known as ysandeep | 02:26 | |
| openstackgerrit | melissaml proposed openstack/diskimage-builder master: Switch from unittest2 compat methods to Python 3.x methods https://review.opendev.org/739645 | 03:12 |
| *** factor has quit IRC | 03:26 | |
| *** factor has joined #opendev | 03:27 | |
| *** factor has joined #opendev | 03:28 | |
| *** ykarel|away is now known as ykarel | 04:04 | |
| *** sgw1 has quit IRC | 04:39 | |
| *** bhagyashris is now known as bhagyashris|brb | 05:50 | |
| *** marios has joined #opendev | 06:01 | |
| *** bhagyashris|brb is now known as bhagyashris | 06:16 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Cleanup grafana.openstack.org https://review.opendev.org/739625 | 07:01 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: graphite: statsd timer settings https://review.opendev.org/739665 | 07:05 |
| *** dtantsur|afk is now known as dtantsur | 07:18 | |
| *** tosky has joined #opendev | 07:36 | |
| *** hashar has joined #opendev | 07:42 | |
| *** fressi has quit IRC | 07:53 | |
| *** moppy has quit IRC | 08:01 | |
| *** moppy has joined #opendev | 08:01 | |
| *** kevinz has joined #opendev | 08:13 | |
| openstackgerrit | Sorin Sbarnea (zbr) proposed opendev/base-jobs master: Avoid rsync chgrp errors on promote https://review.opendev.org/739674 | 08:19 |
| *** DSpider has joined #opendev | 08:24 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Use kpartx option to update partition mappings https://review.opendev.org/728824 | 08:27 |
| openstackgerrit | Riccardo Pittau proposed openstack/diskimage-builder master: Convert multi line if statement to case https://review.opendev.org/734479 | 08:31 |
| zbr | Any infra-root online to check my AFS rsync patch? ^ | 08:33 |
| *** ysandeep is now known as ysandeep|lunch | 08:34 | |
| *** priteau has joined #opendev | 08:49 | |
| kevinz | Hi Infra, the Linaro-us mirror looks died | 08:51 |
| kevinz | error: Status code: 403 for https://mirror.regionone.linaro-us.opendev.org/centos/8/AppStream/aarch64/os/repodata/repomd.xml (IP: 2604:1380:4111:3e54:f816:3eff:fea4:899d) (https://mirror.regionone.linaro-us.opendev.org/centos/8/AppStream/aarch64/os/repodata/repomd.xml) | 08:53 |
| kevinz | Ajaeger:ianw: This mirror has strangely shutdowned...And I reboot it but there is no files on the link: http://mirror01.regionone.linaro-us.opendev.org/ | 08:57 |
| hashar | zbr: I am not an infra root but I have done a quick review. Those options can be passed as parameters to the ansible module ( see https://docs.ansible.com/ansible/2.7/modules/synchronize_module.html ) | 09:02 |
| zbr | true, leftovers from when i was testing more advanced options :D | 09:03 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed opendev/base-jobs master: Avoid rsync chgrp errors on promote https://review.opendev.org/739674 | 09:13 |
| AJaeger | kevinz: I cannot help, let's ask infra-root to help with the linaro mirror ^ | 09:16 |
| *** fressi has joined #opendev | 09:25 | |
| frickler | kevinz: infra-root: there's a kernel panic from the afs module on mirror.regionone.linaro-us.opendev.org , I'm trying another reboot, if that doesn't help, we'll need some kernel expert I guess | 09:26 |
| frickler | kevinz: seems to be working better now | 09:44 |
| *** ysandeep|lunch is now known as ysandeep | 09:48 | |
| *** ryohayakawa has quit IRC | 09:53 | |
| *** tosky has quit IRC | 10:00 | |
| *** tosky has joined #opendev | 10:01 | |
| *** hashar has quit IRC | 10:07 | |
| hrw | frickler: thanks | 10:34 |
| *** bhagyashris is now known as bhagyashris|brb | 10:37 | |
| frickler | infra-root: fyi, that's the kernel oops, it looks to me like it might have choked on some corrupted cache file after the previous unclean shutdown. probably difficult to reproduce but maybe worth forwarding to afs developers? http://paste.openstack.org/show/795602/ | 10:45 |
| *** priteau has quit IRC | 11:00 | |
| *** tkajinam has quit IRC | 11:04 | |
| *** bhagyashris|brb is now known as bhagyashris | 11:12 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Drop support for ansible 2.6 https://review.opendev.org/727158 | 11:32 |
| *** ysandeep is now known as ysandeep|afk | 11:41 | |
| *** hashar has joined #opendev | 11:52 | |
| *** priteau has joined #opendev | 11:55 | |
| *** ysandeep|afk is now known as ysandeep | 12:06 | |
| *** priteau has quit IRC | 12:12 | |
| *** priteau has joined #opendev | 12:19 | |
| fungi | just a heads up, i'm around but catching up after a few days off, and today is grocery day plus a number of meetings, so i can help with urgent issues but likely not much else before tomorrow | 13:03 |
| *** sgw1 has joined #opendev | 13:11 | |
| *** rh-jelabarre has joined #opendev | 13:21 | |
| *** wrenchyfrenchy has left #opendev | 13:39 | |
| fungi | okay, stepping out to fetch our grocery order, shouldn't be more than an hour. back soon | 13:44 |
| openstackgerrit | Thierry Carrez proposed openstack/project-config master: Define maintain-github-mirror job https://review.opendev.org/738228 | 13:53 |
| openstackgerrit | Thierry Carrez proposed openstack/project-config master: Define maintain-github-mirror job https://review.opendev.org/738228 | 14:01 |
| kevinz | frickler: thanks a lot :-D | 14:04 |
| *** bhagyashris is now known as bhagyashris|dinn | 14:09 | |
| *** _marc-antoine_ has joined #opendev | 14:27 | |
| openstackgerrit | Slawek Kaplonski proposed opendev/irc-meetings master: Update neutron team meeting time https://review.opendev.org/739780 | 14:40 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed opendev/base-jobs master: Avoid rsync chgrp errors on promote https://review.opendev.org/739674 | 14:41 |
| *** mlavalle has joined #opendev | 14:46 | |
| *** fressi has quit IRC | 14:52 | |
| *** bhagyashris|dinn is now known as bhagyashris | 14:56 | |
| openstackgerrit | Drew Walters proposed openstack/project-config master: Publish Airship governance https://review.opendev.org/739790 | 15:17 |
| openstackgerrit | gugug proposed openstack/diskimage-builder master: Remove glance-registry https://review.opendev.org/739796 | 15:26 |
| openstackgerrit | gugug proposed openstack/diskimage-builder master: Remove glance-registry https://review.opendev.org/739796 | 15:30 |
| *** ysandeep is now known as ysandeep|away | 15:32 | |
| openstackgerrit | gugug proposed openstack/diskimage-builder master: Remove glance-registry https://review.opendev.org/739796 | 15:33 |
| *** dtantsur is now known as dtantsur|brb | 15:49 | |
| zbr | corvus: i discovered with pabelanger that fedora-31 lots ability to create swaps over last night :) affecting all jobs on ansible zuul. | 15:54 |
| zbr | i was trying to identify if same issue happened on opendev too | 15:55 |
| corvus | zbr: there's a "build_node" field in logstash, but i don't see any fedora-31 entries for it | 15:56 |
| corvus | zbr: but i can check the zuul scheduler logs | 15:56 |
| *** ykarel is now known as ykarel|away | 15:57 | |
| corvus | zbr: here are the builds from today: http://paste.openstack.org/show/795628/ | 15:58 |
| corvus | zbr: and yesterday: http://paste.openstack.org/show/795629/ | 15:59 |
| corvus | zbr: looks like it's just a couple of jobs, so you can probably query the zuul dashboard for those jobs and find the builds (rather than going to each build id individually) | 16:00 |
| *** marios has quit IRC | 16:02 | |
| *** ysandeep|away is now known as ysandeep | 16:21 | |
| *** icarusfactor has joined #opendev | 16:29 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Allow deleting workspace after running terraform destroy https://review.opendev.org/738771 | 16:29 |
| *** factor has quit IRC | 16:32 | |
| *** factor has joined #opendev | 16:32 | |
| *** icarusfactor has quit IRC | 16:34 | |
| *** factor has quit IRC | 16:34 | |
| *** chandankumar is now known as raukadah | 16:38 | |
| *** dtantsur|brb is now known as dtantsur | 16:41 | |
| *** _marc-antoine_ has quit IRC | 17:05 | |
| *** dtantsur is now known as dtantsur|afk | 17:38 | |
| *** hashar has quit IRC | 17:47 | |
| *** priteau has quit IRC | 17:52 | |
| hrw | morning | 18:09 |
| hrw | is there a way to add other fonts to etherpad.opendev.org list of fonts? | 18:09 |
| hrw | I can write userscript to do it on client side but prefer to check first is it doable on server side ;D | 18:10 |
| fungi | hrw: possible it can be done with a plugin, if you want to look at the available etherpad plugins | 18:10 |
| fungi | we can add plugins to our image build fairly easily, and hold a test node for more thorough interactive exercisinfg | 18:10 |
| fungi | exercising | 18:10 |
| hrw | fungi: thanks, will check | 18:13 |
| fungi | also possible it's configurable server-side, i haven't explored the etherpad config file that deeply, nor what new features came with the theming enhancements in recent versions | 18:14 |
| hrw | fungi: is config somewhere on opendev.org git? | 18:19 |
| hrw | cause when I look at etherpad plugins list I feel bad | 18:19 |
| fungi | heh, yeah just a sec, it's in opendev/system-config but i'll find it | 18:20 |
| hrw | thanks | 18:21 |
| fungi | hrw: right now we only have the headings plugin added: https://opendev.org/opendev/system-config/src/branch/master/docker/etherpad/Dockerfile#L20-L24 | 18:23 |
| hrw | looked at plugins. will write userscript | 18:31 |
| fungi | heh, okay | 18:37 |
| fungi | i totally understand | 18:38 |
| fungi | some aspects of etherpad are not as configurable as others | 18:38 |
| hrw | yep | 18:39 |
| fungi | out of curiosity, is the change you need to make an accessibility related one? maybe we could help with a pr upstream, and/or some css changes on our deployment | 18:47 |
| ianw | meeting in 2 minutes @ #opendev-meeting | 18:58 |
| *** diablo_rojo has joined #opendev | 19:00 | |
| *** hashar has joined #opendev | 19:34 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Don't install the track-upstream cron on review-test https://review.opendev.org/739840 | 19:40 |
| openstackgerrit | Michael Johnson proposed openstack/diskimage-builder master: Fix DIB scripts python version https://review.opendev.org/739844 | 19:53 |
| ianw | huh, arm64 job on 739840 failed with "fatal: unable to access 'https://github.com/infraly/k8s-on-openstack/': GnuTLS recv error (-54): Error in the pull function." ... transient error i hope | 20:07 |
| fungi | that seems arbitrary enough at least | 20:08 |
| fungi | or maybe the request had to pass through kentucky | 20:08 |
| corvus | hrm, starting to smell like wildfire here... | 20:19 |
| ianw | were the iptables drops that are currently on gitea-lb persisted anywhere? | 20:34 |
| ianw | ok there is a ~clarkb/china_unicom_ranges | 20:37 |
| clarkb | ianw I did a for loop over the contents of that file. Check my user history gor the exact command | 20:38 |
| ianw | yeah, i did :) just thinking about how we can test now | 20:39 |
| fungi | well, there are many options for testing | 20:44 |
| fungi | we could tcpdump to see which of those cidrs has a lot of requests coming from it and then just delete the rule covering it | 20:44 |
| fungi | but in reality i don't think we need to fiddle with the current ruleset to test | 20:45 |
| ianw | yeah, just switch one of the backends in haproxy to 3081 | 20:45 |
| fungi | there are similar requests coming from other netblocks (last we checked) | 20:45 |
| fungi | which is why we can still see a significant open connections count in cacti | 20:45 |
| fungi | and elevated (but now not dire) processor utilization across the backends | 20:46 |
| ianw | once it deploys i'll poke at a manual redirect and see what happens, then propose a config file update | 20:47 |
| openstackgerrit | Merged opendev/system-config master: gitea: Add reverse proxy option https://review.opendev.org/738721 | 21:07 |
| openstackgerrit | Merged opendev/system-config master: gitea: crawler UA reject rules https://review.opendev.org/738725 | 21:10 |
| openstackgerrit | Merged opendev/system-config master: graphite: statsd timer settings https://review.opendev.org/739665 | 21:10 |
| *** tobiash has quit IRC | 22:12 | |
| *** DSpider has quit IRC | 22:22 | |
| *** avass has quit IRC | 22:25 | |
| *** calcmandan has quit IRC | 22:42 | |
| *** calcmandan has joined #opendev | 22:42 | |
| *** jentoio has quit IRC | 22:43 | |
| *** jentoio has joined #opendev | 22:45 | |
| *** seongsoocho has quit IRC | 22:47 | |
| *** gmann has quit IRC | 22:47 | |
| *** mwhahaha has quit IRC | 22:50 | |
| *** mnaser has quit IRC | 22:50 | |
| *** jbryce has quit IRC | 22:50 | |
| *** knikolla has quit IRC | 22:50 | |
| *** johnsom has quit IRC | 22:51 | |
| *** jrosser has quit IRC | 22:51 | |
| *** hillpd has quit IRC | 22:51 | |
| *** aannuusshhkkaa has quit IRC | 22:51 | |
| *** rpittau has quit IRC | 22:51 | |
| *** wendallkaters has quit IRC | 22:52 | |
| *** rm_work has quit IRC | 22:52 | |
| *** rajinir_ has quit IRC | 22:52 | |
| *** jentoio has quit IRC | 22:52 | |
| *** Open10K8S has quit IRC | 22:53 | |
| *** lseki has quit IRC | 22:53 | |
| *** zbr has quit IRC | 22:53 | |
| *** diablo_rojo_phon has quit IRC | 22:53 | |
| *** donnyd has quit IRC | 22:53 | |
| *** tkajinam has joined #opendev | 22:54 | |
| *** mlavalle has quit IRC | 22:59 | |
| *** tosky has quit IRC | 23:01 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: gitea: install proxy https://review.opendev.org/739865 | 23:14 |
| *** hashar has quit IRC | 23:15 | |
| gouthamr | hello o/ anyone aware of any problems with rax-iad? We have a test job in openstack/manila that fails only on that cloud with likely an OOM issue: rabbitmq halts, systemd services are restarted (i notice a reboot in the logs when this happens) | 23:37 |
| fungi | i'd be surprised if it happens in rax-iad but not rax-dfw or rax-ord | 23:40 |
| fungi | any idea how long it's been going on? | 23:40 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!