Tuesday, 2021-01-19

*** guillaumec has quit IRC05:40
*** guillaumec has joined #opendev-meeting05:44
*** sboyron has joined #opendev-meeting06:33
*** hashar has joined #opendev-meeting08:22
*** sboyron has quit IRC09:06
*** sboyron_ has joined #opendev-meeting09:06
*** sboyron has joined #opendev-meeting09:43
*** sboyron_ has quit IRC09:43
*** hashar is now known as hasharOut09:59
*** sboyron_ has joined #opendev-meeting10:05
*** sboyron has quit IRC10:08
*** hasharOut is now known as hashar12:45
*** hashar is now known as hasharKids15:28
*** diablo_rojo has joined #opendev-meeting16:01
*** zbr3 has joined #opendev-meeting17:18
*** zbr has quit IRC17:20
*** zbr3 is now known as zbr17:20
*** hamalq has joined #opendev-meeting17:40
*** hasharKids has quit IRC18:34
clarkbhello anyone else here for the meeting?19:00
clarkbwe will get started shortly19:00
fungiyeah, sure, why not?19:00
clarkbwell it is a nice day outside >_>19:00
ianwo/19:00
corvusit's a *weird* day outside19:00
fungihere it's a nice day inside19:00
clarkb#startmeeting infra19:01
openstackMeeting started Tue Jan 19 19:01:09 2021 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link http://lists.opendev.org/pipermail/service-discuss/2021-January/000166.html Our Agenda19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
zbro/19:01
clarkbI didn't have any announcements. Continuing on19:01
clarkb#topic Actions from last meeting19:02
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:02
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2021/infra.2021-01-12-19.01.txt minutes from last meeting19:02
clarkbiane had an action to check on wiki backups after the borg migration19:02
clarkbianw ^19:02
clarkbtyping is hard sometimes19:02
clarkbianw: has that happened or should we tack it back on (I know last week was crazy)19:03
ianwi don't believe it is backing up to borg, but i haven't fixed it, so yeah, leave it on19:03
clarkb#action ianw confirm wiki is still backed up after bup to borg migration19:03
clarkband now we'll jump right into mnaser's topic to avoid a conflict19:04
clarkb#topic General topics19:04
*** openstack changes topic to "General topics (Meeting topic: infra)"19:04
clarkb#topic Discuss infra-core (on behalf of OpenStack)19:04
*** openstack changes topic to "Discuss infra-core (on behalf of OpenStack) (Meeting topic: infra)"19:04
clarkbwe'll resume regularly scheduled agenda planning after this item19:04
mnaserthanks.  so it was recently brought up that andreas who's done a ton of infra reviews will not be able to help out as much19:04
mnaserand the biggest load of project-config changes right now happen to openstack/project-config (i assume) and the opendev team is already a small one so .. wondering if it's an issue right now?  the reviews that come in seems to be very little right now19:05
mnasermostly: how do we make sure we stay on top of this, i guess19:05
clarkbactive infra-core is roughly myself, fungi, ianw, corvus, and mnaser I think19:05
clarkband by active I mean active in the community but not necessarily with reviews on that particular project19:05
fungier, project-config-core?19:05
clarkbfungi: ya sorry19:06
clarkbconfig-core is the general term we use for highlights iirc19:06
corvusandreas did a lot of things, but in addition to reviewing project-config changes that added or updated projects, he also reviewed zuul-jobs changes, especially ones that affected openstack's use of zuul.19:06
fungiand translations and docs tooling19:06
corvusso there's likely to be some "opendev maintenance" work shortfall as well as openstack tact work shortfall19:07
corvusyep those too19:07
zbrmy concern on reviews was around small projects which were nurtured by infra historically, getting a review on these requires a lot of effort, as infra-cores do always have more urgent priorities. what can we do to improve?19:07
fungibut yeah, one thing which came up during the openstack tc meeting was that if we have some volunteers from the openstack community with bandwidth to review project-config changes and who can show some understanding of the material in there, we could likely expand the core review team for that fairly quickly19:07
clarkbzbr: I think that is a separate but related concern (it has its own agenda item)19:07
clarkbnot sure if we want to combine them due to the bit of overlap (basically lack of resources overall being the overlap)_19:08
corvusfungi: ++19:08
corvusclarkb: separate i think19:08
fungimnaser: also it's not just openstack/project-config, as corvus points out there's zuul/zuul-jobs and also openstack/openstack-zuul-jobs (and to a lesser extent, opendev/base-jobs)19:08
mnaseryes, i believe the concern here isn't around tooling but more around the project config reviews (tooling is another valid, but seperate issue)19:09
fungiwell, the bulk of the reviewing is job content, job definitions, job roles... and some project creation changes19:10
clarkbI agree we could probably bootstrap a few interested individuals quickly. Maybe start by identifying who they are and ask them to do reviews with +1/-1 and they can reach out to us with questions or concerns with changes?19:10
clarkbbasically get them involved then once we've established comms and some trust we can bump up to +2?19:10
corvusmnaser: i also think the revies he did on project-config are a little unique -- a *lot* of the technical stuff of that is actually automated or rigidly documented.  most of what he did there was, i feel, a form of hand-holding folks through the process.19:10
clarkbcorvus: yes that was definitely a big part of what ajaeger was doing19:10
corvusthat is in contrast to all the other repos which involve things like "knowing what tox is"19:11
fungihelping folks who have submitted config changes understand the job failures which arise from those19:11
mnasercorvus: i agree, it was a lot of "enforcing" the process19:11
fungiand being aware of openstack policy (pti, guidelines in the ptoject teams guide, release policies, et cetera)19:11
fungiand processes outlined in the infra manual19:12
corvusclarkb: i also agree that it's the easiest team to expand19:12
corvusand that the knowledge needed is a mix of community and tech.19:13
fungii've tried to step up my reviewing of project-config and related repos in recent weeks, but that's also eating into my available time for other opendev work19:13
clarkbmaybe the thing to do then is write down these things that we've identified ajaeger was doing in an email and request volunteers from the openstack community. Then once we've identified people interested do our best to help them do reviews and gain confidence in the processes and tech?19:14
mnasersounds good to me clarkb.19:14
ianwthat's also not a bad idea to try and find things we can reconsider/rework in 2020 that we've haven't looked at in a long time because ajaeger was doing such a good job keeping things humming along19:15
ianw2021 even19:15
fungii'm not so keen to go back and try 2020 again19:15
clarkbianw: ++ perspective from some new volunteers might help us identify those areas too19:15
clarkbanyone want to volunteer to write that email? I can probably do it though unlikely today19:16
clarkbbut probably this week19:16
fungiif one of us drafts it in an etherpad i suppose others can help flesh it out too19:16
clarkb++ to an etherpad19:17
clarkbI'll go ahead and put myself down for it and if someone beats me to it they can just let the group know19:17
fungias tact sig chair, i'll consider it my duty19:17
fungiyou can look it over once i have a go at it19:17
clarkbk19:17
mnaseri'll bring up this discussion in the tc meeting on thursday too19:17
*** gmann has joined #opendev-meeting19:18
corvusmnaser: thanks for being on top of this :)19:18
clarkb#action fungi Draft email describing ajaeger's tasks and asking for new volunteers to help with those tasks. Also open possibility for improving processes and tools19:18
clarkbAnything else on this subject or should we move on?19:18
mnaserno problem :) i'll try to look at project-config changes too on my side as well19:18
mnaserthat's it from me for this19:18
fungi#link https://etherpad.opendev.org/p/tact-sig-2021-rfh OpenStack TaCT SIG 2021 Request for Help19:18
gmanno/19:18
fungii'll write something there19:18
clarkbfungi: thank you!19:18
clarkb#topic Priority Efforts19:18
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:18
clarkb#topic OpenDev19:19
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:19
clarkbA reminder that nominations for the Service Coordinator position are open.19:19
clarkb#link http://lists.opendev.org/pipermail/service-discuss/2021-January/000161.html19:19
clarkbPlease let us know if you are interested19:19
clarkbI was going to bring up the zuul status change list, but it looks like ianw got that landed and applied19:19
clarkbianw: anything else to bring up on the subject of the zuul status work in gerrit?19:20
ianwnot, good team effort with clarkb getting it across the line, thanks :)19:20
clarkbit looks great in gerrit too, thank you for writing thatp lugin19:20
corvusit looks great, thanks!19:20
clarkbZuul's gerrit WIP change support should be deployed as of this last weekend19:21
fungidiscussion on the openstack-discuss ml thread about it suggested reviewing the zuul status plugin as a potential next candidate19:21
clarkbfungi: another candidate to deploy in our installation?19:21
fungiright, the live test progress display in the gerrit webui change view19:21
clarkbgotcha19:21
ianwfungi: that one might be a bit harder to get screen shots of in testing, but i think the framework for including it is fairly straight forward now19:22
clarkbfor WIP support has anyone tested that yet?19:22
clarkbI believe we want to confirm that zuul will not enqueue a wip change to the gate19:22
corvusianw: is non-voting handled?19:23
corvus(by handled i mean displayed)19:23
fungii did just approve zbr's gerritbot change today to start ignoring wip changes, and start announcing them if they transition out of wip (there's a separate event for that now)19:23
ianwcorvus: in the in-progress plugin, or our summary-status?19:23
corvusianw: summary -- and yes, i see it now19:24
corvusshows up on the right with the rest of the time comment19:24
corvusfungi: oh um19:24
corvusfungi: can we not do that?19:24
corvusi really want to see wip changes announced19:24
fungiwe can revert it19:24
fungiit was up for review for a while19:24
ianwcorvus: yeah, i did think looking at that yesterday on a massive cinder change it could a) sort the NV together and b) show them somehow differently19:25
corvusfungi: well, i may be getting ahead of things here, but i don't want anything about gerritbot to change :)19:25
fungisure, i'm pushing up a revert now and then we can discuss further19:25
clarkbfungi: I think you tested if zuul ignored WIP chagnes in the gate previously. do you know if there is a good change to test that on?19:26
corvusand i realize we may be getting into personal preferences, but i have found it really useful when, say, we're having some convo in irc, and someone pushes up a wip patch, to go look at it right then.19:26
clarkbI guess we can maybe WIP that revert and approve it and see what zuul does then unwip it and merge it?19:26
clarkbthat should sufficiently exercise the new zuul behavior19:27
fungii've pushed up a revert and a revert of the revert, approving the former now and we can discuss it in the latter19:27
clarkbok19:27
clarkbThe last major opendev item I wanted to bring up is gitea 1.13.1 upgrade. I've put this off as fires have crept up and needed attention. I think we're largely stabilized and I hope to approve the upgrade tomorrow (when I'll be able to monitor it)19:28
clarkb#link https://review.opendev.org/c/opendev/system-config/+/76922619:28
zbrwip change on gerritbot was open for 2 months19:28
clarkbthe upgrade is reasonable well tested so I don't expect issues but the upgrade is fairly large in terms of new features and stuff in gitea19:28
clarkbAny other opendev items or should we continue on?19:30
fungiclarkb: i think zbr may have had a wip change which was approved, and we saw zuul attempt to merge it and fail leaving a verified +2 behind... i don't recall which change it was off the top of my head though19:30
zbrcorvus: in fact regarding WIP on gerritbot, i think that revert is not the best idea as it would be easy to enable announcement with two lines of code.19:30
zbryep, we experimented it and found that zuul was trying to merge as wip change and gerrit was not letting it do it.19:31
corvusfungi: to be clear, are you saying that happened *after* we restarted zuul with the wip change support, or before?19:32
fungicorvus: before19:32
fungibefore the wip support change was written19:32
corvusokay, so that's a description of the behavior to be on the lookout for19:32
fungiright19:32
fungizbr: on the gerritbot change, we can edit the revert of the revert to perhaps make it configurable by channel19:33
corvuszbr: i certainly don't oppose the option for gerritbot users to disable wip notices19:33
zbrdo we really need it configurable?19:34
fungiif some channels want to see wip changes announced and others don't19:34
corvusi'm just volunteering that i'm happy with the current workflow, don't think it's spammy, and would miss functionality if it changed.  i would probably run my own gerritbot or something like it to make up for the loss of functionality.19:34
clarkblets move on to ensure we get to the other items, but can get back to this discussion if we have time at the end of the meeting19:35
clarkb#topic Update Config Management19:35
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:35
zbrcurrent behavior is that is even missing to anounce a change getting out of WIP, which is quite important IMHO19:35
clarkbianw has started work to ansible our afs servers19:35
zbrbecause that is the moment that marks "that is ready for review"19:35
clarkb#link https://review.opendev.org/c/opendev/system-config/+/77126819:35
clarkbthis is related to the next topic whcih is openafs cluster updates19:36
clarkbwhy don't we just jump ahead there so that meeting notes are a bit cleaner19:36
fungithere's been a flurry of openafs configuration management improvement yeah19:36
clarkb#topic General Topics19:36
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:36
clarkb#topic OpenAFS Cluster Status19:36
*** openstack changes topic to "OpenAFS Cluster Status (Meeting topic: infra)"19:36
ianwi can give a quick summary19:36
clarkbianw: please do19:36
fungiprobably better not to burn meeting time getting into the history of what's transpired, but current status would be great i think19:37
ianwyes, current summary sorry :)19:37
ianwso afs01/02.dfw and afs01.ord are now all running afs 1.8 from our ppa19:37
ianwwe should have the changes in system-config to manage them via ansible, but when i left yesterday the CD jobs were blocked19:37
ianw(think that's fixed now)19:37
ianwso i will chase up on that19:37
fungias are all clients (at least installed, many will run it on next reboot)19:37
ianwtoday, i'd like to upgrade afsdb01/02 in turn to 1.8, then we'll be 100% 1.819:38
ianwi have the system-config change to manage them with ansible too, and remove the puppet bits19:38
clarkbianw: that would be great19:38
clarkbthis way we'll be able to respond to future bugs more directly19:38
fungiand jobs are using either patched 1.8.6 or 1.8.7 which has the patches officially applied19:38
clarkband as you mentioend it likely simplifies upgrading the servers19:38
ianwthen i'd like to upgrade the hosts one-by-one to focal, in place.  i don't think it's worth making new servers, especially for the db which are annoying to change ip addresses19:38
clarkbianw: yup we upgraded them in place previously so I expect that is still our best option19:39
ianwi had planned to start that with afs01.ord and see how it goes.  this should be zero downtime as we can now do it one-by-one19:39
ianwthat's about it19:39
fungiwe're holding off the focal upgrades though until we have fully recovered from the storage outage on afs02.dfw19:40
fungiwhich is still in progress19:40
clarkbthere were two things I wanted to call out the first is debian buster test nodes will use our bionic ppa package until debain fixes their distro packages19:40
ianwoh yes, sorry, after we're back in a regular release schedle19:40
clarkbsecond is the openafs-client role seems to not try to install our ppa packages on xenial19:40
fungiby my estimate we've got full releases of roughly half the mirror volumes left to complete, everything else is back on track19:40
clarkbevidence from the zuul executors which are our only xenial openafs-clients shows taht we seem to be using the ppa there anyway19:41
fungiso it's progressing faster than i'd expected anyway19:41
clarkbmaybe someone else can look at that and we can decide if we should remove the xenial exclusion condition from the ansible role?19:41
ianwclarkb: huh, wasn't aware of that.  we can chase up after meeting, definitely sounds like a bug19:41
fungione theory on the executors is that puppet installed the ppa config, and so it's lingered there after we switched them to ansible19:41
clarkbianw: cool I'll get links together for that after the meeting19:41
corvusfungi: is that true for ze12?19:42
fungicorvus: was it built after the ansibilification? if so then i gues sthat theory can be discounted19:42
fungibut even ze12 has a lot of ppa config crufy in place19:42
corvusi don't know, but worth looking into if we need to verify/falsify that19:42
clarkbit does have the ppa installed and did update the package at least19:42
fungier, cruft19:42
corvusbut also maybe we don't care about the past and only the future in this case :)19:43
fungion ze12 there are three different files all adding that same ppa19:43
ianwall of the afs servers also had some old cruft from when we tried something that added all the ubuntu repos twice19:44
clarkbso maybe thats a larger general cleanup (the extra ppa configs)19:44
clarkbwe can follow up on that after the meeting just wanted to make people aware19:44
clarkb#topic Picking up steam on Puppet -> Ansible rewrites19:45
*** openstack changes topic to "Picking up steam on Puppet -> Ansible rewrites (Meeting topic: infra)"19:45
fungithe ppa addition files on ze12 all date from may 202019:45
fungioops, sorry, will discuss after the meeting19:45
clarkbno worries just want to get through the agenda :)19:45
clarkbone thing I noticed when we looekd at the openafs stuff last week is we have a number of servers still running xenial and for many of them not called zuul or review they are still running puppet19:45
clarkbwe did not solve puppet on bionic or newer which means that one of the things we need to keep in mind is xenial -> newer upgrades will want ansible (and maybe docker) on the config managment side19:46
clarkbThis is an early call out that we need to start looking at this (work like ianw's openafs role is great!) so that we can also do server upgrades19:46
fungithere was renewed interest expressed in last week's storyboard meeting about finishing the docker deployment automation for sb at least19:47
clarkbI'm hoping that if things remain calm I can start putting together and audit so that we have a todo list we can pick things off of and annotated with interest like ^19:47
clarkbthere is also interest from refstack folks19:47
fungias that's blocking us from deploying a number of other fixes to production at this point19:47
ianw++; things like docker for storyboard seem like something that is good for us, but good for everyone else too19:47
clarkb#action clarkb put together audit/todo list for ansiblification and server upgrades19:47
fungiin the case of storyboard, we already have docker images publishing on each new change merged (thanks mordred!)19:48
clarkbI've got a few things I need to get done in my backlog but thats near the top of the next items to pull off19:48
clarkb#topic two-review rule impact on low-activity projects19:48
*** openstack changes topic to "two-review rule impact on low-activity projects (Meeting topic: infra)"19:48
clarkbzbr added this to the agenda and this is the topic I was referring to during mnaser's topic as being related but separate19:49
clarkbwe've got a number of low activity projects like git review and bindep and gerritbot etc that don't get regular reviews19:49
fungifwiw, we've already said it's okay to single-core approve stuff in these as long as we observe a quick-revert policy should concerns be expressed by another core reviewer later (as we've just done with gerritbot, for example)19:49
clarkba consequence of that is that it is hard to get two +2s to land changes19:49
clarkbfungi: yes, I think the bigger issue is more a lack of time for any reviews on those change due to higher priority issues/fires19:50
clarkbat least for me it is rare that I'm able to make a chunk of time for reviews in those projects19:50
fungiyep, i've been trying to strafe some of them as i get time. i should probably convert a number of my solo +2's to approvals19:50
corvusi think we've always been flexible with this.  i think we can continue to do so.  i think that procedural/minor technical changes are especially good candidates for sigle-core reviews.  i think substantial changes benefit from more reviews.  again, using the gerritbot change as an example: more reviewers might have surfaced an alternate view.19:51
fungifor example, someone e-mailed an archlinux fix for bindep to openstack-discuss, i proposed it to gerrit for them in https://review.opendev.org/771108 and waffled on whether i should just approve it too19:51
clarkbsome things are pretty tightly coupled to our service (git review and gerritbot for example) so I'd be wary of putting them up to adoption.19:52
corvusbut i'd also say that some of the low activity projects are low activity for a reason: in my view some of them are pretty close to "done" and i view them as being in maintenance mode.19:52
clarkbprojects like bindep are much less so (and we really only got bindep beacuse no one elsewanted ti it seems like)19:52
corvusi don't want git-review, gerritbot, and gear to change.19:52
fungigerritlib is also in that space a bit19:53
fungisince jeepyb depends on it fairly heavily19:53
fungi(as does gerritbot)19:53
clarkbI know some projects have relied heavily on priority lists (either generated by gerrit and a review priority category) or manually curated in things like an etherpad19:53
fungiwe do get soe good bug fixes for those from time to time, but also they tend to attract feature additions which are harder for me to justify as they almost always imply some scope creep19:54
clarkbwe could try something like that to not just surface changes but more publicly communicate what we think is important right now and maybe people will unerstand why other changes are being passed over19:54
zbrsomeone said a sw is done only when is dead (nobody is using it) :p19:54
clarkbI think the big win with something like ^ would be more to expose "this is important and keeping us from other work" more so than helping us get the other work done quicker19:54
corvuszbr: i recognize that's a common view of folks who get paid to write software, but i don't entirely agree with it19:55
fungii use a ton of 'nix utilities day in and day out which haven't added a new feature in countless years19:55
fungidoesn't make them less useful19:55
zbrbe aware that not reviewing patches from others does not help us get more people to join our effort19:55
corvusi think if people want to join our effort, then focusing work on where our priorities are would be best19:56
corvusand we have quite a few ways of conveying that19:56
zbrin fact is quite an effective deterrent, and creates a vicious cycle19:56
clarkbcorvus: ya thats why I'm wondering if pointing at the priorities more clearly will help19:56
clarkb"this is where you can help us" also "this is why we are busy"19:56
fungizbr: yep, in my case i think i struggle to break the bad news to people that i'm not interested in approving their pet feature addition. if i harden my heart a bit maybe i can find more time to approve bug fixes in those projects and not worry about spending as much time explaining to contributors why their proposal doesn't fit with the project19:57
corvusclarkb: yeah, it looks like we need a link to infra-specs in https://docs.opendev.org/opendev/system-config/latest/project.html#priority-efforts19:57
fungisometimes it's easier to just put off leaving a review on something i know i'm not going to approve, because i don't have time to write an essay explaining why i'm rejecting it19:57
clarkbcorvus: ya I think specs as well as current fires19:58
zbrfungi: yep, that is tricky, but is still better in long term to cut it short (but politely)19:58
clarkbat least the last few months have felt more like fire fighting than proper maintenance work. I'll think on how to expose that19:58
clarkbmaybe just start with a simple etherpad then look into automating it osmehow19:58
corvushttps://docs.opendev.org/opendev/infra-specs/latest/#priority-efforts19:58
fungizbr: i totally agree19:58
fungisomething i need to improve at personally19:58
corvusclarkb: i'd rather just see that kept up to date19:59
corvusi don't think we need more places to share our priorities19:59
clarkbcorvus: I agree keeping it up to date is a good thing, but I'm not sure it captures "here are the five things we're dealing with to unbreak afs"19:59
clarkbwhich lately seems to be the type of thing consuming my time19:59
zbri would be very upset if spend 10 days on making something work, and failing to deliver it than just a couple of hours and being notified politely that that feature is outside the scope of the project.19:59
ianwi also do not mind if people have a change that seems stuck calling it out as a meeting topic.  i have found that very useful over the years, and i think it's reasonable to expect some outcome from calling a specific change out in a topic20:00
clarkb(as a time check we are at the end of our scheduled hour)20:00
corvusclarkb: ianw has tried to keep storyboard up to date for some efforts like that20:00
clarkbcorvus: good point, so maybe its tagging in storyboard and makign a dashboard out of that20:01
clarkband then linking to that in the docs next to the long term maintenance priority efforts20:01
clarkband being better about filing bugs and tagging them appropriately20:01
clarkb(I know I'm really bad at that bit myself)20:01
corvusyeah, if you want to go down that road, i think that may fit the bill20:01
clarkbI like that idea, I'll see if I can make that work reasonably well20:02
clarkbanother thing to keep in mind on this topic is that we are all human and trying our best to keep up with what at times is a flood of inputs. I doubt anything we do will ever make this problem go away20:02
clarkbwhich means we probably need to both set expectations appropriately but also figure out methods for doing better20:03
corvusthe only thing that has even remotely made a dent is having the ability to hire more people to do more work.20:03
clarkbyup20:03
corvusand that was just less of a backlog than now.20:04
clarkbI'll leave this open for another minute or two if there are any last thoughts20:05
clarkbBut feel free to continue any conversations had in this meeting in #opdnev20:05
clarkb* #opdnev20:05
clarkbugh I cannot type today. #opendev20:05
fungisomething like that20:05
fungithanks clarkb!20:05
zbrthank!20:06
clarkbthanks everyone!20:07
clarkb#endmeeting20:07
*** openstack changes topic to "Incident management and meetings for the OpenDev sysadmins; normal discussions are in #opendev"20:07
openstackMeeting ended Tue Jan 19 20:07:47 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:07
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2021/infra.2021-01-19-19.01.html20:07
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2021/infra.2021-01-19-19.01.txt20:07
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2021/infra.2021-01-19-19.01.log.html20:07
*** zbr5 has joined #opendev-meeting20:15
*** zbr has quit IRC20:16
*** zbr5 is now known as zbr20:16
*** sboyron_ has quit IRC21:26

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!