19:01:37 <clarkb> #startmeeting infra
19:01:37 <opendevmeet> Meeting started Tue Feb 20 19:01:37 2024 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:37 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:37 <opendevmeet> The meeting name has been set to 'infra'
19:01:44 <clarkb> #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/DHYKPAV5EU5SAIYRYPR6YUHNTZIIKJSD/ Our Agenda
19:01:49 <clarkb> #topic Announcements
19:01:53 <clarkb> Service coordinator nominations open February 6, 2024 - February 20, 2024
19:01:58 <clarkb> #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/TB2OFBIGWZEYC7L4MCYA46EXIX5T47TY/
19:02:10 <clarkb> it is ~19:01 UTC now and nominations end at the end of day today UTC time
19:03:01 <clarkb> I suspect this is a case of people being happy with the status quo and I should make that official?
19:03:08 <clarkb> or do we have any last minute voluinteers?
19:03:46 * frickler needs 10min break after tc meeting ran long, bbl
19:04:37 <clarkb> If no one indicates they will volunteer by the end of the meeting I guess I'll make it official
19:05:09 <clarkb> #topic Server Upgrades
19:05:25 <clarkb> #link https://review.opendev.org/c/opendev/system-config/+/905510 Upgrading meetpad service to jammy
19:05:29 <clarkb> still waiting on this stack.
19:05:45 <clarkb> The change to filter out nodes with insufficient cpu flags did merge though
19:07:50 <clarkb> Don't really have anything else to add here. tonyb I guess let me or others know if we can help with the meetpad stuff at all
19:08:08 <clarkb> and with that we can dive into the prePTG summary. But I'll pause for a few minutes here since frickler may be interested in this
19:08:13 <clarkb> #topic PrePTG Summary
19:08:19 <clarkb> #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/GJFCZJ5QECXYNIIF62RCOT2DDKRGTAC6/ PrePTG Summary
19:08:40 <fungi> thanks for sending that out!
19:09:07 <corvus> i see a mention of hound on the agenda
19:09:26 <corvus> i think that's basically a completely stateless server?  and a redeploy from scratch with new os/versions should be fine?
19:09:59 <clarkb> corvus: oh yup I think I need to prune that as we did end up resolving that
19:10:02 <fungi> i vaguely recall discussing that recently, and such was the determination
19:10:07 <clarkb> corvus: the tldr is yes that server can just be replaced without issue
19:10:21 <corvus> then yes i agree :)
19:10:34 <corvus> (i probably just didn't say that last time cause someone else did :)
19:10:52 <clarkb> We had our Pre PTG event last week and I think it went well and was useful.
19:11:17 <clarkb> If you disagree on that or have feedback to make it more useful that would be useful info if we plan another. Feel free to PM me that info if you prefer
19:11:53 <clarkb> One thing I wanted to call out is that we seemed to have general agreement on review policies and practices which is good. I'm glad there were no surprises there
19:12:16 <clarkb> tl;dr is that reviewers should use judgement when single core approving changes to ensure they are low impact or safe (based on testing etc)
19:12:36 <clarkb> and that we should continue to do our best to approve things when we acn watch them land and deploy. Holding off on approvals is appropriate if you can't do that
19:13:22 <corvus> as a part-time contributor, i found it very useful to ensure everyone was up to speed on what's going on, who's working on what, what could use attention, sharing what we're interested and working on, other expectations, etc.  i didn't leave with a lot of work assigned to me (some, but not a huge amount), but nonetheless found the whole time useful.
19:13:44 <corvus> and i think the summary reflects that
19:14:30 <clarkb> that is good to hear. It wasn't a ton of effort to put together but it was non zero effort so I'm happy people foudn it useful
19:14:59 <clarkb> I did identify three topics I wanted to followup on here in our meeting, but afterwards we can dive into any others if there is interest
19:15:07 <fungi> i found it useful for talking through some stuff that i may have pushed to the back burner and forgotten about
19:15:07 <clarkb> #topic MariaDB Upgrades
19:15:36 <clarkb> fungi: agreed I found it useful to have a refresher and also sitrep on whether or not stuff ahs changed since we lst considered them
19:15:50 <clarkb> Paste, Etherpad, Gitea, Gerrit, Refstack, and Mailman databases could use upgrades.
19:16:11 <clarkb> During the preptg we agreed that we would go ahead and try this with the container image's MARIADB_AUTO_UPGRADE flag
19:16:23 <clarkb> I'ev since held a paste node to test this, but haven't done any testing yet.
19:16:47 <clarkb> My rough plan there is to add some pastes of varying size to the held node, then manually attempt an upgrade from 10.4 to 10.11 setting that flag
19:17:05 <clarkb> if the data survives then I think we can proceed with upgrading production this way
19:17:30 <fungi> also unicode snowman ;)
19:17:33 <clarkb> ++
19:17:58 <clarkb> look for more info this week and maybe even an upgrade or two
19:18:07 <clarkb> #topic AFS Mirror cleanups
19:18:23 <clarkb> This is sort of the evolution of the prior meetings' afs quotas topic
19:18:50 <clarkb> During the PTG we agreed that OpenSUSE Leap, Debian Buster, CentOS 7 and Ubuntu Xenial could be cleaned up from nodepool and our mirrors
19:19:13 <clarkb> this should free up quite a bit of disk for more modern mirroring that we may want to perform (for example ubuntu has a release coming up and rocky etc have asked about mirroring too)
19:19:46 <clarkb> The other idea (I think from fungi) related to this was that we can process our apache logs on the mirrors to determine which mirror content is most used
19:19:57 <clarkb> then using that data determine if there are things we are hosting that we probably don't need to host any longer
19:20:31 <clarkb> for example usage data on python wheel cache/mirror content would be useful to determien if we can just claen all that stuff up
19:21:08 <clarkb> #topic OpenDev Email Hosting
19:21:24 <clarkb> this is the last one I wanted to bring up today. And the reason is that we didn't reach consensus during the pre ptg
19:21:49 <clarkb> we did agree that having email hosting for opendev.org is something we should address. We can either host it ourselves or pay a reasonable amount for a hosted service to do it for us
19:22:03 <clarkb> for example I personally pay $45/year for fastmail service that I think would be sufficient here
19:22:38 <clarkb> it would be great if everyone can think about that over the next week or two then we can see if we've got a preference for one or the other
19:22:53 <frickler> would the foundation pay that?
19:23:02 <clarkb> frickler: I haven't confirmed that yet, but I suspect they would
19:23:33 <fungi> it would definitely be one of the first things we'll check if interest is leaning that direction
19:23:39 <frickler> we also might want to consider bot accounts like for bounce handling of lists
19:24:07 <frickler> or are those local on lists.o.o anyway?
19:24:07 <clarkb> the way my $45/year fastmail service works is all email to my domain ends up in one account's inbox. But I can filter it from there
19:24:08 <fungi> bot accounts for bounce handling of which lists?
19:24:18 <fungi> for mailman, it's integrated into how mailman works
19:24:36 <clarkb> so we probably wouldn't try to have proper accounts for everything but setup a single inbox that can handle the traffic to opendev.org and organize it appropriately
19:24:38 <fungi> it creates verp addresses on the fly for bounce checking
19:25:14 <fungi> and also dynamic reply addresses for certain kinds of dmarc mitigation features
19:25:23 <fungi> all of that is internal to mailman itself
19:25:29 <frickler> ah, ok
19:26:09 <fungi> but yes, for things like gerrit notification reply addresses, we currently funnel those into a sub-inbox of infra-root
19:26:42 <clarkb> And that was all I had on the agenda
19:26:48 <clarkb> #topic Open Discussion
19:26:57 <fungi> may need something similar for keycloak if we turn on e-mail validation features
19:26:57 <clarkb> Anything else? I'm happy to dig more into the preptg as well
19:28:16 <fungi> 30 minutes to prepare for my next meeting, i guess ;)
19:28:27 <frickler> I didn't get to read all of the pad and mail yet. and sorry again for now showing up, just didn't work out
19:28:36 <frickler> s/now/not/
19:28:57 <clarkb> that said given this was useful I think it may be good to try and do them more often. Maybe twice a year or quarterly?
19:29:14 <clarkb> if we increase the regularity we may not need as much time either which hopefully makes it easier to attend
19:29:21 <clarkb> I'll have to think about this and look at calendars
19:30:17 <clarkb> frickler: when you do read the etherpad and email feel free to reach out with questions etc. Happy to do my best to recount
19:30:36 <frickler> twice a year in between ptgs would sound good I think
19:31:09 <frickler> clarkb: ack, will do
19:31:32 <clarkb> I'll leave the meeting open a few more minutes for any other items. But ya maybe we can get a break between meetings fungi  :)
19:32:00 <fungi> a rare treat ;)
19:34:31 <clarkb> sounds like that is it. Thanks everyone. We'll be back here next week at the same time and location
19:34:42 <clarkb> #endmeeting