19:02:26 #startmeeting infra 19:02:26 Meeting started Tue Apr 11 19:02:26 2023 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:02:26 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:02:26 The meeting name has been set to 'infra' 19:02:44 \o 19:02:48 As mentioned yesterday I didn't expect this to be well attended but did want to write down some of what has happened recently 19:02:58 #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/LC5RWSADKSROICHN7YZKPOZX4AV55N34/ Our Agenda 19:04:18 #topic Announcements 19:04:22 #link https://meetpad.opendev.org/opendev-contributor-bootstrap-202304 New Contributor Onboarding Wednesday April 12, 2023 at 14:00 UTC 19:04:47 We've had at least two people interested in contributing and I'll have a bootstrapping session on meetpad tomorrow morning (for me) 19:05:03 Others are welcome to join if interested. Definitely not limited to those who have already chimed in 19:05:21 * frickler plans to join 19:05:29 ++ i've mentioned to a few others too 19:05:44 #link https://etherpad.opendev.org/p/opendev-contributor-bootstrap-202304 Add your thoughts and ideas here 19:05:54 thats a link to the etherpad we'll use for ease of use if you have anything to add to the agenda 19:06:12 do we want to order that a bit? looks pretty overwhelming to me 19:07:08 yes, my idea was to go over basics like gerrit and zuul and ensuring everyone knows how to communicate, push changes, join the meeting etc. Then dig into the ideas for contributions. Grouping the contribution ideas by interest is probably a good idea 19:07:17 like select some (<5 ?) most important topics? 19:07:51 the getting started section is fine, but some more structure to the other ideas 19:07:59 yes, sorry that's probably largely my fault dumping a lot of todo things 19:08:24 I hesitate to prioritize on our side as I think that might discourage people who are interested in other things. More like here is some CI related stuff, here is some container related stuff, here is some something else stuff? 19:08:58 or maybe just reorder so that the first things you see are priorities and then the list can go on if you don't find something that interests you to start 19:09:09 I'll have to look at it more closely since it got updated 19:09:34 yes i agree, i think the point is that doing any one thing is a way into contributing, where you'll start to see the bigger picture 19:09:50 ++ 19:10:08 I'll take a look at organizing it a bit more after lunch today 19:10:12 even zuul/zuul-jobs things expose you to a lot of the mechanics of ci, etc. 19:10:53 I do think we've got a good set of tasks that don't require root involvement. And some that do as things progress 19:11:16 i think the question of where we keep these backlog items is a good one 19:11:32 i don't think we have a good answer atm 19:11:36 ianw: ya I was thinking a bit about that after the storyboard question yesterday 19:12:03 I think there are options but nothing is used consistently today. (options include wiki, etherpad, less formal section of the specs repo, etc) 19:12:43 ok lets continue with the other items on the agenda. Looking forward to seeing people tomorrow 19:12:49 #topic Migrating to Quay 19:13:24 At last check all of the outstanding changes to update jobs merged. However, I think corvus may still have one to implement the intermediate registry promotion path that isn't quite ready? 19:13:40 yep that's on my list 19:13:48 I think ianw and corvus were also working on testing the updates that had landed using zuul/zuul-client. 19:14:01 corvus: ianw: is there anything the rest of us can do at this point to help? 19:14:49 that does work now with zuul-client, so that path of pushing and re-tagging via the api is ok 19:15:03 also, just a friendly reminder from the last topic... there's some zuul stuff on there which is great, but keep in mind that zuul is an independent project and opendev's task list is kind of a separate thing :) 19:15:54 yes, I see those as tasks that would benefit opendev and we would work on anyway 19:16:05 but that would be contribution to zuul that aids opendev not direct contribution to opendev 19:16:17 and yep, zuul-client is looking good with that container build path 19:16:22 so i think the intermediate registry thing is the next step; i should have time for that this week. 19:17:04 clarkb: yep. 95% of what i see there matches that. a few of those are... kind of zuul-project level work. 19:17:36 excellent re intermediate registry stuff. 19:17:41 Let me know if I can help with reviews etc 19:17:48 i don't think we need to get into specifics, but if we're indoctrinating new contributors, it might be good to highlight the separation. 19:17:55 corvus: can do 19:18:30 Also worth noting that April 14 (Friday) was the original you can't push to these orgs day on docker hub. They have said they will not do that anymore but we should keep our eyes open for anything unexpected 19:19:11 #topic Bastion Host Updates 19:19:18 #link https://review.opendev.org/c/opendev/system-config/+/879388 Fix rax rdns 19:19:37 This change is related to launching new servers recently and could use a second review if anyone has time 19:19:55 Also ianw you reported the fixes to reinstalling the launch venv have rax volume listing working now? 19:20:29 yeah, it has now updated itself to the latest everything 19:21:04 i think that's probably appropriate for this venv; as it is a bit of a dogfood for communicating with all our various clouds 19:21:38 sorry, i mean to say, it updating itself periodically is probably appropriate .... 19:22:03 yup and we've identified specific versions of things where necessary too 19:22:05 which pip should respect 19:23:24 Anything else bridge related? 19:24:13 not really; the backup stuff still hasn't had a lot of review, but i don't expect movement on that for a bit 19:24:46 everyone has been taking vacation :) 19:24:52 #topic Mailman 3 19:25:09 related to my last comment I don't think there is any movement on this either. I expect fungi should be able to get to it once he returns 19:25:22 #topic Gerrit Upgrade and Project Renames 19:25:52 We upgraded Gerrit and renamed three projects last Thursday/Friday (depending on timezone). As far as I can tell this went well though we did need almost all of our alotted time 19:26:21 One thing Gerrit did with its notedb migration was update all labels to have a copyCondition including changekind:NO_CHANGE 19:26:37 I feel like this was a bit over the top and filed a bug about it. But haven't seen any resposne from upstream on whether or not they think it is a bug 19:27:00 fungi reverted this update in All-Projects but we had updates to all of the labels in project specific configs too that we need to evaluate 19:27:42 Additionally gerrit updated the project specific config files to use hard tabs in the config file that our normalize config script removes. I think we should actually match what gerrit wants as much as possible to keep diffs as small as possible in situations like this 19:27:44 yeah i need to finish that audit, sorry, got distracted 19:27:46 what's wrong with it? (why not leave it?) 19:27:49 #link https://review.opendev.org/c/openstack/project-config/+/879906 Gerrit also normalized indentation of config files we should consider this to be in sync 19:28:20 i still don't understand fully how a series of client events leads to a NO_CHANGE event being matched 19:28:23 corvus: two things. The first is that many of our copyConditions already include changekind:TRIVIAL_REBASE which is a superset of NO_CHANGE so it is confusingly redundant 19:28:56 it=copycondition 19:28:57 corvus: the second is that for Verified and Workflow we explicitly had no copyConditions and I don't think it is appropriate for Gerrit to add any copy conditions in those cases. 19:29:19 got it, thx. 19:29:21 We may decide to add copyConditions to Verified and/or Workflow but that should be done by users not automation 19:29:58 ianw: ya I don't think we need to rush on this as I suspect in projects specific configs it is rare to lack a copycondition 19:30:09 i think it may relate to merge changes 19:30:11 and yeah, that sounds potentially dangerous for project-specific "core approval" types of labels too. :( 19:31:03 is there some kind of "copycondition:never" that we should look for that might prevent this? 19:31:23 there isn't afaik 19:31:48 ... but yeah, i think fully understanding the conditions that lead to NO_CHANGE might help evaluate the impact 19:31:59 ++ 19:32:15 and if we do bulk updates we should consider bundling in a hard tab change along with that 19:32:23 "Matches when a new patch set is uploaded that has the same parent tree, code delta, and commit message as the previous patch set." 19:32:48 i'm not sure how you do that, because i feel like gerrit usually rejects that as "nothing changed"? 19:33:07 ianw: it has and hasn't depending on the version. Maybe it won't reject it today anymore? Might be worth testing that 19:33:40 In any case doing an audit of what changed and understanding the situation when NO_CHANGE matches is the first step then we can plan any updates 19:33:45 ++ 19:33:51 copycondition: "before:1970-01-01" 19:34:16 #link https://groups.google.com/g/repo-discuss/c/kkbyIKkzlAU/m/4t789yerBAAJ new usp query parameters 19:34:20 (^ amusing way to get a copycondition that never matches?) 19:34:46 This is something frickler called out today. Gerrit is adding query strings to change urls to annotate where the link originated from. I've sent email asking them to clarify how this is actually used 19:35:47 As far as positives go the web ui now supports bulk actions (something gertty did first). Comments support markdown syntax. 19:35:57 perhaps used by google's gerrit and they expect everyone else to ignore it / don't really care 19:36:22 Some of the diff annotations are nice too giving you a better indication of what is changing if not just file content 19:36:28 they should do threaded change relationships next :) 19:36:46 ++ 19:37:32 The only other next step I'ev currently got in mind is cleaning up the 3.6 image and adding 3.8 images and upgrade testing when we are ready 19:38:00 Thank you to everyone who helped, test, review, upgrade, fix etc during this process 19:39:04 #topic Upgrading Old Servers 19:39:14 #link https://etherpad.opendev.org/p/opendev-bionic-server-upgrades Notes 19:39:47 static02 should be serving all of the static.o.o and related content now. I used the LE cert config to determine what all needed updating and think I've got that all changed at this point 19:40:35 yesterday I shutdown apache2 on static01 as a sanity check. corvus found that zuul-ci.org was struggling this morning possibly due to stale dns pointing at static01? So I'll leave static01 alone for another day or two before cleaning it up to be sure there aren't any gremlins in the works 19:40:59 One thing to note is that Jammy's openafs-client package is newer than our PPAs which means static02 is using the upstream distro package version 19:41:32 I've also got an etherpad02 in the works which should be deployed now with an empty db. I'll need to sort out an etherpad downtime and db migration and dns update soon (hopefully tomorrow) 19:42:03 please don't put anything you want ot persist in etherpad02's db as the db migration from 01 will dump what is in 02 19:42:23 ianw: nameservers are the other push on this item. Anything new to report for them? 19:42:45 no but soon i hope! 19:43:11 thanks, let me know if I can help. This generally has become relatively high priority for me now that docker doom is cancellled/delayed 19:43:22 #topic AFS volume quotas and utilization 19:43:47 I've been keeping an eye on this and there hasn't been any dramatic change but he trend is up slowly 19:44:03 I know that with osme of the wheel mirror cleanup work ideas we will delete content from AFS 19:44:12 And also that fedora 36 mirroring will go away soon 19:44:33 I think that those are good next steps. But please if you see something going sideways more quickly say something as we might need to intervene in different ways 19:45:36 i also have about 20gb of wheels to remove when i get back to it 19:46:18 yup I think the current status/plan is good I jsut want to keep this on the agenda in case anyone has specific concerns or wants to tackle this in a different way. I'm not hearing anything like that so we can probably move on 19:46:25 #topic Gitea 1.19 19:46:31 #link https://review.opendev.org/c/opendev/system-config/+/877541 Upgrade opendev.org to 1.19.0 19:47:06 This change is ready I think. There is a held node for it too if anyone else wants to look. But it looks like 1.19.1 is going to happen very soon judging by their issue tracker's milestone for 1.19.1 19:47:23 I feel like we have enough going on and there isn't a presssing reason to upgrade quickly so waiting for the bugfix version is a good idea 19:48:06 That said I don't expect the change we land to upgrade to differ much between 1.19.0 and 1.19.1 if you have time to review the existing change that may be a good way to get ahead of things 19:48:54 #topic Quo vadis Storyboard 19:49:15 As expected a number of projects decided to move to launchpad at the PTG. You may see openstack/project-config changes to update their projects.yaml metadata to reflect this 19:49:34 I think we can also continue to encourage projects to work together if they end up building tools to migrate content or similar. 19:50:03 Oh also we can set projects read only in storyboard via admin config stuff in the storyboard UI (and likely API) should we get requests for that 19:50:48 #topic Open Discussion 19:50:51 anything else? 19:51:31 As noted previously my priorities this week are largely static02, etherpad02, and then supporting the work everyone else has going on. 19:51:45 I'm going to attempt to take Friday off as my kids are out of school Thursday and Friday. We'll see how that goes 19:54:44 Sounds like that may be everything. Thank you everyone! We'll be back here same time and location next week. 19:54:47 #endmeeting