19:01:32 #startmeeting infra 19:01:33 Meeting started Tue Sep 17 19:01:32 2019 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:35 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:37 The meeting name has been set to 'infra' 19:01:45 o/ 19:01:53 #link http://lists.openstack.org/pipermail/openstack-infra/2019-September/006479.html Our Agenda 19:03:10 #topic Announcements 19:04:37 * corvus sneaks in late 19:04:46 I'll be traveling starting on Friday. Will likely need someone to chair the meeting next week as I'll be doing zuul stuff on ansiblefest 19:06:12 i'll be there too so not me either 19:06:28 same probably for corvus and mordred? 19:06:56 might also just be that there are so many infra folks at ansiblefest that the need for a meeting next week is reduced 19:07:17 meet in real life instead ;) 19:07:19 ya could be that cancelling makes sense too. I'll leave that up to those that will be around (ianw frickler AJaeger?) 19:07:50 probably cancel if so many away i'd say 19:08:21 agreed 19:08:33 ye 19:08:34 yep 19:09:38 #agreed Cancel the meeting on September 24 19:09:47 that may not be a meetbot directive, oh well 19:10:13 #topic Actions from last meeting 19:10:23 #link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-09-10-19.01.txt Meeting notes from last meeting 19:10:38 I didn't mark this as an action btu I took an action to draft up opendev governance email on etherpad. I have not done this yet 19:13:03 $topic Specs Approval 19:13:08 #topic Specs Approval 19:13:23 quick mention here that we are getting some interest in the zuulv3 for third party ci spec 19:14:44 ianw: rajinir was aksing about it earlier today 19:14:52 interest from operators in seeing someone do it, but so far no interest in volunteering to do it from what i've seen 19:15:02 right 19:15:39 pabelanger mentioned some more windmill documentation, not sure if that happened as yet 19:15:51 mostly just calling it out as we might be able to convert some of the interset into volunteers 19:17:16 #topic Priority Efforts 19:17:19 #topic OpenDev 19:17:39 We renamed kayobe* and ansible-role-cloud-launcher projects in opendev yseterday 19:18:02 all looks fine, Zuul config errors have been fixed 19:18:04 that seemed to go well. The one outstanding item from that is we need to fix the rename playbook to deal with creating new orgs if the renames are to an org that does not exist yet 19:18:06 it went remarkably well, yeah 19:18:30 if this sounds familiar to you it is because we already fixed this once, but then the rewrite of gitea org management into python deleted teh tasks file we were using in the rename playbook 19:18:47 we can probably just undelete that file? 19:19:22 corvus: ya that is one option. It won't be tested by normal new project creation but we don't create new orgs often anyway 19:19:39 and we shouldn't care too much about the performance impact in the case of renaming projects 19:19:52 yeah, takes however long it takes 19:20:12 I can push that chagne up after the meeting 19:20:19 cause i think the project creation module is sort of optimized for what it does; it may be difficult to convince it to just create an org 19:20:35 corvus: ya 19:22:12 the other opendev item which fungi brought up in the infar channel just a little while ago is managing gpg signing keys for openstack releases 19:22:32 yeah, we're coming up on time to generate another one 19:22:44 fungi: since my original thought of have the ptl create the key and sign it and have others attest to it another idea (more effort on our side) is to use one of those signing service tools like what LF uses 19:23:04 signing service tools? 19:23:15 like a third-party signatory? 19:23:30 ya let me try and find the details on that, but basically a server that fronts the keys and signing for you 19:23:40 then you make api requests to sign blobs or generate keys etc 19:23:45 but still needs someone to generate/rotate the keys 19:23:57 I think the tool can do htat for you 19:24:34 so like barbican with a hsm? 19:24:49 i mean, we could probably just stand up a barbican instance in that case 19:24:50 and the added api to sign blobs too 19:25:47 https://docs.releng.linuxfoundation.org/en/latest/index.html is where I expected to find this info but not seeing it there 19:26:15 well, in the short term, i mainly wanted to know if we wanted to make any adjustments to the key data since any major undertaking is going to require more time than the current train cycle openstack release signing key is valid 19:26:18 anyway can figure that out later if it is a path we want to take 19:26:34 fungi: in the short term I think the existing process is fine 19:27:00 like using a different e-mail address than the current infra-root alias 19:27:15 or switcning to per-year instead of tied to openstack release cycles 19:27:24 using a more opendev-oriented key name 19:27:27 that sort of thing 19:27:46 I don't think we should use an opendev oriented key name unless we want opendev to attest all the releases. 19:27:55 yeah, at a minimum, if we keep the approximate current system, i think yearly opendev keys would be the way to go 19:28:09 it's been used to sign non-openstack release artifacts too, which is why i wondered 19:28:10 clarkb: well, aiui, we're attesting that opendev's ci system built this artifact? 19:28:17 corvus: that is a good point 19:28:33 like, zuul packages have detached signatures for this key 19:28:51 fungi: ah I didn't realize that 19:29:30 changing the e-mail address might be tough to work out quickly if we want an opendev.org address, since we don't currently have an mta accepting e-mail for that domain and would want it to double as a valid contact address 19:30:21 but we might be able to get some basic solution in place if that's desired, or could probably easily target the following rotation for something like that 19:30:52 i'd be okay targeting the following rotation 19:31:59 okay, i'll start figuring out what we want to talk through for a working opendev.org contact address in the coming months, in that case 19:32:22 thanks, fungi ! 19:32:33 and we can just stick with the openstack/cycle-based key name for this time through 19:32:56 "OpenStack Infra (Ussuri Cycle)" 19:33:02 fungi: sounds good, thanks 19:33:09 is the uid we'd be using 19:33:14 thanks everyone! 19:34:43 #topic Update Config Management 19:35:03 corvus: do you think you can proxy the updates on gerrit dockerization for mordred ? 19:35:15 aiui bazel updated and we need to update bazel to match? 19:35:45 somewhat yes 19:36:07 bazel had a security issue, so gerrit updated its dependencies to require a fixed version 19:36:38 that was initially a 1.0 rc, but mordred and i found that they also cut a 0.29.1 release with the fix, so we're converging on that 19:37:05 our dockerfiles temporarily overwrite gerrit's dependency to that, while we wait for gerrit to finish merging the patches which update it upstream 19:37:20 that's almost done (or maybe done? haven't checked today) so should be back to normal soon 19:37:38 otherwise, i think mordred is proceeding with the general task of "make gerrit images that work for us in our setup" 19:38:19 [end] 19:38:29 thanks 19:38:40 any other items related to docker or ansible in our config management? 19:39:06 oh look https://review.opendev.org/682469 is green so i think it's all done 19:39:41 (that's a self-testing change of the removal of our local version hack) 19:40:28 yay for removing hacks 19:41:07 #topic General Topics 19:41:15 fungi: any progress on the wiki? 19:41:44 nope 19:42:06 AJaeger: has been pushing on udpates to publishing jobs that I believe are related to the static server cleanups 19:42:49 yes, going to promote jobs will make going to AFS easier IMHO 19:43:06 #link https://etherpad.openstack.org/p/static-services Still need volunteers there 19:43:27 unfortunately I feel like I've been chasing fires around openstack release and other commitments (like the renames and ansiblefest) lately 19:43:39 hopefully I'll be able to sign up for stuff and of next week with all of that behind me/us 19:44:02 ianw: did you make progress on how the layout in AFS should be for the sites? 19:44:30 AJaeger: sorry, haven't looped back on that yet 19:44:45 * AJaeger pushed some first changes and that showed that ianw and myself had different ideas... 19:44:54 happy to adjust if I know how ;) 19:45:56 very little infra-root help is needed for this, right? 19:46:21 so we could be getting help from the maintainers of the relevant governace, security, service-types, releases projects, right? 19:46:24 I think we may need to create an new afs volumes but other than that it should all be driven by config management 19:46:31 we just need to decide what volumes things go into, and if required create them 19:46:47 we can do that en-masse too 19:47:11 is it worth letting the tc know that there's some infrastructure maintenance we need the projects' help on? 19:47:21 and then update jobs, and copy from old site to new and publish (or run old and new jobs in parallel) 19:47:26 certainly can't hurt to tell them 19:47:30 or would we rather just do this ourselves? 19:47:31 seems reasonable 19:47:47 I don't think we need to touch any repos besides infra repos. 19:48:06 If we name the new jobs like the old ones, all is fine... 19:48:19 and if we duplicate jobs temporarily, we can add in project-config ;) 19:48:45 mostly i'm thinking this is a pretty clear case of something the contributors to the projects should be interested and involved in 19:48:52 this isn't "figure out how to run gerrit in docker" 19:49:07 this is well inside of the wheelhouse of "help maintain your projects deliverables" 19:49:21 i guess it's a question of whether we need those projects deciding which repos should publish into separate afs volumes 19:49:25 corvus: it's updating the promote job - and very few know anything about that 19:49:42 and it's setting up AFS volumes 19:50:02 AJaeger: yeah, let's take the afs volumes out of the conversation. one of us can make all of them in 30 seconds. 19:50:37 AJaeger: so my remaining question then is: is this something where we should be encouraging more folks to get involved? 19:50:51 so that more people know how the promote job works :) 19:51:37 like, to take the governance.openstack.org site for an example, openstack/governance, openstack/election, openstack/governance-uc and openstack/governance-sigs all publish into subtrees of it. those could be separate volumes one mounted in each other which release discretely, or could all go into a single volume with a vos release on a cron timer like docs.openstack.org uses 19:52:24 for that matter, if using multiple volumes they wouldn't necessarily have to be nested, apache could map the urls to parallel mounts 19:52:25 corvus: I'd love to see that ;) 19:53:13 AJaeger: corvus in that case I think we should ask the TC nicely 19:53:23 so those could be decisions we make on behalf of the openstack project, or we could ask openstack project leadership to have some liaisons assigned to work with us on redesigning that 19:53:28 and we can work through the volume details when that specific decision needs to be made 19:53:45 and i don't think we have to block on this 19:54:05 corvus: meaning upgrade the server in the mean time or? 19:54:33 maybe ask the tc if they can help come up with some folks to pitch in on this, and if they can't, we do it ourselves 19:54:43 oh I see 19:54:45 that wfm 19:55:26 but if we're going to have an opendev team, we need to make the bench deeper on the openstack-infra team 19:57:31 ++ 19:57:37 We are almost at time so really quickly 19:57:55 #link https://etherpad.openstack.org/p/OpenDev-Shanghai-PTG-2019 Feel free to add PTG planning stuff there 19:58:03 and I'll give us ~2 minutes for anything else 19:58:07 #topic Open Discussion 19:58:18 does someone want to volunteer to take that to the tc? 19:59:05 ianw: ^ you may have the best grasp of the work involved since you did the audit. But I can probably manage to bring that up during the next office hours (though I haven't checked when those are) 19:59:30 01:00 utc 19:59:41 "wednesday" (so your tonight) 19:59:52 though it's ianw's wednesday for sure 19:59:54 how about i first come up with a lightweight spec to explain what's going on and the rough plan ... 20:00:10 ianw: I think that works since we'll need that documentation anyway 20:00:16 (to guide volunteers) 20:00:19 and we are at time 20:00:23 Thank you everyone! 20:00:24 there's a more active tc office hour 15:00z thursday 20:00:30 #endmeeting