19:01:16 #startmeeting infra 19:01:17 Meeting started Tue Dec 10 19:01:16 2019 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:18 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:20 The meeting name has been set to 'infra' 19:01:22 #link http://lists.openstack.org/pipermail/openstack-infra/2019-December/006546.html Our Agenda 19:01:28 #topic Announcements 19:01:52 There will be no meeting December 24 or December 31. That means next week will be our last official meeting of the calendar year 19:02:23 #topic Actions from last meeting 19:02:28 #link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-12-03-19.01.txt minutes from last meeting 19:02:44 ianw: have we created AFS volumes for static.o.o replacement? 19:02:58 yes, mostly ... the docs sites all done 19:03:19 mnaser is working on jobs i see, and i'm working on bringing up the roles & server to publish them 19:03:40 tarballs and releases are still something we need to think about 19:04:01 that was where we ran into the 50GB fs problem on rax test nodes right? 19:04:07 (the role and server) 19:04:31 o/ 19:04:35 what's the complication with the releases site? 19:04:59 i thought it was statically-generated content like the other sites 19:05:34 fungi: no complication, as such, just need to look at the details of the jobs and how they are going to publish to afs 19:06:08 ahh, okay 19:07:23 ok we've got an agenda item to go in depth for this topic in a bit too 19:07:30 #topic Priority Efforts 19:07:35 #topic Update Config Management 19:07:54 Mordred reports being unwell and hasn't made much progress here 19:08:02 remember to cook your turkeys to 165F 19:08:41 #topic OpenDev 19:08:49 #link http://lists.openstack.org/pipermail/openstack-infra/2019-December/006537.html Governance email 19:08:58 The governance email has gone out. Good feedback so far 19:09:40 I've made one revision to the proposal already and will prbably make at least one more to make things a bit clearer 19:09:45 if you've got feedback I'm happy to hear it 19:12:46 (sorry distracted by zuul upgrade fallout) 19:13:03 The other opendev item I wanted to call out is that the gitea upgrade does not seem to have fixed tonyb's nova repo fetching problem 19:13:29 but at least we were able to rule that out 19:13:29 however the gitea upgrade doesn't seem to have had any other reported issues and looking at cacti graphs gitea 1.10 results in fewer tcp connections 19:13:43 on the plus side, nobody else has reported anything? 19:14:37 ianw: not that I have seen 19:15:58 yeah, seems to have gone very smoothly 19:16:06 thanks to mordred in absentia! 19:16:10 ianw: at this point I think we are probably going to have to dig in with tracing and do our best to see where gitea fails to close those connections? 19:17:12 clarkb: i think we know that git upload-pack is dying, at the bottom layer 19:17:22 but then there's so much ontop of that ... 19:17:58 and we know that gitea doesn't close the tcp connection to the client (whcih is why the command hangs and doesn't do anything) 19:18:17 I guess if the connection was closed the fetch would still fail so we need to fix the dying upload-pack to properly solve the issue 19:20:59 Anything else on Opendev or should we continue? 19:22:10 #topic General Topics 19:22:24 fungi: anything new on the wiki upgrade? 19:22:37 negatory 19:23:23 Next is the static.o.o replacement update 19:23:44 we got a short update earlier. ianw anything else causing problems for the replacement server or was that filesystem thing the only problem? 19:24:36 i think it's going ok ... https://review.opendev.org/#/c/698128/ is the change in question that was not setting a small cache for all afs clients in the gate tests 19:24:54 #link https://review.opendev.org/#/c/697587/ 19:25:12 is the static.opendev.org role ... i still want to iterate on that some more, better testinfra 19:25:28 but initially, everything should be in place to switch governance.openstack.org to that 19:25:50 we should be able to test it before we switch DNS, once the server is up 19:27:35 ++ and since we do dns acme we can even get ssl set up before updating the A/AAAA/CNAME records 19:27:57 yep, i will need to setup the acme records before the server goes live 19:28:13 with that process proven out, i think the other sites will be easy to switch too 19:28:26 sounds like good progress. Thank you 19:30:05 The nodepool dib item was on the agenda but yseterday ianw said we don't need to go over it in the meeting 19:30:09 ianw: ^ is that still the case? 19:30:54 i don't think so ... we have progress merging the changes that build sibling projects with python-builder 19:31:22 that is now all working, and the nodepool-builder container can build an image 19:31:53 nodepool-launcher hasn't managed boot it in the functional test yet, but so i'd say we're half-way :) 19:33:19 ok 19:33:34 The last thing on the agenda is to point this list of services out again 19:33:38 #link https://etherpad.openstack.org/infra-service-list 19:33:52 as mentioned this isn't urgent but I'd like to pick up on some of that "spring cleaning" in the new year 19:35:01 and that is it for the scheduled agenda 19:35:08 I think others have been distracted by soe of the zuul fallout. 19:35:12 #topic Open Discussion 19:35:21 I'll open it up for anything we missed or went past too quickyl 19:35:41 i wanted to bring up dib gate testing 19:35:48 this only occurred yesterday afternoon 19:36:01 #link https://review.opendev.org/#/c/698136/ 19:36:21 because devstack has dropped xenial, we're left without anywhere to run functional test for suse-minimal jobs 19:36:27 bionic does not have zypper on the host 19:37:06 two options here are to pin the suse jobs to devstack train -- this will require work in the install-devstack role the test uses to install branches 19:37:08 can an older devstack be used? 19:37:12 yeah, that 19:37:22 got it 19:37:25 this is where mordred's container bootstrap would help too 19:37:36 or, implement some of the other ideas about using containers as minimal images, etc 19:37:38 then we could use a suse container to get zypper and not care what platform we run dib on 19:37:57 I expect that the container bootstrap work will take time and not be a drop in replacement for a while 19:38:13 probably best to use train devstack while we work on the container bootstrapping? 19:38:37 right, so yeah, for immediate "gate is broken" purposes, i'm going to have to put it into experimental 19:39:14 i reckon a branch option for install-devstack could be generally useful. esp since most public clouds don't run master. 19:39:27 i'm more than happy to expedite review, but i'm not sure how much time i have to focus on fixing it 19:40:11 if we have to triage dev/review time, i think that the container bootstrap is the best long-term investment 19:41:00 if we have "extra" time, then adding branch support for install-devstack is also a good investment, but not as important as the first 19:41:18 (but, obivously, something we can do relatively quickly and can do it first) 19:41:22 that's something i'd definitely like to be testing under a nodepool+containers functional test, as we might be looked at nested containers in that case 19:41:39 ianw: yep, there be dragons 19:43:30 on a somewhat related note, we'll also have to think about when dib can go python3 only 19:43:50 it can probably go ahead and do that today? 19:44:05 at least for nodepool users it is already ypthon3 only 19:44:12 the problem is the branchless nature, and so it will basically kill any centos7 users 19:44:49 #link https://review.opendev.org/#/c/697211 19:45:04 is an example of the work-around we'll have to do to keep python2 19:45:44 ianw: maybe put out a warning and then switch in a few months then? 19:45:52 give people time to go to centos 8 or similar 19:46:58 yep, i think so, anyway, just a heads-up on that 19:49:12 I wanted to ask if there was any interest in a less formal activity with the team/group? One idea I had was everyone playing a few rounds of geoguessr (though that requires online registration with an email address and use of google maps) 19:50:36 i feel like i wandered into a parallel universe now 19:51:16 (or missed a transition in the discussion) 19:51:17 fungi: I also compiled mangband but I couldn't get the curses version to build and the sdl version doesn't allow me to move my character :) 19:51:24 sorry the last discussion seemed to have tapered off 19:51:39 and was wondering if anyone was interested in end of year "office party" except we are remote so what do 19:51:52 got it 19:52:21 i suppose geoguessr won't work with osm 19:52:37 geoguessr came to mind because its a browser game that tests geographical knowledge and a bunch of us travel a bit 19:52:42 or maybe a pose on rdr2? 19:52:44 fungi: ya I think gmaps is built into their web app 19:52:55 zbr: I expect we might be the only two with rdr2 :) 19:53:06 I hated rdr2 19:53:16 I only just started playing it sunday night 19:53:32 i loved rdr2 single player, online sucks. 19:54:11 fungi: basically you get a streetview that you can navigate around but have to determine where you are on a world map only from what you see in the streetview 19:54:23 fungi: and osm doesn't do streetview as far as I know 19:54:41 as long nobody propose a candycrush competition we should be safe 19:54:46 oh, i see. i'd never heard of geoguessr before 19:55:10 me neither, i am now looking at geoguessr for the first time 19:55:53 well let me know. If a hour of geoguessr interests people I can pay for the pro version so we can share common maps together in challenges 19:56:06 or feel free to suggest alternatives. 19:56:15 I also tried supertuxracer but my gpu isn't good enoguh for it 19:56:38 how is your gpu good enough for rdr2 but not tuxracer? 19:56:40 i just did an anonymous trial round of geoguessr, and guessed new zealand for tokelau. only 2500km off, but it drew a funny picture 19:56:46 ianw: ps4 for rdr2 19:57:17 https://imgur.com/a/wr0LPP6 19:57:25 ahh! i have it on pc ... though I'd have to kick my kid off :) 19:57:29 i am on ps4, posse /dev/null -- id "ssbarnea" 19:57:34 corvus: nice 19:58:10 We'll call that the meeting then 19:58:12 #endmeeting