| prometheanfire | thanks, that works | 00:00 |
|---|---|---|
| clarkb | corvus: fwiw this being the first ever tiem I've read that code your change seems to make sense. The redirect is installed when transfering the repo and not something dynamically sorted out on every request right? | 00:02 |
| *** jamesmcarthur has joined #openstack-infra | 00:02 | |
| clarkb | corvus: in the transfer ownership code you install a redirect (into something the db?) and then whatever web hadnling code happens on the requests will look that up | 00:02 |
| corvus | clarkb: right, that makes a row in the db. somewhere there's a web router that hits a point that says "there's no repo here, let's see if there's a redirect object here" and pulls that row out of the db. | 00:03 |
| clarkb | then ya lgtm | 00:03 |
| *** sdake has joined #openstack-infra | 00:04 | |
| corvus | clarkb: here's the db model defn: https://github.com/go-gitea/gitea/blob/master/models/repo_redirect.go#L10-L15 | 00:04 |
| prometheanfire | clarkb: fungi corvus https://review.openstack.org/639868 | 00:04 |
| prometheanfire | I'm going to be afk, but if it's passing feel free to merge | 00:04 |
| prometheanfire | tonyb should be around too | 00:04 |
| corvus | clarkb: the lucky/well-designed thing is that it references the target repo by id, which means that even though it's currently only used when the target repo is in the same org, it shouldn't necessarily be restricted to that. | 00:05 |
| clarkb | corvus: ya | 00:05 |
| corvus | (if that were a string instead of a repo id, i was just going to file a bug and call it a day :) | 00:06 |
| clarkb | prometheanfire: I've +2'd can let tonyb be second reviewer and hopefully approve | 00:06 |
| prometheanfire | kk | 00:06 |
| prometheanfire | may need backports, we'll see there I guess | 00:06 |
| *** dave-mccowan has quit IRC | 00:07 | |
| *** anteaya has quit IRC | 00:07 | |
| clarkb | that chagne merging should trigger post jobs and we'll know quickl.y My guess is it will work for master changes at that point but then we still want backports for stable merges | 00:07 |
| corvus | prometheanfire: will probably need a backport for each branch you want to fix publishing | 00:07 |
| corvus | clarkb: yeah that. | 00:07 |
| prometheanfire | corvus: feel free if needed | 00:07 |
| clarkb | mordred: thinking out loud a bit about that image update thing, we may want to suggest to mnaser that they update their bionic image to a version that has that netplan stuff instaleld so that other users don't have to work around it too? | 00:09 |
| ianw | bridge.o.o : 1.4G Feb 28 00:05 puppet_run_cloud_launcher.log ... should probably rotate that | 00:09 |
| clarkb | mnaser: ^ tl;dr is current vexxhost bionic images fail to update packages because of a package thing with netplan. I expect htat newer bionic images have this sorted already | 00:09 |
| clarkb | ianw: I thought we did rotate that | 00:09 |
| clarkb | ianw: maybe that was only in the puppet nad didn't get ansibled | 00:10 |
| ianw | clarkb: yeah, will look when i find out why graphite's not puppeting :) | 00:10 |
| ianw | - ze08.openstack.org # clarkb is testing jemalloc | 00:11 |
| ianw | .. did i misremember but didn't we do this on all hosts? | 00:11 |
| clarkb | fungi pushed a chagne to do it on all hosts, did that merge? if so we can remoev that | 00:11 |
| * clarkb checks | 00:11 | |
| clarkb | https://review.openstack.org/#/c/638044/ not merged. I will review it now | 00:11 |
| fungi | thanks! | 00:12 |
| fungi | i have plenty of changes which aren't merged, but i expect that goes for all of us | 00:12 |
| clarkb | fungi: ianw I just approved it after double checking the new defaults file contents match what I put in place on ze08 | 00:13 |
| *** mriedem has quit IRC | 00:13 | |
| fungi | awesome | 00:13 |
| fungi | presumably we'll need executor restarts for it to take effect | 00:14 |
| clarkb | yup | 00:14 |
| ianw | clarkb: oh you know what, i think that's an old cloud_launcher file, it's all in /var/log/ansible which we do rotate | 00:14 |
| clarkb | ianw: ah | 00:14 |
| * tonyb cap in hand ... can someone approve my message to openstack-discuss 'Too many recipients to the message' #oops | 00:14 | |
| clarkb | fungi: I expect taht zuul will end up wanting another one of those soon as I think people want a release as soon as we base64 encode the commit message in the job vars | 00:14 |
| fungi | yup | 00:15 |
| clarkb | tonyb: how do we know you aren't spamming:P | 00:15 |
| fungi | tonyb: i already did | 00:15 |
| clarkb | with mailman not deduping I've been trying to remove those extra recipients | 00:15 |
| fungi | tonyb: if your mua has a reply-to-list option, i recommend it. i couldn't get by without mine | 00:15 |
| prometheanfire | L | 00:16 |
| tonyb | fungi: Ahh thanks | 00:16 |
| ianw | FAILED! => {... "msg": "puppet did not run", ...} ... thanks ansible :) | 00:16 |
| clarkb | ianw: check syslog on the target host | 00:16 |
| clarkb | ianw: I like sudo grep puppet-user /var/log/syslog | 00:16 |
| tonyb | fungi: I humharred about it but figured it was worth going to all the lists | 00:16 |
| clarkb | this doesn't work on centos because its puppet-something-else iirc | 00:17 |
| tonyb | fungi: which was probably a mistake | 00:17 |
| *** jamesmcarthur has quit IRC | 00:17 | |
| fungi | yeah, i trimmed the recipient list all the way down to just openstack-discuss on my reply, for good measure | 00:17 |
| ianw | clarkb: yeah ... and doh i didn't update the heira vals, so that explains that | 00:17 |
| clarkb | ianw: is it safe for me to edit the emergency file? I don't want ot stomp on your changes. BUt i can remove kdc01 from that file now and we can remoev ze08 as soon as fungi's change merges | 00:18 |
| clarkb | ianw: oh I ran into that with health too | 00:18 |
| *** jamesmcarthur has joined #openstack-infra | 00:18 | |
| fungi | i doubt the starlingx, kata, airship and zuul discussion lists care about this ongoing row | 00:18 |
| clarkb | which reminds me I can also delete the old host_vars/health.openstack.rog as soon as you aren't editing things | 00:18 |
| ianw | clarkb: yeah, i'm not using it, thanks | 00:18 |
| clarkb | ok updating emergency now | 00:18 |
| fungi | clarkb: did i have changes in the emergency file? | 00:18 |
| clarkb | done with emergency, let me know when I can edit ansible vars | 00:18 |
| clarkb | fungi: ya old openstackid.org is in there | 00:19 |
| clarkb | fungi: I am done with the file if you want to remove that | 00:19 |
| fungi | oh, i'll remember to clean that up as soon as i delete the server itself | 00:19 |
| *** mriedem has joined #openstack-infra | 00:19 | |
| fungi | wanted to be sure smarcet had a chance to copy anything he might have left on it | 00:19 |
| clarkb | sounds good | 00:20 |
| ianw | clarkb: ok, i just made a copy and similar will remove the old one when it's retired | 00:20 |
| mnaser | larsks: are you telling me that uh | 00:20 |
| mnaser | err sorry for that highlight ^ | 00:20 |
| clarkb | ianw: I'm good to make my changes now then? | 00:20 |
| ianw | clarkb: yep, i'm done for now, thanks | 00:20 |
| *** rascasoft has joined #openstack-infra | 00:20 | |
| mnaser | clarkb: are you telling me that ubuntu 18.04 images magically started shipping with netplan at some arbitrary point? | 00:20 |
| fungi | and in an un-upgradeable state, it seems like | 00:21 |
| clarkb | mnaser: aiui netplan is a dep of network config things | 00:21 |
| clarkb | mnaser: so yes I expect that current bionic images will ship with an up to date an upgradeable netplan as they are past the point of needing interaction | 00:21 |
| clarkb | I haven't confirmed this though | 00:21 |
| mnaser | https://i.ytimg.com/vi/xz3PbpPR6DY/hqdefault.jpg | 00:22 |
| *** jamesmcarthur has quit IRC | 00:22 | |
| clarkb | I think it was added as a dep to a preexisting package? | 00:23 |
| clarkb | something like that which forces you to say yes its ok to install | 00:23 |
| fungi | right, it needs apt full-upgrade (formerly apt or apt-get dist-upgrade which is more recently deprecated) | 00:24 |
| *** rascasoft has quit IRC | 00:27 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Update gentoo-releng gpg key https://review.openstack.org/639833 | 00:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul-website master: Add a promotional message banner and events list https://review.openstack.org/639871 | 00:35 |
| openstackgerrit | Merged openstack-infra/puppet-zuul master: Use jemalloc on Zuul v3 executors https://review.openstack.org/638044 | 00:40 |
| ianw | oh my god, my cable/pole situation has gone from mildly annoying to kafkaesque farce | 00:40 |
| ianw | due to the "national broadband network" taking over the coax cable outside my house at some unspecified future point, the telecommunications company who presently owns and maintains it won't work on it | 00:42 |
| clarkb | ianw: I think you need a Grade 34 bureaucrat | 00:43 |
| ianw | somebody could come and plug in, run and terminate the cable to the house in 15 minutes and be done. but now there are at least 4 people somehow involved, apart from the people they have already physically sent out | 00:43 |
| ianw | who weren't aware of the situation, but weren't qualified to work on the pole | 00:44 |
| *** ijw has joined #openstack-infra | 00:47 | |
| *** agopi has joined #openstack-infra | 00:48 | |
| fungi | get harry tuttle. he doesn't care if you've filed a twenty-seven b stroke six | 00:57 |
| *** sdake has quit IRC | 01:08 | |
| ianw | ++ | 01:11 |
| *** yamamoto has quit IRC | 01:11 | |
| ianw | "[remote 1.43.155.180:27803] ImportError: No module named whisper ..." ... guess graphite puppet needs some debugging | 01:15 |
| *** ijw has quit IRC | 01:15 | |
| ianw | ailed to call refresh: Could not find command 'pip' ... interesting, this might be similar to clarkb's issues | 01:16 |
| *** betherly has joined #openstack-infra | 01:17 | |
| *** ijw has joined #openstack-infra | 01:17 | |
| clarkb | ianw: my low tech fix was to delete the repos and let next puppet run find the pip that was indtalled | 01:18 |
| clarkb | and my complictaed real fix is pushed to the puppet health repo | 01:18 |
| ianw | gcc is also not installed | 01:20 |
| ianw | how do the rspec test work :/ | 01:21 |
| *** betherly has quit IRC | 01:22 | |
| ianw | so, breaking it down, first issue is that whisper install doesn't depend on the install::pip -> https://git.openstack.org/cgit/openstack-infra/puppet-graphite/tree/manifests/init.pp#n101 | 01:23 |
| *** hwoarang has quit IRC | 01:23 | |
| ianw | that "install ::pip" doesn't bring in build-essential is interesting | 01:25 |
| clarkb | rspec worls because we install that stuff in our images aiui | 01:26 |
| *** hwoarang has joined #openstack-infra | 01:29 | |
| tonyb | corvus, clarkb: for the record setting CR=0,W=0 didn't work on https://review.openstack.org/#/c/639011/6 gerrit stilled -2'd the new patchset | 01:32 |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-graphite master: Fix pip ordering and build requirements https://review.openstack.org/639877 | 01:33 |
| ianw | clarkb: yeah, two points that we miss in testing of course hit in production :) | 01:34 |
| fungi | it should be a fundamental law of ci | 01:35 |
| *** gyee has quit IRC | 01:38 | |
| clarkb | tonyb that is quite weird I wonder what happens if someone else pushes a patch | 01:41 |
| tonyb | clarkb: Yeah | 01:42 |
| * tonyb tries to re-create it in the sandox | 01:42 | |
| tonyb | sandbox even | 01:42 |
| *** yamamoto has joined #openstack-infra | 01:47 | |
| tonyb | clarkb: Yup https://review.openstack.org/#/c/639880/ is doing the same thing | 01:52 |
| *** yamamoto has quit IRC | 01:52 | |
| tonyb | clarkb, corvus: it does have soemthing to do with switching from 0,0 -> -2,-1 -> 0,-1 | 01:52 |
| tonyb | clarkb: clarkb I messed up https://review.openstack.org/#/c/639880/ and it doesn't do it | 01:53 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Add API endpoint to get frozen jobs https://review.openstack.org/607077 | 01:59 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 01:59 |
| *** rascasoft has joined #openstack-infra | 02:00 | |
| *** sdake has joined #openstack-infra | 02:09 | |
| *** betherly has joined #openstack-infra | 02:10 | |
| *** rascasoft has quit IRC | 02:11 | |
| *** betherly has quit IRC | 02:14 | |
| *** yamamoto has joined #openstack-infra | 02:27 | |
| *** roman_g has quit IRC | 02:28 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:29 | |
| *** bhavikdbavishi has quit IRC | 02:31 | |
| *** hongbin has joined #openstack-infra | 02:32 | |
| *** yamamoto has quit IRC | 02:39 | |
| *** rlandy|bbl is now known as rlandy | 02:45 | |
| *** sdake has quit IRC | 02:46 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-graphite master: Fix pip ordering and build requirements https://review.openstack.org/639877 | 02:46 |
| hogepodge | fungi: I can send a patch up to disable loci jobs except for requirements. We're not doing functional testing of loci containers so we don't even know if the artifacts work (well, those running in production know but it's not the same as a gate job to be sure) | 02:55 |
| fungi | hogepodge: do you happen to know which projects it was running on? the alternative i think is to move the job definition and playbooks into the openstack-infra/project-config repo | 02:56 |
| fungi | though that may still require a job rename dance touching all those repos since zuul won't let us remove an in-use job nor create a non-variant job with the same name | 02:57 |
| *** psachin has joined #openstack-infra | 03:02 | |
| *** wolverineav has quit IRC | 03:06 | |
| *** dave-mccowan has joined #openstack-infra | 03:06 | |
| tonyb | hogepodge: https://review.openstack.org/#/c/639868/ we're in the process of removing it from requirements ATM happy to put it back when we can see it working | 03:10 |
| *** ijw has quit IRC | 03:13 | |
| *** ijw_ has joined #openstack-infra | 03:13 | |
| *** udesale has joined #openstack-infra | 03:15 | |
| *** yamamoto has joined #openstack-infra | 03:15 | |
| *** apetrich has quit IRC | 03:15 | |
| hogepodge | tonyb: I've been afk for a bit tonight, so I need to catch up on exactly what's happening. | 03:16 |
| clarkb | hogepodge: its the same thing as with the cinder image build job | 03:16 |
| hogepodge | requirements is the one job we actually do want to run and publish since everything else depends on it | 03:16 |
| clarkb | just a different job | 03:17 |
| hogepodge | is there a writeup of the issue or a link to where the discussion happened in IRC? | 03:17 |
| clarkb | prometheanfire: made a bug let me find a link | 03:18 |
| clarkb | hogepodge: https://storyboard.openstack.org/#!/story/2005108 | 03:19 |
| hogepodge | Ok, so if I understand, we won't automatically trigger a loci-requirements build with the requirements repo updates, but our regular loci requirements build is ok? | 03:19 |
| tonyb | hogepodge: I believe that is correct | 03:19 |
| hogepodge | this kind of fits in with how I want to do builds anyway, rather than segment across projects I'd rather segment across distros and add functional testing into the mix also somehow. | 03:20 |
| fungi | yes, running it on loci changes is fine | 03:20 |
| hogepodge | I'm cool with all those changes | 03:20 |
| clarkb | yes and we werent triggering it anyway for just over a month | 03:20 |
| clarkb | since the zuul security fix went in | 03:20 |
| hogepodge | that explains the sense of... emptiness I was feeling ;-) (kidding) | 03:20 |
| *** sdake has joined #openstack-infra | 03:22 | |
| *** sdake has quit IRC | 03:23 | |
| *** bhavikdbavishi has joined #openstack-infra | 03:24 | |
| *** takamatsu has quit IRC | 03:25 | |
| *** sdake has joined #openstack-infra | 03:27 | |
| *** dave-mccowan has quit IRC | 03:32 | |
| *** mriedem has quit IRC | 03:32 | |
| *** ijw_ has quit IRC | 03:38 | |
| *** sdake has quit IRC | 03:40 | |
| *** wolverineav has joined #openstack-infra | 03:40 | |
| *** wolverineav has quit IRC | 03:47 | |
| *** markvoelker has joined #openstack-infra | 03:48 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: WIP: add job filter to trigger event https://review.openstack.org/639905 | 03:49 |
| *** rascasoft has joined #openstack-infra | 03:58 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-graphite master: Fix pip ordering and build requirements https://review.openstack.org/639877 | 04:08 |
| *** wolverineav has joined #openstack-infra | 04:08 | |
| *** rascasoft has quit IRC | 04:08 | |
| *** spsurya has joined #openstack-infra | 04:14 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 04:16 |
| *** wolverineav has quit IRC | 04:17 | |
| *** yamamoto has quit IRC | 04:19 | |
| *** ykarel|away has joined #openstack-infra | 04:19 | |
| *** ykarel|away is now known as ykarel | 04:20 | |
| *** markvoelker has quit IRC | 04:20 | |
| *** betherly has joined #openstack-infra | 04:26 | |
| *** betherly has quit IRC | 04:31 | |
| *** eernst has joined #openstack-infra | 04:42 | |
| *** rlandy has quit IRC | 04:50 | |
| tonyb | :qa! | 04:56 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add initial driver and event https://review.openstack.org/555153 | 05:00 |
| *** armax has quit IRC | 05:00 | |
| *** raukadah has quit IRC | 05:03 | |
| *** chandankumar has joined #openstack-infra | 05:04 | |
| *** takamatsu has joined #openstack-infra | 05:05 | |
| *** takamatsu has quit IRC | 05:10 | |
| *** wolverineav has joined #openstack-infra | 05:14 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add web route and rpclistener https://review.openstack.org/554839 | 05:15 |
| *** hongbin has quit IRC | 05:16 | |
| *** markvoelker has joined #openstack-infra | 05:17 | |
| *** wolverineav has quit IRC | 05:19 | |
| *** sdake has joined #openstack-infra | 05:30 | |
| *** rascasoft has joined #openstack-infra | 05:43 | |
| *** ramishra has joined #openstack-infra | 05:45 | |
| *** yamamoto has joined #openstack-infra | 05:46 | |
| *** rascasoft has quit IRC | 05:47 | |
| *** markvoelker has quit IRC | 05:51 | |
| *** psachin has quit IRC | 05:58 | |
| *** psachin has joined #openstack-infra | 06:00 | |
| *** ramishra has quit IRC | 06:01 | |
| *** sdake has quit IRC | 06:02 | |
| *** ramishra has joined #openstack-infra | 06:08 | |
| *** janki has joined #openstack-infra | 06:15 | |
| *** yamamoto has quit IRC | 06:19 | |
| *** bhavikdbavishi has quit IRC | 06:19 | |
| *** bhavikdbavishi has joined #openstack-infra | 06:20 | |
| *** rascasoft has joined #openstack-infra | 06:20 | |
| *** yamamoto has joined #openstack-infra | 06:22 | |
| *** sdake has joined #openstack-infra | 06:22 | |
| *** gregoryo has joined #openstack-infra | 06:24 | |
| *** sdake has quit IRC | 06:26 | |
| *** rascasoft has quit IRC | 06:32 | |
| *** ijw has joined #openstack-infra | 06:32 | |
| *** sdake has joined #openstack-infra | 06:42 | |
| *** markvoelker has joined #openstack-infra | 06:48 | |
| openstackgerrit | Merged openstack-infra/grafyaml master: Add capability to use grafana folderid parameter https://review.openstack.org/637520 | 06:54 |
| *** eernst has quit IRC | 07:00 | |
| *** tkajinam_ has joined #openstack-infra | 07:02 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build button to trigger job https://review.openstack.org/635716 | 07:04 |
| *** tkajinam has quit IRC | 07:05 | |
| *** dpawlik has joined #openstack-infra | 07:08 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Explanatory comment for Editable Git install https://review.openstack.org/639649 | 07:09 |
| *** mandre is now known as mandre_away | 07:09 | |
| *** rascasoft has joined #openstack-infra | 07:09 | |
| *** apetrich has joined #openstack-infra | 07:12 | |
| *** sdake has quit IRC | 07:15 | |
| *** markvoelker has quit IRC | 07:20 | |
| *** aojea has joined #openstack-infra | 07:25 | |
| *** ijw has quit IRC | 07:25 | |
| *** ijw has joined #openstack-infra | 07:25 | |
| prometheanfire | clarkb: fungi ping me if backport needed, sleep now, nn | 07:28 |
| *** sshnaidm|afk is now known as sshnaidm | 07:32 | |
| *** kopecmartin|off is now known as kopecmartin | 07:34 | |
| *** ijw has quit IRC | 07:34 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: switch jobs list to a tree view https://review.openstack.org/633437 | 07:41 |
| *** quiquell is now known as quiquell|brb | 07:48 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add jobs list filter https://review.openstack.org/633652 | 07:48 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: scheduler: add job's tags to the rpc job_list method https://review.openstack.org/633653 | 07:48 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add tags to jobs list https://review.openstack.org/633654 | 07:48 |
| *** roman_g has joined #openstack-infra | 07:52 | |
| *** ccamacho has joined #openstack-infra | 07:59 | |
| *** jpich has joined #openstack-infra | 08:00 | |
| *** jtomasek has joined #openstack-infra | 08:02 | |
| *** ginopc has joined #openstack-infra | 08:13 | |
| *** quiquell|brb is now known as quiquell | 08:14 | |
| *** markvoelker has joined #openstack-infra | 08:18 | |
| zbr | prometheanfire: hi! can you give a kick to https://review.openstack.org/#/c/635590/ ? | 08:27 |
| *** pcaruana has joined #openstack-infra | 08:32 | |
| *** helenafm has joined #openstack-infra | 08:37 | |
| *** bhavikdbavishi has quit IRC | 08:38 | |
| *** tkajinam_ has quit IRC | 08:38 | |
| *** bhavikdbavishi has joined #openstack-infra | 08:38 | |
| *** tosky has joined #openstack-infra | 08:39 | |
| *** ykarel is now known as ykarel|lunch | 08:43 | |
| *** markvoelker has quit IRC | 08:50 | |
| *** jpena|off is now known as jpena | 08:54 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum and dnf atoms for platforms https://review.openstack.org/639951 | 08:58 |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum and dnf atoms for platforms https://review.openstack.org/639951 | 09:00 |
| *** gregoryo has quit IRC | 09:05 | |
| *** jaosorior has joined #openstack-infra | 09:06 | |
| *** iurygregory_mtg is now known as iurygregory | 09:06 | |
| *** electrofelix has joined #openstack-infra | 09:11 | |
| *** elod has quit IRC | 09:12 | |
| *** sdake has joined #openstack-infra | 09:15 | |
| *** olivierbourdon38 has joined #openstack-infra | 09:20 | |
| *** olivierbourdon38 has quit IRC | 09:22 | |
| *** iurygregory has quit IRC | 09:24 | |
| *** iurygregory has joined #openstack-infra | 09:28 | |
| *** udesale has quit IRC | 09:28 | |
| *** udesale has joined #openstack-infra | 09:29 | |
| *** e0ne has joined #openstack-infra | 09:32 | |
| *** derekh has joined #openstack-infra | 09:35 | |
| *** yboaron has joined #openstack-infra | 09:45 | |
| *** gfidente has joined #openstack-infra | 09:45 | |
| *** sdake has quit IRC | 09:46 | |
| *** markvoelker has joined #openstack-infra | 09:47 | |
| *** rcernin has quit IRC | 09:48 | |
| *** ykarel|lunch is now known as ykarel | 09:48 | |
| *** dtantsur|afk is now known as dtantsur | 09:49 | |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: Adds task title to new story modal https://review.openstack.org/639962 | 09:50 |
| *** panda|ruck|off is now known as panda|ruck | 09:51 | |
| openstackgerrit | M V P Nitesh proposed openstack/diskimage-builder master: Adding 'oel' as a new dib element https://review.openstack.org/625501 | 09:56 |
| *** iurygregory has quit IRC | 10:03 | |
| *** yamamoto has quit IRC | 10:04 | |
| dulek | Hi folks! Do we happen to have infra mirrors of k8s builds? | 10:06 |
| dulek | I'm working on running upstream k8s tests on kuryr-k8s gates. | 10:06 |
| *** yboaron_ has joined #openstack-infra | 10:09 | |
| *** iurygregory has joined #openstack-infra | 10:09 | |
| *** elod has joined #openstack-infra | 10:10 | |
| *** yboaron has quit IRC | 10:12 | |
| *** markvoelker has quit IRC | 10:21 | |
| *** udesale has quit IRC | 10:24 | |
| *** udesale has joined #openstack-infra | 10:24 | |
| *** sdake has joined #openstack-infra | 10:37 | |
| *** luizbag has joined #openstack-infra | 10:38 | |
| *** yamamoto has joined #openstack-infra | 10:38 | |
| *** iurygregory has quit IRC | 10:40 | |
| *** iurygregory has joined #openstack-infra | 10:41 | |
| *** wolverineav has joined #openstack-infra | 10:42 | |
| *** iurygregory has quit IRC | 10:42 | |
| *** iurygregory has joined #openstack-infra | 10:42 | |
| *** ccamacho has quit IRC | 10:43 | |
| *** yamamoto has quit IRC | 10:43 | |
| *** wolverineav has quit IRC | 10:46 | |
| *** cmoura has quit IRC | 10:47 | |
| *** ccamacho has joined #openstack-infra | 10:49 | |
| *** dpawlik has quit IRC | 11:00 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Support requests for specific providers https://review.openstack.org/639418 | 11:02 |
| *** bhavikdbavishi has quit IRC | 11:03 | |
| *** agopi has quit IRC | 11:06 | |
| *** mhu has joined #openstack-infra | 11:10 | |
| openstackgerrit | Merged openstack-infra/zuul master: Request child nodes of paused jobs at higher priority https://review.openstack.org/638796 | 11:10 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] Add AuthorizationRule Configuration Object, claim validation https://review.openstack.org/639855 | 11:11 |
| *** yboaron_ has quit IRC | 11:15 | |
| *** markvoelker has joined #openstack-infra | 11:18 | |
| *** sdake has quit IRC | 11:18 | |
| *** yboaron_ has joined #openstack-infra | 11:19 | |
| *** sdake has joined #openstack-infra | 11:20 | |
| openstackgerrit | M V P Nitesh proposed openstack/diskimage-builder master: Adding 'oel' as a new dib element https://review.openstack.org/625501 | 11:22 |
| *** e0ne has quit IRC | 11:30 | |
| *** rossella_s has quit IRC | 11:31 | |
| *** rossella_s has joined #openstack-infra | 11:32 | |
| *** e0ne has joined #openstack-infra | 11:36 | |
| *** jpich has quit IRC | 11:47 | |
| *** markvoelker has quit IRC | 11:50 | |
| openstackgerrit | Merged openstack-infra/bindep master: Adds job to test bindep on Fedora https://review.openstack.org/635590 | 11:51 |
| *** dpawlik has joined #openstack-infra | 11:52 | |
| *** sdake has quit IRC | 11:55 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum, dnf and zypper atoms for platforms https://review.openstack.org/639951 | 11:57 |
| *** gfidente has quit IRC | 12:07 | |
| *** bhavikdbavishi has joined #openstack-infra | 12:12 | |
| *** yboaron_ has quit IRC | 12:14 | |
| *** bhavikdbavishi has quit IRC | 12:14 | |
| *** bhavikdbavishi has joined #openstack-infra | 12:14 | |
| *** yamamoto has joined #openstack-infra | 12:18 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum, dnf and zypper atoms for platforms https://review.openstack.org/639951 | 12:18 |
| *** bhavikdbavishi1 has joined #openstack-infra | 12:19 | |
| *** bhavikdbavishi has quit IRC | 12:19 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 12:19 | |
| *** yamamoto has quit IRC | 12:22 | |
| *** gfidente has joined #openstack-infra | 12:31 | |
| *** ccamacho has quit IRC | 12:38 | |
| *** jpena is now known as jpena|lunch | 12:41 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum, dnf and zypper atoms for platforms https://review.openstack.org/639951 | 12:41 |
| *** ramishra_ has joined #openstack-infra | 12:41 | |
| *** ramishra has quit IRC | 12:44 | |
| *** panda|ruck is now known as panda|ruck|lunch | 12:44 | |
| *** markvoelker has joined #openstack-infra | 12:47 | |
| *** rh-jelabarre has joined #openstack-infra | 12:52 | |
| *** yamamoto has joined #openstack-infra | 12:58 | |
| *** trown|outtypewww is now known as trown | 12:58 | |
| *** luizbag has quit IRC | 12:58 | |
| *** yamamoto has quit IRC | 13:03 | |
| *** ccamacho has joined #openstack-infra | 13:06 | |
| *** dave-mccowan has joined #openstack-infra | 13:10 | |
| *** yamamoto has joined #openstack-infra | 13:11 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] Add AuthorizationRule Configuration Object, claim validation https://review.openstack.org/639855 | 13:14 |
| *** luizbag has joined #openstack-infra | 13:16 | |
| *** jamesmcarthur has joined #openstack-infra | 13:16 | |
| *** rtjure has joined #openstack-infra | 13:17 | |
| *** markvoelker has quit IRC | 13:20 | |
| *** jamesmcarthur has quit IRC | 13:23 | |
| *** rossella_s has quit IRC | 13:24 | |
| *** rossella_s has joined #openstack-infra | 13:24 | |
| *** jamesmcarthur has joined #openstack-infra | 13:26 | |
| *** yboaron_ has joined #openstack-infra | 13:26 | |
| guilhermesp | hey there! I've asked a while ago a hold on https://review.openstack.org/#/c/627782/ which has 23.253.218.253 ip | 13:29 |
| guilhermesp | just to let you know it can be destroyed | 13:29 |
| *** panda|ruck|lunch is now known as panda|ruck | 13:30 | |
| *** jamesmcarthur has quit IRC | 13:31 | |
| *** rh-jelabarre has quit IRC | 13:31 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Update dns message to make sense for opendev https://review.openstack.org/640026 | 13:34 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add control plane clouds to nodepool builder clouds.yaml https://review.openstack.org/640027 | 13:34 |
| *** rlandy has joined #openstack-infra | 13:35 | |
| *** quiquell is now known as quiquell|lunch | 13:38 | |
| *** Shrews_ is now known as Shrews | 13:43 | |
| *** jpena|lunch is now known as jpena | 13:43 | |
| fungi | thanks guilhermesp! i've released it back to the pool for deletion | 13:45 |
| *** jamesmcarthur has joined #openstack-infra | 13:46 | |
| *** yamamoto has quit IRC | 13:46 | |
| *** yamamoto has joined #openstack-infra | 13:48 | |
| *** yamamoto has quit IRC | 13:48 | |
| *** yamamoto has joined #openstack-infra | 13:49 | |
| *** dpawlik has quit IRC | 13:51 | |
| *** quiquell|lunch is now known as quiquell | 13:52 | |
| *** yamamoto has quit IRC | 13:53 | |
| *** rfolco|rover is now known as rfolco|afk | 13:54 | |
| *** ekultails has joined #openstack-infra | 14:01 | |
| *** betherly has joined #openstack-infra | 14:02 | |
| *** jamesmcarthur has quit IRC | 14:04 | |
| *** yamamoto has joined #openstack-infra | 14:04 | |
| *** yamamoto has quit IRC | 14:04 | |
| *** yamamoto has joined #openstack-infra | 14:04 | |
| *** jamesmcarthur has joined #openstack-infra | 14:05 | |
| *** jamesmcarthur has quit IRC | 14:10 | |
| *** jistr is now known as jistr|call | 14:11 | |
| *** mriedem has joined #openstack-infra | 14:13 | |
| *** dpawlik has joined #openstack-infra | 14:14 | |
| *** ykarel has quit IRC | 14:15 | |
| *** ykarel has joined #openstack-infra | 14:15 | |
| zbr | AJaeger: finished implementing your bindep comments on https://review.openstack.org/#/c/639951/ -- looks ok now? | 14:16 |
| *** markvoelker has joined #openstack-infra | 14:18 | |
| *** wolverineav has joined #openstack-infra | 14:18 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum, dnf and zypper atoms for platforms https://review.openstack.org/639951 | 14:18 |
| *** yboaron_ has quit IRC | 14:19 | |
| *** yboaron_ has joined #openstack-infra | 14:19 | |
| *** ianychoi has joined #openstack-infra | 14:20 | |
| *** yboaron_ has quit IRC | 14:21 | |
| *** yboaron_ has joined #openstack-infra | 14:21 | |
| *** wolverineav has quit IRC | 14:22 | |
| *** bogdando has joined #openstack-infra | 14:23 | |
| bogdando | hi folks | 14:25 |
| bogdando | corvus: I know your time was spent enough on the ci pipelines topic, but some updates and numbers I posted probably worth reading and re-evaluating | 14:26 |
| bogdando | http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003336.html | 14:26 |
| bogdando | and a few more on top of that | 14:27 |
| bogdando | thanks | 14:27 |
| bogdando | clarkb: ^^ | 14:27 |
| *** bogdando has left #openstack-infra | 14:29 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Build ubuntu bionic minimal images for control plane https://review.openstack.org/640044 | 14:30 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Build ubuntu bionic minimal images for control plane https://review.openstack.org/640044 | 14:30 |
| mordred | infra-root: I pushed up a couple of patches ^^ that contain a radical approach - namely, using nodepool to build minimal base images for our control plane | 14:31 |
| pabelanger | mordred: I'm behind that, last we discussed (many years ago) there was some concerns around nodepool.o.o using passwords to control plane. At one point we might have discussed a 2nd nodepool just for that. However, +1 from me for doing it | 14:32 |
| fungi | mordred: having consistent images across providers for our persistent servers sounds great, though i do wonder what fun gotchas we'll encounter once we're using them for systems we don't delete in an hour | 14:32 |
| mordred | infra-root: this was spurred by 2 things - a) we keep talking about eventually building our own images b) I spent all of yesterday trying to boot a new server and kept failing on clarkb's unattended-upgrades - no matter what I tried something kept bombing out. I manually built a minimal base image and uploaded it to see if that worked better - and it worked GREAT | 14:32 |
| mordred | pabelanger: yeah - I thnik the thing that clicked for me is that the now the builders are separate from the launchers and have a separate config, so we can pretty safely configure only the builders to know about the control plane | 14:33 |
| mordred | fungi: ++ ... if we are ok with this- I'd like to update launch-node to know how to look for an image by its logical name assuming it's been uploaded by nodepool - so basically get the image list and look for the newest image prefixed with the requested name | 14:34 |
| mordred | fungi: since step 1 of using launch-node today is "openstack image list" to find what image is the right one to use | 14:34 |
| fungi | more wondering, e.g., where our model for things like glean may fall down for long-lived systems | 14:35 |
| mordred | fungi: the nice thing is that it means that when we boot a server it'll have the latest packages already | 14:35 |
| mordred | fungi: yeah - although I think as long as glean survives reboots we should be fine (and I think we already require this) | 14:36 |
| fungi | it's also the case that we can set up access controls so that the credentials nodepool uses can only upload and delete images, the actual security risks i think come in where a compromised nodepool builder is able to compromise the images we use to boot any new server | 14:37 |
| *** jamesmcarthur has joined #openstack-infra | 14:38 | |
| mordred | fungi: I'm not sure we have that ability actually -at least not across all of our clouds | 14:38 |
| fungi | oh, good point, i may be conflating rackspace's account access controls with openstack's | 14:39 |
| mordred | yeah. heaven forbid we have such acl support in openstack :) | 14:40 |
| mordred | to be fair, the app-creds work aims to provide such a thign in a sane manner | 14:40 |
| cmurphy | it's this close -->||<-- | 14:42 |
| mordred | infra-root: in any case, we don't specifically have to agree to that approach today - but at the very least I'd like either an ack from folks on using manually uploaded minimal base images, because otherwise I'm pretty dead in the water with the gitea servers | 14:42 |
| mordred | or - I said either - I guess the or would be "or would love to hear an alternate suggestion for getting unstuck" | 14:43 |
| *** bhavikdbavishi has quit IRC | 14:43 | |
| *** bhavikdbavishi has joined #openstack-infra | 14:44 | |
| *** jamesmcarthur has quit IRC | 14:44 | |
| mordred | although I guess if we do decide we want to do that, I should expand those patches to include the control plane accounts on all of our clouds | 14:46 |
| *** kgiusti has joined #openstack-infra | 14:46 | |
| mordred | also - my manually built image used the elements listed in that definition | 14:48 |
| mordred | fungi, pabelanger: https://review.openstack.org/#/c/640026/ should be a safe review to land | 14:48 |
| *** markvoelker has quit IRC | 14:50 | |
| *** lujinluo has joined #openstack-infra | 14:50 | |
| fungi | mordred: i'm certainly cool with us uploading images for our servers to get consistency, whether we build them ourselves or fetch them from an image provider | 14:50 |
| mordred | cool. if we're not ok with using nodepool builder - we could make a script that could be run manually to ensure we get consistently built images | 14:51 |
| fungi | i even do that for my personal servers because i don't trust clouds to have the exact same build of debian image, even if they all download their debian images from the same place | 14:52 |
| mordred | fungi: ++ | 14:52 |
| fungi | (i don't build them myself... i could but my needs aren't quite that unusual) | 14:52 |
| *** rh-jelabarre has joined #openstack-infra | 14:54 | |
| *** lujinluo has quit IRC | 14:55 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add control plane clouds to nodepool builder clouds.yaml https://review.openstack.org/640027 | 15:00 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Clean up boot-from-volume volumes on error https://review.openstack.org/640054 | 15:00 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Change the default for config-drive to True https://review.openstack.org/640055 | 15:00 |
| mordred | infra-root: https://review.openstack.org/#/q/topic:infra-base-images updated a few things and made a topic | 15:02 |
| *** rossella_ has joined #openstack-infra | 15:06 | |
| *** rossella_s has quit IRC | 15:06 | |
| *** gfidente has quit IRC | 15:06 | |
| mordred | fungi, clarkb, corvus, pabelanger: while I'm bugging people - in zone-opendev.org we seem to be half aligning rocketships and half not. I do not care at all which thing we do - but should we pick one? if we pick one - I think I'd vote for not aligning rocketships | 15:07 |
| fungi | mordred: i think it depends on what your tabstop is set to | 15:09 |
| fungi | since it's all hard tabs | 15:09 |
| fungi | but we could certainly just treat it as tab-separated fields rather than tab-aligned fields | 15:10 |
| mordred | fungi: yeah - some of the lines seem to just be tab-separated, some seem to be tab aligned - but even the tab aligned have chosen different arbitrary alignments | 15:13 |
| mordred | fungi: I'll leave it for now, but maybe do a cleanup after the gitea servers go in | 15:14 |
| fungi | doesn't surprise me | 15:14 |
| fungi | if we do decide it's important, then we should make a simple linter | 15:14 |
| fungi | though there's also nslint i used to use when i had a bunch of operators managing hundreds of bind zonefiles | 15:14 |
| mordred | yeah. I mean - I mostly don't think it's important - but thought I'd ask :) | 15:15 |
| *** e0ne has quit IRC | 15:16 | |
| *** jamesmcarthur has joined #openstack-infra | 15:18 | |
| *** e0ne has joined #openstack-infra | 15:19 | |
| *** jamesmcarthur has quit IRC | 15:22 | |
| *** amorin has joined #openstack-infra | 15:23 | |
| corvus | mordred: generally it's tab-aligned, but some of the names got long and stick out a bit. i'd be fine rounding up to the next tab so everything is aligned again. | 15:27 |
| mordred | corvus: nod. so the intent was tab-aligned | 15:27 |
| corvus | mordred: i think uploading a base image is fine. i also like the idea of auto-building them via some method in general. but i haven't looked at your patches yet. | 15:29 |
| *** gfidente has joined #openstack-infra | 15:29 | |
| mordred | corvus: cool. so - for now I'm booting the gitea servers using a manually built and uploaded image - but the image was built using the same elements as the proposed patches, so if we decide to move forward with that they shouldn't be terribly divergent | 15:30 |
| *** pgaxatte has quit IRC | 15:33 | |
| *** kashyap has joined #openstack-infra | 15:36 | |
| corvus | mordred: re boot from volume -- is there not an argument to tell it to auto delete volumes when the server is deleted? | 15:39 |
| corvus | mordred: (i thought both that there was, and we were using it) | 15:40 |
| mordred | corvus: no - there is for ips, and there is a way to tell openstack when you _create_ a root volume via boot-from-volume to delete the volume when the server is deleted | 15:40 |
| *** yboaron_ has quit IRC | 15:40 | |
| mordred | but we don't set that flag because it would mean we wouldn't be able to delete the server keeping the root volume and boot a new server using it as a root volume | 15:41 |
| *** yboaron_ has joined #openstack-infra | 15:41 | |
| corvus | oh. i don't recall ever doing that, but i could see that being useful to "resize" a server... | 15:42 |
| *** jamesmcarthur has joined #openstack-infra | 15:42 | |
| mordred | yeah | 15:42 |
| corvus | mordred: i'm +2 on your first 2 patches | 15:42 |
| mordred | \o/ | 15:43 |
| *** jamesmcarthur has quit IRC | 15:46 | |
| *** armax has joined #openstack-infra | 15:46 | |
| *** jamesmcarthur has joined #openstack-infra | 15:47 | |
| *** markvoelker has joined #openstack-infra | 15:47 | |
| *** janki has quit IRC | 15:52 | |
| clarkb | I'm not opposed to building our own images though we may not want to use the nodepool builders for that? Also some have recently suggested we move away from custom images entirely which would be complicated by doubling down on building our own images for the control plane (so we should consider if going back to upstream images for testing is actually something to consider) | 15:54 |
| *** dpawlik has quit IRC | 15:55 | |
| corvus | clarkb: heh, i'm in favor of *simplifying* our nodepool images, but i'm not in favor of going to upstream images. we've been there. we wrote an entire program because of how bad that was. | 15:56 |
| clarkb | Another thing to consider if we use glean in the control plane is that we only support static ipv6 on debuntu and gentoo iirc. So there will be work there to get non debuntu control plane servers running in various regions with ipv6 | 15:56 |
| corvus | in fact, i mean, we're talking about using that program now because upstream images don't even work for a *regular server* much less a ci system :) | 15:56 |
| clarkb | corvus: ya thats sort of where I sit today too. Also we publish our custom images to make it easier for people to reconsume them for local reproduction (which is the case that I think started the discussion from dmsimard) | 15:56 |
| *** ykarel is now known as ykarel|away | 15:58 | |
| mordred | clarkb: we don't really have any need of non-debuntu servers in control plane currently, do we? | 15:59 |
| mordred | clarkb: and if we did, updating glean to properly support them wouldn't be terrible | 15:59 |
| *** jistr|call is now known as jistr | 16:00 | |
| mordred | also - I am also not in favor of ditching custom images. and by not in favor, I mean categorically opposed | 16:00 |
| clarkb | mordred: we do not, just pointing out the corner cases of the proposed plan | 16:00 |
| mordred | totes | 16:00 |
| *** armstrong has joined #openstack-infra | 16:00 | |
| mordred | (I also agree that simplifying our nodepool images is not a bad idea) | 16:01 |
| corvus | mordred: i'm +1 on the nodepool plan, but sounds like we should wait for others to weigh in, and maybe consider alternatives if there are concerns. but meanwhile, i left comments on how to fix the test job. | 16:02 |
| mordred | corvus: sweet.I will read them | 16:03 |
| *** rfolco|afk is now known as rfolco|rover | 16:10 | |
| *** udesale has quit IRC | 16:13 | |
| *** yamamoto_ has joined #openstack-infra | 16:15 | |
| clarkb | I'm catching up on email (in particular the reorg test jobs stuff) but will review changes after | 16:15 |
| *** e0ne has quit IRC | 16:16 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum, dnf and zypper atoms for platforms https://review.openstack.org/639951 | 16:17 |
| *** yamamoto has quit IRC | 16:17 | |
| *** dave-mccowan has quit IRC | 16:17 | |
| *** SpamapS has quit IRC | 16:18 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/bindep master: Adds yum, dnf and zypper atoms for platforms https://review.openstack.org/639951 | 16:18 |
| *** bhavikdbavishi has quit IRC | 16:18 | |
| *** bhavikdbavishi has joined #openstack-infra | 16:19 | |
| *** wolverineav has joined #openstack-infra | 16:19 | |
| *** SpamapS has joined #openstack-infra | 16:19 | |
| *** quiquell is now known as quiquell|off | 16:20 | |
| *** markvoelker has quit IRC | 16:21 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Update dns message to make sense for opendev https://review.openstack.org/640026 | 16:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Clean up boot-from-volume volumes on error https://review.openstack.org/640054 | 16:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add control plane clouds to nodepool builder clouds.yaml https://review.openstack.org/640027 | 16:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Change the default for config-drive to True https://review.openstack.org/640055 | 16:22 |
| mordred | corvus: ^^ I had to fix a line in the first patch - then I fixed your review comments on the third | 16:22 |
| *** e0ne has joined #openstack-infra | 16:23 | |
| *** gfidente has quit IRC | 16:23 | |
| *** quiquell|off is now known as quique | 16:23 | |
| *** quique is now known as quiquell|off | 16:23 | |
| *** wolverineav has quit IRC | 16:23 | |
| openstackgerrit | Ivoline Ngong proposed openstack-infra/storyboard master: Updated_at content modified https://review.openstack.org/638840 | 16:26 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Use publish-to-pypi for instack-undercloud https://review.openstack.org/638459 | 16:27 |
| *** IvensZambrano has joined #openstack-infra | 16:27 | |
| *** psachin has quit IRC | 16:28 | |
| *** dpawlik has joined #openstack-infra | 16:30 | |
| clarkb | I have removed ze08 from the ansible emergency file | 16:30 |
| clarkb | once caught up on mordred's changes I'm going to try and do a cleaning pass of our ansible vars/hieradata since I've got a bunch of that context paged in after doign server rebuilds | 16:30 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Use publish-to-pypi for instack-undercloud https://review.openstack.org/638459 | 16:31 |
| corvus | clarkb: ooooh nice | 16:31 |
| * mordred supports this activity from clarkb | 16:32 | |
| *** chandankumar is now known as raukadah | 16:34 | |
| *** dpawlik has quit IRC | 16:34 | |
| corvus | i'm launching a zuul-preview server | 16:35 |
| corvus | (in rax) | 16:35 |
| *** sthussey has joined #openstack-infra | 16:35 | |
| *** hamzy has quit IRC | 16:40 | |
| clarkb | mordred: comment on https://review.openstack.org/#/c/640026/2 | 16:41 |
| mordred | clarkb: hrm. that's a good question. will those rdns command work in rax for a domain not hosted in their dns? | 16:43 |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: Adds task title to new story modal https://review.openstack.org/639962 | 16:43 |
| clarkb | mordred: for reverse dns, yes | 16:44 |
| mordred | clarkb: kk. yeah - let's do that - I'll do a followup patch | 16:44 |
| fungi | i need to disappear for lunch now that all my meetings have wrapped up, but will be back in an hour or so | 16:45 |
| *** bhavikdbavishi1 has joined #openstack-infra | 16:45 | |
| *** bhavikdbavishi has quit IRC | 16:46 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 16:46 | |
| corvus | mordred, clarkb: what should we call the zuul preview server? "preview.opendev.org" ? | 16:46 |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: Adds task title to new story modal https://review.openstack.org/639962 | 16:47 |
| clarkb | corvus: preview might be too generic? aritfact-preview.opendev.org ? | 16:47 |
| corvus | remember that the visible hostnames will be site.75031cad206c4014ad7a3387091d15ab.openstack.preview.opendev.org | 16:48 |
| corvus | so: site.75031cad206c4014ad7a3387091d15ab.openstack.artifact-preview.opendev.org | 16:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Print reverse dns instructions for rackspace servers https://review.openstack.org/640094 | 16:49 |
| corvus | could also call it "zuul-preview". that might be better marketing. :) | 16:49 |
| clarkb | I'd be fine with that. Mostly thinking about how status.openstack.org caused confusion in dns the other day | 16:49 |
| clarkb | granted we'll have review now so less of a concern | 16:49 |
| clarkb | mordred: I also left a note that config drive implies no live migration in older clouds. I don't think this is a big issue for us | 16:50 |
| clarkb | mordred: but may be unexpected side effect of that chagne for some | 16:51 |
| *** lujinluo has joined #openstack-infra | 16:51 | |
| mordred | clarkb: I don't expect other people who are not us to be using launch-node ... are they? | 16:51 |
| clarkb | mordred: no I think its mostly for our info | 16:52 |
| mordred | coolio | 16:52 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zone-opendev.org master: Add gitea servers and load balancer https://review.openstack.org/640095 | 16:52 |
| mordred | clarkb, corvus: ^^ done booting gitea servers and lb | 16:52 |
| *** helenafm has quit IRC | 16:53 | |
| corvus | mordred: cool, we'll need one more change to add the gitea servers to the load balancer config; there should be a placeholder for them in system-config hostvars | 16:53 |
| clarkb | infra-root I'm taking the hieradata "lock" now | 16:54 |
| *** dtantsur is now known as dtantsur|afk | 16:55 | |
| mordred | corvus: yup. and also to add the servers to the inventory. that change is on its way | 16:55 |
| corvus | oh yeah, i forgot that for zp01. :) | 16:55 |
| mordred | you know - we shoudl update launchnode to include a yaml snippet | 16:55 |
| mordred | when it prints things | 16:55 |
| openstackgerrit | Merged openstack-infra/zuul master: Request dependent job nodes from paused parent provider https://review.openstack.org/639467 | 16:56 |
| *** lujinluo has quit IRC | 16:56 | |
| corvus | i thought someone did that? | 16:56 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack-infra/zuul-jobs master: Add tag in upload image name https://review.openstack.org/640096 | 16:56 |
| evrardjp | corvus: I am back | 16:56 |
| evrardjp | Do you think https://review.openstack.org/640096 is a reasonable approach ? | 16:57 |
| evrardjp | not tested | 16:57 |
| corvus | clarkb, fungi: i thought someone wrote a change to launch-node to spit out an inventory blob but i can't find it. is that ringing a bell? | 16:57 |
| corvus | evrardjp: https://review.openstack.org/639779 | 16:58 |
| clarkb | corvus: I think it was mordred? | 16:58 |
| clarkb | corvus: I think it was a script that would write out a full inventory not just the delta | 16:58 |
| evrardjp | corvus: haha sorry | 16:58 |
| corvus | evrardjp: np, and thanks :) | 16:59 |
| *** betherly has quit IRC | 16:59 | |
| *** electrofelix has quit IRC | 16:59 | |
| corvus | clarkb, mordred: huh. i can't find any evidence of that. must be a glitch in the matrix. | 17:00 |
| evrardjp | corvus: different approach I see | 17:00 |
| *** ginopc has quit IRC | 17:02 | |
| openstackgerrit | Merged openstack-infra/zone-opendev.org master: Add gitea servers and load balancer https://review.openstack.org/640095 | 17:02 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add gitea servers to inventory and load-balancer hostvars https://review.openstack.org/640100 | 17:02 |
| corvus | i unwisely decided to align the tabs right before mordred proposed his zone change :) | 17:04 |
| mordred | corvus, clarkb: it's worth noting - given how we do hostnames now - and how we're doing dns ... we could stop putting the ip addresses into our inventory | 17:04 |
| mordred | it really COULD just be gitea01.opendev.org: | 17:04 |
| mordred | with no additional info | 17:04 |
| *** e0ne has quit IRC | 17:05 | |
| clarkb | mordred: we actually use the ip addr as the connection iirc | 17:05 |
| clarkb | mordred: which is nice beause you can bootstrap things before changing IPs | 17:05 |
| clarkb | er changing dns | 17:05 |
| mordred | good point | 17:05 |
| mordred | kk. that's a good reason to keep it then :) | 17:05 |
| clarkb | I think if we continue to support that use case too then I would be fine with simplifying in general | 17:05 |
| openstackgerrit | James E. Blair proposed openstack-infra/zone-opendev.org master: Add zuul-preview https://review.openstack.org/640102 | 17:06 |
| corvus | mordred: did you mean to put tabs in gitea-lb.yaml? | 17:06 |
| corvus | mordred: of course not -- yaml forbids tabs! | 17:07 |
| mordred | ugh | 17:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add gitea servers to inventory and load-balancer hostvars https://review.openstack.org/640100 | 17:08 |
| mordred | corvus: silly me :) | 17:08 |
| clarkb | fungi: ttx: odsreg.openstack.org is not a service/server anymore right? | 17:09 |
| openstackgerrit | Merged openstack-infra/zuul master: Fix multiple jobs with provides/requires https://review.openstack.org/639472 | 17:09 |
| *** ykarel|away has quit IRC | 17:09 | |
| clarkb | There is no DNS A record for it and I would like ot clean out its hieradata file | 17:09 |
| clarkb | similarlry pholio01.openstack.org does not exist right? | 17:09 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Encode zuul.message with base64 https://review.openstack.org/638801 | 17:09 |
| clarkb | infra-root ^ | 17:09 |
| mordred | clarkb: I believe pholio01.openstack.org is not a thing anymore | 17:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add zp01 to inventory https://review.openstack.org/640111 | 17:10 |
| clarkb | and proposal.slave.openstack.org has been replaced with zuul secrets | 17:11 |
| clarkb | this is fun doing spring cleaning | 17:11 |
| clarkb | Delete all the things | 17:11 |
| corvus | mordred: ah crap, sorry, i led you astray. one more thing to fix in 640100 | 17:12 |
| mordred | clarkb: so much delete. many many things | 17:12 |
| mordred | corvus: on it | 17:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add gitea servers to inventory and load-balancer hostvars https://review.openstack.org/640100 | 17:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add zp01 to inventory https://review.openstack.org/640111 | 17:13 |
| mordred | corvus: ^^ - and rebased yours | 17:13 |
| corvus | thx | 17:13 |
| mordred | corvus, clarkb: feel like revoting on https://review.openstack.org/#/c/640026/2 (and or voting for the first time?) | 17:14 |
| corvus | clarkb: https://review.openstack.org/640102 https://review.openstack.org/640100 and https://review.openstack.org/640111 are ready for you now :) | 17:14 |
| mordred | corvus: https://review.openstack.org/#/c/640054/2 had a zuul sad | 17:14 |
| *** dpawlik has joined #openstack-infra | 17:15 | |
| corvus | mordred: re 640026, you gonna do the reverse dns fix? | 17:15 |
| corvus | i think clarkb is right, we still want to run those | 17:15 |
| clarkb | corvus: there was a followup change for that | 17:16 |
| mordred | corvus: yes - I did a followup at the end of the stack | 17:16 |
| mordred | although I guess I could just squash them | 17:16 |
| mordred | corvus: ImportError: No module named 'six'\n" was the zuul sad | 17:16 |
| *** luizbag has quit IRC | 17:16 | |
| mordred | was that maybe just an unlucky blip of something updating? | 17:16 |
| *** luizbag has joined #openstack-infra | 17:16 | |
| clarkb | I've just rechecked https://review.openstack.org/#/c/639740/1 since cryptography has wheels again. That change adds secrets editing script to bridge so that we don't have the gpg-agent problem (as much) | 17:17 |
| *** kopecmartin is now known as kopecmartin|off | 17:17 | |
| * clarkb goes back to cleaning out hieradata | 17:17 | |
| corvus | mordred: yeah, seems likely. maybe an ansible release :) | 17:17 |
| openstackgerrit | Alex Litvinov proposed openstack-infra/project-config master: add upstream reference https://review.openstack.org/640118 | 17:17 |
| mordred | \o/ | 17:17 |
| corvus | no ansible release. maybe a zuul install. weird. doesn't usually do that. | 17:18 |
| mordred | yeah | 17:18 |
| *** markvoelker has joined #openstack-infra | 17:18 | |
| clarkb | fungi: ttx similarly is design-summit-prep.openstack.org replaced by the forum CFP thing? | 17:18 |
| mordred | I rechecked. we can keep our eyes out | 17:18 |
| corvus | mordred: +2s on the stack | 17:19 |
| corvus | mordred: though... tbh... i kinda think you should squash it | 17:19 |
| *** dpawlik has quit IRC | 17:19 | |
| mordred | corvus: ok. will do real qu-ick | 17:19 |
| corvus | mordred: it's a regression if the first change lands and it takes us a while to get the middle of the stack in | 17:19 |
| corvus | (and since the middle needs rechecking anyway) | 17:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Update dns message to make sense for opendev https://review.openstack.org/640026 | 17:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Clean up boot-from-volume volumes on error https://review.openstack.org/640054 | 17:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add control plane clouds to nodepool builder clouds.yaml https://review.openstack.org/640027 | 17:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Change the default for config-drive to True https://review.openstack.org/640055 | 17:20 |
| mordred | clarkb, corvus: ^^ there's the squished version | 17:21 |
| openstackgerrit | Alex Litvinov proposed openstack-infra/project-config master: add cinder-backup-swift charm and interface https://review.openstack.org/639831 | 17:21 |
| *** ccamacho has quit IRC | 17:22 | |
| *** gfidente has joined #openstack-infra | 17:22 | |
| corvus | mordred: on 640027 you're still missing the files i mentioned in the commit message comment on ps 2 | 17:23 |
| clarkb | fungi: can I remove group_vars/signing.yaml ? git grep for signing in system-config doesn't show that used in an obvious way | 17:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Print yaml inventory instructions https://review.openstack.org/640126 | 17:26 |
| mordred | corvus: really? crappit | 17:27 |
| *** jamesmcarthur has quit IRC | 17:27 | |
| *** wolverineav has joined #openstack-infra | 17:27 | |
| clarkb | infra-root double check me on this. I'm going to delete host_vars/review.openstack.org whcih has the same content as group_vars/review.yaml because gerrit currently runs on review01.openstack.org which doesn't match host_vars/review.openstack.org.yaml | 17:27 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add control plane clouds to nodepool builder clouds.yaml https://review.openstack.org/640027 | 17:28 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Change the default for config-drive to True https://review.openstack.org/640055 | 17:28 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Print yaml inventory instructions https://review.openstack.org/640126 | 17:28 |
| mordred | corvus: maybe this time I've learned how to use git add | 17:28 |
| mordred | clarkb: yes. I believe that is correct | 17:29 |
| *** wolverineav has quit IRC | 17:31 | |
| clarkb | ok I'm making that change now | 17:32 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add control plane clouds to nodepool builder clouds.yaml https://review.openstack.org/640027 | 17:36 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Change the default for config-drive to True https://review.openstack.org/640055 | 17:36 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Print yaml inventory instructions https://review.openstack.org/640126 | 17:36 |
| corvus | mordred: see what i did in the last patchset on 640027 | 17:37 |
| *** jamesmcarthur has joined #openstack-infra | 17:37 | |
| *** wolverineav has joined #openstack-infra | 17:37 | |
| clarkb | fungi: ttx: ok I think I've completed by hieradata spring cleaning except for group_vars/signing.yaml and host_vars/design-summit-prep.openstack.org.yaml and host_vars/odsreg.openstack.org.yaml. IF we can confirm some or all of those can be removed I will clean them up | 17:37 |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: removes # for cards in automatic worklists https://review.openstack.org/640128 | 17:38 |
| clarkb | infra-root I tried to be organized in my cleanup so that any chagnes can be easily reverted without bringin in a bunch of other cleanups should something go unexecptedly wrong | 17:38 |
| *** hwoarang has quit IRC | 17:38 | |
| *** hwoarang has joined #openstack-infra | 17:39 | |
| ttx | clarkb: I think it's safe to drop them now. Forum org moved to the CFP tooling and that worked out pretty well all considered | 17:41 |
| *** panda|ruck is now known as panda|ruck|off | 17:42 | |
| clarkb | ttx: thanks | 17:43 |
| *** gyee has joined #openstack-infra | 17:44 | |
| *** jamesmcarthur has quit IRC | 17:45 | |
| *** lujinluo has joined #openstack-infra | 17:46 | |
| *** derekh has quit IRC | 17:46 | |
| tosky | from time to time, when I look at jobs in my gerrit dashboard, I see jobs whose update time does not match the last recorded event when I open that review | 17:47 |
| tosky | for example, right now the "Updated" time in the table for this job: https://review.openstack.org/639774 is 6:23 PM | 17:48 |
| *** jamesmcarthur has joined #openstack-infra | 17:48 | |
| corvus | tosky: when you say "job" do you mean "change"? | 17:48 |
| tosky | corvus: ups, sorry; yes, s/job/change/ or s/job/review/ | 17:49 |
| tosky | but when I open the review, the last event happened at 4:06 PM | 17:49 |
| clarkb | tosky: my hunch is that gets updated when gerrit does background things like finding a new conflicts with change | 17:49 |
| corvus | well, let's be really specific about terms here, because leaving a "review" on a "change" updates its last updated time, so it could be important. :) | 17:49 |
| tosky | corvus: you are totally right, my bad | 17:49 |
| corvus | but anyway, i think i understand the question now :) | 17:50 |
| clarkb | tosky: so nothing has been updated in your change, the nsomeone else pushes another change which conflicts with your change. Gerrit updates both sides of that relationship but there is no review commit to go along with it | 17:50 |
| *** bhavikdbavishi has quit IRC | 17:50 | |
| tosky | but is this something that has been going on for a while, or a new behavior? I think I noticed only recently, but maybe I didn't pay attention :) | 17:50 |
| *** markvoelker has quit IRC | 17:51 | |
| corvus | when was tom barron added as a reviewer? | 17:51 |
| tosky | oh, good question | 17:51 |
| tosky | I don't know, I didn't receive a notification for that | 17:51 |
| tosky | so that's probably the event | 17:51 |
| *** bhavikdbavishi has joined #openstack-infra | 17:52 | |
| corvus | yeah, i think i've seen newer gerrit record that event, but i think ours doesn't? but it still might update the timestamp? | 17:52 |
| *** jamesmcarthur has quit IRC | 17:52 | |
| *** gfidente has quit IRC | 17:53 | |
| *** jamesmcarthur has joined #openstack-infra | 17:53 | |
| tosky | there was another review this morning where I'm sure I didn't see any additional reviewers added, but it was updated since then, so it can't be used to check | 17:53 |
| tosky | anyway, not really important, I was just confused because the default ordering is by timestamp and I was weird for me to see reviews moving up without any apparent change | 17:54 |
| tosky | now I know that there are silent events | 17:54 |
| openstackgerrit | Merged openstack-infra/zuul master: Fix missing semaphore release around dequeue https://review.openstack.org/639023 | 17:54 |
| corvus | tosky: thanks, i hadn't noticed either :) | 17:55 |
| *** luizbag has quit IRC | 17:56 | |
| *** gfidente has joined #openstack-infra | 17:56 | |
| *** yboaron_ has quit IRC | 17:57 | |
| *** yboaron_ has joined #openstack-infra | 17:57 | |
| *** lujinluo has quit IRC | 17:59 | |
| openstackgerrit | Merged openstack-infra/zuul master: Fix rare semaphore leak during reconfiguration https://review.openstack.org/639118 | 17:59 |
| *** trown is now known as trown|lunch | 18:03 | |
| *** jmorgan1 has quit IRC | 18:04 | |
| openstackgerrit | Merged openstack-infra/zuul master: Encode zuul.message with base64 https://review.openstack.org/638801 | 18:06 |
| *** rossella_ has quit IRC | 18:07 | |
| *** bhavikdbavishi has quit IRC | 18:09 | |
| *** aojea has quit IRC | 18:10 | |
| *** jpena is now known as jpena|off | 18:12 | |
| *** hamzy has joined #openstack-infra | 18:13 | |
| *** ykarel|away has joined #openstack-infra | 18:16 | |
| openstackgerrit | Merged openstack-infra/system-config master: Update dns message to make sense for opendev https://review.openstack.org/640026 | 18:16 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Print yaml inventory instructions https://review.openstack.org/640126 | 18:21 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add control plane clouds to nodepool builder clouds.yaml https://review.openstack.org/640027 | 18:21 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Change the default for config-drive to True https://review.openstack.org/640055 | 18:21 |
| mordred | clarkb: can has re-review on ^^ 640027 (missed adding file to the linter script) and review on https://review.openstack.org/#/c/640100/ ? | 18:22 |
| *** bhavikdbavishi has joined #openstack-infra | 18:22 | |
| mordred | corvus: (you too on re-review) | 18:22 |
| clarkb | mordred: the cloud names are different in control_plane_cloudns and the nodepool clouds group files. I *think* this is ok because we'll just load the bits out of both when we need them in the nodepool builder ndoes? | 18:26 |
| clarkb | they aren't exclusive right? | 18:27 |
| pabelanger | mordred: re: launch-node, I've been trying to think of a better way of managing out SSH host keys on bridge, what are you thoughts on maybe adding the ssh-ed25519 host key for each host, as an inventory host variable, then populating SSH known_hosts file from it? This is to work around step we need to manually accept the hostkey on bridge for each new server | 18:27 |
| pabelanger | s/out/our | 18:28 |
| mordred | clarkb: that's right. I think we could also just use the clouds. dict in the builder file - it has all of the things in it | 18:28 |
| *** ramishra_ has quit IRC | 18:29 | |
| *** wolverineav has quit IRC | 18:30 | |
| *** wolverineav has joined #openstack-infra | 18:30 | |
| clarkb | mordred: +2. Haven't approved as I think we should consider if we want a dedicate builder for this, but looks to be put together properly if we reuse the builders | 18:32 |
| *** bhavikdbavishi has quit IRC | 18:32 | |
| mordred | clarkb: sweet | 18:32 |
| clarkb | pabelanger: I think a simpler option is to just have launch-node accept the key for root | 18:33 |
| *** gfidente has quit IRC | 18:33 | |
| clarkb | we are already running the script as root so this may automagicvally be happening | 18:33 |
| *** wolverineav has quit IRC | 18:33 | |
| *** wolverineav has joined #openstack-infra | 18:33 | |
| IvensZambrano | AJaeger: ping | 18:35 |
| *** jamesmcarthur has quit IRC | 18:37 | |
| mordred | clarkb, pabelanger: it's already happening best I can tell | 18:37 |
| mordred | although - it could be improved so that we don't accept the ssh hostkey for real until the script is successful - accepting the hostkey, having the server boot, fail, then later booting a different server that gets the same ip is ... fun | 18:38 |
| mordred | but mostly only a problem when iterating on launch-node issues | 18:38 |
| *** rh-jelabarre has quit IRC | 18:40 | |
| *** rh-jelabarre has joined #openstack-infra | 18:41 | |
| *** sdake has joined #openstack-infra | 18:45 | |
| *** markvoelker has joined #openstack-infra | 18:47 | |
| *** armstrong has quit IRC | 18:50 | |
| *** e0ne has joined #openstack-infra | 18:51 | |
| zbr | AJaeger: pabelanger : please have a look at https://review.openstack.org/#/c/639951/ - thanks. | 18:54 |
| fungi | time to find out what i missed in here' | 18:55 |
| fungi | that's rather more scrollback than i expected! | 18:55 |
| *** luizbag has joined #openstack-infra | 18:57 | |
| fungi | clarkb: yes i believe odsreg, pholio01 and proposal.slave can all go away | 18:58 |
| fungi | clarkb: also yes to cleaning up design-summit-prep | 18:59 |
| fungi | clarkb: i may be mistaken, but i think i spotted a phabricator in dns earlier this week too | 18:59 |
| fungi | clarkb: and yes, group_vars/signing.yaml can be deleted | 18:59 |
| fungi | we have a zuul job for all of those bits now | 19:00 |
| fungi | and i also agree host_vars/review.openstack.org should be safe to delete | 19:00 |
| *** ykarel|away has quit IRC | 19:01 | |
| *** luizbag_ has joined #openstack-infra | 19:02 | |
| *** e0ne has quit IRC | 19:03 | |
| mordred | fungi: wanna review https://review.openstack.org/#/c/640100/ ? | 19:03 |
| *** luizbag has quit IRC | 19:04 | |
| mordred | (and the one after it too) | 19:04 |
| *** IvensZambrano has quit IRC | 19:05 | |
| *** kgiusti has quit IRC | 19:05 | |
| openstackgerrit | Merged openstack-infra/system-config master: Clean up boot-from-volume volumes on error https://review.openstack.org/640054 | 19:05 |
| *** ijw has joined #openstack-infra | 19:06 | |
| *** iurygregory has quit IRC | 19:08 | |
| fungi | mordred: oh, you know i always want to review your changes, don't you? ;) | 19:09 |
| mordred | fungi: oh I do. I Do! | 19:09 |
| fungi | fire in the hole | 19:10 |
| mordred | fungi: wcpgw? | 19:12 |
| fungi | not much since it's not a production service yet anyway | 19:12 |
| mordred | good point | 19:12 |
| fungi | my main review criteria for system-config changes is "can this break anything we care about?" | 19:12 |
| fungi | (i mean, i review for more than _just_ that, but...) | 19:13 |
| mordred | fungi: I just review for "has clarkb already +2d it" | 19:14 |
| * mordred hides | 19:14 | |
| *** pcaruana has quit IRC | 19:14 | |
| fungi | well, to be fair, that's on my list of criteria too ;) | 19:15 |
| *** jtomasek has quit IRC | 19:16 | |
| *** dpawlik has joined #openstack-infra | 19:16 | |
| clarkb | fungi: ok signing has been removed | 19:18 |
| clarkb | I think I've cleaned up everything that can be cleaned up now | 19:19 |
| *** dpawlik has quit IRC | 19:20 | |
| *** whoami-rajat has quit IRC | 19:20 | |
| *** markvoelker has quit IRC | 19:21 | |
| openstackgerrit | Merged openstack-infra/system-config master: Print yaml inventory instructions https://review.openstack.org/640126 | 19:21 |
| *** trown|lunch is now known as trown | 19:22 | |
| mordred | infra-root: I've been up and working since 4am local time (for no good reason) - so I'm going to step away for a bit | 19:22 |
| fungi | clarkb: thanks! | 19:28 |
| fungi | mordred: enjoy! | 19:28 |
| *** ijw has quit IRC | 19:35 | |
| *** wolverineav has quit IRC | 19:39 | |
| *** wolverineav has joined #openstack-infra | 19:40 | |
| mriedem | silly question, but if a job only runs on nova changes, is there value in it being in the gate queue? | 19:41 |
| mriedem | or just voting + check? | 19:41 |
| mriedem | multiple nova changes get merged in the gate queue and would run the job right? | 19:41 |
| mriedem | so value+ ? | 19:41 |
| clarkb | mriedem: the value is in checking the merge results yes | 19:42 |
| mriedem | ok i thought so | 19:42 |
| clarkb | mriedem: basically it ensures that the order we've ended up in still works | 19:42 |
| mriedem | i don't have to think about this very often... | 19:42 |
| fungi | the risk is that if the change passed that job a month ago but the state of the repo has changed since such that a recheck would show that job broken for that change, then approving it would allow it to merge without a more recent exercise of that job and render the job broken for all other changes benig checked | 19:43 |
| fungi | or one of the risks anyway | 19:43 |
| mriedem | gate queue it is | 19:43 |
| fungi | more generally, the main risk of running jobs only in check and not in gate is that it becomes much easier to merge changes which wedge that check job | 19:44 |
| fungi | because you only know that it passed that job some time before and in relative isolation | 19:44 |
| fungi | not that it actually still passes that job at the time it merges | 19:44 |
| fungi | or rather, represents a state of the repository which will be capable of passing that job | 19:45 |
| *** wolverineav has quit IRC | 19:48 | |
| *** wolverineav has joined #openstack-infra | 19:48 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add gitea servers to inventory and load-balancer hostvars https://review.openstack.org/640100 | 19:49 |
| openstackgerrit | Merged openstack-infra/system-config master: Add zp01 to inventory https://review.openstack.org/640111 | 19:49 |
| *** dpawlik has joined #openstack-infra | 19:50 | |
| *** ijw has joined #openstack-infra | 19:51 | |
| clarkb | infra-root can I get reviews on https://review.openstack.org/#/c/639740/ to add a script to edit the secrest file? It passes tests now | 19:51 |
| *** wolverineav has quit IRC | 19:53 | |
| *** dpawlik has quit IRC | 19:54 | |
| *** ijw has quit IRC | 19:56 | |
| *** jamesmcarthur has joined #openstack-infra | 19:56 | |
| fungi | i'll let someone who uses emacs for that review it. i still just back up the original, decrypt to a mode 0600 file, edit that quickly while confirming nobody else is making changes to the list, encrypt it and then shred and delete the plaintext copy | 19:58 |
| fungi | i haven't actually looked at emacs's pgp mode to see what it does under the covers | 19:59 |
| clarkb | fungi: the thing that convinced me to let emacs do it for me is it decrypts in memory only | 20:01 |
| clarkb | whcih there are vim plugins for but I figured emacs already does it so good enough | 20:01 |
| fungi | that's an excellent feature. granted it's still no guarantee it won't be written to disk unless you disable or encrypt swap for the server | 20:02 |
| clarkb | you can also mlockall though I have no idea if emacs does that for this use case | 20:03 |
| corvus | emacs will also tell you if someone is editing it at the same time, or if it changed out from under you. | 20:05 |
| fungi | also convenient. maybe it's time i learned to ctrl-x | 20:05 |
| *** sdake has quit IRC | 20:07 | |
| corvus | switching between openstack, docker, k8s is starting to muddle a few things in my mind... i just typed "docker show foo" and "docker examine foo" before i finally hit on "docker inspect foo" | 20:09 |
| fungi | can't we get a unified command-line client? | 20:09 |
| corvus | fungi: yes. we'll have one written in go. and python. and rust. | 20:10 |
| *** jtomasek has joined #openstack-infra | 20:10 | |
| fungi | wait, can't we get a unified programming language? | 20:11 |
| clarkb | ingy.net wrote one of those in yaml | 20:11 |
| clarkb | (I'm not even joking, compiles your yaml to js/ruby/perl/python) | 20:12 |
| fungi | i wrote one on a bar napkin but promptly lost it. in retrospect, it was likely crummy and full of bugs | 20:12 |
| *** luizbag_ has quit IRC | 20:13 | |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: removes # for cards in automatic worklists https://review.openstack.org/640128 | 20:15 |
| *** e0ne has joined #openstack-infra | 20:16 | |
| *** wolverineav has joined #openstack-infra | 20:19 | |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: removes # for cards in automatic worklists https://review.openstack.org/640128 | 20:22 |
| *** wolverineav has quit IRC | 20:24 | |
| fungi | infra-root: i've generated a new openstack release signing key for use during the upcoming train development cycle: https://sks-keyservers.net/pks/lookup?op=vindex&search=0xcdc08088c3cb45a9be08332b2354069e5b504663&fingerprint=on please follow our attestation directions at your earliest convenience: https://docs.openstack.org/infra/system-config/signing.html#attestation | 20:25 |
| ianw | while we're infra-rooting, could i get some eyes on the graphite puppet fixes in https://review.openstack.org/#/c/639877/ | 20:30 |
| *** wolverineav has joined #openstack-infra | 20:30 | |
| *** wolverineav has quit IRC | 20:37 | |
| *** wolverineav has joined #openstack-infra | 20:38 | |
| *** wolverineav has quit IRC | 20:38 | |
| *** wolverineav has joined #openstack-infra | 20:38 | |
| *** hamzy has quit IRC | 20:39 | |
| *** hamzy has joined #openstack-infra | 20:43 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use publish-to-pypi for instack-undercloud https://review.openstack.org/638459 | 20:45 |
| *** kgiusti has joined #openstack-infra | 20:51 | |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: allows editing for any length of story title https://review.openstack.org/640187 | 20:52 |
| *** kgiusti has quit IRC | 20:58 | |
| *** sdake has joined #openstack-infra | 21:02 | |
| *** dave-mccowan has joined #openstack-infra | 21:10 | |
| *** gondoi has joined #openstack-infra | 21:14 | |
| *** kgiusti has joined #openstack-infra | 21:18 | |
| *** wolverineav has quit IRC | 21:18 | |
| *** e0ne has quit IRC | 21:19 | |
| *** gondoi has left #openstack-infra | 21:19 | |
| *** dave-mccowan has quit IRC | 21:22 | |
| *** sdake has quit IRC | 21:23 | |
| *** kjackal has joined #openstack-infra | 21:23 | |
| *** wolverineav has joined #openstack-infra | 21:25 | |
| *** sdake has joined #openstack-infra | 21:25 | |
| *** wolverineav has quit IRC | 21:28 | |
| *** wolverineav has joined #openstack-infra | 21:28 | |
| *** wolverineav has quit IRC | 21:29 | |
| *** ijw has joined #openstack-infra | 21:32 | |
| clarkb | Oh hey today was the day I said I would merge the letsencrypt spec if there wasn't any major concerns | 21:33 |
| clarkb | if you haven't yet and would like to rereview that spec please do so now. I'll get to that towards the end of my day | 21:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Add zuul user to bridge.openstack.org https://review.openstack.org/604925 | 21:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Manage user ssh keys from urls https://review.openstack.org/604932 | 21:34 |
| clarkb | Rebased the CD prereq changes (adding zuulcd user to bridge) | 21:35 |
| *** wolverineav has joined #openstack-infra | 21:35 | |
| *** kgiusti has quit IRC | 21:36 | |
| *** jtomasek has quit IRC | 21:42 | |
| *** e0ne has joined #openstack-infra | 21:42 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-graphite master: graphite.wsgi : update for django 1.7 https://review.openstack.org/640199 | 21:43 |
| *** kjackal has quit IRC | 21:44 | |
| *** e0ne has quit IRC | 21:44 | |
| *** e0ne has joined #openstack-infra | 21:45 | |
| *** hamzy has quit IRC | 21:47 | |
| *** eernst has joined #openstack-infra | 21:48 | |
| *** markvoelker has joined #openstack-infra | 21:48 | |
| clarkb | ianw: 'READ THIS" ok I guess :) | 21:50 |
| openstackgerrit | Merged openstack-infra/puppet-graphite master: Fix pip ordering and build requirements https://review.openstack.org/639877 | 21:51 |
| *** dpawlik has joined #openstack-infra | 21:51 | |
| *** wolverineav has quit IRC | 21:51 | |
| ianw | clarkb: heh, i would say that i didn't put that there :) | 21:52 |
| *** eernst has quit IRC | 21:52 | |
| ianw | there's still something wrong, and whatever should be on port 2003 isn't on the new server ... debugging | 21:52 |
| clarkb | fwiw we don't ahve that WSGIScriptImport in our vhost but that isn't a new regression | 21:52 |
| *** zul has quit IRC | 21:53 | |
| *** dpawlik has quit IRC | 21:55 | |
| ianw | yeah, not sure what that's about | 21:56 |
| *** slaweq has quit IRC | 21:58 | |
| *** wolverineav has joined #openstack-infra | 22:01 | |
| *** e0ne has quit IRC | 22:01 | |
| *** slaweq has joined #openstack-infra | 22:04 | |
| ianw | can we clone cinder volumes in rax? | 22:05 |
| clarkb | I'm not aware of such a feature | 22:05 |
| fungi | i can't say i've tried | 22:05 |
| fungi | we can definitely snapshot them | 22:05 |
| fungi | and i can't imagine what good that does you if you can't make a new volume from a snapshot | 22:06 |
| ianw | fungi: now i've hit a couple of things and there's a few new versions of django in the way, i'm not thrilled about just detaching and reattaching the stats data | 22:06 |
| fungi | so in a sense you could make a copy of a cinder volume but you may need a lengthy outage to do so | 22:06 |
| ianw | so i think either your rsync idea, or if i can clone them and then rsync that would be even better | 22:07 |
| ianw | oh, i'm not sure i've snapshotted; it's not an atomic op where the parent just continues on? | 22:07 |
| *** wolverineav has quit IRC | 22:09 | |
| *** slaweq has quit IRC | 22:09 | |
| ianw | even still a volume-level snapshot might still be inconsistent given a busy host like statsd. i think this is pointing to a userspace rsync solution | 22:10 |
| clarkb | does whsiper do a merge? | 22:11 |
| clarkb | we might be able to start writing to the new host, then merge in the content of the old host? | 22:11 |
| *** slaweq has joined #openstack-infra | 22:11 | |
| ianw | hrm, maybe, they're pretty flat files, i wouldn't be surprised if you can cat them together | 22:11 |
| ianw | python-whisper : whisper-merge | 22:12 |
| *** rh-jelabarre has quit IRC | 22:12 | |
| *** sdake has quit IRC | 22:13 | |
| *** slaweq has quit IRC | 22:16 | |
| fungi | if you're committed to doing some rsync, i'd just do it all via rsync. it's only 0.45tb and 2.1m inodes in /var/lib/graphite/storage | 22:17 |
| clarkb | oh thats less than I thoguht it was, ya not too bad | 22:18 |
| fungi | the logical volume is 2tb but it's only 22% used | 22:18 |
| fungi | in theory, at least, subsequent rsync after the initial pass or two should result in a fairly brief lag to grab the remaining delta | 22:19 |
| fungi | so we shouldn't need to shut down writing services on it for long | 22:19 |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-graphite master: graphite.wsgi : update for django 1.7 https://review.openstack.org/640199 | 22:19 |
| fungi | plus, i think we need to reload iptables on everything it's polling anyway, right? | 22:20 |
| ianw | i don't think so, it's ports are open for the servers that want to talk to it | 22:20 |
| clarkb | ya it doesn't poll | 22:20 |
| fungi | oh, nm. i was thinking cacti for a sec | 22:21 |
| ianw | i will take the rsync approach, i'll add some volumes and do an initial sync now | 22:21 |
| clarkb | as long as things lookup new dns values we should be fine | 22:21 |
| fungi | ianw: fair warning, i dunno what our cinder quota looks like these days but you may need to hunt for things to clean up first | 22:21 |
| clarkb | (I have no idea if they will do that though) | 22:21 |
| ianw | if everything's in place, i can probably do this on my monday, which is the quietest time | 22:21 |
| ianw | this being cutting over dns and final rsync | 22:21 |
| clarkb | so we may need to restart services, but otherwise should work | 22:21 |
| fungi | ianw: alternative, just do a 1tb volume initially and we can expand it later (but maybe format it for a larger inode percentage just to be safe) | 22:22 |
| *** markvoelker has quit IRC | 22:22 | |
| ianw | recovery of graphite.o.o partitions. noticed during pypi testing | 22:22 |
| ianw | that it had a volume in indeterminite state. rebuild cinder | 22:22 |
| ianw | volumes, resync. in progress | 22:22 |
| fungi | er, nevermind about the inodes. looks like we're at 2% inode utilization on the existing volume anyway | 22:23 |
| ianw | my notes from 2017-08-04 ... maybe the second volume was only in service of fixing it at the time? | 22:23 |
| ianw | graphite.o.o recovery. take different approach of swapping out | 22:24 |
| ianw | failed disk in lvm. cleanup other cinder volumes | 22:24 |
| fungi | well, we extended the logical volume to use both physical volumes | 22:24 |
| fungi | or so it appears | 22:24 |
| fungi | yeah, /dev/xvdc1 and /dev/xvdf1 are both present in the main vg | 22:24 |
| ianw | so i'm not sure if it started at 1tb and got an extra disk during all that for no particular reason | 22:25 |
| ianw | i think probably a single 1tb lvm is fine, can always grow it | 22:25 |
| *** slaweq has joined #openstack-infra | 22:25 | |
| fungi | my thinking as well | 22:25 |
| clarkb | and if you attach it to the old host you can rsync "locally" right? | 22:26 |
| fungi | we won't even be at 50% utilization on a 1tb volume anyway | 22:26 |
| fungi | clarkb: it's still going over their iscsi network which i haven't observed being especially faster | 22:26 |
| fungi | but sure, it's an option | 22:27 |
| fungi | could also use the private addresses instead of public? also no idea if that will be appreciably better throughput though | 22:27 |
| ianw | i think i'll just do it on host to avoid having to move lvm pvs's around etc | 22:27 |
| *** wolverineav has joined #openstack-infra | 22:28 | |
| fungi | yeah, i would prioritize your own time spent fiddling with stuff over time spent waiting for the rsync to complete when you can just work on other things anyway | 22:29 |
| *** wolverineav has quit IRC | 22:29 | |
| *** wolverineav has joined #openstack-infra | 22:29 | |
| fungi | er, that is to say, i favor simplicity over complex solutions to make it happen faster | 22:29 |
| *** dpawlik has joined #openstack-infra | 22:29 | |
| *** slaweq has quit IRC | 22:30 | |
| clarkb | ++ | 22:30 |
| *** ijw has quit IRC | 22:32 | |
| *** ijw has joined #openstack-infra | 22:33 | |
| *** wolverineav has quit IRC | 22:33 | |
| *** dpawlik has quit IRC | 22:34 | |
| *** ijw has quit IRC | 22:35 | |
| *** ijw has joined #openstack-infra | 22:35 | |
| corvus | clarkb, mordred: i found an issue with my gitea patch and made a followup: https://github.com/go-gitea/gitea/pull/6216 -- but with that, my rename playbook works no matter what crazy ordering i throw at it. | 22:35 |
| mordred | corvus: neat~! | 22:35 |
| corvus | clarkb: would you mind +3ing https://review.openstack.org/640102 ? | 22:36 |
| clarkb | corvus: looking | 22:36 |
| clarkb | you can have globs in zone files? how have I never known this | 22:37 |
| clarkb | I mean it makes sense | 22:37 |
| corvus | it's a wildcard | 22:37 |
| corvus | fungi: ^ maybe you want to double check that too, it's our first wildcard entry :) | 22:37 |
| openstackgerrit | Merged openstack-infra/zone-opendev.org master: Add zuul-preview https://review.openstack.org/640102 | 22:37 |
| clarkb | oh I just approve it | 22:37 |
| clarkb | :) | 22:37 |
| mordred | fungi: no doublecheck for you! | 22:38 |
| corvus | there's still plenty of time to revert :) | 22:38 |
| fungi | indeed, i'm looking now | 22:38 |
| fungi | the reindenting makes it a little hard to follow | 22:39 |
| fungi | i guess it's only whitespace changes except for the serial and the last record? | 22:39 |
| clarkb | fungi: last 3 records | 22:40 |
| fungi | yep, i see that now | 22:40 |
| corvus | sorry about that... there was a conversation about tabs and i just wanted to get it out of the way. i think i did drop a couple of ".opendev.org" from the right hand side. | 22:40 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/openstack-zuul-jobs master: Drop legacy-grenade-dsvm-neutron-multinode-live-migration https://review.openstack.org/640209 | 22:40 |
| corvus | graphite and pbx | 22:40 |
| fungi | yep, lgtm | 22:41 |
| fungi | times were i wouldn't advocate for a cname rr to another record in the same zone, but this is sort of a special case and these days nameservers know to provide additional results when they're likely to be iteratively requested otherwise | 22:42 |
| fungi | so it's not actually likely to result in many additional queries, if any | 22:43 |
| fungi | not to mention, we're not in any danger of blowing up these nameservers with requests in the first place | 22:43 |
| clarkb | fungi: ya and the dns over tls is gonna hold a connection open forever anyway | 22:44 |
| fungi | dns over tls? | 22:44 |
| fungi | i've really been out of the loop | 22:44 |
| *** wolverineav has joined #openstack-infra | 22:45 | |
| clarkb | fungi: there is also dns over https but I'm a little annoyed browsers thinkt they should ignore my system settings for dns | 22:45 |
| *** slaweq has joined #openstack-infra | 22:45 | |
| fungi | oh, is dns over tls for securing the last hop between the local resolver and your configured recursive resolver? | 22:46 |
| clarkb | fungi: ya to prevent your isp from snooping or injecting things (though with dnssec its mostly the snooping) | 22:46 |
| clarkb | infra-root I'm not sure if ya'll follow the tc channel but not too long ago a request from the internets came in to hand over the "nimble" package on pypi. Looking at it we renamed our nimble project to mogan due to a name conflict with a contributing company. We only ever published one version 0.0.1 to nimble on pypi. Our nimble was never an official project and renamed anyways so I think we can | 22:47 |
| clarkb | hand it over | 22:47 |
| clarkb | (request was sent to the openstack-discuss list if you want to read it) | 22:47 |
| clarkb | can you all think of a reason not to hand it over? | 22:47 |
| corvus | clarkb: did anyone ask https://pypi.org/user/Zhenguo/ ? | 22:49 |
| *** wolverineav has quit IRC | 22:49 | |
| clarkb | I don't think anyone has yet, but smcginnis asked on the huawei side. | 22:50 |
| clarkb | But maybe we respond to the request and cc zhenguo and see what happens | 22:50 |
| *** slaweq has quit IRC | 22:51 | |
| corvus | my thinking is that since it *wasn't* an official project, we should defer to the original registrant first. but if they are awol after earnest attempts to contact them, i think we can then defer to the tc as the next best placed entity to make a determination of non-interest. | 22:51 |
| clarkb | ok in that case I'll respond to the mailing list request with a cc to zhenguo and see if they can ack/nak | 22:52 |
| fungi | yeah, that sounds fine | 22:53 |
| *** tkajinam has joined #openstack-infra | 22:55 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add gitea to project rename playbook https://review.openstack.org/640218 | 22:56 |
| smcginnis | No real reason to wait. It was nimble, then renamed to mogan, then died. | 22:57 |
| smcginnis | Nimble definitely is not needed anymore. | 22:57 |
| smcginnis | But maybe good to copy zhenguo on the response just as a courtesy I guess. | 22:57 |
| clarkb | smcginnis: ya I'm gonna write an email offering to do the legwork is zhenguo niu responds with an ack and leave an out if we don't get a response in a reasonable amount of time | 22:58 |
| clarkb | generally I think we should follow a process like this so that it isn't ambiguous should it happen again. | 22:58 |
| smcginnis | Sounds like a good plan. | 22:58 |
| mordred | corvus: there isn't any gitea running on the gitea servers | 22:58 |
| mordred | did I miss something in the patches earlier? | 22:58 |
| mordred | corvus: also - should we add admins to the docker group? | 22:59 |
| *** wolverineav has joined #openstack-infra | 23:01 | |
| corvus | mordred: i also didn't see zuul-preview running, but i haven't dug into it yet | 23:01 |
| mordred | corvus: oh! I see a failed=1 in the run_all logs | 23:01 |
| clarkb | mordred: iirc you are strongly recommended to use sudo | 23:01 |
| clarkb | (rather than be in the docker group) | 23:02 |
| clarkb | I think because you get sudo accounting? I dunno if we really care | 23:02 |
| mordred | clarkb: oh - ok. if using sudo is right, totes cool | 23:02 |
| mordred | corvus: fatal: [gitea08.opendev.org]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'gitea_root_db_password' is undefined"} | 23:02 |
| mordred | corvus: fatal: [zp01.opendev.org]: FAILED! => {"changed": false, "module_stderr": "/bin/sh: 1: /usr/bin/python: not found\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 127} | 23:04 |
| mordred | corvus: is the other one | 23:04 |
| clarkb | for zp01 probably need to tell ansible to use python3 | 23:04 |
| *** ijw has quit IRC | 23:06 | |
| corvus | mordred: i think on bridge we need to copy gitea-cluster.yaml to hosts/group_vars/gitea.yaml | 23:06 |
| *** ijw has joined #openstack-infra | 23:06 | |
| corvus | gitea-cluster.yaml is used by the k8s cluster, since it doesn't fit into the normal inventory | 23:07 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/openstack-zuul-jobs master: Drop legacy-grenade-dsvm-neutron-multinode-live-migration https://review.openstack.org/640209 | 23:07 |
| *** wolverineav has quit IRC | 23:07 | |
| corvus | but i tihnk i kept all the variable names the same, so it should be a subset of the original | 23:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Set ansible_python_interpreter for zuul-preview https://review.openstack.org/640221 | 23:08 |
| mordred | corvus, clarkb: I think that should fix zuul-preview | 23:08 |
| *** wolverineav has joined #openstack-infra | 23:08 | |
| *** wolverineav has quit IRC | 23:08 | |
| corvus | mordred: then i'll fix your thing :) | 23:08 |
| clarkb | mordred: +2 | 23:08 |
| *** wolverineav has joined #openstack-infra | 23:08 | |
| mordred | corvus: yay! cause I can't find gitea-cluster ... likely cause I'm daft | 23:09 |
| mordred | oh - hosts/gitea-cluster.yaml | 23:09 |
| corvus | mordred: done. the only variable i dropped was the xtradb backup password | 23:10 |
| mordred | cool | 23:10 |
| *** slaweq has joined #openstack-infra | 23:11 | |
| *** rascasoft has quit IRC | 23:11 | |
| ianw | infra-root: if i could get a couple of eyes on https://review.openstack.org/#/c/640199/ for the graphite django update, once it rolls around graphite01.opendev.org should be reporting the stats it's currently rsyncing | 23:14 |
| *** slaweq has quit IRC | 23:15 | |
| *** ijw has quit IRC | 23:16 | |
| *** ijw has joined #openstack-infra | 23:16 | |
| *** markvoelker has joined #openstack-infra | 23:18 | |
| *** wolverineav has quit IRC | 23:23 | |
| mordred | ianw: +A | 23:24 |
| *** markvoelker has quit IRC | 23:24 | |
| *** markvoelker has joined #openstack-infra | 23:24 | |
| ianw | thanks; probably a good initial target for letsencrypt certs too | 23:25 |
| ianw | i will leave upgrading everything to python3 to future work when it's run in a container ... that's a bridge too far ATM :) | 23:25 |
| openstackgerrit | Ivoline Ngong proposed openstack-infra/storyboard master: Updated_at content modified https://review.openstack.org/638840 | 23:27 |
| *** ijw has quit IRC | 23:29 | |
| *** wolverineav has joined #openstack-infra | 23:29 | |
| *** ijw has joined #openstack-infra | 23:29 | |
| *** ijw has quit IRC | 23:31 | |
| *** ijw has joined #openstack-infra | 23:32 | |
| *** ekultails has quit IRC | 23:38 | |
| *** jamesmcarthur has quit IRC | 23:46 | |
| openstackgerrit | Merged openstack-infra/puppet-graphite master: graphite.wsgi : update for django 1.7 https://review.openstack.org/640199 | 23:50 |
| clarkb | I need to write a firefox plugin that autocloses logs.o.o tabs after a day | 23:53 |
| clarkb | mordred: http://logs.openstack.org/25/604925/10/check/system-config-run-base-ansible-devel/4df913a/job-output.txt.gz#_2019-02-28_21_58_43_579799 do you think ansible would look at us silly if we suggested that a third aprty ci job? | 23:55 |
| clarkb | it appears ansible devel has broken testinfra, not sure if that is intentional or that anyone knows but seems like maybe that is important to not break? | 23:56 |
| *** hwoarang has quit IRC | 23:57 | |
| clarkb | I'm guessing we might have to bring the scope of that job down a bit for use as third aprty ci, but so far its caught all kinds of fun stuff that ansible probably would like to know about | 23:57 |
| *** hwoarang has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
