| *** tosky has quit IRC | 00:01 | |
| mnaser | so this is weird but i was just scrolling through opendev | 00:04 |
|---|---|---|
| mnaser | and noticed 4 nodes that were in rax-ord which are 3 days old | 00:04 |
| mnaser | they're also ubuntu-bionic so its not like its a super unpopular nodeset | 00:05 |
| mnaser | (they're "ready" too) | 00:05 |
| clarkb | possible that nodepool isn't properly reusing those if they've been originally procurred for a dropped request? | 00:06 |
| clarkb | typically older nodes would be due to deletes that refuse to work or holds | 00:07 |
| clarkb | being ready implies neither of those cases are the case | 00:07 |
| mnaser | yeah, that's why it semed curious to me | 00:14 |
| ianw | well, the problem starts @ https://opendev.org/openstack/python-openstackclient/src/branch/master/openstackclient/volume/v2/volume.py#L737 ... but after that i'm not so sure | 00:29 |
| ianw | #status log removed old files02 and static.openstack.org servers | 00:43 |
| openstackstatus | ianw: finished logging | 00:43 |
| *** mlavalle has quit IRC | 00:54 | |
| mordred | ianw: bother. so - it's entirely possible osc isn't pulling things from clouds.yaml fully properly here. might be worth trying volume_endpoint_override as well .. or there's also potentially just a bug | 01:30 |
| ianw | mordred: volume_endpoint_override seems to give the same error | 01:31 |
| mordred | ianw: blerg | 01:32 |
| ianw | auth_config_hook(): ... 'volume_api_version': '2', 'volume_endpoint_override': 'https://dfw.blockstorage.api.rackspacecloud.com/v2/', | 01:32 |
| ianw | (same for block_) ... so it gets into the config that far | 01:32 |
| mordred | ianw: try setting --os-endpoint-override=https://dfw.blockstorage.api.rackspacecloud.com/v2/ on the command line? I'm betting it's an internal mapping issue | 01:33 |
| mordred | ianw: (since osc is ultimately instantiating python-cinderclient, some stuff can go to the bad place) | 01:33 |
| ianw | umm, it doesn't like os-endpoint-override | 01:34 |
| mordred | nod | 01:34 |
| mordred | ok. well - I'll swing back around to it in the morning if you haven't magically solved it :) | 01:34 |
| ianw | --os-endpoint doesn't work | 01:35 |
| *** diablo_rojo_phon has joined #opendev | 02:29 | |
| ianw | ok wow i got a volume list out but OMG this is all so so so terrible ... | 03:07 |
| *** diablo_rojo has quit IRC | 04:27 | |
| ianw | mordred: in short; i have filed https://storyboard.openstack.org/#!/story/2007459. it seems that cinder does not pass the api endpoint correctly, which i have prosed a fix for in https://review.opendev.org/714570 | 05:13 |
| ianw | mordred: however, it still seems that openstacksdk/openstackclient doesn't correctly pass the os-endpoint information on to cinderclient correctly. i will leave that to you as i'm out of brain power for now :) | 05:14 |
| ianw | mordred: details in task #39137 | 05:15 |
| *** DSpider has joined #opendev | 05:50 | |
| *** diablo_rojo has joined #opendev | 07:02 | |
| *** dpawlik has joined #opendev | 07:16 | |
| *** diablo_rojo has quit IRC | 07:35 | |
| *** roman_g has quit IRC | 07:48 | |
| *** ralonsoh has joined #opendev | 08:13 | |
| *** rpittau|afk is now known as rpittau | 08:26 | |
| *** tosky has joined #opendev | 08:41 | |
| *** roman_g has joined #opendev | 10:59 | |
| *** dpawlik has quit IRC | 11:33 | |
| *** dpawlik has joined #opendev | 11:34 | |
| mordred | ianw: awesome - thanks! | 11:44 |
| *** roman_g has quit IRC | 12:03 | |
| mordred | ianw: https://review.opendev.org/714630 Update Rackspace vendor profile for cinder v2 | 12:57 |
| mordred | ianw: https://review.opendev.org/714624 Honor endpoint override from config for volume | 12:57 |
| mordred | clarkb: ^^ | 12:57 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Stop running against devel of ansible/ansible https://review.opendev.org/714631 | 13:01 |
| mordred | config-core: could you please approve that real quick ^^? | 13:02 |
| mordred | corvus: when you're up - we should maybe try to debug why zuul ran a job on https://github.com/ansible/ansible/pull/68122/files | 13:04 |
| *** hashar has joined #opendev | 13:05 | |
| mnaser | mordred: +2 | 13:05 |
| mordred | mnaser: thanks. I went ahead and clicked +A since right now we're making noise on ansible PRs | 13:05 |
| openstackgerrit | Merged openstack/project-config master: Stop running against devel of ansible/ansible https://review.opendev.org/714631 | 13:20 |
| mordred | mnaser: since you're using python-base and python-builder images, https://review.opendev.org/#/c/714532/ is worth being aware of | 13:35 |
| mordred | mnaser: as well as https://review.opendev.org/#/c/711073/ | 13:35 |
| *** hashar has quit IRC | 13:37 | |
| openstackgerrit | Artom Lifshitz proposed openstack/project-config master: Add whitebox-tempest-plugin under QA https://review.opendev.org/714478 | 13:47 |
| corvus | mordred: did anything regarding that sdk job change recently? | 13:55 |
| corvus | mordred: (or anything in the sdk repo? other branches?) | 13:55 |
| mnaser | mordred: both lgtm :) | 13:56 |
| openstackgerrit | Sagi Shnaidman proposed openstack/project-config master: Remove devel branch for ansible openstack jobs https://review.opendev.org/714647 | 13:59 |
| mordred | corvus: not since jan 20 | 13:59 |
| mordred | lemme check other branches | 13:59 |
| mordred | corvus: nope. nothing in .zuul.yaml changed in sdk repo | 14:01 |
| corvus | mordred: any new branches created or deleted? | 14:01 |
| mordred | not in sdk | 14:01 |
| mordred | there were some branch shuffles in ansible - but that was mostly a branch creation and deletion in the ansible repo | 14:02 |
| mordred | corvus: and then all of the files which match the matchers we had in that definition were deleted yesterday | 14:02 |
| mordred | or maybe a little while before that - but recently-ish | 14:07 |
| corvus | mordred: since the zuul configuration has been removed, i'm not sure i can determine the cause | 14:15 |
| mordred | corvus: ok. do you think it's worth re-adding the config for a bit to help with that? (I could update the job content to stop failing so nobody on the ansible side would notice) | 14:16 |
| corvus | mordred: the logs don't have any clues. what i would do next is to re-equeue that change with the repl debug console open and inspect it. | 14:16 |
| *** hashar has joined #opendev | 14:16 | |
| openstackgerrit | Sabeel Ansari proposed openstack/project-config master: Add cert-manager to StarlingX https://review.opendev.org/714652 | 14:19 |
| corvus | mordred: or you could change the 3pc pipeline not to report to github | 14:19 |
| mordred | corvus: in other news ... the jeepyb change is successfully running system-config upload changes | 14:20 |
| mordred | corvus: well - we still want to run and report on changes to stable branches | 14:20 |
| mordred | upload jobs | 14:21 |
| corvus | mordred: if we want to debug this, i would suggest we turn off all github reporting | 14:22 |
| mordred | ah. nod | 14:22 |
| corvus | i think that's what we should do. | 14:22 |
| mordred | corvus: ok. I'll turn off reporting and re-enable that job | 14:23 |
| AJaeger | are we still in freeze regarding repo changes and new reps? | 14:23 |
| corvus | mordred: ++ i think that can be one change? | 14:23 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Disable github reporting and re-add devel ansible job https://review.opendev.org/714654 | 14:25 |
| mordred | AJaeger: yes - but we're _really_ close | 14:25 |
| mordred | corvus: does that look right? | 14:25 |
| AJaeger | mordred: thanks | 14:30 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Revert "Disable github reporting and re-add devel ansible job" https://review.opendev.org/714660 | 14:33 |
| mordred | and there's a followup revert revert | 14:33 |
| mordred | infra-root: mirror.iad looks like it's having sads | 14:34 |
| mordred | I can't ssh to it | 14:35 |
| * mordred is logging in to the web console to check there | 14:35 | |
| fungi | is that mirror.iad.rax.openstack.org or mirror.iad.rax.opendev.org? both seem broken but in slightly different ways | 14:37 |
| mordred | fungi: .openstack | 14:37 |
| mordred | https://zuul.opendev.org/t/openstack/build/61b24d96e0ec4fc7b1fe67841c64dbd6/log/job-output.txt#459 is a job log output | 14:37 |
| mordred | but I've seen it in a couple now | 14:37 |
| mordred | there is an "apache task hung" message on the console | 14:38 |
| mordred | should we give it a cloud reboot? | 14:38 |
| fungi | yeah, cacti says it ceased responding to snmp polls around 01:41z | 14:39 |
| mordred | kk. rebooting | 14:39 |
| fungi | the opendev.org version has apparently been offline since around 05:00z on 2020-02-06 | 14:40 |
| corvus | mordred: lgtm; AJaeger can you push https://review.opendev.org/714654 through? | 14:41 |
| mordred | k. back up | 14:42 |
| fungi | thanks! | 14:42 |
| fungi | status log it? | 14:42 |
| fungi | or i can | 14:42 |
| AJaeger | corvus: checking... | 14:43 |
| mordred | #status log rebooted mirror.iad.rax.openstack.org because it was hung | 14:43 |
| AJaeger | +3 | 14:43 |
| openstackgerrit | Merged openstack/project-config master: Disable github reporting and re-add devel ansible job https://review.opendev.org/714654 | 14:55 |
| corvus | mordred: ^ i'll manually enqueue and debug that after breakfast | 14:59 |
| mordred | corvus: cool | 15:06 |
| openstackgerrit | Merged opendev/jeepyb master: Clean up some more python3 things https://review.opendev.org/714318 | 15:08 |
| clarkb | re reboots it seems like if live migrations fail the servers end up in those weird states | 15:08 |
| clarkb | mordred: the sdk changes to fix volume things lgtm | 15:40 |
| clarkb | re the ansible thing its due to pulling all the useful bits out of devel aiui | 15:42 |
| clarkb | I assume we'll want to fix that job once upstream stablizes as we'll need to do that anyway to upgrade | 15:42 |
| clarkb | infra-root wasn't there a change up of some sort to update the text at opendev.org's root page? I thought there was and it would be a good candidate for monitoring the new gitea restart procedures | 15:49 |
| clarkb | I'm thinking we (I) should do that sooner than later to ensure it isn't buggy before we need it | 15:49 |
| openstackgerrit | Merged openstack/project-config master: Revert "Disable github reporting and re-add devel ansible job" https://review.opendev.org/714660 | 15:52 |
| corvus | mordred: ^ maybe shoulda put a wip on that, eh? | 15:53 |
| corvus | i literally just connected to the repl to start debugging | 15:53 |
| mordred | corvus: crappit | 15:54 |
| corvus | i'm reverting | 15:54 |
| clarkb | oh sorry | 15:54 |
| clarkb | the commit message said you were done | 15:54 |
| mordred | yeah - sorry - I wrote it to be right for when we landed it - my bad, I should have marked it WIP | 15:55 |
| openstackgerrit | James E. Blair proposed openstack/project-config master: Revert "Revert "Disable github reporting and re-add devel ansible job"" https://review.opendev.org/714685 | 15:55 |
| fungi | clarkb: the only one i was aware of already merged | 15:57 |
| fungi | (updating the irc channel listed there) | 15:57 |
| clarkb | fungi: that may have been it. I suppose there is likely some other edit that makes sense I can come up with :) | 15:58 |
| clarkb | mordred: did you see ianw's comment on https://review.opendev.org/#/c/714553/2 that has me reconsidering if the sdk changes are correct | 15:58 |
| fungi | has anybody put any thought into a new discussion ml for opendev yet? | 15:58 |
| clarkb | mordred: I think the most correct thing is to keep using v1 volume api based on that | 15:58 |
| fungi | like service-discuss@ maybe | 15:58 |
| clarkb | fungi: we probably should. I've got a meeting agenda item to discuss similar things (meeting location in particular) | 15:59 |
| fungi | awesome. will remember to bring it up again in a few hours | 15:59 |
| openstackgerrit | Sabeel Ansari proposed openstack/project-config master: Add nginx-ingress-controller armada app to StarlinX https://review.opendev.org/714686 | 16:00 |
| mordred | clarkb: the v2 api is there - I have used it just fine | 16:06 |
| mordred | clarkb: the v2 api does not have an accessible version discovery document | 16:06 |
| clarkb | mordred: is that the only difference? the rax doc implies its not a full v2 api | 16:06 |
| clarkb | the doc seems to imply there are quite a few deltas | 16:07 |
| clarkb | its possible they don't matter muc though | 16:07 |
| clarkb | mordred: rereading it though I'm not sure their v1 cinder api is any better? | 16:09 |
| clarkb | this document is comparing rax's api to cinder v2, I assume cinder v1 has similar deltas? | 16:09 |
| mordred | probably? it's a great question tbh | 16:10 |
| clarkb | its unfortunate that its rax api v1 | 16:11 |
| clarkb | which may be confusing if that doesn't map onto cinder api v1 | 16:11 |
| openstackgerrit | Merged openstack/project-config master: Revert "Revert "Disable github reporting and re-add devel ansible job"" https://review.opendev.org/714685 | 16:20 |
| openstackgerrit | Sabeel Ansari proposed openstack/project-config master: Add Cert-Manager Armada app to StarlingX https://review.opendev.org/714689 | 16:25 |
| mordred | clarkb: also - with my patch to openstackclient- openstack volume list works just fine | 16:32 |
| mordred | clarkb: we should maybe test whatever other osc commands we use | 16:32 |
| mordred | clarkb: (well, with the sdk patch and the osc patch) | 16:32 |
| openstackgerrit | Clark Boylan proposed opendev/system-config master: Note about gitea functionality on opendev.org https://review.opendev.org/714691 | 16:33 |
| clarkb | infra-root ^ maybe something like that to exercise the gitea restart changes | 16:33 |
| mordred | clarkb: sounds good | 16:37 |
| openstackgerrit | Tobias Henkel proposed opendev/gear master: Move handleDisconnect into BaseClientServer https://review.opendev.org/714709 | 16:44 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add jeepyb to file matchers for gerrit-base https://review.opendev.org/714711 | 16:48 |
| mordred | clarkb: sigh ^^ | 16:48 |
| mordred | clarkb: we built images on the jeepyb patch - but we didn't build the gerrit-base image which is the thing that actually includes the jeepyb content :) | 16:48 |
| clarkb | mordred: hrm that won't rebuild if top level files of jeepyb change? | 16:50 |
| clarkb | thats a weird one. The fix is definitely an improvement but probably won't catch all cases | 16:50 |
| mordred | clarkb: yeah - but it's probably ok for the time being I hope | 16:50 |
| mordred | clarkb: we could also put in more file matchers in teh project-config pipeline definition | 16:50 |
| clarkb | mordred: maybe we can add something like ^[^/]+$ | 16:54 |
| clarkb | I think that will match all top level files | 16:54 |
| mordred | clarkb: yeah - but I don't think we want that to be true when we run it from system-config | 16:54 |
| clarkb | ya so would be in the project-config definition for the jeepyb application? though maybe there we can just say .* | 16:55 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Add more file matchers for jeepyb https://review.opendev.org/714713 | 16:57 |
| mordred | clarkb: like that ^^ | 16:57 |
| clarkb | mordred: can we reduce it to ^.*$ there? | 16:58 |
| mordred | clarkb: worth a shot | 17:00 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Always build gerrit images on jeepyb changes https://review.opendev.org/714713 | 17:01 |
| mordred | clarkb: ^^ | 17:01 |
| *** mlavalle has joined #opendev | 17:06 | |
| *** rpittau is now known as rpittau|afk | 17:18 | |
| *** hashar has quit IRC | 17:29 | |
| AJaeger | mordred: +3 | 17:38 |
| AJaeger | config-core, a few reviews for your consideration: https://review.opendev.org/679082 https://review.opendev.org/714506 https://review.opendev.org/714543 | 17:41 |
| *** openstackstatus has quit IRC | 17:45 | |
| *** openstackstatus has joined #opendev | 17:46 | |
| *** ChanServ sets mode: +v openstackstatus | 17:46 | |
| clarkb | fungi on https://review.opendev.org/#/c/714506/ (noted above) can you ack or nack that? I wasn't quite clear on whether you were saying that is ok yesterday | 17:46 |
| openstackgerrit | Merged openstack/project-config master: Always build gerrit images on jeepyb changes https://review.opendev.org/714713 | 17:46 |
| fungi | clarkb: ahh, i was talking about the related change for zuul's encrypt_secret.py but sure, i'll review this too | 17:48 |
| openstackgerrit | Merged opendev/base-jobs master: Remove env var for tarballs mirror https://review.opendev.org/714543 | 17:59 |
| openstackgerrit | Monty Taylor proposed opendev/jeepyb master: Just use flake8 instead of hacking https://review.opendev.org/714729 | 18:00 |
| mordred | clarkb, fungi: that's not a very meaningful change - but should exercise our latest project-config update | 18:01 |
| openstackgerrit | Merged zuul/zuul-jobs master: Trim whitespace from uri password for docker promote https://review.opendev.org/714506 | 18:06 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed opendev/lodgeit master: Add lodgeit-db script https://review.opendev.org/714732 | 18:16 |
| *** diablo_rojo has joined #opendev | 18:42 | |
| mordred | clarkb, ianw: https://review.opendev.org/714624 has merged - so even without the rax profile update in the sdk change - it should allow testing the use of block_storage_endpoint_override in clouds.yaml with osc | 18:49 |
| mordred | (it works for me on my laptop - but obviously further testing isn't a bad idea) | 18:49 |
| *** dpawlik has quit IRC | 18:53 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed opendev/lodgeit master: Add lodgeit-db script https://review.opendev.org/714732 | 18:57 |
| *** ralonsoh has quit IRC | 19:07 | |
| mordred | ianw: when you get a chance, if you can confirm that master of osc works for you I'll get a release cut | 19:21 |
| ianw | mordred: will it work without the cinderclient changes? | 19:22 |
| mordred | yes | 19:22 |
| mordred | at least - it worked for me without those locally | 19:22 |
| mordred | you'll need the cinderclient changes if you want to use cinderclient cli with an override I believe | 19:22 |
| ianw | ok interesting, so it uses something other than cinderclient keystone session? or it sets up its own? | 19:23 |
| mordred | it's .... complicated :) | 19:23 |
| ianw | haha yes, that i will agree on! | 19:23 |
| mordred | but yeah- it constructs a Client object from cinderclient from its own parameter processing | 19:23 |
| mordred | that processing itself is too complex because it's also double-processing things that sdk will do for it for historical reasons | 19:24 |
| mordred | and I haven't unwound that yet | 19:24 |
| mordred | ianw: I *almost* rage-removed cinderclient from osc this morning ... but I figured I should just fix it in place as-is :) | 19:27 |
| ianw | mordred: sorry, so should "--os-cloud=openstackci-rax --os-region=DFW --os-volume-api=2 --os-endpoint=https://dfw.blockstorage.api.rackspacecloud.com/v1/...... volume list" work *without* block_storage_endpoint_override in clouds.yaml? | 19:30 |
| mordred | nope | 19:33 |
| mordred | just the block_storage_endpoint_override in clouds.yaml | 19:33 |
| mordred | ianw: the other stuff is a much larger rats-nest to unwind | 19:33 |
| mordred | (also - be sure it's /v2/$project_id | 19:34 |
| mordred | ianw: /v2/610275 | 19:34 |
| mordred | or whatever it is | 19:34 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Actually write out the projects.ini config file https://review.opendev.org/714760 | 19:39 |
| mordred | clarkb, corvus: ^^ silly mistake | 19:39 |
| ianw | mordred: huh, ok .. so "/v2" actually works, despite not being documented as an endpoint either in their catalog or documentation? | 19:42 |
| mordred | yup | 19:43 |
| mordred | it's in the service discovery doc | 19:43 |
| *** hashar has joined #opendev | 19:43 | |
| mordred | if you do: | 19:43 |
| mordred | ianw: http://paste.openstack.org/show/791101/ | 19:45 |
| mordred | ianw: you can see the version discovery document | 19:45 |
| mordred | ianw: acutlaly - it's curl-able | 19:45 |
| mordred | curl https://dfw.blockstorage.api.rackspacecloud.com/ | 19:45 |
| mordred | {"versions": [{"status": "SUPPORTED", "updated": "2014-06-28T12:20:21Z", "id": "v1.0", "links": [{"href": "http://dfw.blockstorage.api.rackspacecloud.com/v1/", "rel": "self"}]}, {"status": "CURRENT", "updated": "2012-11-21T11:33:21Z", "id": "v2.0", "links": [{"href": "http://dfw.blockstorage.api.rackspacecloud.com/v2/", "rel": "self"}]}]} | 19:45 |
| ianw | well, TIL ... | 19:52 |
| clarkb | mordred: corvus https://review.opendev.org/#/c/714691/1 that job failure was due to connectivity problems to dockerhub. We don't use our caches there bceause its using the system-config roles not the zuul roles | 19:53 |
| clarkb | I'm not sure if there is a good fix for that, but wanted to call it out | 19:53 |
| ianw | mordred: i can confirm volume_endpoint_override: https://dfw.blockstorage.api.rackspacecloud.com/v2/610275 makes it work for me with master openstackclient | 19:53 |
| mordred | ianw: woot! | 19:53 |
| mordred | I'll get a release cut | 19:53 |
| openstackgerrit | Monty Taylor proposed opendev/jeepyb master: Just use flake8 instead of hacking https://review.opendev.org/714729 | 20:00 |
| clarkb | corvus: to be clear I Think the meetpad idea is an excellent one and will likely capture the process of inperson brainstorming and discussion at PTGs and design summits far better than many of the other options available to us. I'm simply anticipating that as plans congeal there will be more askes for tools or tweaks to existing tools | 20:02 |
| corvus | clarkb: ok, i wasn't sure if you knew of other efforts that might impact or relate to this | 20:03 |
| clarkb | I do not | 20:03 |
| corvus | i thought it may be useful, but don't want to needlessly spin wheels or step on other peoples toes | 20:03 |
| corvus | but sounds like we're good :) | 20:03 |
| clarkb | I think the first lareg community brainstorm is happening next week? | 20:03 |
| clarkb | fungi: ^ would probably know for sure | 20:04 |
| openstackgerrit | Monty Taylor proposed opendev/jeepyb master: Trigger new images https://review.opendev.org/714763 | 20:04 |
| clarkb | as far as anticipating specific things I know video recording for zoom calls are a thing some communities (starlingx) rely on. I also expect that scheduling due to time zones may be a tricky problem people look to tooling to help solve | 20:05 |
| mordred | clarkb, corvus: cool - that seems to trigger the right things ^^ | 20:05 |
| fungi | there's a "community meeting" (conference call) which i think it going to be announced for 15:00z on thursday april 2 | 20:05 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Override RAX block storage endpoint https://review.opendev.org/714553 | 20:05 |
| mordred | (we were missing a job config the last time we built images- we dont' actually have new content, but we need new images built) | 20:05 |
| fungi | clarkb: not sure if that's what you're referring to | 20:05 |
| ianw | mordred: ^^ i think that's now right to update configs | 20:06 |
| clarkb | fungi: ya I think that is the one | 20:06 |
| mordred | ianw: ++ | 20:06 |
| clarkb | ianw: one small fix on that change | 20:08 |
| clarkb | (noted it inline and now I'm finding lunch) | 20:08 |
| clarkb | and probably a short indoor bike ride because its cold outside | 20:08 |
| * mordred is going to take an outdoor bike ride - because it is not cold outside | 20:11 | |
| fungi | i love that it must now be explicitly differentiated from an "indoor | 20:17 |
| fungi | " bike ride | 20:17 |
| mordred | clarkb, corvus, fungi, ianw: if anybody wants to +A https://review.opendev.org/#/c/714763/ - we should have new images by the time I'm back | 20:17 |
| fungi | that's... intruiging. is it really a diffless commit, or is gertty lying to me? | 20:20 |
| mordred | fungi: really a diffless commit | 20:20 |
| mordred | I just did a git commit --allow-empty | 20:20 |
| fungi | yeah, gerrit's webui confirms | 20:20 |
| fungi | wow | 20:20 |
| fungi | i'm approving just because i want to see this "merge" | 20:21 |
| fungi | wcpgw? | 20:21 |
| AJaeger | mordred: will jobs run - of will filematcher not find any files? | 20:21 |
| mordred | AJaeger: it looks like it's running all of them | 20:21 |
| mordred | (including gerrit-base which is what was missing before) | 20:22 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Override RAX block storage endpoint https://review.opendev.org/714553 | 20:22 |
| * mordred afks | 20:22 | |
| AJaeger | mordred: indeed - http://zuul.opendev.org/t/openstack/status/change/714763,1 | 20:22 |
| openstackgerrit | sebastian marcet proposed opendev/puppet-openstackid master: Fixing results from ZAP Scanning Report https://review.opendev.org/714351 | 20:33 |
| openstackgerrit | sebastian marcet proposed opendev/puppet-openstackid master: Fixing results from ZAP Scanning Report https://review.opendev.org/714351 | 20:40 |
| corvus | clarkb, fungi: who's organizing that community meeting? | 20:40 |
| openstackgerrit | Merged opendev/system-config master: OpenStackID 3.0.6 production deployment https://review.opendev.org/714263 | 20:50 |
| fungi | corvus: allison price is working on an agenda for it which she says she hopes to send to project mailing lists tomorrow | 20:54 |
| corvus | fungi: thanks | 20:55 |
| fungi | tentative agenda is the openstack project 10th anniversary activities and an events update, plus a lot of q&a time for anyone on the call who has input/suggestions/ideas | 20:55 |
| fungi | so sounds like virtual ptg is on topic | 20:55 |
| *** hashar has quit IRC | 21:08 | |
| mordred | ianw: sad: https://review.opendev.org/#/c/714553 | 21:33 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Override RAX block storage endpoint https://review.opendev.org/714553 | 21:35 |
| ianw | this time for sure! :) | 21:35 |
| clarkb | ianw: yup +2 | 21:55 |
| openstackgerrit | Merged opendev/jeepyb master: Trigger new images https://review.opendev.org/714763 | 21:56 |
| mordred | clarkb, ianw: https://review.opendev.org/#/c/714760/ ? | 22:03 |
| mordred | clarkb, ianw: also Release 5.1.0 of python-openstackclient https://review.opendev.org/714781 | 22:03 |
| clarkb | mordred: 760 done | 22:04 |
| mordred | thanks! | 22:04 |
| openstackgerrit | Merged opendev/system-config master: Note about gitea functionality on opendev.org https://review.opendev.org/714691 | 22:04 |
| openstackgerrit | Merged opendev/puppet-openstackid master: Fixing results from ZAP Scanning Report https://review.opendev.org/714351 | 22:04 |
| clarkb | I'll watch 691 | 22:05 |
| corvus | mordred: i'm not sure what to do about that ansible issue | 22:18 |
| mordred | corvus: me either. maybe we'll learn something when gundalow replies | 22:19 |
| corvus | mordred: at this point, i think it's going to take quite a bit of debugging.... do we want to do that, or write this off as a blip since we're stopping testing on that repo anyway? | 22:19 |
| mordred | corvus: I think I'm leaning more towards writing it off as a blip | 22:19 |
| corvus | mordred: i want to find the right balance of being supportive but not imposing on the ansible project | 22:21 |
| corvus | mordred: i notice from the debug logs that we used to see a lot of github traffic related to ansible/ansible, and i think it may have stopped yesterday | 22:21 |
| corvus | mordred: did you log in using the opendev superuser access to investigate the app? | 22:22 |
| mordred | corvus: I did - but I couldn't find anywhere to see where it was installed | 22:23 |
| corvus | mordred: 2020-03-23 19:23:30,044 was the last time we saw a GithubTriggerEvent from ansible/ansible | 22:24 |
| corvus | we have seen them from kata since then | 22:24 |
| mordred | corvus: there are _many_ payloads in recent deliveries since that date/time - but spot checking I don't see any from ansible/ansible | 22:26 |
| corvus | mordred: i'm leaning toward thinking they've uninstalled it | 22:27 |
| mordred | I'm leaning that way too | 22:27 |
| corvus | mordred: hopefully gundalow can confirm; and if that's the case, that at least explains the invalid change thing | 22:27 |
| corvus | then the question is, do we ask to reinstall it so we can debug the original error, or just say 'nevermind' and revert the revert revert revert? | 22:28 |
| mordred | yeah - although not so much why it went weird in the first place :( | 22:28 |
| mordred | that's a good question - and one for which I don't know that I have an answer for | 22:29 |
| corvus | okay, well, we can pick this up again tomorrow if we want; i'll stand down for today | 22:30 |
| mordred | cool - I'll let you know what I hear from gundalow when I do | 22:31 |
| corvus | mordred: thx | 22:31 |
| clarkb | currently gitea nodes are getting the base server playbook applied to them | 22:33 |
| clarkb | should update the deployment soon I expect | 22:33 |
| clarkb | I'm going to take the zuul user survey now for OpenDev | 22:34 |
| *** bolg has joined #opendev | 22:39 | |
| clarkb | corvus: we aren't using the pagure driver are we? | 22:40 |
| openstackgerrit | Merged openstack/diskimage-builder master: Use rpm -e instead of dnf for cleaning old kernels https://review.opendev.org/713988 | 22:42 |
| clarkb | codesearch says no | 22:42 |
| *** DSpider has quit IRC | 22:44 | |
| openstackgerrit | Merged opendev/system-config master: Actually write out the projects.ini config file https://review.opendev.org/714760 | 22:46 |
| openstackgerrit | Merged opendev/system-config master: Override RAX block storage endpoint https://review.opendev.org/714553 | 22:46 |
| clarkb | ps shows gitea-web process started after sshd but ansible syslog logs show things running in the correct sequence. I'm going to guess there is some sort of forking related there that may throw off the ps timestamp? | 22:49 |
| clarkb | replication around that timeframe seems to have worked though | 22:52 |
| clarkb | the rax block storage endpoint change above is an example https://gitea01.opendev.org:3000/opendev/system-config | 22:52 |
| clarkb | that started about 40 seconds after the gitea-ssh container was started though so its not quite in the bit we think might have problems | 22:52 |
| clarkb | at the very least this doesn't seem to regress worse than the behavior we had before | 22:53 |
| clarkb | which was my biggest concern and wanted to rule out before it bit us later unexpectedly. So yay this seems happy | 22:53 |
| mordred | clarkb: nope | 22:55 |
| mordred | clarkb: sorry - lag - I was answering "we arent' using pagure" | 23:01 |
| openstackgerrit | Ian Wienand proposed opendev/infra-specs master: Move static.openstack.org replacement spec to completed https://review.opendev.org/714789 | 23:02 |
| mordred | clarkb: yay! | 23:02 |
| clarkb | mordred: ya I figured that was what you were responding to | 23:04 |
| mordred | clarkb, corvus: could I interest you in https://review.opendev.org/#/c/714532/ ? | 23:07 |
| clarkb | oh ya that was on my list yesterday then it got lost | 23:08 |
| clarkb | mordred: left a thought and a clarification for other reviews but looks great to me | 23:19 |
| mordred | clarkb: setting version arg is a good idea | 23:24 |
| mordred | corvus: ansible app was disabled - is reinstalled | 23:24 |
| mordred | corvus: s/ansible/github/ | 23:24 |
| corvus | mordred: thanks! | 23:25 |
| clarkb | corvus: rereading the meetpad spec, it might be a good idea to make note of what changes we're carrying locally that prevent us from usign the upstream images (since we want to switch to the upstream images at some point) | 23:28 |
| corvus | clarkb: there is exactly one two-line patch | 23:28 |
| clarkb | corvus: we can probably do that on the change to use the images though not necessarily in the spec | 23:28 |
| corvus | there's a comment in the change about it | 23:29 |
| clarkb | ah ok I must've missed it when looking over the meetpad changes | 23:29 |
| clarkb | is it in the docker image change? | 23:29 |
| corvus | clarkb: https://review.opendev.org/714505 | 23:29 |
| clarkb | thanks! | 23:30 |
| corvus | easy :) | 23:30 |
| corvus | now, getting redhat to agree to the cla -- i dunno | 23:30 |
| openstackgerrit | Merged opendev/infra-specs master: Move static.openstack.org replacement spec to completed https://review.opendev.org/714789 | 23:31 |
| mordred | maybe clarkb could clean-room write the same patch? | 23:33 |
| clarkb | I haven't looked at the PR yet at least | 23:33 |
| corvus | well, that might be an option if the cla situation is different for clarkb | 23:35 |
| corvus | i've sent off an email to redhat legal to see about setting the wheels in motion | 23:35 |
| clarkb | I can ask jbryce if you don't make progress | 23:37 |
| clarkb | corvus: in the docker-compose file you've got ${CONFIG} as a var. Where is that value set? | 23:38 |
| clarkb | is docker-compose reading the .env file? | 23:38 |
| clarkb | (I'm assuming so and thats an interesting feature I had not come across before yet) | 23:38 |
| corvus | clarkb: yes in https://review.opendev.org/714238 | 23:38 |
| clarkb | yup thats what I'm looking at. Ok making sure I grok what is going on there | 23:39 |
| clarkb | and that must just be a docker-compose behavior | 23:39 |
| corvus | clarkb: and yes, it is interesting. i like it because it's nice separation of inputs. i don't like that it's a dotfile and i don't see a way to change that. | 23:39 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Start making 3.8 python images https://review.opendev.org/714532 | 23:41 |
| mordred | corvus, clarkb: updated per clarkb's comment | 23:41 |
| mordred | clarkb: ok. manage-projects now works right via the image on review-dev | 23:44 |
| mordred | BLERG | 23:45 |
| mordred | ok. maybe if I were smarter I'd test more than one thing at a time | 23:45 |
| openstackgerrit | Ghanshyam Mann proposed openstack/project-config master: Replace python-charm-jobs to py3 job https://review.opendev.org/714796 | 23:50 |
| clarkb | and I'm through the meetpad stack. I think it is ready to go but will keep the spec open until thursday as mentioned in the meeting | 23:51 |
| openstackgerrit | Monty Taylor proposed opendev/jeepyb master: Update StringIO to use six https://review.opendev.org/714797 | 23:52 |
| mordred | clarkb: sigh ^^ | 23:52 |
| mordred | should have checked update-bug and update-blueprint and not just manage-projects | 23:53 |
| clarkb | yay testing | 23:53 |
| mordred | yeah | 23:53 |
| *** mlavalle has quit IRC | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!