| *** marios is now known as marios|ruck | 05:41 | |
| *** amoralej|off is now known as amoralej | 06:45 | |
| *** rpittau|afk is now known as rpittau | 07:55 | |
| *** ykarel_ is now known as ykarel | 08:52 | |
| *** ykarel_ is now known as ykarel | 09:03 | |
| opendevreview | Andreas Jaeger proposed openstack/project-config master: Retire ara projects https://review.opendev.org/c/openstack/project-config/+/777675 | 10:44 |
|---|---|---|
| *** dviroel|out is now known as dviroel | 11:53 | |
| *** amoralej is now known as amoralej|lunch | 12:55 | |
| opendevreview | Ananya proposed opendev/elastic-recheck master: Run elastic-recheck container https://review.opendev.org/c/opendev/elastic-recheck/+/729623 | 13:31 |
| opendevreview | Ananya proposed opendev/elastic-recheck master: Run elastic-recheck container https://review.opendev.org/c/opendev/elastic-recheck/+/729623 | 13:33 |
| *** amoralej|lunch is now known as amoralej | 13:50 | |
| *** gibi is now known as gibi_pto | 14:03 | |
| opendevreview | Ananya proposed opendev/elastic-recheck master: Run elastic-recheck container https://review.opendev.org/c/opendev/elastic-recheck/+/729623 | 14:11 |
| *** ykarel is now known as ykarel|away | 14:56 | |
| clarkb | infra-root as indicated last week I intend on running the external id conflict cleanup script against the accounts that were retired recently today. Probably after I settle in a bit | 14:58 |
| clarkb | As far as I know we haevn't seen any of those users complain since their accounts were retired so I don't need to reduce the list | 14:59 |
| fungi | yep, not a peep afaik | 15:00 |
| fungi | sounds like a good plan | 15:00 |
| clarkb | I am proceeding with the external id cleanups now. I will rerun the consistency check after. I expect we'll have a couple of new errors in the consistency checks from the accounts we addressed during the downtime. Specifically the new accounts we didn't want will likely have preferred email addresses set without corresponding external ids | 15:53 |
| clarkb | if those two pop up as issues I'll clean them up too | 15:53 |
| clarkb | And then I think there are a few accounts like our old infra stackalytics account that I can also ninja cleanup | 15:54 |
| fungi | yeah, that can certainly go | 16:00 |
| clarkb | then maybe next week it is time to start putting together a large all in one fixup change for our active users | 16:03 |
| clarkb | I think the list is approaching a reasonable size after this cleanup round | 16:03 |
| clarkb | cleanup script is done running. Log is in the usual location | 16:05 |
| rpittau | hi all! Quick question, is the "branches" option supposed to work on zuul templates? | 16:07 |
| clarkb | rpittau: I don't think so, but you can double check the zuul docs to confirm | 16:08 |
| rpittau | clarkb: I was afraid of that, I did a quick test and zuul complained :/ | 16:09 |
| rpittau | I'll check the docs again | 16:09 |
| rpittau | thanks! | 16:09 |
| dtantsur | I couldn't find anything in the docs, maybe I haven't looked well enough | 16:10 |
| fungi | what specifically are you trying to make it do? | 16:10 |
| rpittau | I checked before too, maybe I missed it | 16:10 |
| rpittau | fungi: we'd like to run the lower-constraints job in master branch only | 16:10 |
| fungi | and when you say "zuul templates" you mean project-templates? | 16:10 |
| rpittau | fungi: yes, project-templates | 16:11 |
| fungi | what's the template, and which lower-constraints job is it including? any reason not to set the branches parameter in the job definition? | 16:11 |
| clarkb | config consistency checks run in about 90 seconds now. Much quicker than before \o/ confirmed the two accounts we addressed during the downtime have preferred email problems | 16:11 |
| clarkb | I'll cross check against my notes then properly retire those two unneeded accounts | 16:12 |
| rpittau | fungi: the template is openstack-lower-constraints-jobs | 16:12 |
| rpittau | we can't exactly modify that as it's run by the entire openstack community | 16:12 |
| fungi | well, don't be so hasty to assume... there seemed to be some consensus on the ml to switch to only running lower-constraints jobs on master branches | 16:13 |
| rpittau | ah! :) | 16:13 |
| fungi | maybe everyone would be okay with that? | 16:13 |
| rpittau | well I guess I cant try to submit a change in zuul and create a master-only template | 16:14 |
| rpittau | s/cant/can | 16:14 |
| fungi | but yeah, worst case, make a openstack-lower-constraints-master-branch-jobs temlpate which includes master-branch-only variants of the same jobs | 16:14 |
| rpittau | thanks fungi , if the community is interested maybe they will start adopting that template instead | 16:15 |
| fungi | rpittau: that's also something for the qa team and tact sig to potentially push on | 16:17 |
| gmann | rpittau: fungi consensus on l-c testing is - it is up to project to test it or not or even test on master only | 16:20 |
| gmann | not sure what is benefit of making master-on template as openstack-lower-constraints-jobs can be removed from stable branch gate if any project do not want to run it on stable ? | 16:21 |
| rpittau | gmann: yes, speaking for the ironic project, we removed the l-c tests but we'd like to re-introduce it only for the master branch, that's why the questioning | 16:21 |
| rpittau | gmann: one less task to do during the stable branch cut | 16:21 |
| gmann | rpittau: i see. | 16:22 |
| gmann | rpittau: +1 | 16:22 |
| rpittau | TL;DR I'm lazy :D | 16:22 |
| rpittau | thanks gmann :) | 16:22 |
| rpittau | btw change is up https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/802335 | 16:23 |
| clarkb | alright the three additional cleanups are done now. I've copied updated logs to the typical location. Rerunning consistency checks now | 16:28 |
| clarkb | consistency checking looks a bit better after the followup cleanups. Those results and all the lgos are in the usual location now. Now to rerun the audit script to produce a current list of conflicts. But then I think this task is done for today | 16:31 |
| *** amoralej is now known as amoralej|off | 16:34 | |
| *** rpittau is now known as rpittau|afk | 16:37 | |
| opendevreview | Merged opendev/puppet-infra-cookiecutter master: Fix gitreview host name https://review.opendev.org/c/opendev/puppet-infra-cookiecutter/+/708452 | 16:38 |
| clarkb | There are 105 remaining conflicts. | 16:42 |
| clarkb | Maybe still a bit much for doing an all in one fixup change? I'll have to look through the audit results and see if there are additional accounts we can cleanup in a more brute force manner | 16:43 |
| *** marios|ruck is now known as marios|out | 16:54 | |
| *** dviroel is now known as dviroel|afk | 16:56 | |
| sshnaidm | clarkb, hi, do you know if zuul has a user on galaxy.ansible.com for publishing collections? | 17:04 |
| opendevreview | Merged openstack/project-config master: Retire ara projects https://review.opendev.org/c/openstack/project-config/+/777675 | 17:05 |
| sshnaidm | I want to create a job which will publish collection after it's tagged, wonder if I can do it on behalf of zuul user | 17:05 |
| fungi | i don't think we have any jobs publishing collections yet, but that would probably need to be done with per-project credentials anyway... the collections repository is namespaced right? | 17:06 |
| fungi | would likely need to be done similar to how we push to project-specific namespaces on dockerhub or github | 17:07 |
| clarkb | fungi: ++ | 17:08 |
| fungi | doing it with zuul secrets also puts the credentials management squarely in the hands of the project leadership or their delegates, so that the opendev sysadmins don't have to create or maintain those namespaces and accounts | 17:11 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Test the rename_repos playbook https://review.opendev.org/c/opendev/system-config/+/802112 | 17:13 |
| clarkb | fungi: ^ thats not quite converged between testing and prod yet and I half expect it to fail due to the large scale chagnes I made to the test job side of things. But its getting closer | 17:14 |
| clarkb | fungi: two things come up from that. FIrst is do you know if the first user is always 'admin' in development mode? And the other is do you know if we need to update our host lists in rename_repos.yaml to be review:!disabled or similar to prevent it from trying to run against review01 as well? | 17:14 |
| clarkb | Actually review01 may not be in inventory anymore /me checks | 17:14 |
| clarkb | ya looks like review01 is out of the inventory so the second thing is probably a non issue | 17:15 |
| fungi | i think in development mode it ignores the username entirely? | 17:15 |
| clarkb | cool in that case my change may work | 17:15 |
| clarkb | which was basically s/admin/openstack-project-creator/ to make testing look more like prod | 17:16 |
| fungi | looking to see what we did with git-review tests now | 17:16 |
| fungi | we seem to use admin:secret as the username:password combo in the git-review tests with auth.type=DEVELOPMENT_BECOME_ANY_ACCOUNT and then call create-account via the ssh cli to create additional non-admin users | 17:19 |
| clarkb | if the account name change works then I think the last remaining piece to make that mergable is figuring out how to run rename_repos.yaml in a more general way | 17:20 |
| clarkb | fungi: you may want to look over the change to get a general idea for the types of change that are necessary | 17:20 |
| clarkb | I do think testing that if we can is a really good idea | 17:20 |
| fungi | agreed, if nothing else it will highlight where new gerrit releases may make our renaming process harder | 17:21 |
| sshnaidm | fungi, I will make credentials, but I think maybe to use a specific CI user for that? I definitely don't want to use my own. Which user do you use for publishing to dockerhub/github? | 17:51 |
| fungi | sshnaidm: it depends on the namespace to which we're publishing. each namespace has its own publishing account and associated credentials | 17:57 |
| sshnaidm | ok, so I need just to create a new one | 17:58 |
| sshnaidm | fungi, also can you please give a link to how to create job that is triggered by tagging? I think I saw such (post-release?) | 17:58 |
| fungi | project leaders or someone to whom they've delegated the responsibility creates the account at the publication site and then encodes the related authentication credentials into a zuul secret to be used when that project calls the publication job | 17:58 |
| sshnaidm | fungi, ack | 17:59 |
| fungi | if you're thinking of the release-post pipeline, that's not a pipeline triggered by tagging, but rather a dedicated post-merge pipeline for the openstack/releases repository so it can be given priority for node requests | 18:00 |
| fungi | i'd have to know more about how collections publication works to suggest a particular mechanism. for example publishing to dockerhub we recommend doing in the gate pipeline, and then using the promote and release pipelines to tag the created artifacts with certain labels or versions in the publication site | 18:01 |
| fungi | or maybe it's that we build images in the gate pipeline and then have the promote jobs push those into dockerhub, i'd have to look back at the flowchart | 18:02 |
| sshnaidm | fungi, well, I plan just to push a tag with collection version, zull job will be triggered, will build a tarball with collection and will publish it to galaxy.ansible.com with API token (secret) | 18:07 |
| sshnaidm | fungi, it will build collection with a tag version inside | 18:07 |
| sshnaidm | so I need a job that is triggered by tag and the tag itself can be discovered inside the job | 18:09 |
| fungi | sshnaidm: in that case any job run in the release pipeline will be triggered on a tag which looks like a 3-part semver version number, the pre-release pipeline will match on tags which look like semver pre-release strings, and the tag pipeline will match on any tag | 18:11 |
| sshnaidm | fungi, will be like that: https://review.opendev.org/c/openstack/ansible-collections-openstack/+/802379/1/tox.ini | 18:12 |
| sshnaidm | fungi, sounds like release pipeline is what I need | 18:12 |
| sshnaidm | fungi, what is tag variable in the job? | 18:13 |
| fungi | easiest way to find out is to look at an example by filtering the zuul builds ui for the release pipeline and then look at the inventory it includes: https://zuul.opendev.org/t/openstack/build/37c1cd547e9d467986d519bfd5d29fc0/log/zuul-info/inventory.yaml#101 | 18:17 |
| fungi | according to that i think it's zuul.tag in vars | 18:17 |
| fungi | you could also parse it out of the zuul.ref var | 18:18 |
| clarkb | fungi: seems that the account being admin is implied | 18:23 |
| clarkb | this means I need to do a bit of work to add another user | 18:23 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Test the rename_repos playbook https://review.opendev.org/c/opendev/system-config/+/802112 | 18:31 |
| clarkb | maybe that will work | 18:32 |
| clarkb | fungi: do you think project rename email announcement is worth going to -announce? I suppose so | 18:36 |
| clarkb | yuriys: hello | 18:39 |
| yuriys | hey clarkb, at least got this set up now lol | 18:40 |
| yuriys | was the oft forceconvert? do i just nuke all of my previous freenode autojoins | 18:40 |
| yuriys | or is it per project | 18:40 |
| yuriys | as it usually is | 18:41 |
| clarkb | This was pretty global. We can't easily run bots for both networks and projects like openstack decided to switch entirely | 18:41 |
| yuriys | noted | 18:42 |
| clarkb | in this case the writing on the wall awas pretty clear then a few weeks after we moved freenode dropped the original network too and switched to a new cluster with new services | 18:42 |
| clarkb | infra-root I've grabbed the user cleanup notes I wanted from review-test. Would probably be good if others that used that server for testing etc double check it too before we clean it up? cc fungi mordred in particular I think | 18:54 |
| clarkb | but then we should be able to clean up review01 and review-test together | 18:54 |
| clarkb | I still need to look at other info on review01 outside of the user cleanup stuff to see if I want to preserve any of that | 18:55 |
| mordred | clarkb: I don't think I have anything remaining on review-test | 19:05 |
| clarkb | mordred: thanks for confirming | 19:07 |
| clarkb | anything else to add to the infra meeting agenda? I'll probably wait for ianw to start the day before sending it out but get your updates now :) | 19:29 |
| clarkb | that is really odd, the rename repo playbook test timed out after doing apparently nothing for 12 minutes? https://zuul.opendev.org/t/openstack/build/f08764252a2e4849a00779e7c62707ab/log/job-output.txt#30869 | 19:50 |
| clarkb | the sshd_log is empty making me think that we aren't actually successfully doing those ssh commands | 19:55 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Test the rename_repos playbook https://review.opendev.org/c/opendev/system-config/+/802112 | 19:59 |
| clarkb | maybe a known_hosts issue? | 19:59 |
| *** dviroel|afk is now known as dviroel | 20:04 | |
| clarkb | I took afs disk usage off the agenda beacuse we seem to have stablized below 90% utilization | 20:08 |
| clarkb | and have plans to clean up stretch in the near future | 20:09 |
| fungi | yeah, i've seen no objections (nor responses of any kind) to my proposal to do that | 20:09 |
| clarkb | to answer my question about the github PR closer that is happening with the openstack-mirroring account I assume via the zuul jobs that mirror stuff | 20:11 |
| fungi | and so probably only happens for the repos being synced that way? | 20:13 |
| clarkb | yup, but I think those are the only ones we need to worry about | 20:15 |
| fungi | good point | 20:16 |
| clarkb | ok I think it was a known_hosts issue because I haev successful jobs now looks like | 20:48 |
| clarkb | I guess that means I have to figure out a good way to run the rename playbook. Maybe I just do that from test-review.yaml | 20:48 |
| clarkb | and test-gitea.yaml I guess | 20:49 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Test the rename_repos playbook https://review.opendev.org/c/opendev/system-config/+/802112 | 21:00 |
| clarkb | if ^ works I think we may be in a mergable state. The only real delta on the production side is we use an ssh -p 29418 localhost instead of ssh -p 29418 review.opendev.org when reindexing | 21:00 |
| clarkb | that and the wait for gerrit to start up code is changed to use wait_for | 21:01 |
| opendevreview | Alex Schultz proposed opendev/system-config master: Add cdn0{1,2}.quay.io https://review.opendev.org/c/opendev/system-config/+/744038 | 21:01 |
| fungi | clarkb: huh, not sure if you saw my comments, but surprised openssh didn't mind /home/gerrit2/.ssh/known_hosts being root-owned | 21:31 |
| clarkb | fungi: ya I thought about that but I suspect it is because it is root. If it was another user it would probably complain? | 21:48 |
| clarkb | https://review.opendev.org/c/opendev/system-config/+/802112 passes testing now and I think it is a mergable state. Going to double check the job logs to make sure it does what we expect | 22:03 |
| clarkb | yup seems to do what we want and actually runs the rename_repos.yaml playbook without redoing the base job framework | 22:06 |
| clarkb | fungi: I responded to your comments on the earlier ps | 22:11 |
| opendevreview | Merged opendev/system-config master: Add cdn0{1,2}.quay.io https://review.opendev.org/c/opendev/system-config/+/744038 | 22:20 |
| clarkb | ianw: when your day starts can you let me know if you want ot add anything to the meeting agenda? I'll send that out once you've given the all clear (or in the next hour if not received otherwse) | 22:20 |
| ianw | clarkb: nothing from me | 22:37 |
| clarkb | thanks | 22:38 |
| clarkb | and agenda sent | 22:40 |
| clarkb | donnyd: hey if you are around I'd like to fix a gerrit email address conflict between two accounts for you. | 22:48 |
| clarkb | donnyd: if you can confirm the account id number you get when you login to https://review.opendev.org that would help me confirm that my plan here is valid | 22:49 |
| clarkb | fungi: as a heads up I'm putting together anothe batch that I think we can do without much user involvement. I'm getting bolder now that we've tested the fixup for other accounts recently | 22:53 |
| clarkb | definitely still a number I'm skipping over as good for reaching out though | 22:54 |
| fungi | sure, sounds great! | 22:54 |
| fungi | its in the home stretch now | 22:54 |
| *** prometheanfire is now known as Guest2352 | 22:56 | |
| clarkb | I also figure that once we get all of these done we'll be able to do online updates as well so getting there opens things up a bit more | 22:56 |
| *** ChanServ changes topic to "OpenDev is a space for collaborative Open Source software development | https://opendev.org/ | channel logs https://meetings.opendev.org/irclogs/%23opendev/" | 22:57 | |
| clarkb | what changed in that topic update? | 22:58 |
| clarkb | fungi: ^ do you know why we seemed to update our channel topics to their current channel topics? | 23:04 |
| clarkb | dpawlik: same question as for donnyd above. You have a couple of accounts that conflict on an email address. I'm fairly certain I see the correct way to clean it up but if you can pm me the account id you see on the gerrit settings page when you log in that would help me to confirm it | 23:10 |
| fungi | clarkb: my guess is a services restart did that | 23:14 |
| clarkb | ok I've got new proposed cleanups list short dpawlik and donnyd's cleanups though I'm fairly certain we can do those two as well. But scp is failing because my ssh key manager is telling me its been a day already | 23:24 |
| clarkb | I'll push these up tomorrow when I reload keys | 23:24 |
| fungi | yep, it's time to knock off | 23:24 |
| fungi | (i'm not even sure why i'm here!) | 23:25 |
| clarkb | of course if I stop keyboarding I'll have to figure out dinner | 23:28 |
| fungi | that already happened to me, so i made (and consumed) phat si-io | 23:29 |
| clarkb | that sounds really good | 23:30 |
| clarkb | ianw: https://review.opendev.org/c/opendev/system-config/+/802112/ may interst you as it touches gerrit testing in particular. | 23:31 |
| clarkb | ianw: trying to make a change that is mergable that will run our rename repos playbook against gerrit and gitea when we touch those services | 23:31 |
| fungi | it's nearly impossible to find chinese broccoli here, and we tried growing it with minimal success (thanks to a proliferation of cabbage moths/worms), so we either use store-bought western broccoli (and alter the cooking process with an additional step of steaming it by pouring sake in and slapping a lid on the wok) or we use greens from our garden (chard actually substitutes well) | 23:33 |
| ianw | clarkb: hrm, perhaps we should abstract things into a role so we don't have to call another nested ansible? | 23:45 |
| Clark[m] | ianw: but we'll run it during the downtime with that playbook... If we can keep it as much like what we'll actually run that may be better? | 23:54 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!