How to ensure that release tags are uniformly created throughout a cohort of closely-related Git repositories?

When two repositories depend so much on each other that tags must correspond to each other, I think that's a strong indication using one instead of two repos would be the better choice for your system. Going that route, you may end with a single repo for all your components. As long as you structure that repository in a sensible manner, this can be a viable option (known as mono-repo).

If you have some factual reasons against this approach, you could alternatively

• write a script which tags certain groups of repos with a similar or corresponding tag (so make it easy for the team to tag correctly)

• write a verification script which identifies wrong or missing tags, and run this script regularly.

I doubt you need anything like Git hooks for this. And of course, you may also question the need of corresponding tags at all, as Ewan wrote in his answer. Individual repos should be used for components which have their own life line, each one with their own version or tag history. Which version of X requires which version of Y is best expressed by the build configuration, not by tags in version control.

If you have a project (A) which consumes the published version of other projects (B) there is no need to tag the consumed projects (B) with a "this version is used by A v1.X"

The build of project A will have a dependency list which includes the required version for each library. You don't need to sync the versions.

You wouldn't phone up bill gates and say "please tag Microsoft.SqlClient with Ewan'sAmazingApp v0.3.112"

I regularly run into situations where one or more of the satellite repositories are missing their own matching tag for a new release version published at the primary repo.

There is a core contradiction here in keeping separate Git repositories for things that you wish to have a shared versioning lifecycle.

My initial attempts to curb this phenomenon with more detailed release process documentation and checklist style workflows have sadly failed to move the needle on this

Manual workflows tend not be ideal solutions, especially in the field of software development which is all about automation and streamlining in as much as is possible. Manual box ticking should be a last resorts, not an ideal fix.

how best to approach the matter programmatically to have the creation of a new tagged release in the top-most repository in the hierarchy create a condition in all the others for the next commit pushed to include an identically named tag.

I'm going to call this a potential XY problem, relating back to the first point in this answer. The inherent setup of the repositories works against your intention to synchronize the tag versions.

Repositories are relatively hard boundaries for independent application lifecycles. You can have smaller lifecycles (e.g. monorepo), but you can't feasibly have lifecycles that span multiple separate git repositories since they inherently have different commit histories and there is no direct relation between commits made to both histories.

Why would a release of one application force the release pattern for another? That makes little sense to me on the face of it, and your question is light on an example context.

In a scenario where application B (the consumer) depends on application A (the library), it's relatively easy to agree that an update in B does not necessitate an update in A.
Harder to see, but still correct is that an update in A also does not inherently necessitate an update in B. B might not be able to pivot into the new version of A immediately because of B's own roadmap.

If A is not a library but an application with its own runtime, that means that A and B have a tight coupling between them to the point of you expecting them to have joint releases to keep things in working order.
That strongly suggests that A and B are not, in fact, independent applications, and what you're dealing with here is what is often tongue-in-cheek referred to as a distributed monolith.

To extend the last point, that is not to say that independent applications are not allowed to call each other as part of their runtime. It's even possible that this is a two-way street.
But you need to safeguard the independent SDLC of each application. In order to maintain the ability to deploy one application without the other, that generally entails a careful API versioning strategy which ensures that while one application can release an upgraded version of its API, if the changes are breaking changes it should expose the upgraded API on different endpoints, without changing the original endpoints.

This ensures that the consumer does not immediately break when the other service is deployed. Instead, it keeps making use of the old endpoints until it sees fit to upgrade itself to the newer endpoint.

Realistically, for your new feature to be fully implemented you obviously will need to upgrade both the API and its consumer, but it can be done in a way that you don't have to align their deployment to be synchronized.

Having separate SDLCs doesn't mean that you can't cooperate to deliver a feature, but it does mean that you shouldn't need to coordinate identical release patterns.

Put all satellites in submodules of an aggregate repository and tag the aggregate.

Instead of making a decision to link to a commit in a sattelite per release, you would have to do it on each commit. This is not necessarily a bad thing, because a commit without a clear indication of dependency version is less useful.

I know that Git submodules are disliked but this is the exact scenario they are supposed to implement - to make a cohesive link across commits in multiple repositories.

What it seems you're trying to achieve is to release all the applications from all the repositories every time one of the applications needs a release. If that's the case that makes each "satelite" repository a module of the project using it and one possible solution is to change the setup so that there is one repository hosting the project with corresponding modules for each "satelite" repository and release every time from the root of the project.