Machine-Enforced Version Consistency in Release Engineering

Executive Summary

A release has one version number but usually many places that claim to state it: the package manifest, the lockfile, one or more metadata/registry descriptors, the changelog header, sometimes a __version__ string or a Helm chart. When these "version faces" disagree, the failure can stay silent until a downstream consumer reads the wrong face. The motivating case is an anonymized internal incident observed on a component-platform team: a release bumped 2 of 4 required version files; an installed-version registry read a stale descriptor and generated repeat-upgrade prompts for users who had already upgraded. This is operational experience from that team, not an independently published incident report. The analysis found two layers: release consistency existed only as documentation policy, and the project's scaffold had copied only a subdirectory of the template, leaving later template rules and tests without an automatic propagation path into the generated repo.

This article surveys how mature release tooling (release-please, changesets, semantic-release, npm's version lifecycle, cargo-release, bump2version/bumpver) keeps multi-file versions synchronized; how lockfile drift specifically breaks CI; how to build a permanent consistency-gate test with positive controls (known-bad mutants) so the gate can be proven to fail; where the boundary sits between what a machine gate can prove and what only human/process review can prove; and how scaffold tools differ in whether they let engineering rules propagate into already-generated projects over time, or freeze them at generation time. It closes with concrete recommendations for agent-component platforms, which are exactly the kind of environment — many small generated repos, each carrying release/version machinery copied from a shared template — where this failure mode recurs.

The Problem: Version-Face Drift and Template Orphaning

Call each file that states a version a version face. A typical component/package has at minimum:

a package manifest (package.json, Cargo.toml, pyproject.toml)
a lockfile that pins the manifest's declared version indirectly, or in monorepo/registry setups, directly (package-lock.json, Cargo.lock)
a metadata/registry descriptor consumed by something other than the package manager — a plugin manifest, a component descriptor, an "installed version" registry entry, frontmatter in a doc
a CHANGELOG entry, which is prose but is still a face that reviewers and consumers read as ground truth

Nothing forces these to agree. A manual release process — "bump the version, update the changelog, tag it" — depends on the releaser remembering every face that exists. As a project accretes files (a scaffold added a component.yaml for a registry two years after the original manifest existed), the tribal-knowledge list of "files you must touch on release" grows and the probability that any one release misses one of them grows with it. This is exactly what happened in the motivating incident: two of four required files were bumped, and the two that were missed included the one an installed-version registry actually reads, so the registry believed the component was still at the old version and kept prompting installed users to upgrade to a version they already had.

Two distinct failures compound here, and it matters to separate them:

No failing test. The rule "these four files must state the same version" existed only in a document. Documentation cannot fail a CI run; it can only be forgotten. A rule that cannot fail is not a gate, it is a suggestion.
Template orphaning. The consistency rule did eventually get written as policy in the source template repository, but the scaffold tool that generated this component's repo had copied only a subdirectory (e.g., the app skeleton) and not the parent's CI/test scaffolding. So even after the rule existed conceptually, it had no path into the generated repo. The generated repo was orphaned from the template at generation time and stayed orphaned forever — later fixes to the template (including this consistency test itself) could not reach it without someone manually re-diffing the template against every already-generated repo.

Both failures are structural, not personnel failures — no amount of "be more careful" fixes either one. The fix accordingly has two parts: a machine-checkable consistency test with real positive controls (known-bad mutants, fixed below), and closing the propagation gap so the test and the rule it encodes actually reach every generated repo, not just the template.

Tooling Landscape Survey

Mature release tools converge on the same idea — designate one file as the source of truth for the version, then mechanically propagate it everywhere else as part of the release step, rather than trusting a human to edit N files by hand.

release-please (Google) generates release PRs from Conventional Commits and, on merge, updates CHANGELOG.md, bumps the version in the package manifest, and creates a tag/GitHub release (release-please docs). For every other file that must carry the version, it exposes an extra-files / generic-updater mechanism: you list additional files, and it edits them via format-aware updaters (JSONPath for JSON/YAML/TOML, XPath for XML) or via inline x-release-please-version / block x-release-please-start-* … x-release-please-end annotations that mark exactly where in an arbitrary text file the version string lives (release-please customizing.md). In multi-package repos it uses a manifest-driven mode keyed off .release-please-manifest.json so each component's current version is tracked centrally rather than re-derived by scanning files (release-please manifest-releaser.md). The releasable unit is the PR itself: nothing is bumped until the release PR merges, which means the version bump and the changelog update land in one dedicated, reviewable commit — a process guarantee, not just a file-sync guarantee.

Changesets takes a different authoring model suited to JS/TS monorepos: each contributor drops a small markdown "changeset" file describing the semver impact of their change at PR time. A separate changeset version step aggregates all pending changesets, computes the correct bump per package (including bumping dependents whose internal dependency versions must move too), rewrites every affected package.json, and regenerates CHANGELOG.md — keeping inter-package version references from drifting away from what's actually published (changesets docs). Because the changeset files are committed alongside the code change, review of "does this change deserve a major bump" happens at PR time, not at release time.

semantic-release fully derives the version from commit messages (Conventional Commits) and runs a fixed pipeline of plugins (analyze-commits → verify-release → generate-notes → prepare → publish). The official @semantic-release/npm plugin's prepare step updates the version in package.json and npm-shrinkwrap.json within the package root (semantic-release/npm docs); for anything else — Docker tags, a Python __init__.py, a Helm Chart.yaml, a Kubernetes manifest — the ecosystem relies on community plugins such as semantic-release-plugin-update-version-in-files (placeholder substitution against calculated version) or semantic-release-replace-plugin / semantic-release-update-file (regex/string replace across arbitrary files) (semantic-release plugins list; npm package pages). The design principle is the same as release-please's extra-files: one computed version, N declarative write targets.

npm's built-in version lifecycle is the oldest, lowest-level mechanism: npm version patch|minor|major bumps package.json (and the lockfile) and runs preversion → version → postversion hooks around one automatic git commit and tag (npm docs; libnpmversion). version runs after the manifest is bumped but before the commit, making it the natural place to hand-roll extra-file propagation on projects too small for release-please/semantic-release — e.g. "version": "node scripts/sync-version-files.js && git add -A", with postversion doing git push --follow-tags. Most bespoke internal consistency scripts end up replicating this shape even without using npm version directly.

cargo-release (Rust) automates the crates.io flow: bump Cargo.toml, test, tag, publish, and optionally move to a next development version. Its README notes a narrower caveat under dry run: because the dry-run bump is held in memory, packaging/verification output can still reflect the pre-bump version. That statement does not generalize to --execute or to arbitrary hooks. Cargo.lock also does not affect downstream consumers of a library the way Cargo.toml does (Cargo.toml vs Cargo.lock, The Cargo Book) — not every lockfile is an externally meaningful version face.

bump2version / bumpversion / bumpver (Python ecosystem, but language-agnostic in practice) use a "single source of truth, many write targets" configuration: a .bumpversion.cfg can list multiple file sections and one command performs their replacements. In bump2version, the default replacement uses an unrestricted string replacement and can update repeated identical version strings in one file; custom search/replace is useful when only selected occurrences should change. The multi-file operation is not atomic: after preflight, the implementation iterates targets and writes them individually, so interruption or a later write failure can leave a partially changed tree. Commit and tag creation are optional follow-on steps, not rollback protection for those writes.

Across the tools surveyed, the consistent lesson is to pick one source of truth and make every other registered face a generated artifact of a release step rather than a second value edited by hand.

Consistency Gates as Permanent Tests

Automated propagation at release time removes most opportunities for drift, but it does not eliminate the need for a standing test, for two reasons: propagation tooling can itself be misconfigured (a new version face gets added to the repo and nobody adds it to extra-files/.bumpversion.cfg), and some repos — like the one in the incident — do releases partly by hand. The permanent safety net is a small, pure, dependency-free test that reads every known version face and asserts they agree.

Two properties make such a test worth keeping in the suite indefinitely rather than treating it as a one-time fix:

Purity. The checker should be a pure function — file contents in, boolean/diagnostic out — with no network calls, no reliance on git state, no reliance on which CI system runs it. That makes it trivially unit-testable itself and portable if the release process changes.
Controls in both polarities. The test suite must contain positive controls — fixtures that are known to be inconsistent and must make the checker report failure — not just consistent fixtures that the checker accepts. It should also retain a negative control, a known-good consistent baseline that must not trigger an alarm. This directly answers "why does a gate that never fails prove anything?": a check that has only ever seen passing input has never demonstrated it can detect failing input, and code changes (a refactor, a regex tightened or loosened) can silently turn it into a no-op that always returns true without any test noticing. This is the same argument mutation testing makes about application test suites — a green suite proves the tests ran, not that they would catch a real defect, because coverage records which lines executed, not which claims were actually checked (CircleCI mutation testing explainer). The sharper framing: a probe needs three possible verdicts, not two — GREEN (claim holds), RED (claim doesn't hold yet), and BROKEN (the probe itself stopped measuring anything) — and every checkable claim needs a permanently retained counterexample, a "trap," that the probe must keep rejecting forever; a gate only closes when the new case passes, all previously-passing cases still pass, and all known-bad traps still fail (bordumb, "A probe that has never failed proves nothing"). Removing or "temporarily" disabling a positive-control known-bad fixture is exactly as dangerous as removing an assertion from a security check — it should require the same review scrutiny as touching the checker's pass path.

A generic sketch (language-agnostic pseudocode, deliberately minimal):

# version_consistency.py — pure function, no I/O side effects beyond reading input paths

def extract_versions(paths: dict[str, ExtractRule]) -> dict[str, str | None]:
    """paths: face name -> (file path, extraction rule e.g. JSONPath/regex/frontmatter key)
    Returns face name -> extracted version string, or None if the face is missing/unparseable."""
    return {name: rule.extract() for name, rule in paths.items()}

# Maintained from the release contract, independently of whichever keys an
# extractor happened to return in this run.
REQUIRED_VERSION_FACES = frozenset({
    "manifest", "lockfile", "registry_descriptor", "changelog"
})

def check_consistent(versions: dict[str, str | None],
                     required_faces=REQUIRED_VERSION_FACES) -> Result:
    absent = sorted(required_faces - versions.keys())
    unreadable = sorted(k for k in required_faces
                        if k in versions and versions[k] is None)
    missing = absent + unreadable
    if missing:
        return Result.fail(f"version face(s) missing or unreadable: {missing}")
    present = {k: versions[k] for k in required_faces}
    distinct = set(present.values())
    if len(distinct) > 1:
        return Result.fail(f"version faces disagree: {present}")
    return Result.ok(distinct.pop())

# --- tests, permanent, never deleted ---

def test_negative_control_all_agree():
    assert check_consistent({"manifest": "1.4.0", "lockfile": "1.4.0",
                              "registry_descriptor": "1.4.0", "changelog": "1.4.0"}).ok

def test_positive_control_one_face_stale():
    # Known-bad fixture modeling the actual incident: registry descriptor not bumped.
    result = check_consistent({"manifest": "1.4.0", "lockfile": "1.4.0",
                                "registry_descriptor": "1.3.2", "changelog": "1.4.0"})
    assert not result.ok
    assert "registry_descriptor" in result.message

def test_positive_control_face_missing_entirely():
    # Real omission: the required registry_descriptor key is absent, not None.
    result = check_consistent({"manifest": "1.4.0", "lockfile": "1.4.0",
                                "changelog": "1.4.0"})
    assert not result.ok
    assert "registry_descriptor" in result.message

The positive-control known-bad fixtures should be checked-in files (or literal dicts, as above) that are never "fixed". The required-face inventory must be maintained independently from the extractor's runtime result; otherwise deleting a configured key can shrink the observed set and make omission pass. When a fifth required face is added, update the release-contract inventory and extraction configuration, then retain positive controls for both an absent key and a stale value, plus a negative-control known-good baseline. This gate proves agreement and presence only for the inventoried faces. It cannot discover an arbitrary new consumer or version-bearing file that nobody registers; repository ownership and release review still have to keep that inventory complete.

Machine Gate vs. Process Gate: What a Working-Tree Check Can and Cannot Prove

The consistency test above proves a working-tree invariant limited to its independently maintained required-face inventory: at the commit under test, those registered faces are present, readable, and agree. It cannot discover unknown, unregistered consumers, and it cannot answer several related process questions:

Was the bump made in the same commit/PR as the change it releases? A working-tree check only sees the final state of files at HEAD; it cannot distinguish "version bumped in this PR" from "version bumped in a prior PR, and this PR just happens not to touch it." Enforcing "the release commit is dedicated and contains exactly the bump plus changelog" is a review/process concern — it's why release-please's PR-based model and changesets' changeset-file-at-PR-time model bake the bump into a distinguishable, reviewable artifact instead of relying on a generic CI check to infer intent from a diff (release-please docs; changesets docs). Community discussion of ad-hoc version-bump CI gates converges on the same limit: a check can confirm a bump exists in a diff, but confirming it was produced by an authorized, intentional release step — as opposed to a bot, a bypass, or an unrelated commit — requires either branch protection plus required-reviewer policy, or a bot identity with restricted push scope; the CI check alone cannot establish provenance or intent (community discussion on version-bump CI gates, GitHub Community #43460 thread and related engineering write-ups).
Was the correct semver magnitude chosen (patch vs. minor vs. major for the actual change)? Consistency checking is orthogonal to correctness of judgment; that's what conventional-commit analysis (semantic-release) or changeset-authoring-at-PR-time exists to structure, and ultimately what human review approves.
Is the release itself authorized / from the right branch / by the right actor? That's branch protection, required status checks, and CODEOWNERS — repository policy, not a file-content invariant.

The practical rule of thumb: a machine gate should assert something entirely self-contained in the tree — "these strings match" — because that's the class of claim a pure function can decide without ambiguity and without needing to model process, timing, or intent. Everything about how and when and by whom the bump was made belongs to branch protection, PR templates, required reviewers, or a dedicated release-automation identity (as release-please/changesets provide) — process gates that sit around the machine gate, not inside it. Trying to fold process guarantees into the file checker (e.g., trying to infer "was this commit dedicated to the release" from git log) produces a fragile, stateful test that breaks under rebases and squash-merges; keep the pure check pure, and enforce process with process tools.

Scaffold Propagation Strategies: One-Shot Copy vs. Updatable Templates

The second half of the incident — the scaffold copying only a subdirectory, orphaning the generated repo from the template's release rules — is a separate, equally common failure class: template drift. It shows up whenever a "one-shot" scaffold is used to seed many repos that are expected to keep evolving engineering standards in common.

Cookiecutter / degit / GitHub template repositories are primarily one-shot generation mechanisms; ongoing synchronization needs another layer such as Cruft or a project-specific update process. Yeoman is more programmable than that grouping suggests: its generators can locate an existing project through .yo-rc.json, resolve conflicts against existing files, and implement AST-based updates or migrations. What Yeoman does not automatically provide is Copier-style template-history reconciliation; rerunnable behavior depends on the generator author implementing it.
Copier treats scaffolding as an ongoing lifecycle. It records template/answer state and copier update compares the prior template state, a selected newer template revision, and the current project, surfacing conflicts for review. Git-backed templates are required for updates, but the selected revision need not be a tag: --vcs-ref=HEAD or another Git ref is allowed, while tags are part of the default revision-selection path. This supports reapplying template changes; it does not unconditionally guarantee that every local edit is preserved without conflict or manual resolution.

The generalizable lesson, independent of which specific tool is chosen: decide, at scaffold-design time, whether the generated repos are expected to keep receiving engineering-rule updates from the template. If yes, the scaffold mechanism must be update-capable (copier-style, or an internal equivalent — e.g., a script that vendors a "shared CI rules" subtree via a mechanism like git subtree/submodule/monorepo-shared-package that can be re-synced) — and, critically, the scaffold must copy the whole relevant surface (CI config, test scaffolding, the consistency-gate test itself), not just the application code subdirectory, or there is nothing to update into. Copying only a subdirectory of the template — as happened in the incident — silently strips out exactly the machinery (tests, CI wiring, shared config) that would otherwise have propagated later fixes.

Practical Recommendations for Agent-Component Platforms

Platforms that generate many small components/repos from a shared scaffold — which describes most agent-component ecosystems — should treat this as a two-layer problem, matching the two-layer root cause above:

Make version consistency a machine gate, not documentation, in every generated repo. Ship a pure consistency-check test with an independently maintained required-face inventory, permanent positive controls for both stale values and truly absent keys, and a negative-control known-good baseline. Wire it into CI as a required check, and prefer a release tool that mechanically bumps every registered face. The guarantee remains scoped to that inventory; adding a new consumer requires an explicit inventory-and-control update.
Make the scaffold copy — and be able to re-sync — the whole rule surface, not just app code. Audit what the scaffold actually copies into generated repos: CI workflow files, the consistency-gate test, any shared lint/release config. If the scaffold currently copies only an app subdirectory, that is the propagation gap; fix it structurally (copy the full template surface, or adopt an update-capable scaffolding tool) rather than only fixing the one incident's missing file. Decide explicitly whether generated repos are meant to receive future template fixes; if so, choose scaffold tooling that supports re-sync (copier-style three-way merge, or an internal equivalent) instead of a one-shot copy tool, since the population of generated repos will otherwise silently diverge from every future improvement to the template — this incident's root cause will recur in a different shape.
Keep the process guarantees separate from the file check. Use branch protection, required reviewers, and/or a dedicated release-bot identity to guarantee bumps happen in a distinguishable, reviewable release commit/PR (release-please- or changesets-style); do not try to encode "bumped in the right commit, by the right process" inside the pure consistency test — that check should stay simple, portable, and entirely tree-local, precisely so it stays trustworthy as the population of generated repos grows.
Treat the positive-control known-bad fixtures as permanent, protected artifacts. Require the same review scrutiny to modify or delete a known-bad fixture as to modify the checker's pass-path logic — deleting an inconvenient positive control is functionally identical to disabling the gate, just less visible in a diff.