Qualify UK model and data candidates automatically before promotion

[MaxGhenis]

Fixes #573

Draft, stacked on #571 (receipt schema v1). The diff includes #571's commits until that merges; review origin/w4/receipt-schema-v1...HEAD.

Part of the UK Autumn Budget 2026 CI plan (workstream W4). The Budget is on 28 October and there is no model freeze, so the latest qualified model/data pair is what runs that day. This PR builds the workflow that qualifies a candidate pair and promotes it.

What it adds

Workflow (.github/workflows/certify-uk-release.yaml, scripts/check_uk_candidate.py)

• Triggered by a policyengine-uk or UK data release dispatch, by hand, and on a schedule that catches up anything missed.
• resolve finds the candidate pair and deduplicates: a pair that is already certified, stale or in flight is not run twice.
• prepare certifies the candidate on a branch: manifest, pins, lockfile and TRO are regenerated and tested together, and a draft PR is opened.
• gates runs the local gates; request-replay and poll-replay run the scorecard replay; finalize writes the qualification receipt to the ledger.
• promote is serialized (concurrency promote-uk) and picks the dominating promotable pair, whatever order the candidates finished in. accept records a human acceptance of a needs_review receipt.

Gates (provenance/uk_candidate.py). All eleven are mandatory; a gate that cannot run reports unavailable, which makes the verdict incomplete, never qualified.

Gate	What it checks
installed_versions, core_compatibility	The installed model and core are the candidate's, and the model's own requirement admits that core.
wheel_hashes	The downloaded wheels match PyPI and the pin, their members match their RECORDs, and the installed files match the wheels.
data_access, dataset_bytes, supported_years	The release manifest and dataset are reachable; the simulated file has the pinned bytes and the UK layout; the required years are supported.
budget_inputs	52 inputs the Budget measures depend on are present and usable, compared with the previous pair.
reform_contract	W1's contract runner passes on every route. The gate reads the whole report: cases, sampled properties and population checks. It rejects a report that is not a full run of this candidate (other versions, dataset, seed or sample size; a property that did not run; a holds or summary that disagrees with the rows).
metrics_review	W3's metrics tool, vendored from a pinned policyengine-uk commit, on the candidate and the previous pair, including W3's standard reform costings. Hard checks fail; moves outside the review bounds need review.
reform_responsiveness	Every one of W3's ten standard reforms ran and moved its target variable in the stated direction by at least its yearly minimum, in every year from 2025-26 to 2031-32. The set, targets and minimums are the metrics tool's own (standard_reforms.yaml); the tool checks them as responsive:<id>. This gate requires those checks to be present and passing, and recomputes each verdict from the tool's per-year numbers. A reform that returns £0 or is nearly inert fails the candidate; there is no review path.
replay	W6's replay corpus, through an immutable request whose id is the digest of its content. The receipt must name that request, match the status file's digest, and name the candidate's wheels, data and bundle. Structural, coverage, mapping, inertness and construction flags block; numerical and support flags need review.

Certification (certification.py, manifest.py)

• A new basis, qualified_exact_pair, records the pair id and its ledger URL and certifies exactly one model version, one core version and one dataset digest. It is never broadened to a range.
• The runtime accepts that basis only for the exact model version and build id.

Ledger (provenance/qualification_ledger.py, .github/qualification-ledger.sh): an orphan branch, uk-qualification-ledger, with one directory per pair and one per attempt.

Consumers (push.yaml, scripts/release_consumers.py): on release, a policyengine-release dispatch carries the UK bundle identity (model, core, data package version, dataset SHA-256, certificate digest) and the qualification verdict, promotable flag and receipt digest. It goes to sim-api and api as before, and now to the dashboard and the scorecard, which also gets a released-bundle replay request.

Invariants (each tested)

1. The verdict is a pure, monotone function of the gates: rejected > incomplete > needs_review > qualified. Making any gate worse never improves it (Hypothesis).
2. Promotion is independent of arrival order and never moves to a dominated pair (Hypothesis over permutations).
3. A request's id is the RFC 8785 digest of its document, and the scorecard's golden request and receipt reproduce byte for byte.
4. The mirrors and the scorecard's schemas agree (differential): both accept the golden documents and reject the same mutated ones.
5. An exact-pair certificate needs evidence: validation refuses one without a promotable ledger receipt for that pair.
6. The contract gate never passes a partial or foreign report. Tests use W1's real reports: the full all-pass run passes, the pinned 2.125.1 run fails on its 101 defects, and each of ten tamperings is an error.
7. A reform that does nothing never qualifies. The responsiveness gate is monotone in each reform's response: shrinking a value toward zero or flipping its sign never turns a failure into a pass (Hypothesis). Tests cover exactly £0, nearly inert, one inert year, wrong sign, a missing year, a reform that did not run and a report with no reform checks at all, on W3's real reform list. A differential test makes the gate error when the tool's verdict disagrees with its own numbers.
8. expected_release_bundle is computed as the scorecard computes it: no tokens and no network for the manifest request, so the packaged certification is used. A test runs the manifest fetch in that environment and sees it fail over.

Open items

• Decision d1344. Automatic promotion and the review bounds wait on it. Until then promotion.automatic is false, the bounds in uk_review_bounds.json are proposals, and promotion is a reviewed PR merge. The responsiveness minimums are W3's proposals in their tool, also pending d1344.
• Metrics tool pin. The tool is pinned to policyengine-uk#2335's head when I read it (fd944159), an unmerged commit. It moves to the merged commit when that PR lands.
• A real run is queued. One end-to-end run of the runner and this gate on the certified pair is waiting in the host's heavy-run queue; I'll add its result here.
• Admin, with the coordinating session: the GitHub App on policyengine-scorecard and uk-autumn-budget-dashboard-2026 with Contents: write; sender tokens that cover policyengine.py; the scorecard's Hugging Face secret.
• 2031. The Budget horizon includes 2031, but policyengine-uk projects to 2030. The supported_years gate reports that as needs_review.
• Dependencies. The reform_contract gate reports unavailable until W1's runner (Add the UK Budget reform contract runner #577, after Route UK dated reforms through policyengine-uk's statutory pipeline #575) is importable from the installed wrapper. The replay schemas are pinned to an unmerged scorecard commit (dc040de).

Checks run locally

• 166 tests pass in the qualification, gate, pipeline and certification suites.
• ruff is clean; actionlint is clean on the workflow.
• Independent review (GPT-6.1 Sol) is running at def8b8d. The responsiveness gate (28c90ea, 4da37e4) landed after it started and will get its own review pass.

axiom: n/a: release/qualification infrastructure, no policy change.

🤖 Generated with Claude Code