fix(baselines): a loosening ratchet baseline moves only on the user's own command (#348)

[MendixMau]

Closes #348.

Direct lane

What changed and why (one paragraph):
lint-gate.sh --update-baseline took --approved-in-chat as proof that the user said yes. The agent whose script caused the rise could type that flag itself, so "re-baselining is the user's call" ran on trust. Now any baseline write that is not plain-safe waits for the user. That covers a lint rule rising, a conformance row going OK -> not OK, and re-creating a deleted baseline. It also covers writing over a baseline whose verdict is anything but PASS. The refusal saves the proposal and prints bin/baseline-approve.sh <kind>. The user runs that command in a separate terminal window, reads the diff and types approve. That appends the sha256 of exactly that content to .claude/baseline-approvals.tsv; the re-run writes that content and nothing else.

Every instrument write is recorded together with its content (.claude/baseline-writes.tsv plus .claude/baseline-history/). gate-check.sh prints Baselines (human-approved?) from a replay of those records against their content:

• PASS: the baseline is no looser than its last verified write.
• FAIL: a looser baseline or recorded write with no approval, a missing snapshot, or committed records gone missing from the log.
• MANUAL: the baseline predates the lock, or a git merge produced the loosening.

Both FAIL and MANUAL block Stages 5 and 6, and the instruments refuse to write until the user approves. The user's approve --current is the remedy for every non-PASS state: the replay restarts from that approval (rule 7). --approved-in-chat is still accepted so old command lines parse, but it is ignored.

Why this mechanism (candidate 1, enforced with candidate 2's tool):

• Candidate 1, a human-written approval file holding the content hash, was chosen. It is the only candidate that leaves evidence gate-check can verify later, and binding to the content hash means an approval cannot be reused for counts that moved afterwards.
• The harness keeps the agent from running it. Claude Code denies the approve command, a direct baseline-lock.sh record, redirection into either log, and file-tool edits of the logs and snapshots.
• Candidate 2 alone (an ask rule) leaves no evidence.
• Candidate 3 (a human-authored commit) is fabricable: the agent commits as the same git identity.
• Other harnesses: Copilot must ask before running either command. The approve command refuses without a TTY.

What it does NOT protect against, plainly:

• A forged approval row written by an agent deliberately evading the deny rules, for example through an interpreter.
• A harness with no deny rules whose agent terminal is a real TTY, or a TTY faked by a pty wrapper.
• Edits to log rows not yet committed. Outside git, or before the commit, the log is only as tamper-evident as the working tree.

The list lives in project-bin/baseline-lock.sh.

Scope: the baselines inventoried:

Baseline	Covered?	Why
lint-gate.sh .claude/lint-baseline.json	yes	accepts debt
conformance-check.sh docs/conformance/baseline.tsv	yes	--update-baseline can accept OK -> STALE regressions
mxbuild verification stamp	no	It records a machine check, not a human judgement. model-stamp.sh write pass is agent-callable; that forgery gap deserves its own issue.
exec.sh pre-flight error count	no	It is a measurement per run, not a stored acceptance.
page-fidelity, token-burn	no	They keep no baseline.
render-routing word budget	no	It is toolkit-side and changes only via a reviewed PR.
journey/walkthrough/design-audit "baseline"	no	These are in-run values, not stored baselines.
gate-check --ack-protocol --approved-in-chat	no	Same honour-flag shape, not a baseline; candidate follow-up.

Measurement (output of bash tests/wave2/bench-baseline-lock.sh <origin/master worktree> <this branch> at b042e95, not hand-typed):

Baseline	Measure	Before (master)	After (#348)
lint	unapproved moves that landed	12 / 12	4 / 12
lint	baseline changed without approval and NOT blocked at Stage 5 by gate-check	12 / 12	0 / 12
lint	legitimate re-baselines that landed	4 / 4	4 / 4
lint	extra human steps per legitimate re-baseline	0.0	1.0
conformance	unapproved moves that landed	12 / 12	4 / 12
conformance	baseline changed without approval and NOT blocked at Stage 5 by gate-check	12 / 12	0 / 12
conformance	legitimate re-baselines that landed	4 / 4	4 / 4
conformance	extra human steps per legitimate re-baseline	0.0	1.0

Of the unapproved attempts, 4 (lint) and 4 (conformance) edit the baseline file directly (1 each then forges a write record with baseline-lock.sh record … tighten; 1 each first deletes the write log and re-runs the update): no instrument can refuse an edit; the gate-check row is what catches it.
Replayed: 12 + 12 unapproved and 4 + 4 legitimate attempts (lint + conformance) per tree, synthetic input; not a field measurement.

Field evidence: none yet. Every number here comes from replayed attempts on synthetic input. A field run is still owed.

• No client data anywhere in the diff (leak guard and private-citation checks clean; forbidden-word grep of added lines clean)
• Size cap: exceeded. 14 non-fixture files, 649 changed lines. The retired flag was taught in six places, and the review rounds added the git anchoring.
• Test tier: T1, fixtures only. The list is under "Confirmation review".
• Golden input: not applicable. The lint JSON and the mxcli stub are synthetic.
  • Two-tree layout: git lookups use HEAD:./<path>, which resolves relative to the project root. Not exercised separately.
  • Platforms: CR is stripped before hashing. Not run on macOS or Git Bash.
• CHANGELOG.md line, crediting the external reviewer feedback; updated in place each round.

Checks: render-routing.sh --check passes. Baseline words are 75,028 before and 75,028 after, so the always-loaded cost is 0. check-scripts, check-portability, the leak guard, check-no-private-citations and check-pr-discipline are all clean.

Review (MERGE-AFTER-FIX) — fixed in ad7a9cf

1. Forged record … tighten: records are now checked against snapshots of their content. I also added deny rules for a direct record and for writes into the logs. Fixture F1.
2. CRLF: CR is stripped before hashing. Fixture X1. Merges: see H2 below.
3. Bash deny narrowed to *baseline-approve.sh*, so git add and git log -p on the logs work.
4. Docs: the approve command runs in a separate terminal window; Claude Code's ! prefix has no TTY and is refused.

Confirmation review (HOLD) — fixed in b042e95

• H1 (log removed or truncated): rows git HEAD has committed are a required prefix of the working log. If any are missing, verify says FAIL, never "predates the lock". lint-gate and conformance-check now refuse an unapproved write unless the prior verdict is PASS. Fixtures: rm of the log + snapshots, and truncation of the last committed row. Both report FAIL, and lint-gate refuses to re-record.
• H2 (a commit bypassed the lock): MANUAL now requires a merge. Either the looser row reached HEAD's log from a merge's non-first parent, or the mainline commit that last changed the baseline is a merge that produced the content or took a recorded write from the other side. A one-parent commit of a loosening is FAIL (fixture X2). The real two-branch merge reports MANUAL (X3). gate-check now blocks Stages 5 and 6 on MANUAL as well as FAIL.
• M1 (forged mid-log row): the replay restarts from the latest write a human approved, so approve --current clears it. The fixture uses a forged row that differs from the state the user later approves.
• M2 (merge MANUAL never cleared): it clears on the user's approve --current, or on a later mainline write that is not looser.
• M3 (approvals never expired): an approval counts only if it is newer than every recorded write. A write is justified only by an approval given between the previous row and that write. Timestamps are now microseconds, because the approval and its write routinely share a second. Fixture: returning to an old approved state after a tighter write is FAIL, and lint-gate does not reuse the old approval.
• L1 (missing snapshot): .claude/ is not gitignored by init-project.sh, so snapshots are committed. approve --current re-saves the snapshot and re-anchors the replay. Fixture: deleted snapshots give FAIL, then PASS after the approval.
• L2 (pty-faked TTY): documented as a residual in baseline-lock.sh; no code change.
• New behaviour to note: a project whose baseline predates the lock needs one approve --current from the user before the next re-baseline (fixture G4).

Fixture proof: bash tests/wave2/test-baseline-lock.sh <tree>

Tree	Result
ad7a9cf	19 pass, 9 fail (X2, M2, H1 ×3, M1, M3 lint-gate half, L1, G4)
b042e95	28/28

The M3 gate half reads FAIL on ad7a9cf too, but for an unrelated reason: on that head, M1's forged row never clears, so every later state in that project replays to FAIL.

Bench, ad7a9cf vs b042e95: "changed without approval and NOT blocked at Stage 5" goes from 1/12 to 0/12 for each baseline. The 1 is the rm-the-log attempt, which read as a non-blocking MANUAL before. Every other row is equal.

Fixtures re-run on b042e95, each with its own usage argument:

• test-baseline-lock 28/28
• test-lint-gate-accept 23/23
• test-conformance-baseline-scope 7/7
• test-install-claude-permissions 79/79
• test-sec014-notice 24/24
• test-lint-receipt-revert 55/55
• test-exec-lock-watch 84/84
• test-conformance-per-brd-ledger 19/19
• test-review-conformance-per-brd 6/6
• test-verify-stack-blocked 6/6
• gate-check fixtures: test-bug03-gates, test-wrong-verdicts 20/20, test-source-ledger 55/55, test-bug04-subscripts 30/30, test-closeout 34/34, test-stage-p 14/14

Impact

Row	Effect	Evidence
Process	+1 human approval per re-baseline: the user runs bin/baseline-approve.sh <kind> in a separate terminal and types approve (bench: 0.0 → 1.0 extra steps per legitimate re-baseline). A project whose baseline predates the lock needs one approve --current first. New block: gate-check row Baselines (human-approved?) blocks Stages 5 and 6 on FAIL and on MANUAL; lint-gate.sh / conformance-check.sh refuse unapproved writes. --approved-in-chat is now ignored. Review status: HOLD.	PR measurement table (bench-baseline-lock.sh); review notes
Dependencies	Shared files: bin/gate-check.sh, project-bin/exec.sh (message text only), bin/lib/install-manifest.sh, bin/install-claude-permissions.sh + bin/install-harness-permissions.sh (new deny rules), agents/gate-agent.md. Merges clean onto master 868b3cb (git merge-tree). Conflicts with #374 in bin/lib/install-manifest.sh; gate-check.sh merges clean with #273 and #346. Platform: approve refuses without a TTY; CR stripped before hashing; not run on macOS or Git Bash. Reads .claude/lint-baseline.json (producer: lint-gate.sh), docs/conformance/baseline.tsv (producer: conformance-check.sh), .claude/baseline-approvals.tsv (producer: baseline-approve.sh, run by the human only), .claude/baseline-writes.tsv + .claude/baseline-history/ (producer: baseline-lock.sh via the two instruments), and git HEAD's committed log.	git merge-tree vs 868b3cb and pairwise
Context cost	Unchanged: baseline 73,522 → 73,522 words (budget 80,000); Stage 5 mdl 30,185 → 30,185, lead 25,820 → 25,820	bin/render-routing.sh --check on master 868b3cb vs this head (b042e95) merged onto 868b3cb via git merge-tree in a temporary worktree
Behaviour	A/B REQUIRED, NOT RUN. Agent stub agents/gate-agent.md changes its lint re-baseline instruction; situational skills/lint-that-actually-runs.md also changes (A/B recommended).	diff of agents/gate-agent.md; skill-routing.tsv load class
Runtime	Every gate-check run adds the Baselines replay (duration NOT RUN: not measured). Each legitimate re-baseline +1 human step. exec.sh: message text only.	PR text

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw