From 9fa8020951ccb85cf8bdf882959ce8a428313154 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 14:12:05 +0000 Subject: [PATCH 1/4] chore(context): runbook slice for helpers; runbook drops its copy of the baseline table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit T2: skills/conversion-runbook.md is no longer a render-routing surface; its rendered baseline table (a fourth copy) is replaced by a one-line pointer to CLAUDE.local.md and ROUTING.md. T3: bin/runbook-slice.sh prints §1b plus that stage's section. The span logic moves from gate-check.sh into bin/lib/runbook-span.sh, shared by both and by the per-role count. The runbook routing row is split: lead/ba/architect keep the runbook, mdl/gate/test/review get the slice; routing_baseline_pack counts the slice's printed words. Adds tests/wave2/test-runbook-slice.sh and tests/bench/trim-noloss.sh; updates test-routing-tier.sh. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 1 + README.md | 3 +- ROUTING.md | 3 +- agents/gate-agent.md | 2 +- agents/mdl-agent.md | 2 +- agents/review-agent.md | 2 +- agents/test-agent.md | 2 +- bin/check-skill-routing.sh | 4 +- bin/gate-check.sh | 14 ++--- bin/init-project.sh | 3 +- bin/lib/runbook-span.sh | 46 +++++++++++++++ bin/lib/skill-routing.sh | 22 +++++++- bin/lib/skill-routing.tsv | 3 +- bin/render-routing.sh | 4 +- bin/runbook-slice.sh | 45 +++++++++++++++ skills/conversion-runbook.md | 55 +----------------- tests/bench/trim-noloss.sh | 79 ++++++++++++++++++++++++++ tests/wave2/test-routing-tier.sh | 54 ++++++++++-------- tests/wave2/test-runbook-slice.sh | 93 +++++++++++++++++++++++++++++++ 19 files changed, 339 insertions(+), 98 deletions(-) create mode 100644 bin/lib/runbook-span.sh create mode 100755 bin/runbook-slice.sh create mode 100755 tests/bench/trim-noloss.sh create mode 100644 tests/wave2/test-runbook-slice.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index ff60ea83..0fbc0bc7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- chore(context): **helpers read the runbook slice for their stage, and the runbook no longer carries a copy of the baseline table — mdl at Stage 5 reads 56,236 → 43,352 words; baseline 75,028 → 73,233.** `skills/conversion-runbook.md` held a fourth rendered copy of the baseline routing table (README, ROUTING.md and every project's CLAUDE.local.md carry it too); it is out of the render surfaces and replaced by a one-line pointer. New `bin/runbook-slice.sh ` prints §1b plus that stage's section, using the span logic gate-check already had (now `bin/lib/runbook-span.sh`, shared). The runbook routing row is split: lead, ba and architect keep the runbook; mdl, gate, test and review get the slice, and `render-routing.sh --check` counts the slice's printed words for them, so the per-role numbers stay honest. New `tests/bench/trim-noloss.sh` checks that every removed line is found again or is a still-routed table row. Behaviour A/B owed before merge — context-cost research (r-stub-diet, T2+T3) - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau diff --git a/README.md b/README.md index 618fe870..27d927d8 100644 --- a/README.md +++ b/README.md @@ -775,7 +775,8 @@ Then filter by Role(s). A helper dispatched with a role stub (ba, architect, mdl | Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict | `skills/degrade-to-judgement.md` | every stage | every role | | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | `skills/tool-output-is-not-ground-truth.md` | every stage | every role | | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | `skills/retesting-learned-rules.md` | every stage | every role | -| Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | `skills/conversion-runbook.md` | P,0,1,2,3,4,5,6,7 | every role | +| Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | `skills/conversion-runbook.md` | P,0,1,2,3,4,5,6,7 | lead, ba, architect | +| Any pipeline work at all, as a helper — every dispatch, before producing anything: run the toolkit's bin/runbook-slice.sh , which prints §1b plus your stage's section of the conversion runbook; never open skills/conversion-runbook.md whole | `bin/runbook-slice.sh` | P,0,1,2,3,4,5,6,7 | mdl, gate, test, review | | Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order | `skills/query-the-model.md` | P,0,1,2,3,4,5,6,7 | every role | | Putting a question TO the user — any gate, any stage: ask in chat not in a file, two named options plus your recommendation, one batch per gate then end the turn | `skills/interview-protocol.md` | P,0,1,2,3,4,5,6,7 | lead, ba, architect | | The user gives feedback or asks for a change mid-project — "change X", "it also needs Y", a remark on a screenshot or the running app, a stakeholder's ask relayed in chat: log a CR row in PROJECT.md ## Change log BEFORE any edit, size it, then edit the highest artifact it alters (BRD, decision, architecture, wireframe), never the MDL first | `skills/change-intake.md` | 2,3,4,5,6,7 | lead, ba | diff --git a/ROUTING.md b/ROUTING.md index c99f42a7..ba0f7e17 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -32,7 +32,8 @@ picks the row up. That is the whole procedure — there is no second list to rem | Always relevant for | Load this | Agent(s) | Stage(s) | Tier | |---|---|---|---|---| -| Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | `skills/conversion-runbook.md` | all | P,0,1,2,3,4,5,6,7 | baseline | +| Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | `skills/conversion-runbook.md` | lead,ba,architect | P,0,1,2,3,4,5,6,7 | baseline | +| Any pipeline work at all, as a helper — every dispatch, before producing anything: run the toolkit's bin/runbook-slice.sh , which prints §1b plus your stage's section of the conversion runbook; never open skills/conversion-runbook.md whole | `bin/runbook-slice.sh` | mdl,gate,test,review | P,0,1,2,3,4,5,6,7 | baseline | | Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order | `skills/query-the-model.md` | all | P,0,1,2,3,4,5,6,7 | baseline | | Before writing any .js or .sh for a check, gate or report — and before adding a rule to an existing one: judgement goes in a skill, code only fetches facts a reader cannot | `skills/skills-over-scripts.md` | all | - | baseline | | Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict | `skills/degrade-to-judgement.md` | all | - | baseline | diff --git a/agents/gate-agent.md b/agents/gate-agent.md index b7da6484..0fbccec0 100644 --- a/agents/gate-agent.md +++ b/agents/gate-agent.md @@ -30,7 +30,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | Load this | When | |---|---| -| `skills/conversion-runbook.md` | Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | +| `bin/runbook-slice.sh` | Any pipeline work at all, as a helper — every dispatch, before producing anything: run the toolkit's bin/runbook-slice.sh , which prints §1b plus your stage's section of the conversion runbook; never open skills/conversion-runbook.md whole | | `skills/query-the-model.md` | Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order | | `skills/skills-over-scripts.md` | Before writing any .js or .sh for a check, gate or report — and before adding a rule to an existing one: judgement goes in a skill, code only fetches facts a reader cannot | | `skills/degrade-to-judgement.md` | Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index 1aa9e31b..461549dc 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -36,7 +36,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | Load this | When | |---|---| -| `skills/conversion-runbook.md` | Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | +| `bin/runbook-slice.sh` | Any pipeline work at all, as a helper — every dispatch, before producing anything: run the toolkit's bin/runbook-slice.sh , which prints §1b plus your stage's section of the conversion runbook; never open skills/conversion-runbook.md whole | | `skills/query-the-model.md` | Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order | | `skills/skills-over-scripts.md` | Before writing any .js or .sh for a check, gate or report — and before adding a rule to an existing one: judgement goes in a skill, code only fetches facts a reader cannot | | `skills/degrade-to-judgement.md` | Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict | diff --git a/agents/review-agent.md b/agents/review-agent.md index 02245473..ba5ca239 100644 --- a/agents/review-agent.md +++ b/agents/review-agent.md @@ -36,7 +36,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | Load this | When | |---|---| -| `skills/conversion-runbook.md` | Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | +| `bin/runbook-slice.sh` | Any pipeline work at all, as a helper — every dispatch, before producing anything: run the toolkit's bin/runbook-slice.sh , which prints §1b plus your stage's section of the conversion runbook; never open skills/conversion-runbook.md whole | | `skills/query-the-model.md` | Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order | | `skills/skills-over-scripts.md` | Before writing any .js or .sh for a check, gate or report — and before adding a rule to an existing one: judgement goes in a skill, code only fetches facts a reader cannot | | `skills/degrade-to-judgement.md` | Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict | diff --git a/agents/test-agent.md b/agents/test-agent.md index b9fb108d..1bb1985f 100644 --- a/agents/test-agent.md +++ b/agents/test-agent.md @@ -29,7 +29,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | Load this | When | |---|---| -| `skills/conversion-runbook.md` | Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | +| `bin/runbook-slice.sh` | Any pipeline work at all, as a helper — every dispatch, before producing anything: run the toolkit's bin/runbook-slice.sh , which prints §1b plus your stage's section of the conversion runbook; never open skills/conversion-runbook.md whole | | `skills/query-the-model.md` | Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order | | `skills/skills-over-scripts.md` | Before writing any .js or .sh for a check, gate or report — and before adding a rule to an existing one: judgement goes in a skill, code only fetches facts a reader cannot | | `skills/degrade-to-judgement.md` | Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict | diff --git a/bin/check-skill-routing.sh b/bin/check-skill-routing.sh index 11dbbb93..a20a2725 100755 --- a/bin/check-skill-routing.sh +++ b/bin/check-skill-routing.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Proves the skill-routing table actually reaches an agent, not just that it renders. # -# render-routing.sh --check already proves the nine in-repo surfaces match the table. What it +# render-routing.sh --check already proves the in-repo surfaces match the table. What it # does NOT catch: two rows sharing a name (one silently shadows the other in agent: views), # a [[wikilink]] between skills pointing at a file that was never written, or — the part that # actually matters — whether bin/lib/skill-routing.sh's routing_sync_claude_local() genuinely @@ -62,7 +62,7 @@ while IFS= read -r -d '' f; do done < <(find "$ROOT/skills" -name '*.md' -print0) [ "$dangling" -eq 0 ] && ok "no dangling cross-references" || FAIL=1 -echo "== 5. the nine in-repo surfaces match the table ==" +echo "== 5. the in-repo surfaces match the table ==" if "$SCRIPT_DIR/render-routing.sh" --check >/tmp/ckroute.render 2>&1; then ok "README/ROUTING.md/agent templates/gate-check.sh all in sync" else diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 61f648b3..77b06c21 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -2379,12 +2379,10 @@ fi # grepping the runbook's actual headings ("### Stage N — …", "## 1b. …") on every run — never # a hard-coded number, which would be wrong after the next edit. The runbook's own "how to # read this" paragraph states the same rule; this line is what makes it actionable. -runbook_span() { # runbook_span — "A–B" (1-based, inclusive) or "" - awk -v h="$2" -v e="$3" ' - !start && $0 ~ h { start = NR; next } - start && $0 ~ e { end = NR - 1; exit } - END { if (start) { if (!end) end = NR; printf "%d–%d", start, end } }' "$1" 2>/dev/null -} +# The span logic lives in lib/runbook-span.sh, shared with bin/runbook-slice.sh (which prints +# these same lines for a helper that never runs a gate) and with the per-role word count in +# lib/skill-routing.sh. +. "$TOOLKIT_DIR/bin/lib/runbook-span.sh" if [ -n "$REQUESTED_STAGE" ]; then RB_FILE="$TOOLKIT_DIR/skills/conversion-runbook.md" case "$REQUESTED_STAGE" in @@ -2395,8 +2393,8 @@ if [ -n "$REQUESTED_STAGE" ]; then # Keyed on the " — " every stage heading carries: "### Stage 0 runs in every entry mode" # (under Entry Modes) also starts with "### Stage 0", and matched first without it. RB_HEAD="$(grep -m1 "^### Stage $RB_STAGE — " "$RB_FILE" | sed 's/^### //; s/[[:space:]]*$//')" - RB_SPAN="$(runbook_span "$RB_FILE" "^### Stage $RB_STAGE — " "^###? ")" - RB_1B="$(runbook_span "$RB_FILE" "^## 1b\\." "^## ")" + RB_SPAN="$(runbook_stage_span "$RB_FILE" "$RB_STAGE")" + RB_1B="$(runbook_1b_span "$RB_FILE")" if [ -n "$RB_HEAD" ] && [ -n "$RB_SPAN" ] && [ -n "$RB_1B" ]; then printf 'Read for this gate: skills/conversion-runbook.md §"%s" (lines %s) + §1b Live Checklist (lines %s) — not the whole file\n' \ "$RB_HEAD" "$RB_SPAN" "$RB_1B" diff --git a/bin/init-project.sh b/bin/init-project.sh index 6523aabc..cabed48e 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -309,7 +309,8 @@ This project uses the shared toolkit at \`$TOOLKIT_ROOT\`. For ANY pipeline work 1. **Read \`$TOOLKIT_ROOT/skills/conversion-runbook.md\` §1b plus your own stage's section FIRST — every session; not the whole file** (\`bin/gate-check.sh \` prints the line spans). It is the executable spec. The toolkit README and toolkit-guide.html are orientation only; stage - summaries anywhere are routing, not deliverable lists. + summaries anywhere are routing, not deliverable lists. A helper agent (mdl, gate, test, review) + runs \`$TOOLKIT_ROOT/bin/runbook-slice.sh \` instead, which prints exactly those lines. 2. **Before producing any stage artifact, open that stage's owning skill** and follow its full output list (e.g. Stage 3 design work = \`design-artifacts.md\`, incl. one wireframe per screen — not just a design system). diff --git a/bin/lib/runbook-span.sh b/bin/lib/runbook-span.sh new file mode 100644 index 00000000..de021d23 --- /dev/null +++ b/bin/lib/runbook-span.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# runbook-span.sh — locate the runbook sections a stage session needs, by heading, never by a +# hard-coded line number. Source it; do not execute it. +# +# Callers: bin/gate-check.sh (the "Read for this gate" line), bin/runbook-slice.sh (prints the +# lines), bin/lib/skill-routing.sh (counts the slice's words for the per-role budget lines). +# One copy, so the three never disagree about where a section starts or ends. +# +# What a stage session needs is §1b (the Live Checklist Protocol, which applies to every stage) +# plus its own stage's section under §2. Stage headings are keyed on the " — " every one carries: +# "### Stage 0 runs in every entry mode" (under Entry Modes) also starts with "### Stage 0", and +# matched first without it. +# +# bash 3.2 (stock macOS) and POSIX awk. + +# runbook_span — "A–B" (1-based, inclusive) or "" +runbook_span() { + awk -v h="$2" -v e="$3" ' + !start && $0 ~ h { start = NR; next } + start && $0 ~ e { end = NR - 1; exit } + END { if (start) { if (!end) end = NR; printf "%d–%d", start, end } }' "$1" 2>/dev/null +} + +# runbook_stage_token — P, 0-7 as the runbook spells them ("build-ready" reads Stage +# 5's section; "p" is "P"). Prints nothing for anything else. +runbook_stage_token() { + case "$1" in + build-ready) echo "5" ;; + p|P) echo "P" ;; + 0|1|2|3|4|5|6|7) echo "$1" ;; + *) : ;; + esac +} + +# runbook_stage_span — the "A–B" span of "### Stage N — …" +runbook_stage_span() { runbook_span "$1" "^### Stage $2 — " "^###? "; } +# runbook_1b_span — the "A–B" span of "## 1b. …" +runbook_1b_span() { runbook_span "$1" "^## 1b\\." "^## "; } + +# runbook_print_span <"A–B"> — the lines themselves. +runbook_print_span() { + local a b + a="${2%%–*}"; b="${2##*–}" + [ -n "$a" ] && [ -n "$b" ] || return 1 + sed -n "${a},${b}p" "$1" +} diff --git a/bin/lib/skill-routing.sh b/bin/lib/skill-routing.sh index c0f3dbd8..f6fb0320 100644 --- a/bin/lib/skill-routing.sh +++ b/bin/lib/skill-routing.sh @@ -4,7 +4,6 @@ # Every routing surface in this repo is a VIEW of that table, rendered by this library: # ROUTING.md full (human-readable index) # README.md readme-baseline + readme-situational -# skills/conversion-runbook.md readme-baseline # agents/-agent.md agent: # bin/gate-check.sh stage-map (shell case arms) # /CLAUDE.local.md baseline (written by init-project.sh / sync-project.sh) @@ -152,6 +151,19 @@ _routing_stage_arm() { # Optional third argument (lead|ba|architect|mdl|gate|test|review): count only rows # whose agents cell is "all" or names that role — what one helper at that stage is pointed # at. Omitted = every role, the number this function always returned. +# +# One script row IS counted: bin/runbook-slice.sh. Its whole output is read into context — +# it replaces the runbook for the helper roles — so it is counted as the words it prints for +# (about 1.5k at Stage 5), not as zero and not as the script's own text. With an +# unrecognised (the "full" pack) it is counted as the whole runbook, the upper bound. +_ROUTING_READ_SCRIPTS="bin/runbook-slice.sh" +_routing_slice_words() { # _routing_slice_words + if [ "$3" -eq 1 ]; then + LC_ALL=C wc -w < "$1/skills/conversion-runbook.md" 2>/dev/null | tr -d '[:space:]' + else + bash "$1/$_ROUTING_READ_SCRIPTS" "$2" 2>/dev/null | LC_ALL=C wc -w | tr -d '[:space:]' + fi +} routing_baseline_pack() { local stage="$1" root="$2" role="${3:-}" words=0 files=0 paths="" p wc_out local name path when agents stages tier group @@ -159,14 +171,18 @@ routing_baseline_pack() { case "$known_stages" in *" $stage "*) : ;; *) full=1 ;; esac while IFS=$'\t' read -r name path when agents stages tier group; do [ "$tier" = "baseline" ] || continue - case "$path" in *.md) : ;; *) continue ;; esac + case "$path" in *.md|"$_ROUTING_READ_SCRIPTS") : ;; *) continue ;; esac _routing_reaches "$agents" "$role" || continue if [ "$full" -eq 0 ] && [ "$stages" != "-" ]; then _routing_has "$stages" "$stage" || continue fi p="$root/$path" [ -f "$p" ] || continue - wc_out="$(LC_ALL=C wc -w < "$p" 2>/dev/null | tr -d '[:space:]')" # LC_ALL=C: see BASELINE_BUDGET in render-routing.sh + if [ "$path" = "$_ROUTING_READ_SCRIPTS" ]; then + wc_out="$(_routing_slice_words "$root" "$stage" "$full")" + else + wc_out="$(LC_ALL=C wc -w < "$p" 2>/dev/null | tr -d '[:space:]')" # LC_ALL=C: see BASELINE_BUDGET in render-routing.sh + fi [ -n "$wc_out" ] || wc_out=0 words=$((words + wc_out)) files=$((files + 1)) diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index edf47908..6355d9bd 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -68,7 +68,8 @@ # protocol-staleness ack on every routing edit. # # name path when agents stages tier group -conversion-runbook skills/conversion-runbook.md Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only all P,0,1,2,3,4,5,6,7 baseline spine +conversion-runbook skills/conversion-runbook.md Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only lead,ba,architect P,0,1,2,3,4,5,6,7 baseline spine +runbook-slice bin/runbook-slice.sh Any pipeline work at all, as a helper — every dispatch, before producing anything: run the toolkit's bin/runbook-slice.sh , which prints §1b plus your stage's section of the conversion runbook; never open skills/conversion-runbook.md whole mdl,gate,test,review P,0,1,2,3,4,5,6,7 baseline spine query-the-model skills/query-the-model.md Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order all P,0,1,2,3,4,5,6,7 baseline spine skills-over-scripts skills/skills-over-scripts.md Before writing any .js or .sh for a check, gate or report — and before adding a rule to an existing one: judgement goes in a skill, code only fetches facts a reader cannot all - baseline spine degrade-to-judgement skills/degrade-to-judgement.md Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict all - baseline spine diff --git a/bin/render-routing.sh b/bin/render-routing.sh index ecde20ef..eb50ab83 100755 --- a/bin/render-routing.sh +++ b/bin/render-routing.sh @@ -33,10 +33,12 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" # The surfaces. Anything listed here MUST carry at least one ROUTING:BEGIN marker; a surface # that has lost its markers is a structural error (exit 1), not silence — that silent-drop is # precisely how the old six-copy arrangement lost track of a table. +# skills/conversion-runbook.md left this list 2026-10-14: it carried a fourth rendered copy of the +# baseline table (~1,800 words) inside a baseline file every role reads, the same rows already in +# README, ROUTING.md and every project's CLAUDE.local.md. It now points at those instead. SURFACES=" ROUTING.md README.md -skills/conversion-runbook.md agents/ba-agent.md agents/architect-agent.md agents/mdl-agent.md diff --git a/bin/runbook-slice.sh b/bin/runbook-slice.sh new file mode 100755 index 00000000..3ea917a8 --- /dev/null +++ b/bin/runbook-slice.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# runbook-slice.sh — print the part of the conversion runbook a helper at one stage needs: +# §1b (the Live Checklist Protocol, every stage) plus that stage's section under §2. +# +# bin/runbook-slice.sh 5 # §1b + "### Stage 5 — Build" +# bin/runbook-slice.sh build-ready # same as 5 +# bin/runbook-slice.sh --spans 5 # only the line spans, as gate-check.sh prints them +# +# WHY (2026-10-14). skills/conversion-runbook.md is ~14k words. Its own "how to read this" rule +# says a session reads §1b plus its stage's section, and gate-check.sh prints those line spans — +# but a helper (mdl, gate, test, review) never runs a gate, so it never saw the spans and opened +# the whole file: 26% of an mdl helper's Stage 5 reading. This prints the ~1.5k words instead, the +# same way bin/bug-lookup.sh turned the bug ledger into a lookup. Sections are found by heading +# (bin/lib/runbook-span.sh, shared with gate-check.sh), never by line number. +# +# Exit: 0 printed · 1 usage · 2 runbook or heading not found +set -eu +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +. "$ROOT/bin/lib/runbook-span.sh" +RB="${MXTK_RUNBOOK:-$ROOT/skills/conversion-runbook.md}" + +SPANS_ONLY=0 +if [ "${1:-}" = "--spans" ]; then SPANS_ONLY=1; shift; fi +case "${1:-}" in + ""|-h|--help) sed -n '2,8p' "$0" | sed 's/^# \{0,1\}//'; exit 1 ;; +esac +ST="$(runbook_stage_token "$1")" +[ -n "$ST" ] || { echo "runbook-slice: unknown stage '$1' (P, 0-7 or build-ready)" >&2; exit 1; } +[ -f "$RB" ] || { echo "runbook-slice: no runbook at $RB" >&2; exit 2; } + +S1B="$(runbook_1b_span "$RB")" +SST="$(runbook_stage_span "$RB" "$ST")" +if [ -z "$S1B" ] || [ -z "$SST" ]; then + echo "runbook-slice: could not locate '## 1b.' or '### Stage $ST — ' in $RB — read §1b and your stage's §2 section there" >&2 + exit 2 +fi + +if [ "$SPANS_ONLY" -eq 1 ]; then + printf 'skills/conversion-runbook.md §1b lines %s + Stage %s lines %s\n' "$S1B" "$ST" "$SST" + exit 0 +fi +printf '\n\n' "$S1B" "$ST" "$SST" +runbook_print_span "$RB" "$S1B" +echo +runbook_print_span "$RB" "$SST" diff --git a/skills/conversion-runbook.md b/skills/conversion-runbook.md index 1f229619..f8361811 100644 --- a/skills/conversion-runbook.md +++ b/skills/conversion-runbook.md @@ -11,63 +11,12 @@ ## Skills always loaded — every session, before any stage artifact -This table is the canonical baseline set. It is rendered from the toolkit's one routing -table; a skill missing here is a skill no agent will find. - - - -Each row is a trigger, not a reading list: open a row's file when the first column happens in this session, and only rows whose Stage(s) cell says *every stage* or names the stage the register (PROJECT.md) says you are in. Do not read the table ahead — a build session that only writes pages never opens the microflow rows. - -Then filter by Role(s). A helper dispatched with a role stub (ba, architect, mdl, gate, test, review) reads only rows that say *every role* or name its role — its stub lists the same files, and it never opens another role's rows. The main session is **lead**: *every role* and lead rows, plus the rows of any role whose work it does itself instead of dispatching. - -| Always relevant for | Reference this | Stage(s) | Role(s) | -|---|---|---|---| -| Before writing any .js or .sh for a check, gate or report — and before adding a rule to an existing one: judgement goes in a skill, code only fetches facts a reader cannot | `skills/skills-over-scripts.md` | every stage | every role | -| Any pass whose input is missing, stale or unresolvable — before recording UNMEASURED, N/A or a silent skip: name what was missing, say what you assessed against instead, still deliver a verdict | `skills/degrade-to-judgement.md` | every stage | every role | -| Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | `skills/tool-output-is-not-ground-truth.md` | every stage | every role | -| Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | `skills/retesting-learned-rules.md` | every stage | every role | -| Any pipeline work at all — every session, before producing any stage artifact: read §1b plus your own stage's section, not the whole file (gate-check.sh prints the line spans); READMEs and the guide are orientation only | `skills/conversion-runbook.md` | P,0,1,2,3,4,5,6,7 | every role | -| Any question before asking the user or writing anything — query the model, then read the source, then ask the human, in that order | `skills/query-the-model.md` | P,0,1,2,3,4,5,6,7 | every role | -| Putting a question TO the user — any gate, any stage: ask in chat not in a file, two named options plus your recommendation, one batch per gate then end the turn | `skills/interview-protocol.md` | P,0,1,2,3,4,5,6,7 | lead, ba, architect | -| The user gives feedback or asks for a change mid-project — "change X", "it also needs Y", a remark on a screenshot or the running app, a stakeholder's ask relayed in chat: log a CR row in PROJECT.md ## Change log BEFORE any edit, size it, then edit the highest artifact it alters (BRD, decision, architecture, wireframe), never the MDL first | `skills/change-intake.md` | 2,3,4,5,6,7 | lead, ba | -| Any stage transition — the 2+1 format every CAC uses, and the one-register rule (answers land in PROJECT.md, never in a separate state file). The seven CACs themselves are routed per stage in the situational table | `skills/checkpoints/checkpoint-template.md` | 0,1,2,3,4,6,7 | lead, ba, architect | -| Setting up or completing a project's dev-process subagents — once, at project start, not "on demand" | `skills/agent-roles.md` | P,4,5 | lead | -| Starting a subagent, sending it a fix round, or reading an agent-stop notice — name the model, fresh not fork, fresh fixer not resume, coalesce notices | `skills/dispatching.md` | P,1,2,3,4,5,6 | lead | -| Deciding whether to extract at all, before any BRD gets generated | `skills/source-triage.md` | 0 | ba | -| Taking in a new source — before generating anything from it. Grades what the source can support; nothing else in this toolkit reads a source | `bin/source-sufficiency.sh` | 0,1 | ba | -| Closing Stage 1, or adding files to a source folder — every inventoried file must name the artifact that consumed it (text AND embedded diagrams), or carry a waiver; blocks Stages 1–2 until it does | `bin/source-ledger.sh` | 0,1,2 | ba | -| HTML in the source corpus — convert once before anyone reads it: `bin/html-to-md.sh ` writes each page as Markdown under analysis/knowledge-base/text/ (3–10× fewer tokens than the raw export; inline images decoded to files; section list with line numbers as the read's denominator) plus documents-index.md over EVERY file, which one ledger glob mark points at. A session that opens a .html itself has skipped this | `bin/html-to-md.sh` | 0,1 | ba | -| Pictures in the source corpus — decks, Word files, PDFs, screenshots: one worklist of unique images with context, one description file each, coverage checked | `bin/images-to-md.sh` | 0,1,2 | ba | -| Deciding who answers a question — before putting any batch to the user. gap/conflict/choice/user-only is what keeps a gate batch at four questions instead of 127 | `bin/question-kinds.sh` | 1,2,3 | ba | -| Writing BRDs, especially several in parallel — "build" before the fan-out, "check" before any BRD is called done | `bin/facts-lock.sh` | 2 | ba | -| Building any module — before the first script. The mdl-agent's single per-module input | `skills/module-brief.md` | 4,5 | ba, mdl | -| Writing ANY MDL script — before the first line. Step 0 picks the write mode, then the STOP table overrides it for corrupting operations | `skills/learned-mdl-preflight.md` | 5 | mdl | -| Placing any document in a module — before the first `create`. Feature group, then Pages/Microflows/Services/Resources; the path comes from the brief's folder plan, and the table says which types mxcli can actually place | `skills/module-folder-convention.md` | 4,5 | architect, mdl | -| Writing or fixing any microflow — MDL gotchas plus annotation discipline | `skills/learned-microflow-patterns.md` | 5 | mdl | -| Writing a microflow with any loop, a retrieve/commit/call inside a loop, nested or multiple loops, >20 activities counting loop bodies, or a list built from a list — post the checklist before the first MDL line | `skills/microflow-preflight.md` | 5 | mdl | -| Building any page or snippet — before the first widget. Wireframe, tokens, gallery reuse, cross-check; no wireframe means STOP | `skills/ui-preflight-pages.md` | 5 | mdl | -| Writing or reviewing any page or snippet — the spacing scale (8/16/24/32/48), section rhythm, and the page-header scaffold every full page starts with; sections at 0px apart and pages with no H1 are the defects it retires | `skills/design-spacing.md` | 5 | mdl, review | -| After every page-building script, and any time the UI looks wrong — the cheap repeatable look during the build: one page, one screenshot, four questions, scored when a wireframe exists. Feeds Gate: UI, never replaces it | `skills/ui-loop.md` | 5 | mdl, review | -| After the FIRST build that follows any design-system port, and before any page is built on it — reads the BUILT stylesheet and reports how many framework knobs point at a design token, how many tokens arrived, how many component classes arrived, each with its denominator. Measured on a real run: 55 tokens ported correctly into the right file, 0 of 35 knobs bound and 0 of 20 classes present, two build phases shipped in the framework's default blue with mx check, mxcli lint, the MDL suite and two e2e journeys all green | `project-bin/check-design-reaches-app.sh` | 3,5 | architect, mdl, gate, review | -| Before exec'ing ANY page script — compares the drafted MDL's shell against the wireframe's: page column, layout/nav shell, one H1. Measured 0/10 pages on a real first build, repaired wholesale 47 scripts later | `project-bin/check-page-shell.sh` | 5 | mdl, gate, review | -| After drafting and again after exec'ing any page script — scores the page MDL (or `mxcli describe` output on stdin) against its wireframe: headings/actions/content/classes, weighted. The scored companion to check-page-shell's binary gate; 32% median measured without it, 90% first-draft with it. Every run is appended to the project's docs/PAGE-FIDELITY.tsv — first non-stub row per page = first-build score of record vs the ≥80% target (forward-reference stubs score with --stub, exempt) | `project-bin/page-fidelity.js` | 5 | mdl, gate, review | -| Reviewing any module before calling it done — the ONE pass: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | `skills/module-review.md` | 5,6 | mdl, review, test | -| Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | `bin/exec-approval.sh` | 5,6 | mdl, gate | -| Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | `skills/testing-shape.md` | 5,6 | test, gate, review | -| Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | `project-bin/verify-module.sh` | 5,6 | mdl, gate, test, review | -| A style change that appears to have done nothing, a focus ring that paints on every page landing, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | `skills/learned-css-that-never-applied.md` | 5,6 | mdl, review, gate | -| Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | `skills/learned-detection-gaps.md` | 5,6 | mdl, gate, review | -| Creating any entity, or calling a module security-ready — entity and grants land in one script, and ready means SHOW SECURITY MATRIX proves it | `skills/security-is-not-a-later-script.md` | 5 | mdl, gate, review | -| A CE error or behavior that looks like a known mxcli quirk rather than a modeling mistake — `bin/bug-lookup.sh CE0117` / `BUG-102` / "keyword" prints the matching ledger entries, so the session reads one entry, not the 32k-word ledger | `bin/bug-lookup.sh` | 5,6 | mdl, gate | -| The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | `bin/status.sh` | P,0,1,2,3,4,5,6,7 | every role | -| Any session that will push a model to Mendix Team Server, and BEFORE telling the user a Team Server push is blocked — which remote is authoritative, settle-then-push order, and the four checks that have to fail first | `skills/teamserver-alignment.md` | 5,6,7 | mdl, gate, review | - +The baseline set is the toolkit's one routing table (`bin/lib/skill-routing.tsv`), rendered with its Stage(s) and Role(s) columns into your project's `CLAUDE.local.md` and the toolkit's `ROUTING.md` — read it there; it is not copied into this file. **Downstream:** every stage skill listed in §2 — this runbook sequences them, it does not replace their content. **Root pointer:** `CONVERSION-RUNBOOK.md` at the repo root is a thin pointer to this skill plus "how to start"; this file is the executable detail. `toolkit-guide.html` at the repo root is the same journey as a visual page, and doubles as the shared CSS shell/token source for every stage HTML surface. -**How to read this file: not whole.** A session reads **§1b** (the Live Checklist Protocol — it applies to every stage) **plus its own stage's section under §2**, and nothing else until a question sends it elsewhere. `bin/gate-check.sh ` prints the exact spans at the top of its output — `Read for this gate: skills/conversion-runbook.md §"Stage N — …" (lines A–B) + §1b Live Checklist (lines C–D)` — derived from the headings on every run, so they are never stale. "Read the runbook first, every session" in a project's `CLAUDE.local.md` means *those* lines, not the ~11,700 words: the whole-file read was measured cause three of a requirements-driven project's slow Stages 1–4 (2026-09-09). The baseline table above is rendered here so this file is complete, not so it is re-read. +**How to read this file: not whole.** A session reads **§1b** (the Live Checklist Protocol — it applies to every stage) **plus its own stage's section under §2**, and nothing else until a question sends it elsewhere. `bin/gate-check.sh ` prints the exact spans at the top of its output — `Read for this gate: skills/conversion-runbook.md §"Stage N — …" (lines A–B) + §1b Live Checklist (lines C–D)` — derived from the headings on every run, so they are never stale. "Read the runbook first, every session" in a project's `CLAUDE.local.md` means *those* lines, not the ~11,700 words: the whole-file read was measured cause three of a requirements-driven project's slow Stages 1–4 (2026-09-09). A helper that runs no gate gets the same two spans, printed, from `bin/runbook-slice.sh `. **Do not open `toolkit-guide.html` because you read this line.** Opening is governed by the first-touch rule in the toolkit's `CLAUDE.md` — open only if `/.claude/.guide-shown` is absent, then `touch` it. You are reading this file *every session*; an unconditional "open it at kickoff" here means a browser tab every session, which is exactly the bug this wording replaced. diff --git a/tests/bench/trim-noloss.sh b/tests/bench/trim-noloss.sh new file mode 100755 index 00000000..1fc61bfa --- /dev/null +++ b/tests/bench/trim-noloss.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# trim-noloss.sh — the mechanical half of a trim PR's review (CLAUDE.md "Always-loaded skill +# text — the word budget", rule 4): every line a commit range removed from a .md file must be +# found again somewhere in the tree, or be a generated routing row whose path the table still +# routes. Whatever is left is printed for a human to read line by line. +# +# tests/bench/trim-noloss.sh [] [--reviewed ] +# +# the commit the trim started from (the PR's merge base) +# the trimmed commit; default HEAD +# --reviewed a file of removed lines a human has already read and accepted, one per line, +# verbatim (leading/trailing space ignored); they print as REVIEWED, not LEFT +# +# Classes, one line each on stdout: +# FOUND -> +# ROUTING -> a row from inside a ROUTING:BEGIN block; the path is still in +# bin/lib/skill-routing.tsv at head, so the row re-renders elsewhere +# REVIEWED : +# LEFT : needs a human; any protected line (rule, STOP row, denominator, +# failure example) here is a HOLD +# +# Exit: 0 nothing LEFT · 1 at least one LEFT line · 2 usage / git error +# +# Written 2026-10-14 for the stub-diet trim (runbook baseline table out, helper runbook slice); +# generic on purpose, so the next trim PR runs the same check instead of eyeballing a diff. +set -uo pipefail + +BASE=""; HEAD_REF="HEAD"; REVIEWED="" +while [ "$#" -gt 0 ]; do + case "$1" in + --reviewed) REVIEWED="${2:?--reviewed needs a file}"; shift 2 ;; + -h|--help) sed -n '2,24p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) if [ -z "$BASE" ]; then BASE="$1"; else HEAD_REF="$1"; fi; shift ;; + esac +done +[ -n "$BASE" ] || { echo "usage: trim-noloss.sh [] [--reviewed ]" >&2; exit 2; } +git rev-parse --verify -q "$BASE^{commit}" >/dev/null || { echo "trim-noloss: no such commit: $BASE" >&2; exit 2; } +git rev-parse --verify -q "$HEAD_REF^{commit}" >/dev/null || { echo "trim-noloss: no such commit: $HEAD_REF" >&2; exit 2; } + +trim() { sed 's/^[[:space:]]*//; s/[[:space:]]*$//'; } +is_reviewed() { + [ -n "$REVIEWED" ] && [ -f "$REVIEWED" ] || return 1 + trim < "$REVIEWED" | grep -qxF -- "$1" +} + +# The per-line loop runs in a pipeline subshell, so the tallies go through a scratch file. +TMPC="$(mktemp "${TMPDIR:-/tmp}/noloss.XXXXXX")" +trap 'rm -f "$TMPC"' EXIT +TSV_HEAD="$(git show "$HEAD_REF:bin/lib/skill-routing.tsv" 2>/dev/null | cut -f2)" + +for f in $(git diff --name-only --diff-filter=MD "$BASE" "$HEAD_REF" -- '*.md'); do + # Line numbers inside generated ROUTING blocks in the BASE version of the file. + GEN="$(git show "$BASE:$f" 2>/dev/null | awk '/ROUTING:BEGIN/{b=1; next} /ROUTING:END/{b=0} b{print NR}' | tr '\n' ' ')" + # Removed lines with their base line numbers, from a zero-context diff. + git diff -U0 "$BASE" "$HEAD_REF" -- "$f" | awk ' + /^@@/ { h = 1; split($2, a, ","); n = substr(a[1], 2) + 0; next } + !h && /^---/ { next } + /^-/ { printf "%d\t%s\n", n, substr($0, 2); n++ }' | + while IFS=$'\t' read -r ln line; do + t="$(printf '%s' "$line" | trim)" + # Blank lines and bare markup carry no text to lose. + case "$t" in ''|'---'|'|---|'*|'') continue ;; esac + case " $GEN " in + *" $ln "*) + p="$(printf '%s' "$t" | sed -n 's/^|[^`]*`\([^`]*\)`.*/\1/p')" + if [ -n "$p" ] && printf '%s\n' "$TSV_HEAD" | grep -qxF -- "$p"; then + printf 'ROUTING %s -> %s\n' "$f" "$p"; echo R >> "$TMPC"; continue + fi ;; + esac + hit="$(git grep -l -F -e "$t" "$HEAD_REF" -- . 2>/dev/null | head -1 | sed "s|^$HEAD_REF:||")" + if [ -n "$hit" ]; then + printf 'FOUND %s -> %s\n' "$f" "$hit"; echo F >> "$TMPC" + elif is_reviewed "$t"; then + printf 'REVIEWED %s : %s\n' "$f" "$t"; echo V >> "$TMPC" + else + printf 'LEFT %s : %s\n' "$f" "$t"; echo L >> "$TMPC" + fi + done +done 3>&- diff --git a/tests/wave2/test-routing-tier.sh b/tests/wave2/test-routing-tier.sh index f4728a68..ef58b693 100644 --- a/tests/wave2/test-routing-tier.sh +++ b/tests/wave2/test-routing-tier.sh @@ -11,9 +11,13 @@ # project path, and --only render mode WRITES into ROOT in place — so this cannot point the real # script at a project directory the way other project-shaped fixtures do. Instead it copies the # real render-routing.sh and its real lib/skill-routing.sh verbatim into a throwaway scratch -# tree that mimics this repo's own shape (bin/, bin/lib/, the 9 SURFACES files with real markers, +# tree that mimics this repo's own shape (bin/, bin/lib/, the SURFACES files with real markers, # skills/), so ROOT resolves inside the scratch tree and nothing under the real repo is touched. # +# Since 2026-10-14 skills/conversion-runbook.md is NOT a surface: it carried a fourth rendered +# copy of the baseline table inside a baseline file. The scratch runbook below therefore carries +# a pointer and no markers, and T1/T3 assert the table never reaches it. +# # Usage: bash tests/wave2/test-routing-tier.sh [path-to-render-routing.sh] # (lib/skill-routing.sh is taken from alongside it: $(dirname SUT)/lib/skill-routing.sh) @@ -75,7 +79,7 @@ demo skills/demo-tier.md Doing the thing this probes — tier-demotion target al conversion-runbook skills/conversion-runbook.md Any pipeline work — every session all - baseline spine EOF -# ── The 9 SURFACES render-routing.sh hard-codes, each with real markers, minimal otherwise ── +# ── The SURFACES render-routing.sh hard-codes, each with real markers, minimal otherwise ──── cat > "$R/ROUTING.md" <<'EOF' # Routing @@ -99,9 +103,9 @@ mkdir -p "$R/skills" cat > "$R/skills/conversion-runbook.md" <<'EOF' # Runbook - - +The baseline set lives in CLAUDE.local.md and ROUTING.md; it is not copied into this file. EOF +RB_BEFORE="$(cat "$R/skills/conversion-runbook.md")" for a in ba architect mdl gate test review; do cat > "$R/agents/${a}-agent.md" < $BASE2, -$DELTA, body=$DEMO_WORDS)" + if [ "$DELTA" -eq "$DEMO_WORDS" ]; then + ok "baseline word count dropped by exactly demo-tier.md's body ($BASE1 -> $BASE2, -$DELTA)" else - bad "baseline word count delta out of the expected range for demoting one skill" \ - "before=$BASE1 after=$BASE2 delta=$DELTA demo_body_words=$DEMO_WORDS expected range=[$DEMO_WORDS, $((DEMO_WORDS + 40))]" + bad "baseline word count delta is not exactly the demoted skill's body" \ + "before=$BASE1 after=$BASE2 delta=$DELTA demo_body_words=$DEMO_WORDS" fi fi diff --git a/tests/wave2/test-runbook-slice.sh b/tests/wave2/test-runbook-slice.sh new file mode 100644 index 00000000..291288c0 --- /dev/null +++ b/tests/wave2/test-runbook-slice.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# Fixture for bin/runbook-slice.sh — the helper roles' replacement for reading the whole runbook. +# +# Usage: bash tests/wave2/test-runbook-slice.sh bin/runbook-slice.sh +# +# Runs against the REAL runbook (its headings are the golden input — a hand-written one would +# encode the same heading assumption the code makes). Read-only: nothing under the toolkit is +# written. +# T1 every stage token (P, 0-7, build-ready) prints §1b and exactly its own stage section +# T2 the spans agree with gate-check.sh's "Read for this gate" line (one lib, one answer) +# T3 Stage 0 is the "### Stage 0 — " section, not "### Stage 0 runs in every entry mode" +# T4 usage errors: no stage / unknown stage exit 1 and print nothing to stdout +# T5 the per-role word count counts the slice's OUTPUT for helpers, the runbook for lead +set -uo pipefail + +SUT="${1:?usage: test-runbook-slice.sh /path/to/bin/runbook-slice.sh}" +case "$SUT" in /*) ;; *) SUT="$PWD/$SUT" ;; esac +TK="$(cd "$(dirname "$SUT")/.." && pwd)" +RB="$TK/skills/conversion-runbook.md" +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); printf ' ok %s\n' "$1"; } +bad() { FAIL=$((FAIL+1)); printf ' FAIL %s\n' "$1"; [ -n "${2:-}" ] && printf '%s\n' "$2" | sed 's/^/ /'; } + +echo "== T1: §1b plus exactly the requested stage section, for every stage token ==" +for st in P 0 1 2 3 4 5 6 7 build-ready; do + want="$st"; [ "$st" = "build-ready" ] && want=5 + OUT="$(bash "$SUT" "$st" 2>&1)"; RC=$? + n_1b="$(printf '%s\n' "$OUT" | grep -c '^## 1b\.')" + n_own="$(printf '%s\n' "$OUT" | grep -c "^### Stage $want — ")" + n_any="$(printf '%s\n' "$OUT" | grep -c '^### Stage [P0-9] — ')" + n_2="$(printf '%s\n' "$OUT" | grep -c '^## [0-9]')" + if [ "$RC" -eq 0 ] && [ "$n_1b" -eq 1 ] && [ "$n_own" -eq 1 ] && [ "$n_any" -eq 1 ] && [ "$n_2" -eq 1 ]; then + ok "stage $st: exit 0, §1b once, Stage $want once, no other stage, no other ## section" + else + bad "stage $st: rc=$RC 1b=$n_1b own=$n_own stages=$n_any top-sections=$n_2" + fi +done + +echo "== T2: the spans match gate-check.sh's line (both read lib/runbook-span.sh) ==" +SP="$(bash "$SUT" --spans 2)" +S2="$(grep -n '^### Stage 2 — ' "$RB" | head -1 | cut -d: -f1)" +S1B="$(grep -n '^## 1b\.' "$RB" | head -1 | cut -d: -f1)" +case "$SP" in + "skills/conversion-runbook.md §1b lines $S1B–"*" + Stage 2 lines $S2–"*) ok "--spans starts each span at its real heading ($S1B, $S2)" ;; + *) bad "--spans wrong: $SP" ;; +esac +E2="${SP##*–}" +NEXT="$(sed -n "$((E2 + 1))p" "$RB")" +case "$NEXT" in '## '*|'### '*) ok "Stage 2's span ends right before the next heading" ;; + *) bad "line after Stage 2's span is not a heading: $NEXT" ;; esac +if grep -q 'runbook_stage_span\|runbook_1b_span' "$TK/bin/gate-check.sh" && ! grep -q '^runbook_span()' "$TK/bin/gate-check.sh"; then + ok "gate-check.sh sources the shared span lib rather than carrying its own copy" +else + bad "gate-check.sh still defines its own runbook_span" +fi + +echo "== T3: Stage 0 is the §2 section, not the Entry Modes subsection ==" +OUT0="$(bash "$SUT" 0)" +printf '%s\n' "$OUT0" | grep -q '^### Stage 0 runs in every entry mode' \ + && bad "stage 0 printed the Entry Modes subsection" \ + || ok "stage 0 does not print '### Stage 0 runs in every entry mode'" + +echo "== T4: usage errors ==" +O="$(bash "$SUT" 2>/dev/null)"; RC=$? +[ "$RC" -eq 1 ] && ok "no stage: exit 1" || bad "no stage: exit $RC" +O="$(bash "$SUT" 9 2>/dev/null)"; RC=$? +[ "$RC" -eq 1 ] && [ -z "$O" ] && ok "unknown stage: exit 1, empty stdout" || bad "unknown stage: exit $RC, stdout '$O'" + +echo "== T5: the per-role count reads the slice's output for helpers, the runbook for lead ==" +PACK() { MXTK_ROUTING_TSV="$TK/bin/lib/skill-routing.tsv" bash -c '. "$1/bin/lib/skill-routing.sh"; routing_baseline_pack 5 "$1" "$2"' _ "$TK" "$1"; } +for role in mdl gate test review; do + OUT="$(PACK "$role")" + W="$(printf '%s' "$OUT" | cut -f1)"; PATHS="$(printf '%s' "$OUT" | cut -f3)" + case " $PATHS " in *" bin/runbook-slice.sh "*) : ;; *) bad "$role: slice not in its stage-5 pack ($PATHS)"; continue ;; esac + case " $PATHS " in *" skills/conversion-runbook.md "*) bad "$role: still counts the whole runbook"; continue ;; esac + SUM=0 + for p in $PATHS; do + if [ "$p" = "bin/runbook-slice.sh" ]; then n="$(bash "$SUT" 5 | LC_ALL=C wc -w | tr -d ' ')" + else n="$(LC_ALL=C wc -w < "$TK/$p" | tr -d ' ')"; fi + SUM=$((SUM + n)) + done + [ "$SUM" = "$W" ] && ok "$role: stage-5 count ($W) = its .md files + the slice's printed words" \ + || bad "$role: count $W, recomputed $SUM" +done +OUT="$(PACK lead)"; PATHS="$(printf '%s' "$OUT" | cut -f3)" +case " $PATHS " in + *" bin/runbook-slice.sh "*) bad "lead counts the slice; it reads the runbook" ;; + *" skills/conversion-runbook.md "*) ok "lead keeps the whole runbook" ;; + *) bad "lead has no runbook in its pack ($PATHS)" ;; +esac + +printf '\n%s: %d ok, %d FAIL\n' "$(basename "$0")" "$PASS" "$FAIL" +[ "$FAIL" -eq 0 ] From 2ffa27767cdfb02e90cb9ec2b46ff76e08df469e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 14:29:14 +0000 Subject: [PATCH 2/4] test(bench): trim-noloss prints its summary line and exits on LEFT lines Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- tests/bench/trim-noloss.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/bench/trim-noloss.sh b/tests/bench/trim-noloss.sh index 1fc61bfa..bbcd8347 100755 --- a/tests/bench/trim-noloss.sh +++ b/tests/bench/trim-noloss.sh @@ -76,4 +76,9 @@ for f in $(git diff --name-only --diff-filter=MD "$BASE" "$HEAD_REF" -- '*.md'); printf 'LEFT %s : %s\n' "$f" "$t"; echo L >> "$TMPC" fi done -done 3>&- +done + +c() { grep -c "^$1\$" "$TMPC" 2>/dev/null || true; } +printf '\nremoved lines: %s found elsewhere, %s routing rows still routed, %s reviewed, %s LEFT\n' \ + "$(c F)" "$(c R)" "$(c V)" "$(c L)" +[ "$(c L)" = "0" ] From 394f132dd8532d20632ce9f99749d1caddce601a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 15:15:56 +0000 Subject: [PATCH 3/4] =?UTF-8?q?fix(context):=20full=20pack=20counts=20the?= =?UTF-8?q?=20runbook=20once;=20helper=20slice=20carries=20verdicts=20and?= =?UTF-8?q?=20=C2=A73b=20ownership?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review findings on #378: - routing_baseline_pack counted bin/runbook-slice.sh on top of skills/conversion-runbook.md whenever both rows reached the pack (no role). The no-role full pack read 85,797 words, over the 80,000 budget, on gate-check's advisory line. The slice now counts 0 when the runbook row is already counted; full pack 73,233. test-baseline-pack asserts it on a scratch table and on the real one. - runbook-slice.sh now also prints the gate-verdict table (five verdicts + exit codes) and §3b's "Ownership is split" paragraph (a helper reports BRD-touching decisions to the lead), located by heading/anchor in lib/runbook-span.sh. test-runbook-slice T6 asserts both for every stage. mdl at Stage 5: 43,599 words. - CHANGELOG: behaviour A/B result replaces "owed before merge". Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 2 +- bin/lib/runbook-span.sh | 30 +++++++++++++++++++++++++- bin/lib/skill-routing.sh | 16 +++++++++++--- bin/runbook-slice.sh | 31 +++++++++++++++++++++++--- tests/wave2/test-baseline-pack.sh | 36 +++++++++++++++++++++++++++++++ tests/wave2/test-runbook-slice.sh | 21 ++++++++++++++++++ 6 files changed, 128 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fbc0bc7..16f4263f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased -- chore(context): **helpers read the runbook slice for their stage, and the runbook no longer carries a copy of the baseline table — mdl at Stage 5 reads 56,236 → 43,352 words; baseline 75,028 → 73,233.** `skills/conversion-runbook.md` held a fourth rendered copy of the baseline routing table (README, ROUTING.md and every project's CLAUDE.local.md carry it too); it is out of the render surfaces and replaced by a one-line pointer. New `bin/runbook-slice.sh ` prints §1b plus that stage's section, using the span logic gate-check already had (now `bin/lib/runbook-span.sh`, shared). The runbook routing row is split: lead, ba and architect keep the runbook; mdl, gate, test and review get the slice, and `render-routing.sh --check` counts the slice's printed words for them, so the per-role numbers stay honest. New `tests/bench/trim-noloss.sh` checks that every removed line is found again or is a still-routed table row. Behaviour A/B owed before merge — context-cost research (r-stub-diet, T2+T3) +- chore(context): **helpers read the runbook slice for their stage, and the runbook no longer carries a copy of the baseline table — mdl at Stage 5 reads 56,236 → 43,352 words; baseline 75,028 → 73,233.** `skills/conversion-runbook.md` held a fourth rendered copy of the baseline routing table (README, ROUTING.md and every project's CLAUDE.local.md carry it too); it is out of the render surfaces and replaced by a one-line pointer. New `bin/runbook-slice.sh ` prints §1b plus that stage's section, using the span logic gate-check already had (now `bin/lib/runbook-span.sh`, shared). The runbook routing row is split: lead, ba and architect keep the runbook; mdl, gate, test and review get the slice, and `render-routing.sh --check` counts the slice's printed words for them, so the per-role numbers stay honest. New `tests/bench/trim-noloss.sh` checks that every removed line is found again or is a still-routed table row. Behaviour A/B (5 reps × 2 tasks + 5 probes × 3 reps, sonnet): trimmed pack no worse than master on every scored metric — planted STOP refused 10/10 each arm, mxcli-check parse failures page 2/5 vs 3/5, retrieval probes 15/15. — context-cost research (r-stub-diet, T2+T3) - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau diff --git a/bin/lib/runbook-span.sh b/bin/lib/runbook-span.sh index de021d23..b187d080 100644 --- a/bin/lib/runbook-span.sh +++ b/bin/lib/runbook-span.sh @@ -7,7 +7,9 @@ # One copy, so the three never disagree about where a section starts or ends. # # What a stage session needs is §1b (the Live Checklist Protocol, which applies to every stage) -# plus its own stage's section under §2. Stage headings are keyed on the " — " every one carries: +# plus its own stage's section under §2. A helper reading through bin/runbook-slice.sh also gets +# the gate-verdict table and §3b's ownership paragraph (runbook_verdicts_span, runbook_3b_owner_span). +# Stage headings are keyed on the " — " every one carries: # "### Stage 0 runs in every entry mode" (under Entry Modes) also starts with "### Stage 0", and # matched first without it. # @@ -37,6 +39,32 @@ runbook_stage_span() { runbook_span "$1" "^### Stage $2 — " "^###? "; } # runbook_1b_span — the "A–B" span of "## 1b. …" runbook_1b_span() { runbook_span "$1" "^## 1b\\." "^## "; } +# runbook_span_through — "A–B" from the start line through +# the first line matching (inclusive), or through the line before the next heading +# if the anchor is gone, so a reworded anchor shortens the span instead of running to EOF. +runbook_span_through() { + awk -v h="$2" -v a="$3" ' + !start && $0 ~ h { start = NR; next } + start && $0 ~ a { end = NR; exit } + start && /^#/ { end = NR - 1; exit } + END { if (start) { if (!end) end = NR; printf "%d–%d", start, end } }' "$1" 2>/dev/null +} + +# Two spans every helper needs whatever its stage (review of PR #378): +# runbook_verdicts_span — the gate-verdict table under "### What a gate verdict means" +# (PASS/PENDING/FAIL/WAIVED/MANUAL and their exit codes), through "An informational run …". +runbook_verdicts_span() { runbook_span_through "$1" "^### What a gate verdict means" "^An informational run"; } +# runbook_3b_owner_span — §3b's "Ownership is split" paragraph and its bullets ("a helper +# that hits a BRD-touching decision reports it to the lead"), through the next blank line. +runbook_3b_owner_span() { + awk ' + !in3b && /^## 3b\./ { in3b = 1; next } + in3b && /^## / { exit } + in3b && !start && /^\*\*Ownership is split/ { start = NR; next } + start && /^[[:space:]]*$/ { end = NR - 1; exit } + END { if (start) { if (!end) end = NR; printf "%d–%d", start, end } }' "$1" 2>/dev/null +} + # runbook_print_span <"A–B"> — the lines themselves. runbook_print_span() { local a b diff --git a/bin/lib/skill-routing.sh b/bin/lib/skill-routing.sh index f6fb0320..93bc460f 100644 --- a/bin/lib/skill-routing.sh +++ b/bin/lib/skill-routing.sh @@ -154,8 +154,12 @@ _routing_stage_arm() { # # One script row IS counted: bin/runbook-slice.sh. Its whole output is read into context — # it replaces the runbook for the helper roles — so it is counted as the words it prints for -# (about 1.5k at Stage 5), not as zero and not as the script's own text. With an +# (about 1.7k at Stage 5), not as zero and not as the script's own text. With an # unrecognised (the "full" pack) it is counted as the whole runbook, the upper bound. +# When skills/conversion-runbook.md is itself in the pack (no , or lead/ba/architect) +# the slice counts 0: it prints a subset of a file already counted whole. Before this (review +# of PR #378) the no-role full pack counted the runbook twice — 85,797 words, over the 80,000 +# budget, on gate-check's advisory line. _ROUTING_READ_SCRIPTS="bin/runbook-slice.sh" _routing_slice_words() { # _routing_slice_words if [ "$3" -eq 1 ]; then @@ -167,7 +171,7 @@ _routing_slice_words() { # _routing_slice_words routing_baseline_pack() { local stage="$1" root="$2" role="${3:-}" words=0 files=0 paths="" p wc_out local name path when agents stages tier group - local known_stages=" P 0 1 2 3 4 5 6 7 " full=0 + local known_stages=" P 0 1 2 3 4 5 6 7 " full=0 rb_counted=0 slice_words=0 case "$known_stages" in *" $stage "*) : ;; *) full=1 ;; esac while IFS=$'\t' read -r name path when agents stages tier group; do [ "$tier" = "baseline" ] || continue @@ -179,8 +183,13 @@ routing_baseline_pack() { p="$root/$path" [ -f "$p" ] || continue if [ "$path" = "$_ROUTING_READ_SCRIPTS" ]; then - wc_out="$(_routing_slice_words "$root" "$stage" "$full")" + # Held back until the loop ends: whether the runbook row is counted too is known only + # then, whatever the TSV order. + slice_words="$(_routing_slice_words "$root" "$stage" "$full")" + [ -n "$slice_words" ] || slice_words=0 + wc_out=0 else + [ "$path" = "skills/conversion-runbook.md" ] && rb_counted=1 wc_out="$(LC_ALL=C wc -w < "$p" 2>/dev/null | tr -d '[:space:]')" # LC_ALL=C: see BASELINE_BUDGET in render-routing.sh fi [ -n "$wc_out" ] || wc_out=0 @@ -188,6 +197,7 @@ routing_baseline_pack() { files=$((files + 1)) paths="$paths $path" done < <(routing_rows) + [ "$rb_counted" -eq 1 ] || words=$((words + slice_words)) if [ "$full" -eq 1 ]; then printf '%s\t%s\t%s\tfull\n' "$words" "$files" "${paths# }" else diff --git a/bin/runbook-slice.sh b/bin/runbook-slice.sh index 3ea917a8..109b3b0c 100755 --- a/bin/runbook-slice.sh +++ b/bin/runbook-slice.sh @@ -1,8 +1,9 @@ #!/usr/bin/env bash # runbook-slice.sh — print the part of the conversion runbook a helper at one stage needs: -# §1b (the Live Checklist Protocol, every stage) plus that stage's section under §2. +# §1b (the Live Checklist Protocol, every stage) plus that stage's section under §2, then the +# gate-verdict table and §3b's "Ownership is split" paragraph, which every helper needs too. # -# bin/runbook-slice.sh 5 # §1b + "### Stage 5 — Build" +# bin/runbook-slice.sh 5 # §1b + "### Stage 5 — Build" + verdicts + §3b ownership # bin/runbook-slice.sh build-ready # same as 5 # bin/runbook-slice.sh --spans 5 # only the line spans, as gate-check.sh prints them # @@ -13,6 +14,11 @@ # same way bin/bug-lookup.sh turned the bug ledger into a lookup. Sections are found by heading # (bin/lib/runbook-span.sh, shared with gate-check.sh), never by line number. # +# The two extra spans (review of PR #378): gate-agent reports PASS/PENDING/FAIL/WAIVED/MANUAL and +# needs their exit codes; mdl and gate need "a helper that hits a BRD-touching decision reports it +# to the lead" (§3b). Neither is in §1b or a stage section. If either anchor is gone the slice +# still prints, with a line naming the section to read in the runbook, and warns on stderr. +# # Exit: 0 printed · 1 usage · 2 runbook or heading not found set -eu ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -39,7 +45,26 @@ if [ "$SPANS_ONLY" -eq 1 ]; then printf 'skills/conversion-runbook.md §1b lines %s + Stage %s lines %s\n' "$S1B" "$ST" "$SST" exit 0 fi -printf '\n\n' "$S1B" "$ST" "$SST" +SVD="$(runbook_verdicts_span "$RB")" +S3B="$(runbook_3b_owner_span "$RB")" +printf '\n\n' \ + "$S1B" "$ST" "$SST" "${SVD:-not found}" "${S3B:-not found}" runbook_print_span "$RB" "$S1B" echo runbook_print_span "$RB" "$SST" +echo +if [ -n "$SVD" ]; then + runbook_print_span "$RB" "$SVD" +else + echo "runbook-slice: '### What a gate verdict means' not found in $RB" >&2 + echo "(Gate verdict table not found — read \"What a gate verdict means\" in §2 of skills/conversion-runbook.md.)" +fi +echo +echo "From §3b (Requirements drift-sync):" +echo +if [ -n "$S3B" ]; then + runbook_print_span "$RB" "$S3B" +else + echo "runbook-slice: '**Ownership is split' not found under '## 3b.' in $RB" >&2 + echo "(Ownership paragraph not found — read §3b of skills/conversion-runbook.md.)" +fi diff --git a/tests/wave2/test-baseline-pack.sh b/tests/wave2/test-baseline-pack.sh index a20c4ed8..17245d8d 100644 --- a/tests/wave2/test-baseline-pack.sh +++ b/tests/wave2/test-baseline-pack.sh @@ -129,6 +129,42 @@ MODE5="$(printf '%s' "$OUT5" | cut -f4)" && ok "known stages 3 and 5 report no 'full' marker (three fields only)" \ || bad "a known stage carried a 'full' marker (3: '$MODE3', 5: '$MODE5')" +echo "== the runbook slice is never counted on top of the whole runbook (review of PR #378) ==" +# bin/runbook-slice.sh prints a subset of skills/conversion-runbook.md. A pack that already counts +# the runbook whole (no role, or lead/ba/architect) must count the slice as 0 — the no-role full +# pack once counted the runbook twice (85,797 words, over the 80,000 budget). Scratch runbook is +# 50 words; the scratch slice prints 7. +R2="$W/root2"; mkdir -p "$R2/skills" "$R2/bin" +words 10 > "$R2/skills/always.md" +words 50 > "$R2/skills/conversion-runbook.md" +printf '#!/usr/bin/env bash\necho s1 s2 s3 s4 s5 s6 s7\n' > "$R2/bin/runbook-slice.sh" +TSV2="$W/scratch-routing-2.tsv" +cat > "$TSV2" <<'EOF2' +# name path when agents stages tier group +always skills/always.md always relevant all - baseline spine +conversion-runbook skills/conversion-runbook.md any pipeline work lead,ba,architect P,0,1,2,3,4,5,6,7 baseline spine +runbook-slice bin/runbook-slice.sh any pipeline work, as a helper mdl,gate,test,review P,0,1,2,3,4,5,6,7 baseline spine +EOF2 +pack2() { MXTK_ROUTING_TSV="$TSV2" bash -c 'source "'"$SKILL_ROUTING"'"; routing_baseline_pack "$1" "$2" "$3"' _ "$1" "$R2" "${2:-}" | cut -f1; } +[ "$(pack2 GARBAGE)" = "60" ] && ok "no-role full pack counts the runbook once (10+50=60), not runbook+slice" \ + || bad "no-role full pack double-counts the runbook: got $(pack2 GARBAGE), wanted 60 (110 = runbook counted twice)" +[ "$(pack2 5)" = "60" ] && ok "no-role stage-5 pack counts the runbook, not the slice on top (60)" \ + || bad "no-role stage-5 pack: got $(pack2 5), wanted 60 (67 = slice added on top of the runbook)" +[ "$(pack2 5 lead)" = "60" ] && ok "lead stage-5 pack: always + runbook (60)" \ + || bad "lead stage-5 pack: got $(pack2 5 lead), wanted 60" +[ "$(pack2 5 mdl)" = "17" ] && ok "mdl stage-5 pack: always + the slice's printed words (17)" \ + || bad "mdl stage-5 pack: got $(pack2 5 mdl), wanted 17" +[ "$(pack2 GARBAGE mdl)" = "60" ] && ok "mdl full pack: the slice is counted as the whole runbook, once (60)" \ + || bad "mdl full pack: got $(pack2 GARBAGE mdl), wanted 60" +# Same claim on the real table: the no-role full pack equals the sum of its .md files, exactly. +REALOUT="$(MXTK_ROUTING_TSV="$TOOLKIT/bin/lib/skill-routing.tsv" bash -c 'source "'"$SKILL_ROUTING"'"; routing_baseline_pack GARBAGE "$1"' _ "$TOOLKIT")" +REALW="$(printf '%s' "$REALOUT" | cut -f1)"; MDSUM=0 +for p in $(printf '%s' "$REALOUT" | cut -f3); do + case "$p" in *.md) MDSUM=$((MDSUM + $(LC_ALL=C wc -w < "$TOOLKIT/$p" | tr -d ' '))) ;; esac +done +[ "$REALW" = "$MDSUM" ] && ok "real table: no-role full pack ($REALW) = its .md files alone, no slice on top" \ + || bad "real table: no-role full pack $REALW != sum of its .md files $MDSUM (slice double-counted?)" + echo "== field run: the real toolkit table at every runbook stage returns a non-empty pack ==" FIELD_OK=1 for st in P 0 1 2 3 4 5 6 7; do diff --git a/tests/wave2/test-runbook-slice.sh b/tests/wave2/test-runbook-slice.sh index 291288c0..75fbd653 100644 --- a/tests/wave2/test-runbook-slice.sh +++ b/tests/wave2/test-runbook-slice.sh @@ -11,6 +11,8 @@ # T3 Stage 0 is the "### Stage 0 — " section, not "### Stage 0 runs in every entry mode" # T4 usage errors: no stage / unknown stage exit 1 and print nothing to stdout # T5 the per-role word count counts the slice's OUTPUT for helpers, the runbook for lead +# T6 every stage also prints the gate-verdict table (all five verdicts with exit codes) and +# §3b's "a helper ... reports it to the lead" paragraph (review of PR #378) set -uo pipefail SUT="${1:?usage: test-runbook-slice.sh /path/to/bin/runbook-slice.sh}" @@ -89,5 +91,24 @@ case " $PATHS " in *) bad "lead has no runbook in its pack ($PATHS)" ;; esac +echo "== T6: the gate-verdict table and §3b's helper-reports-to-lead paragraph, every stage ==" +for st in P 0 1 2 3 4 5 6 7; do + OUT="$(bash "$SUT" "$st" 2>/dev/null)"; MISS="" + for v in "PASS\` | Checked, and it holds. | 0" "PENDING\`" "FAIL\`" "WAIVED\`" "MANUAL\`"; do + printf '%s\n' "$OUT" | grep -q "^| \`$v" || MISS="$MISS verdict:${v%%\`*}" + done + printf '%s\n' "$OUT" | grep -q 'Exit code on a stage query' || MISS="$MISS exit-code-column" + printf '%s\n' "$OUT" | grep -q 'A helper that hits a BRD-touching decision reports it to the lead' || MISS="$MISS 3b-helper-reports" + [ -z "$MISS" ] && ok "stage $st: verdict table (5 verdicts + exit codes) and §3b helper paragraph present" \ + || bad "stage $st: missing$MISS" +done +# Located by heading/anchor through the shared lib, not by line number. +if grep -q 'runbook_verdicts_span' "$SUT" && grep -q 'runbook_3b_owner_span' "$SUT" \ + && ! grep -Eq 'sed -n ["'"'"']?[0-9]+,[0-9]+p["'"'"']? "?\$\{?RB' "$SUT"; then + ok "runbook-slice.sh finds both spans through lib/runbook-span.sh, no hard-coded line numbers" +else + bad "runbook-slice.sh does not use runbook_verdicts_span/runbook_3b_owner_span, or hard-codes a line range" +fi + printf '\n%s: %d ok, %d FAIL\n' "$(basename "$0")" "$PASS" "$FAIL" [ "$FAIL" -eq 0 ] From 760a423d94802f487eedeed030c57d8c91ab21dd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 15:23:27 +0000 Subject: [PATCH 4/4] =?UTF-8?q?docs(changelog):=20stub=20diet=20A=20?= =?UTF-8?q?=E2=80=94=20mdl=20Stage=205=20figure=20is=2043,599=20after=20th?= =?UTF-8?q?e=20slice=20gained=20verdicts=20+=20=C2=A73b?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured with bin/render-routing.sh --check on this head (master: 56,236; baseline 75,028 → 73,233 unchanged). The line carries no review figure. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 16f4263f..1e6bc305 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased -- chore(context): **helpers read the runbook slice for their stage, and the runbook no longer carries a copy of the baseline table — mdl at Stage 5 reads 56,236 → 43,352 words; baseline 75,028 → 73,233.** `skills/conversion-runbook.md` held a fourth rendered copy of the baseline routing table (README, ROUTING.md and every project's CLAUDE.local.md carry it too); it is out of the render surfaces and replaced by a one-line pointer. New `bin/runbook-slice.sh ` prints §1b plus that stage's section, using the span logic gate-check already had (now `bin/lib/runbook-span.sh`, shared). The runbook routing row is split: lead, ba and architect keep the runbook; mdl, gate, test and review get the slice, and `render-routing.sh --check` counts the slice's printed words for them, so the per-role numbers stay honest. New `tests/bench/trim-noloss.sh` checks that every removed line is found again or is a still-routed table row. Behaviour A/B (5 reps × 2 tasks + 5 probes × 3 reps, sonnet): trimmed pack no worse than master on every scored metric — planted STOP refused 10/10 each arm, mxcli-check parse failures page 2/5 vs 3/5, retrieval probes 15/15. — context-cost research (r-stub-diet, T2+T3) +- chore(context): **helpers read the runbook slice for their stage, and the runbook no longer carries a copy of the baseline table — mdl at Stage 5 reads 56,236 → 43,599 words; baseline 75,028 → 73,233.** `skills/conversion-runbook.md` held a fourth rendered copy of the baseline routing table (README, ROUTING.md and every project's CLAUDE.local.md carry it too); it is out of the render surfaces and replaced by a one-line pointer. New `bin/runbook-slice.sh ` prints §1b plus that stage's section, using the span logic gate-check already had (now `bin/lib/runbook-span.sh`, shared). The runbook routing row is split: lead, ba and architect keep the runbook; mdl, gate, test and review get the slice, and `render-routing.sh --check` counts the slice's printed words for them, so the per-role numbers stay honest. New `tests/bench/trim-noloss.sh` checks that every removed line is found again or is a still-routed table row. Behaviour A/B (5 reps × 2 tasks + 5 probes × 3 reps, sonnet): trimmed pack no worse than master on every scored metric — planted STOP refused 10/10 each arm, mxcli-check parse failures page 2/5 vs 3/5, retrieval probes 15/15. — context-cost research (r-stub-diet, T2+T3) - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau