process(testing): per-project test strategy at CAC-5, unit tests off by default and enforced

CHANGELOG.md

@@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
 
 ## Unreleased
 
+- process(testing): **unit tests are now off by default and `affected-tests.sh` enforces it; each project picks its test strategy with the developer at CAC-5.** Fast-tracked by the maintainer on 2026-10-16 because users report tests firing on every change (the baseline-text A/B is waived; E3 follows as a check). Moments: every change = gate plus what it touched (unit only for affected modules with unit on); module done = that module's journeys (+ its unit tests if on); build complete and before every later deploy = everything once; "run everything" any time. The CAC-5 Offer records it in PROJECT.md `## Test strategy` (Status `ASSUMED` in auto mode). `affected-tests.sh` reads `Unit tests default:` (`no` or absent means unit off, one `unit off` line, never a `.test.mdl` run row); turn unit on per project with `Unit tests default: yes`, per module with `Unit modules: Shop, Admin` in PROJECT.md or `Unit tests: yes` in the module brief; the strategy row's Unit cell is a scope only and never turns unit on. A grant on a microflow or page the same script changes is part of that subject and no longer selects the owning module's tests. `module-review.md` PROVE now runs `--moment module --module <M>`. `testing-shape.md`, `checkpoint-build.md`, `iterative-build-loop.md`, `conversion-runbook.md` and the exec.sh logic lane text updated. `tests/wave2/test-test-strategy-docs.sh` and the extended `test-affected-tests.sh` pin it. Baseline words stay at or below master (75,219 vs 75,225). Split out of #409; found by the field test of the release candidate — the maintainer's decision, 2026-10-16
 - learn(skills): **five lab findings land as compact rows, each paid for by trimming prose in the same files, so always-loaded words do not grow (baseline 75,225 to 75,204, budget 80,000).** Enum value `New` and `AutoCreatedDate`/`AutoOwner` (detection gaps), `[%CurrentDateTime%]` not `currentDateTime()`, expression return types (Decimal from date diffs and `div`, absolute `secondsBetween`, `formatDecimal` on pages), CE2729 from the default home page and page microflow grants, Atlas selector carriers for list views, page body and timelines, `-small` spacing suffixes do not exist, `display: contents` rows paint nothing, CSS pseudo text is invisible to `innerText` — lab field runs, Refs #276 (TK-020, TK-021, TK-022, TK-024)
 - fix(design-check): **a design system whose palette lives only in scoped `:root.<theme>` blocks no longer reads as "0 of N tokens carry the design's value".** Pass 2b compared every token against a bare `:root` of the built sheet, so a themed-only palette always warned, a false warning that teaches agents to ignore the check. A token the design declares only in scoped blocks is now compared block for block against the same-named block in the built sheet, and a wrong value is still named with its block (knob binding stays bare-only: a knob bound inside a theme the app is not wearing is not bound). Synthetic fixtures `ds-themed.css`, `built-themed.css`, `built-themed-wrong.css`; 5 new cases in `test-design-reaches-app.sh` (50 total) — TK-015 part (b)
 - fix(tests): **`test-exec-hash-verify.sh` case K passes on macOS.** `$TMPDIR` ends in `/`, so the bare `mktemp` path held `//` and the exact `MXPATCH_BACKUP_DIR` path assert could not match; `WORK` is now `cd`-ed into and read back with `pwd -P`. Test only, no script change. 42 pass / 1 fail before, 43 / 0 after — #384 review

project-bin/affected-tests.sh

@@ -10,7 +10,7 @@
 #   caller    <Module.Name>  depth N     (SHOW CALLERS OF ... TRANSITIVE)
 #   callers   measured | NOT MEASURED (<reason>)
 #   data      NOT MEASURED  <kind> <Module.Name>  (entity/association/enumeration/access/module/role/userrole/constant subject)
-#   unit      <file.test.mdl>  <why>
+#   unit      <file.test.mdl>  <why>   |   unit  off  (<why unit is off>)
 #   journey   <file.journey.json>  <why>
 #   journeys  unmapped  N of M journey file(s) cannot be selected by name
 #   run       <the command to run that selection>
@@ -27,10 +27,34 @@
 # THE STRATEGY. PROJECT.md "## Test strategy" (skills/checkpoints/checkpoint-build.md writes it):
 #   | Moment | Unit tests | Journeys | Status |
 #   | Every change | affected | affected | CONFIRMED |
-#   | Module close | module | module | CONFIRMED |
-#   | Deploy-ready | full | full | CONFIRMED |
+#   | Module done | module | module | CONFIRMED |
+#   | Build complete | full | full | CONFIRMED |
 # Scopes: none | affected | module | full. No section, or no row for the moment: these defaults,
-# and the strategy row says "default". --moment picks the row (change is the default).
+# and the strategy row says "default". --moment picks the row (change is the default):
+#   change = every change (build and feedback) · module = module done · deploy = build complete and
+#   before every later deploy · all = "run everything", any time (same as deploy).
+# Old row names "Module close" and "Deploy-ready" still read. A label is matched by its first words,
+# case-insensitively: "Every change (build and feedback)" and "Build complete, and before every later
+# deploy" (the Offer's own wording) map to their moment; a cell reads up to its first comma or
+# parenthesis ("affected, modules on only" = affected, "none (decided CAC-5)" = none). An unknown row label warns on stderr.
+#
+# UNIT TESTS ARE OFF BY DEFAULT (field test of the RC, 2026-10-16: nothing read the knob, so a
+# unit test was offered after every change). Precedence for the UNIT scope, highest first:
+#   1. The '## Test strategy' Unit cell is a SCOPE only (none|affected|module|full). It never
+#      turns unit on: at the change and module moments the switches in 2 and 3 always filter it.
+#      Gate open (default yes): the cell is used as written. Gate closed: flipped modules only, with
+#      the moment's built-in scope when the cell says none.
+#   2. A module flipped to yes: that module's tests only, at the change and module moments (deploy
+#      and all run every existing unit test). A flip is any of: a PROJECT.md line
+#      `Unit modules: Shop, Admin` (decided at CAC-5); in the module brief
+#      (architecture/modules/<M>/module-brief.md or architecture/modules/<M>-brief.md) a line
+#      `Unit tests: yes` or the Declared-shape table row `| Unit (...) | yes | ...`.
+#   3. PROJECT.md `Unit tests default:`. `yes` keeps the built-in scope for the moment; `no`, or the
+#      line ABSENT, means unit is OFF for every module not flipped in 2. When off the script prints
+#      one `unit off` line and NEVER a `run ... .test.mdl` row. `Unit modules:` takes names split
+#      by comma or space; `#` and `( ... )` comments, a leading `- ` bullet, any case and `none`
+#      are read; a name that matches no module warns on stderr.
+# Journeys are not affected by any of this.
 #
 # HOW THE SELECTION IS MADE (judgement-free facts only; skills-over-scripts.md):
 #   1. Subjects: `bin/exec.sh --subjects <script>` (the same statement lexer as the change class).
@@ -52,7 +76,11 @@
 #      selection is never read as "nothing owed".
 #   5. module: every test file naming the module, journeys/<M>.journey.json; the modules are the
 #      subjects' modules plus every --module. full: every test file, every journey file.
-#   6. data subjects (entity, association, enumeration, access, module, module role, user role,
+#   6. A grant/revoke whose object (microflow, page, ...) the SAME script creates or modifies is
+#      part of that subject (describe output ends in one): it selects nothing by module. A grant
+#      with no such object in the script (entity, module role, other object) is an access change
+#      and is selected by owning module as in 7.
+#   7. data subjects (entity, association, enumeration, access, module, module role, user role,
 #      constant): selected by the owning module only (a user role has no module and selects
 #      nothing). Their use in another module's rule, page or microflow is not measured, and a
 #      "data NOT MEASURED" row says so for each one.
@@ -81,7 +109,8 @@ while [ $# -gt 0 ]; do
 $(cd "$(dirname "$1")" && pwd)/$(basename "$1")"; shift ;;
   esac
 done
-case "$MOMENT" in change|module|deploy) ;; *) echo "✗ --moment takes change, module or deploy" >&2; exit 2 ;; esac
+[ "$MOMENT" = all ] && MOMENT=deploy
+case "$MOMENT" in change|module|deploy) ;; *) echo "✗ --moment takes change, module, deploy or all" >&2; exit 2 ;; esac
 case "$DEPTH" in ''|*[!0-9]*) echo "✗ --depth takes a number" >&2; exit 2 ;; esac
 SCRIPTS="$(printf '%s\n' "$SCRIPTS" | sed '/^$/d')"
 [ -n "$SCRIPTS" ] || [ "$MOMENT" != change ] || { echo "✗ a change needs at least one script" >&2; usage; }
@@ -92,24 +121,52 @@ WORK="$(mktemp -d "${TMPDIR:-/tmp}/affected.XXXXXX")"; trap 'rm -rf "$WORK"' EXI
 out() { printf '%s\n' "$*"; }
 
 # ── Strategy row ─────────────────────────────────────────────────────────────
-case "$MOMENT" in change) ROW="Every change"; DU=affected; DJ=affected ;;
-                  module) ROW="Module close"; DU=module;   DJ=module ;;
-                  deploy) ROW="Deploy-ready"; DU=full;     DJ=full ;; esac
+case "$MOMENT" in change) ROW="Every change"; ALT="Every change"; DU=affected; DJ=affected ;;
+                  module) ROW="Module done"; ALT="Module close"; DU=module; DJ=module ;;
+                  deploy) ROW="Build complete"; ALT="Deploy-ready"; DU=full; DJ=full ;; esac
 U="$DU"; J="$DJ"; FROM="default: PROJECT.md has no '## Test strategy' row '$ROW' (CAC-5 records it)"
 if [ -f PROJECT.md ]; then
-  _row="$(awk -v r="$ROW" '
+  _row="$(awk -v r="$ROW" -v r2="$ALT" '
     /^## /{ on = ($0 ~ /^## Test strategy/) ; next }
-    on && /^\|/ { split($0, c, "|"); x = c[2]; gsub(/^[ \t]+|[ \t]+$/, "", x)
-                  if (tolower(x) == tolower(r)) { u = c[3]; j = c[4]; s = c[5]
+    on && /^\|/ { split($0, c, "|"); x = tolower(c[2]); sub(/[(,].*/, "", x); gsub(/^[ \t*`]+|[ \t*`]+$/, "", x)
+                  if (x == "moment" || x ~ /^[-: ]*$/) next
+                  if (x !~ /^(every change|module done|module close|build complete|deploy-ready)$/) { y = c[2]; gsub(/^[ \t]+|[ \t]+$/, "", y); print "warning: ## Test strategy row label \"" y "\" is not a known moment, ignored" > "/dev/stderr"; next }
+                  if (!done && (x == tolower(r) || x == tolower(r2))) { done = 1; u = c[3]; j = c[4]; s = c[5]
+                    sub(/[(,].*/, "", u); sub(/[(,].*/, "", j)
                     gsub(/[ \t`*]/, "", u); gsub(/[ \t`*]/, "", j); gsub(/^[ \t]+|[ \t]+$/, "", s)
-                    print tolower(u) "\t" tolower(j) "\t" s; exit } }' PROJECT.md)"
+                    out = tolower(u) "\t" tolower(j) "\t" s } }
+    END { if (done) print out }' PROJECT.md)"
   if [ -n "$_row" ]; then
     _u="$(printf '%s' "$_row" | cut -f1)"; _j="$(printf '%s' "$_row" | cut -f2)"; _s="$(printf '%s' "$_row" | cut -f3)"
     case "$_u" in none|affected|module|full) U="$_u" ;; *) FROM="row '$ROW' unit scope '$_u' unknown, default used; " ;; esac
     case "$_j" in none|affected|module|full) J="$_j" ;; *) FROM="${FROM}row '$ROW' journey scope '$_j' unknown, default used; " ;; esac
     case "$FROM" in default:*) FROM="PROJECT.md ## Test strategy, row '$ROW'${_s:+, $_s}" ;; *) FROM="${FROM}PROJECT.md ## Test strategy" ;; esac
   fi
 fi
+
+# ── Unit gate: off unless the project default is yes, the row says so, or a module is flipped ──
+UNIT_DEFAULT="$(awk '/^[ \t*]*Unit tests default:/ { v = $0; sub(/^[^:]*:/, "", v); gsub(/[ \t*`]/, "", v); print tolower(v); exit }' PROJECT.md 2>/dev/null)"
+FLIPPED="$(awk 'tolower($0) ~ /^[ \t*>-]*unit modules:/ { v = $0; sub(/^[^:]*:/, "", v); sub(/#.*/, "", v); gsub(/\([^)]*\)/, "", v)
+  gsub(/[ \t*`,]+/, " ", v); n = split(v, a, " "); o = ""
+  for (i = 1; i <= n; i++) if (tolower(a[i]) != "none") o = o " " a[i]
+  print o; exit }' PROJECT.md 2>/dev/null)"
+for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md; do
+  [ -f "$_b" ] || continue
+  case "$_b" in */module-brief.md) _m="$(basename "$(dirname "$_b")")" ;; *) _m="$(basename "$_b" -brief.md)" ;; esac
+  if awk '
+      tolower($0) ~ /^[ \t*-]*unit tests:[ \t*]*yes([^a-z]|$)/ { f = 1 }
+      /^\|/ { split($0, c, "|"); k = tolower(c[2]); v = tolower(c[3]); gsub(/[ \t*`]/, "", v)
+              if (k ~ /^[ \t]*unit[ \t]*\(/ && v ~ /^yes/) f = 1 }
+      END { exit !f }' "$_b"; then FLIPPED="$FLIPPED $_m"; fi
+done
+UNIT_GATED=0; UNIT_OFF_WHY=""
+if [ "$MOMENT" != deploy ] && [ "${UNIT_DEFAULT#yes}" = "$UNIT_DEFAULT" ]; then
+  if [ -n "$UNIT_DEFAULT" ]; then _dw="PROJECT.md Unit tests default: $UNIT_DEFAULT"
+  else _dw="PROJECT.md has no 'Unit tests default:' line, treated as no"; fi
+  if [ -n "$FLIPPED" ]; then UNIT_GATED=1; [ "$U" = none ] && U="$DU"
+  else U=none; UNIT_OFF_WHY="($_dw; flip a module: PROJECT.md 'Unit modules: <M>' or 'Unit tests: yes' in its brief)"; fi
+  FROM="$FROM; unit: $_dw"
+fi
 out "strategy	$MOMENT	unit=$U journeys=$J	($FROM)"
 
 # ── Subjects ─────────────────────────────────────────────────────────────────
@@ -121,7 +178,18 @@ if [ -n "$SCRIPTS" ]; then
     done
   done > "$WORK/subjects"
 fi
-while IFS='	' read -r k n w; do out "subject	$k	$n	$w"; done < "$WORK/subjects"
+# A grant/revoke on an object this same script creates or modifies belongs to that subject.
+awk -F'\t' '
+  function scr(w,   a) { split(w, a, " of "); return a[2] }
+  NR == FNR { if ($1 != "access" && $1 != "unmapped") own[$2 "\t" scr($3)] = 1; next }
+  $1 == "access" && (($2 "\t" scr($3)) in own) { print $1 "\t" $2 "\t" $3 "\tPART" ; next }
+  { print $0 "\t" }' "$WORK/subjects" "$WORK/subjects" > "$WORK/subjects.tag"
+: > "$WORK/subjects.sel"
+while IFS='	' read -r k n w tag; do
+  if [ "$tag" = PART ]; then out "subject	$k	$n	$w	(grant on an object this script changes: part of that subject, selects nothing by module)"
+  else out "subject	$k	$n	$w"; printf '%s\t%s\t%s\n' "$k" "$n" "$w" >> "$WORK/subjects.sel"; fi
+done < "$WORK/subjects.tag"
+mv "$WORK/subjects.sel" "$WORK/subjects"
 [ -n "$SCRIPTS" ] && [ ! -s "$WORK/subjects" ] && out "subject	none	-	no model-writing statement in the script(s)"
 mod_of() { printf '%s\n' "${1%%.*}"; }
 awk -F'\t' '$2 != "-" && $1 != "userrole" { split($2, a, "."); print a[1] }' "$WORK/subjects" > "$WORK/modules"
@@ -218,10 +286,34 @@ select_journeys() {  # $1 = scope; prints "<file>\t<why>"
   esac | awk -F'\t' '!seen[$1]++'
 }
 
+# Module names as typed in PROJECT.md or a brief: match case-insensitively, warn on strangers.
+if [ -n "$FLIPPED" ]; then
+  { cat "$WORK/modules"; for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md; do
+      [ -f "$_b" ] || continue
+      case "$_b" in */module-brief.md) basename "$(dirname "$_b")" ;; *) basename "$_b" -brief.md ;; esac
+    done
+    [ -s "$WORK/tests" ] && xargs grep -ohE '(^|[^A-Za-z0-9_.])[A-Za-z_][A-Za-z0-9_]*\.[A-Za-z_]' < "$WORK/tests" 2>/dev/null | sed -E 's/^[^A-Za-z_]//; s/\..*//'
+  } | sort -u > "$WORK/known"
+  _canon=""
+  for m in $FLIPPED; do
+    _k="$(awk -v m="$m" 'tolower($0) == tolower(m) { print; exit }' "$WORK/known")"
+    if [ -n "$_k" ]; then _canon="$_canon $_k"
+    else _canon="$_canon $m"; echo "warning: Unit modules names '$m', which matches no module in the tests, briefs or this change" >&2; fi
+  done
+  FLIPPED="$_canon"
+fi
 select_unit "$U" > "$WORK/unit.sel"
+if [ "$UNIT_GATED" = 1 ]; then   # default no, some modules flipped: only those modules' tests
+  : > "$WORK/unit.gated"
+  for m in $FLIPPED; do
+    while IFS='	' read -r f w; do grep -qE "(^|[^A-Za-z0-9_])$m\." "$f" 2>/dev/null && printf '%s\t%s\n' "$f" "$w"; done < "$WORK/unit.sel"
+  done | awk -F'\t' '!seen[$1]++' > "$WORK/unit.gated"
+  mv "$WORK/unit.gated" "$WORK/unit.sel"
+fi
 select_journeys "$J" > "$WORK/journey.sel"
-if [ -s "$WORK/unit.sel" ]; then sed 's/^/unit	/' "$WORK/unit.sel"
-else out "unit	none	scope $U: $( [ "$U" = none ] && echo 'strategy says none' || echo "0 of $(wc -l < "$WORK/tests" | tr -d ' ') .test.mdl file(s) match")"; fi
+if [ -n "$UNIT_OFF_WHY" ]; then out "unit	off	$UNIT_OFF_WHY"
+elif [ -s "$WORK/unit.sel" ]; then sed 's/^/unit	/' "$WORK/unit.sel"
+else out "unit	none	scope $U: $( [ "$U" = none ] && echo 'strategy says none' || echo "0 of $(wc -l < "$WORK/tests" | tr -d ' ') .test.mdl file(s) match${FLIPPED:+ (unit on only for:$FLIPPED)}")"; fi
 if [ -s "$WORK/journey.sel" ]; then sed 's/^/journey	/' "$WORK/journey.sel"
 else out "journey	none	scope $J: $( [ "$J" = none ] && echo 'strategy says none' || echo "0 of $(wc -l < "$WORK/journeys" | tr -d ' ') journey file(s) in $JDIR/ and tests/e2e/ match")"; fi
 

project-bin/exec.sh

@@ -414,7 +414,7 @@ fi
 change_lane() {
   case "$CHANGE_CLASS" in
     ui-only) echo "re-run the journeys of every page the change reaches (embedders of a changed snippet/layout) once + a LOOK. No .test.mdl; reviewers read the artifacts, they do not re-run" ;;
-    logic)   echo "re-run only the tests whose subject this change touched (changed microflows + their SHOW CALLERS, changed pages' journeys). Full mxcli test at module close, not here" ;;
+    logic)   echo "re-run only the tests whose subject this change touched (changed microflows + their SHOW CALLERS, changed pages' journeys). Unit tests only for affected modules with unit on; everything runs once at build complete and before each later deploy, per PROJECT.md ## Test strategy" ;;
   esac
 }
 [ -n "$CR_ID" ] && echo "→ Change request: $CR_ID (PROJECT.md '## Change log'; cite this run's commit in its Closed-by)"

skills/brd-to-build-plan.md

@@ -340,7 +340,7 @@ Every row is one of five kinds, sharing one numbering and one dependency graph.
   Produces names every document the row writes, shared ones included (pages, navigation,
   after-startup, shared microflows): the parallel ready set excludes overlapping writers.
 - **`PROVE`** — headless and mechanical, mid-phase. `mxcli test`, a `curl`, a `DESCRIBE` read-back.
-  On mxcli ≥ v0.19.0, `.test.mdl` suites are a first-class PROVE instrument: `@setup` really
+  On mxcli ≥ v0.19.0, `.test.mdl` suites are a unit-gated PROVE instrument: `@setup` really
   runs its microflow, `@verify` really evaluates its OQL (holds / fails-with-the-value /
   cannot-evaluate = ERROR), and `@cleanup rollback` really rolls back — **before v0.19 all
   three were parsed and ignored, so no `@verify` could ever fail**. Two consequences: prefer a