From 2aff121511552ca39c23bbabd1b922d037f13b69 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 19:46:26 +0800 Subject: [PATCH 1/4] process(testing): unit tests per flow, listed in the module brief (Unit flows:) Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + project-bin/affected-tests.sh | 71 ++++++++++++++++++++++++-- skills/checkpoints/checkpoint-build.md | 11 ++-- skills/module-brief.md | 8 ++- tests/wave2/test-affected-tests.sh | 51 +++++++++++++++++- tests/wave2/test-test-strategy-docs.sh | 4 +- 6 files changed, 132 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ee699ada..0abfb7c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- process(testing): **unit tests are chosen per flow, not per module: a module brief lists `Unit flows:` (one `- ACT_X: reason` line each) and `affected-tests.sh` runs only the listed flows the change touched.** Follow-up to #417 on the maintainer's policy that unit-testing everything is overkill (logic and integrations only, UI journeys catch the rest). Module done and full runs include every listed flow; `Unit tests: yes` or `Unit modules:` with no list keeps the module-level selection; a flow listed without a reason, or not in the model (`SHOW MICROFLOWS IN`), warns. Test: wave2 `test-affected-tests.sh` L10 (16 checks). Baseline unchanged at 75,197 words (budget 80,000) — MendixMau - process(testing): **unit tests are now off by default and `affected-tests.sh` enforces it; each project picks its test strategy with the developer at CAC-5.** Fast-tracked by the maintainer on 2026-10-16 because users report tests firing on every change (the baseline-text A/B is waived; E3 follows as a check). Moments: every change = gate plus what it touched (unit only for affected modules with unit on); module done = that module's journeys (+ its unit tests if on); build complete and before every later deploy = everything once; "run everything" any time. The CAC-5 Offer records it in PROJECT.md `## Test strategy` (Status `ASSUMED` in auto mode). `affected-tests.sh` reads `Unit tests default:` (`no` or absent means unit off, one `unit off` line, never a `.test.mdl` run row); turn unit on per project with `Unit tests default: yes`, per module with `Unit modules: Shop, Admin` in PROJECT.md or `Unit tests: yes` in the module brief; the strategy row's Unit cell is a scope only and never turns unit on. A grant on a microflow or page the same script changes is part of that subject and no longer selects the owning module's tests. `module-review.md` PROVE now runs `--moment module --module `. `testing-shape.md`, `checkpoint-build.md`, `iterative-build-loop.md`, `conversion-runbook.md` and the exec.sh logic lane text updated. `tests/wave2/test-test-strategy-docs.sh` and the extended `test-affected-tests.sh` pin it. Baseline words stay at or below master (75,219 vs 75,225). Split out of #409; found by the field test of the release candidate — the maintainer's decision, 2026-10-16 - learn(skills): **five lab findings land as compact rows, each paid for by trimming prose in the same files, so always-loaded words do not grow (baseline 75,225 to 75,204, budget 80,000).** Enum value `New` and `AutoCreatedDate`/`AutoOwner` (detection gaps), `[%CurrentDateTime%]` not `currentDateTime()`, expression return types (Decimal from date diffs and `div`, absolute `secondsBetween`, `formatDecimal` on pages), CE2729 from the default home page and page microflow grants, Atlas selector carriers for list views, page body and timelines, `-small` spacing suffixes do not exist, `display: contents` rows paint nothing, CSS pseudo text is invisible to `innerText` — lab field runs, Refs #276 (TK-020, TK-021, TK-022, TK-024) - fix(design-check): **a design system whose palette lives only in scoped `:root.` blocks no longer reads as "0 of N tokens carry the design's value".** Pass 2b compared every token against a bare `:root` of the built sheet, so a themed-only palette always warned, a false warning that teaches agents to ignore the check. A token the design declares only in scoped blocks is now compared block for block against the same-named block in the built sheet, and a wrong value is still named with its block (knob binding stays bare-only: a knob bound inside a theme the app is not wearing is not bound). Synthetic fixtures `ds-themed.css`, `built-themed.css`, `built-themed-wrong.css`; 5 new cases in `test-design-reaches-app.sh` (50 total) — TK-015 part (b) diff --git a/project-bin/affected-tests.sh b/project-bin/affected-tests.sh index 6a2873e0..40c8adb4 100755 --- a/project-bin/affected-tests.sh +++ b/project-bin/affected-tests.sh @@ -54,6 +54,19 @@ # one `unit off` line and NEVER a `run ... .test.mdl` row. `Unit modules:` takes names split # by comma or space; `#` and `( ... )` comments, a leading `- ` bullet, any case and `none` # are read; a name that matches no module warns on stderr. +# 4. PER FLOW (2026-10-16, after #417). A module brief may list the flows that get unit tests: +# Unit flows: +# - ACT_Order_CalculateTotal: money calculation +# (a bullet per flow, `Name: reason`; `Name` may be qualified `Module.Name`; a `` +# name is ignored). A module with a list is flipped by it and selected per flow: at the change +# moment (scope affected) only the tests naming a LISTED flow the change touched (the flow +# itself, or a caller within --depth; a data change in the module touches every listed flow); +# at the module and deploy moments the tests naming any listed flow. A test is mapped to its +# flow the way every selection here is: the test file names `Module.Flow` (grep -w -F). A +# flow listed without a reason warns; a name `SHOW MICROFLOWS IN ` does not list warns +# (best effort: skipped under --no-model or when mxcli cannot run). `Unit tests: yes` or +# `Unit modules:` with NO list keeps the module-level selection. A list is ignored where the +# project default is `Unit tests default: yes` (everything is on). # Journeys are not affected by any of this. # # HOW THE SELECTION IS MADE (judgement-free facts only; skills-over-scripts.md): @@ -146,6 +159,7 @@ fi # ── Unit gate: off unless the project default is yes, the row says so, or a module is flipped ── UNIT_DEFAULT="$(awk '/^[ \t*]*Unit tests default:/ { v = $0; sub(/^[^:]*:/, "", v); gsub(/[ \t*`]/, "", v); print tolower(v); exit }' PROJECT.md 2>/dev/null)" +[ -d "$WORK" ] && : > "$WORK/unitflows" FLIPPED="$(awk 'tolower($0) ~ /^[ \t*>-]*unit modules:/ { v = $0; sub(/^[^:]*:/, "", v); sub(/#.*/, "", v); gsub(/\([^)]*\)/, "", v) gsub(/[ \t*`,]+/, " ", v); n = split(v, a, " "); o = "" for (i = 1; i <= n; i++) if (tolower(a[i]) != "none") o = o " " a[i] @@ -158,7 +172,19 @@ for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md /^\|/ { split($0, c, "|"); k = tolower(c[2]); v = tolower(c[3]); gsub(/[ \t*`]/, "", v) if (k ~ /^[ \t]*unit[ \t]*\(/ && v ~ /^yes/) f = 1 } END { exit !f }' "$_b"; then FLIPPED="$FLIPPED $_m"; fi + # Unit flows: a bullet list `- [Module.]Flow: reason` under a `Unit flows:` line. + awk -v m="$_m" ' + tolower($0) ~ /^[ \t*>-]*unit flows:/ { on = 1; next } + on && /^[ \t]*$/ { if (n) on = 0; next } + on && /^[ \t]*[-*][ \t]/ { l = $0; sub(/^[ \t]*[-*][ \t]+/, "", l); gsub(/`/, "", l) + i = index(l, ":"); nm = (i ? substr(l, 1, i - 1) : l); r = (i ? substr(l, i + 1) : "") + gsub(/^[ \t*]+|[ \t*]+$/, "", nm); gsub(/^[ \t]+|[ \t]+$/, "", r) + if (nm ~ /^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$/) { n++; q = (index(nm, ".") ? nm : m "." nm); print q "\t" r } + next } + on { on = 0 }' "$_b" >> "$WORK/unitflows" done +# a module with a Unit flows list is flipped by it +for _lm in $(cut -f1 "$WORK/unitflows" | sed 's/\..*//' | sort -u); do FLIPPED="$FLIPPED $_lm"; done UNIT_GATED=0; UNIT_OFF_WHY="" if [ "$MOMENT" != deploy ] && [ "${UNIT_DEFAULT#yes}" = "$UNIT_DEFAULT" ]; then if [ -n "$UNIT_DEFAULT" ]; then _dw="PROJECT.md Unit tests default: $UNIT_DEFAULT" @@ -303,17 +329,54 @@ if [ -n "$FLIPPED" ]; then FLIPPED="$_canon" fi select_unit "$U" > "$WORK/unit.sel" +LISTMODS="" +if [ "$UNIT_GATED" = 1 ] && [ -s "$WORK/unitflows" ]; then + # canonical module names (as typed in FLIPPED) for the list rows; warn on a missing reason + for m in $FLIPPED; do + awk -F'\t' -v m="$m" 'tolower(substr($1, 1, index($1, ".") - 1)) == tolower(m) { c = 1 } END { exit !c }' "$WORK/unitflows" && LISTMODS="$LISTMODS $m" + done + awk -F'\t' '$2 == "" { print "warning: Unit flows lists " $1 " without a reason; one line why it needs a unit test (logic or integration)" > "/dev/stderr" }' "$WORK/unitflows" + # unknown flow names: best effort, one batched SHOW MICROFLOWS IN (skipped without a model) + if [ "$NO_MODEL" != 1 ]; then + _mx="${MXCLI:-$(find_project_mxcli 2>/dev/null || true)}"; _mpr="$(find_mpr 2>/dev/null || true)" + if [ -n "$_mx" ] && [ -x "$_mx" ] && [ -n "$_mpr" ]; then + _q="$(cut -f1 "$WORK/unitflows" | sed 's/\..*//' | sort -u | sed 's/.*/SHOW MICROFLOWS IN &/' | awk '{ printf "%s%s", (NR > 1 ? "; " : ""), $0 }')" + if "$_mx" -p "$_mpr" -c "$_q" > "$WORK/mf.out" 2>/dev/null && grep -q '^| ' "$WORK/mf.out"; then + awk -F'|' '/^\| *[A-Za-z_][A-Za-z0-9_]*\.[A-Za-z_][A-Za-z0-9_]* *\|/ { n = $2; gsub(/ /, "", n); print n }' "$WORK/mf.out" | sort -u > "$WORK/mf.names" + cut -f1 "$WORK/unitflows" | while IFS= read -r _f; do + grep -qixF "$_f" "$WORK/mf.names" || echo "warning: Unit flows names '$_f', which is not a microflow in the model" >&2 + done + fi + fi + fi +fi if [ "$UNIT_GATED" = 1 ]; then # default no, some modules flipped: only those modules' tests : > "$WORK/unit.gated" + names_for_unit | sort -u > "$WORK/touched" + modules_for_unit > "$WORK/datamods" for m in $FLIPPED; do - while IFS=' ' read -r f w; do grep -qE "(^|[^A-Za-z0-9_])$m\." "$f" 2>/dev/null && printf '%s\t%s\n' "$f" "$w"; done < "$WORK/unit.sel" - done | awk -F'\t' '!seen[$1]++' > "$WORK/unit.gated" - mv "$WORK/unit.gated" "$WORK/unit.sel" + case " $LISTMODS " in + *" $m "*) # per flow: the tests naming a listed flow (touched ones at scope affected) + awk -F'\t' -v m="$m" 'tolower(substr($1, 1, index($1, ".") - 1)) == tolower(m) { print $1 "\t" $2 }' "$WORK/unitflows" | + while IFS=' ' read -r _fl _why; do + case "$U" in + none) continue ;; + affected) grep -qixF "$_fl" "$WORK/touched" || grep -qixF "$m" "$WORK/datamods" || continue ;; + esac + while IFS= read -r f; do + grep -q -w -F "$_fl" "$f" 2>/dev/null && printf '%s\tnames listed flow %s%s\n' "$f" "$_fl" "${_why:+ ($_why)}" + done < "$WORK/tests" + done >> "$WORK/unit.gated" ;; + *) + while IFS=' ' read -r f w; do grep -qE "(^|[^A-Za-z0-9_])$m\." "$f" 2>/dev/null && printf '%s\t%s\n' "$f" "$w"; done < "$WORK/unit.sel" >> "$WORK/unit.gated" ;; + esac + done + awk -F'\t' '!seen[$1]++' "$WORK/unit.gated" > "$WORK/unit.sel" fi select_journeys "$J" > "$WORK/journey.sel" if [ -n "$UNIT_OFF_WHY" ]; then out "unit off $UNIT_OFF_WHY" elif [ -s "$WORK/unit.sel" ]; then sed 's/^/unit /' "$WORK/unit.sel" -else out "unit none scope $U: $( [ "$U" = none ] && echo 'strategy says none' || echo "0 of $(wc -l < "$WORK/tests" | tr -d ' ') .test.mdl file(s) match${FLIPPED:+ (unit on only for:$FLIPPED)}")"; fi +else out "unit none scope $U: $( [ "$U" = none ] && echo 'strategy says none' || echo "0 of $(wc -l < "$WORK/tests" | tr -d ' ') .test.mdl file(s) match${FLIPPED:+ (unit on only for:$FLIPPED${LISTMODS:+; per listed flow in:$LISTMODS})}")"; fi if [ -s "$WORK/journey.sel" ]; then sed 's/^/journey /' "$WORK/journey.sel" else out "journey none scope $J: $( [ "$J" = none ] && echo 'strategy says none' || echo "0 of $(wc -l < "$WORK/journeys" | tr -d ' ') journey file(s) in $JDIR/ and tests/e2e/ match")"; fi diff --git a/skills/checkpoints/checkpoint-build.md b/skills/checkpoints/checkpoint-build.md index 8fde1c86..74b23e8a 100644 --- a/skills/checkpoints/checkpoint-build.md +++ b/skills/checkpoints/checkpoint-build.md @@ -118,15 +118,14 @@ developer may change any cell in the same answer: > module's own (`--moment module --module `); `full` = everything once; `none` = skip. > Build complete = you say the app is ready for UAT or production; I do not declare it alone. > Say 'run everything' any time. -> Unit tests: only modules you name now, written while the logic is fresh (writing is not -> running). Default: off, except modules with calculation, money/date, mapping, REST/Java logic or -> a fixed bug. Change a cell, name modules, or say 'defaults'." +> Unit tests: only flows you name, a reason each (calculation, money/date, mapping, +> REST/Java logic, a fixed bug), Default none; journeys catch the rest. Change a cell, name flows, or say 'defaults'." **Record as:** `PROJECT.md` → `## Test strategy`, the table plus a `Status` column (`CONFIRMED`, or -`ASSUMED` in auto mode), and the line `Unit modules: Shop, Admin` (`none` if none). Record Unit cells as scopes as shown. +`ASSUMED` in auto mode); each named flow goes in its module brief as `Unit flows:` (`- ACT_X: reason`). Unit cells are scopes. `bin/affected-tests.sh --moment change|module|deploy|all` reads it; no section: these defaults. -Unit is **off** unless `Unit tests default: yes` (whole project; absent = no), a `Unit modules:` -entry, or `Unit tests: yes` in the module brief; the Unit cell is a scope and never turns it on. +Unit is **off** unless `Unit tests default: yes` (absent = no) or a brief lists `Unit flows:` +(`Unit modules:` / `Unit tests: yes` still flip a whole module); the Unit cell is a scope only. At build end ask "full run now?", a confirmation, not a decision. ## Open Question diff --git a/skills/module-brief.md b/skills/module-brief.md index 64b2d2f5..aec7cb97 100644 --- a/skills/module-brief.md +++ b/skills/module-brief.md @@ -150,9 +150,15 @@ all, and never looks at a page. |------|-------------|-----| | UI (Playwright) | **always** | — | | Data (source of truth) | **always** — say which: OQL count delta \| upstream REST response \| documented n/a + the query proving it | `testing-shape.md` §1: "Data" means check the source of truth, not always query a database | -| Unit (`.test.mdl`) | **no** (default) / yes | **no** unless this module has calculation, mapping or date/money logic, calls a Java action or REST, or had a bug fixed in that flow, name which. Project default: intake Q13 (`Unit tests default:`), `no` unless chosen (`testing-shape.md` §2). Flip: line `Unit tests: yes` | +| Unit (`.test.mdl`) | **no** / flows listed | **no** unless a flow has calculation, mapping, date/money logic, a Java action or REST call, or a fixed bug: list those below (`testing-shape.md` §2). | | Trace (OTel) | yes / no | **yes if this module calls out** to REST, an import mapping, or external data — that is where failures get swallowed, and it overrides the project default | +One line and a reason per flow; only these get tests. +``` +Unit flows: +- ACT_Order_CalculateTotal: money calculation +``` + ### Base set — the trigger (`testing-shape.md` §3) The named scripts that constitute this module's base logic. **When every script in this list is done, testing opens.** Scripts added later do not move the goalpost, because the goalpost was diff --git a/tests/wave2/test-affected-tests.sh b/tests/wave2/test-affected-tests.sh index 39d45601..a6047c40 100644 --- a/tests/wave2/test-affected-tests.sh +++ b/tests/wave2/test-affected-tests.sh @@ -63,6 +63,9 @@ D="$(cd "$(dirname "$0")" && pwd)" printf '%s\n' "$*" >> "$D/.stub-log" [ "${STUB_NOPROJ:-}" = 1 ] && { cat "$FX/noproj.out"; cat "$FX/noproj.err" >&2; exit "$(cat "$FX/noproj.exit")"; } q=""; while [ $# -gt 0 ]; do [ "$1" = -c ] && q="$2"; shift; done +case "$q" in "SHOW MICROFLOWS IN"*) # synthetic listing, the real table shape + echo "| Qualified Name | Module | Name | Excluded |"; echo "|---|---|---|---|" + for n in ${STUB_MF:-Shop.SUB_Order_Total Shop.VAL_Order Shop.ACT_Order_Submit Admin.ACT_Reset}; do echo "| $n | ${n%%.*} | ${n#*.} | false |"; done; exit 0 ;; esac cat "$FX/callers.err" >&2 first=1 printf '%s\n' "$q" | tr ';' '\n' | sed -n 's/^ *SHOW CALLERS OF \([^ ]*\) TRANSITIVE *$/\1/p' | while IFS= read -r n; do @@ -73,7 +76,7 @@ printf '%s\n' "$q" | tr ';' '\n' | sed -n 's/^ *SHOW CALLERS OF \([^ ]*\) TRANSI done || exit 99 exit "$(cat "$FX/callers.exit")" MXCLI -chmod +x "$P/mxcli" +chmod +x "$P/mxcli"; cp "$P/mxcli" "$WORK/mxcli.orig" export FX lines() { sed -n "$1p" "$FX/shop.mdl" > "$WORK/$2"; echo "$WORK/$2"; } @@ -332,6 +335,52 @@ echo "== H: usage ==" RC=$(run H --moment later "$TOTAL"); [ "$RC" -eq 2 ] && ok "bad --moment exits 2" || bad "H rc=$RC" RC=$(run H2 "$WORK/no-such.mdl"); [ "$RC" -eq 2 ] && ok "missing script exits 2" || bad "H2 rc=$RC" RC=$(run H3); [ "$RC" -eq 2 ] && ok "a change with no script exits 2" || bad "H3 rc=$RC" +# L10: per-flow unit tests (Unit flows: in the module brief) +echo "== L10: Unit flows: per-flow selection ==" +cp "$WORK/mxcli.orig" "$P/mxcli"; chmod +x "$P/mxcli" # section K swaps the stub +printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md" +fl() { mkdir -p "$BRIEF_DIR/Shop"; printf '# Brief\n\n%b\n' "$1" > "$BRIEF_DIR/Shop/module-brief.md"; } +fl 'Unit flows:\n- SUB_Order_Total: money calculation\n- ACT_Order_Submit: REST call' +run L10a "$TOTAL" >/dev/null 2>&1 +has L10a "unit tests/unit/Total.test.mdl names listed flow Shop.SUB_Order_Total (money calculation)" && ok "L10a listed and touched: its test is selected, with the reason" || bad "L10a" "$(cat "$WORK/out.L10a")" +lacks L10a "Submit.test.mdl" && ok "L10b listed but untouched (depth-2 caller): skipped" || bad "L10b" "$(cat "$WORK/out.L10a")" +lacks L10a "Validate.test.mdl" && ok "L10c unlisted but touched (depth-1 caller): skipped" || bad "L10c" "$(cat "$WORK/out.L10a")" +lacks L10a "Lookalike.test.mdl" && ok "L10d a look-alike flow name is not the listed flow" || bad "L10d" +[ ! -s "$WORK/err.L10a" ] && ok "L10e a clean list, known flows: no warning" || bad "L10e" "$(cat "$WORK/err.L10a")" +run L10f --depth 2 "$TOTAL" >/dev/null 2>&1 +has L10f "Submit.test.mdl" && ok "L10f listed flow reached as a caller within --depth: selected" || bad "L10f" "$(cat "$WORK/out.L10f")" +run L10g "$ENTITY" >/dev/null 2>&1 +has L10g "Total.test.mdl" && has L10g "Submit.test.mdl" && lacks L10g "Validate.test.mdl" && ok "L10g a data change in the module owes every listed flow, not the unlisted ones" || bad "L10g" "$(cat "$WORK/out.L10g")" +run L10h --moment module --module Shop >/dev/null 2>&1 +has L10h "Total.test.mdl" && has L10h "Submit.test.mdl" && lacks L10h "Validate.test.mdl" && lacks L10h "Other.test.mdl" && ok "L10h module done: all listed flows' tests, nothing unlisted" || bad "L10h" "$(cat "$WORK/out.L10h")" +run L10i --moment deploy >/dev/null 2>&1 +for t in Total Validate Submit Lookalike Other; do has L10i "$t.test.mdl" || bad "L10i full run lacks $t.test.mdl"; done; has L10i "unit tests/unit/Other.test.mdl full suite" && ok "L10i full run includes every test, listed or not" || bad "L10i" +fl 'Unit flows:\n- Shop.SUB_Order_Total: money calculation' +run L10j "$TOTAL" >/dev/null 2>&1 +has L10j "Total.test.mdl" && ok "L10j a qualified name in the list is read" || bad "L10j" "$(cat "$WORK/out.L10j")" +fl 'Unit tests: yes' +run L10k "$TOTAL" >/dev/null 2>&1 +has L10k "Total.test.mdl" && has L10k "Validate.test.mdl" && ok "L10k 'Unit tests: yes' without a list keeps the module-level selection" || bad "L10k" "$(cat "$WORK/out.L10k")" +fl 'Unit tests: yes\n\nUnit flows:\n- SUB_Order_Total: money calculation' +run L10l "$TOTAL" >/dev/null 2>&1 +has L10l "Total.test.mdl" && lacks L10l "Validate.test.mdl" && ok "L10l a list beats 'Unit tests: yes' in the same brief" || bad "L10l" "$(cat "$WORK/out.L10l")" +fl 'Unit flows:\n- SUB_Order_Total\n- ACT_Order_Submit: ' +run L10m "$TOTAL" >/dev/null 2>&1 +grep -q "Shop.SUB_Order_Total without a reason" "$WORK/err.L10m" && grep -q "Shop.ACT_Order_Submit without a reason" "$WORK/err.L10m" && has L10m "Total.test.mdl" && ok "L10m a flow listed without a reason warns (and is still selected)" || bad "L10m" "$(cat "$WORK/err.L10m")" +fl 'Unit flows:\n- SUB_Order_Total: money calculation\n- NoSuchFlow: invented' +run L10n "$TOTAL" >/dev/null 2>&1 +grep -q "'Shop.NoSuchFlow'" "$WORK/err.L10n" && ! grep -q "'Shop.SUB_Order_Total'" "$WORK/err.L10n" && ok "L10n an unknown flow name warns, a known one does not" || bad "L10n" "$(cat "$WORK/err.L10n")" +run L10o --no-model "$TOTAL" >/dev/null 2>&1 +! grep -q "NoSuchFlow" "$WORK/err.L10o" && ok "L10o --no-model skips the existence check" || bad "L10o" +fl 'Unit flows:\n- : ' +run L10p "$TOTAL" >/dev/null 2>&1 +lacks L10p "Total.test.mdl" && lacks L10p "Validate.test.mdl" && [ ! -s "$WORK/err.L10p" ] && ok "L10p the unfilled template placeholder lists nothing and warns nothing" || bad "L10p" "$(cat "$WORK/out.L10p" "$WORK/err.L10p")" +printf '# Project\n\nUnit tests default: no\n## Test strategy\n| Moment | Unit tests | Journeys | Status |\n|---|---|---|---|\n| Every change | none | affected | CONFIRMED |\n' > "$P/PROJECT.md" +fl 'Unit flows:\n- SUB_Order_Total: money calculation' +run L10q "$TOTAL" >/dev/null 2>&1 +has L10q "Total.test.mdl" && ok "L10q row none + a Unit flows list: the listed flow's test runs (like L5c)" || bad "L10q" "$(cat "$WORK/out.L10q")" +rm -rf "$P/architecture" +printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md" [ ! -e "$P/.mxcli" ] && ok "wrote nothing into the project" || bad "project written" echo "== wiring ==" diff --git a/tests/wave2/test-test-strategy-docs.sh b/tests/wave2/test-test-strategy-docs.sh index 2b63f55b..2e5023f1 100644 --- a/tests/wave2/test-test-strategy-docs.sh +++ b/tests/wave2/test-test-strategy-docs.sh @@ -51,11 +51,11 @@ has skills/existing-app-change.md "--moment module --module " && has skills/c && ok "the module-moment call names its module" || bad "module call without --module" echo "== unit tests off by default, wired ==" -has skills/checkpoints/checkpoint-build.md "\`Unit modules: Shop, Admin\`" && has skills/checkpoints/checkpoint-build.md "Unit tests default: yes" && has skills/checkpoints/checkpoint-build.md "Unit tests: yes" \ +has skills/checkpoints/checkpoint-build.md "\`Unit flows:\`" && has skills/checkpoints/checkpoint-build.md "Unit tests default: yes" && has skills/checkpoints/checkpoint-build.md "Unit tests: yes" \ && ok "Offer records the three ways to turn unit on" || bad "unit-on syntax missing" has skills/checkpoints/checkpoint-build.md "Build complete = you say the app is ready" && ok "build complete is declared by the developer" || bad "build complete undefined" has skills/module-review.md "--moment module" && ok "module-review PROVE runs the module row" || bad "module-review not wired" -has skills/module-brief.md "Unit tests: yes" && ok "module brief names the flip line" || bad "module brief flip line" +has skills/module-brief.md "Unit flows:" && ok "module brief names the Unit flows list" || bad "module brief flip line" echo; echo "PASS=$PASS FAIL=$FAIL" [ "$FAIL" -eq 0 ] From febc85eef7b6f5d4686035fca89f80e4188509cc Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 20:05:34 +0800 Subject: [PATCH 2/4] fix(testing): per-flow unit tests review fixes (module scope, parser, default yes, owed tests, casing, missing module) Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 +- project-bin/affected-tests.sh | 131 +++++++++++++++++-------- skills/checkpoints/checkpoint-build.md | 5 +- skills/testing-shape.md | 2 +- tests/wave2/test-affected-tests.sh | 38 +++++++ 5 files changed, 132 insertions(+), 46 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e8bb6b39..e47a618d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased -- process(testing): **unit tests are chosen per flow, not per module: a module brief lists `Unit flows:` (one `- ACT_X: reason` line each) and `affected-tests.sh` runs only the listed flows the change touched.** Follow-up to #417 on the maintainer's policy that unit-testing everything is overkill (logic and integrations only, UI journeys catch the rest). Module done and full runs include every listed flow; `Unit tests: yes` or `Unit modules:` with no list keeps the module-level selection; a flow listed without a reason, or not in the model (`SHOW MICROFLOWS IN`), warns. Test: wave2 `test-affected-tests.sh` L10 (16 checks). Baseline unchanged at 75,197 words (budget 80,000) — MendixMau +- process(testing): **unit tests are chosen per flow, not per module: a module brief lists `Unit flows:` (one `- ACT_X: reason` line each) and `affected-tests.sh` runs only the listed flows the change touched.** Follow-up to #417 on the maintainer's policy that unit-testing everything is overkill (logic and integrations only, UI journeys catch the rest). Module done runs the `--module` modules' listed flows only; a list wins for its module even under `Unit tests default: yes`; bullets may be separated by blank lines, carry `(note)` or `Module.` forms, or sit inline (`Unit flows: A, B`), and a line that cannot be read warns with its text; `Unit tests: yes` or `Unit modules:` with no list keeps the module-level selection; a flow listed without a reason, with no test, with a different casing than the model, or not in the model (`SHOW MICROFLOWS IN`), and a module the model lacks, warn. Test: wave2 `test-affected-tests.sh` L10 (26 checks). Baseline unchanged at 75,197 words (budget 80,000) — MendixMau - fix(wire-agents): **a sync re-wire no longer wipes `.claude/settings.json`.** `mxcli init` overwrites that file; wire-agents now snapshots it and merges after init: the project's entries win and keep their order (permission profile, user entries, the `mxcli exec` deny rule, hooks), and entries that only mxcli's new file has are appended. Merged rather than added to PRESERVE because PRESERVE restores old bytes and would also drop anything a newer mxcli adds; an unparsable snapshot is restored byte for byte. Test: `tests/wave2/test-wire-agents-settings.sh` (4 of 6 checks fail on master). — review of #414 (#419) - fix(sync): **the exec-approval item-3 rewrite now works on BSD awk (stock macOS) and `sync-project.sh` no longer prints "Rewrote" for a rewrite that failed.** `_ea_rewrite_item3` passed the multi-line replacement through `awk -v`, which BSD awk rejects ("newline in string", exit 2); the file stayed unchanged and sync still reported the rewrite on every run. The replacement is now streamed from a temp file (as the Q9 repair does), the helper returns 2 when awk fails, and the caller prints a `WARN:` instead of "Rewrote" and does not count a change. `test-exec-approval.sh` gains 3 assertions (failing-awk shim: no "Rewrote", a WARN, file unchanged). No other multi-line `awk -v` call site exists (the rest are single-line values) ([#426](https://github.com/MendixMau/mxcli-project-toolkit/issues/426)) — MendixMau - process(testing): **unit tests are now off by default and `affected-tests.sh` enforces it; each project picks its test strategy with the developer at CAC-5.** Fast-tracked by the maintainer on 2026-10-16 because users report tests firing on every change (the baseline-text A/B is waived; E3 follows as a check). Moments: every change = gate plus what it touched (unit only for affected modules with unit on); module done = that module's journeys (+ its unit tests if on); build complete and before every later deploy = everything once; "run everything" any time. The CAC-5 Offer records it in PROJECT.md `## Test strategy` (Status `ASSUMED` in auto mode). `affected-tests.sh` reads `Unit tests default:` (`no` or absent means unit off, one `unit off` line, never a `.test.mdl` run row); turn unit on per project with `Unit tests default: yes`, per module with `Unit modules: Shop, Admin` in PROJECT.md or `Unit tests: yes` in the module brief; the strategy row's Unit cell is a scope only and never turns unit on. A grant on a microflow or page the same script changes is part of that subject and no longer selects the owning module's tests. `module-review.md` PROVE now runs `--moment module --module `. `testing-shape.md`, `checkpoint-build.md`, `iterative-build-loop.md`, `conversion-runbook.md` and the exec.sh logic lane text updated. `tests/wave2/test-test-strategy-docs.sh` and the extended `test-affected-tests.sh` pin it. Baseline words stay at or below master (75,219 vs 75,225). Split out of #409; found by the field test of the release candidate — the maintainer's decision, 2026-10-16 diff --git a/project-bin/affected-tests.sh b/project-bin/affected-tests.sh index 40c8adb4..e7e8e8b6 100755 --- a/project-bin/affected-tests.sh +++ b/project-bin/affected-tests.sh @@ -57,16 +57,19 @@ # 4. PER FLOW (2026-10-16, after #417). A module brief may list the flows that get unit tests: # Unit flows: # - ACT_Order_CalculateTotal: money calculation -# (a bullet per flow, `Name: reason`; `Name` may be qualified `Module.Name`; a `` -# name is ignored). A module with a list is flipped by it and selected per flow: at the change -# moment (scope affected) only the tests naming a LISTED flow the change touched (the flow -# itself, or a caller within --depth; a data change in the module touches every listed flow); -# at the module and deploy moments the tests naming any listed flow. A test is mapped to its -# flow the way every selection here is: the test file names `Module.Flow` (grep -w -F). A -# flow listed without a reason warns; a name `SHOW MICROFLOWS IN ` does not list warns +# (a bullet per flow, `Name: reason`, `Name (note): reason` or inline `Unit flows: A, B`; blank +# lines between bullets are fine, a heading or any other line ends the list; `Name` may be +# qualified `Module.Name`; a `` is ignored; a line that cannot be read warns). +# A module with a list is flipped by it and selected per flow: at the change moment (scope +# affected) only the tests naming a LISTED flow the change touched (the flow itself, or a caller +# within --depth; a data change in the module touches every listed flow); at the module moment +# only for the --module/subject modules; at deploy every test runs. A test is mapped to its flow +# as every selection here is: the test file names `Module.Flow` (grep -i -w -F). A flow listed +# without a reason warns; a listed flow no test names warns (the test is owed); a name +# `SHOW MICROFLOWS IN ` does not list, a different casing, or a module the model lacks warns # (best effort: skipped under --no-model or when mxcli cannot run). `Unit tests: yes` or -# `Unit modules:` with NO list keeps the module-level selection. A list is ignored where the -# project default is `Unit tests default: yes` (everything is on). +# `Unit modules:` with NO list keeps the module-level selection. Under `Unit tests default: yes` +# a module with a list is still narrowed to it (the list wins for its module); the others stay on. # Journeys are not affected by any of this. # # HOW THE SELECTION IS MADE (judgement-free facts only; skills-over-scripts.md): @@ -172,15 +175,29 @@ for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md /^\|/ { split($0, c, "|"); k = tolower(c[2]); v = tolower(c[3]); gsub(/[ \t*`]/, "", v) if (k ~ /^[ \t]*unit[ \t]*\(/ && v ~ /^yes/) f = 1 } END { exit !f }' "$_b"; then FLIPPED="$FLIPPED $_m"; fi - # Unit flows: a bullet list `- [Module.]Flow: reason` under a `Unit flows:` line. - awk -v m="$_m" ' - tolower($0) ~ /^[ \t*>-]*unit flows:/ { on = 1; next } - on && /^[ \t]*$/ { if (n) on = 0; next } - on && /^[ \t]*[-*][ \t]/ { l = $0; sub(/^[ \t]*[-*][ \t]+/, "", l); gsub(/`/, "", l) - i = index(l, ":"); nm = (i ? substr(l, 1, i - 1) : l); r = (i ? substr(l, i + 1) : "") - gsub(/^[ \t*]+|[ \t*]+$/, "", nm); gsub(/^[ \t]+|[ \t]+$/, "", r) - if (nm ~ /^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?$/) { n++; q = (index(nm, ".") ? nm : m "." nm); print q "\t" r } + # Unit flows: a bullet list `- [Module.]Flow[ (note)][: reason]` under a `Unit flows:` line, or inline + # `Unit flows: A, B`. Blank lines between bullets are fine; a heading or any other line ends the block. + # A line in the block that cannot be read warns with its text, never dropped silently. + awk -v m="$_m" -v b="$_b" ' + function trim(t) { gsub(/^[ \t]+|[ \t]+$/, "", t); return t } + function warn(t) { print "warning: Unit flows in " b ": cannot read \"" t "\"; write `- ACT_Name: reason`" > "/dev/stderr" } + function item(l, nm, rest, par, r, i) { + gsub(/`|\*\*/, "", l); l = trim(l) + if (l == "" || substr(l, 1, 1) == "<") return + if (!match(l, /^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?/)) { warn(l); return } + nm = substr(l, 1, RLENGTH); rest = trim(substr(l, RLENGTH + 1)); par = "" + if (substr(rest, 1, 1) == "(") { i = index(rest, ")"); if (i) { par = substr(rest, 2, i - 2); rest = trim(substr(rest, i + 1)) } } + if (rest == "") r = par + else if (substr(rest, 1, 1) == ":") { r = trim(substr(rest, 2)); if (r == "") r = par } + else if (rest ~ /^-[ \t]/) r = trim(substr(rest, 2)) + else { warn(l); return } + print (index(nm, ".") ? nm : m "." nm) "\t" r + } + tolower($0) ~ /^[ \t*>-]*unit flows:/ { on = 1; v = substr($0, index(tolower($0), "unit flows:") + 11); gsub(/`|\*\*/, "", v); v = trim(v) + if (v != "") { if (index(v, ":")) item(v); else { k = split(v, a, ","); for (j = 1; j <= k; j++) item(a[j]) } } next } + on && /^[ \t]*$/ { next } + on && /^[ \t]*[-*][ \t]/ { l = $0; sub(/^[ \t]*[-*][ \t]+/, "", l); item(l); next } on { on = 0 }' "$_b" >> "$WORK/unitflows" done # a module with a Unit flows list is flipped by it @@ -190,7 +207,7 @@ if [ "$MOMENT" != deploy ] && [ "${UNIT_DEFAULT#yes}" = "$UNIT_DEFAULT" ]; then if [ -n "$UNIT_DEFAULT" ]; then _dw="PROJECT.md Unit tests default: $UNIT_DEFAULT" else _dw="PROJECT.md has no 'Unit tests default:' line, treated as no"; fi if [ -n "$FLIPPED" ]; then UNIT_GATED=1; [ "$U" = none ] && U="$DU" - else U=none; UNIT_OFF_WHY="($_dw; flip a module: PROJECT.md 'Unit modules: ' or 'Unit tests: yes' in its brief)"; fi + else U=none; UNIT_OFF_WHY="($_dw; flip a module: PROJECT.md 'Unit modules: ', or 'Unit tests: yes' or a 'Unit flows:' list in its brief)"; fi FROM="$FROM; unit: $_dw" fi out "strategy $MOMENT unit=$U journeys=$J ($FROM)" @@ -324,27 +341,61 @@ if [ -n "$FLIPPED" ]; then for m in $FLIPPED; do _k="$(awk -v m="$m" 'tolower($0) == tolower(m) { print; exit }' "$WORK/known")" if [ -n "$_k" ]; then _canon="$_canon $_k" - else _canon="$_canon $m"; echo "warning: Unit modules names '$m', which matches no module in the tests, briefs or this change" >&2; fi + else _canon="$_canon $m" # a module named only by a Unit flows list is checked against the model below + awk -F'\t' -v m="$m" 'tolower(substr($1, 1, index($1, ".") - 1)) == tolower(m) { c = 1 } END { exit !c }' "$WORK/unitflows" || + echo "warning: Unit modules names '$m', which matches no module in the tests, briefs or this change" >&2; fi done FLIPPED="$_canon" fi select_unit "$U" > "$WORK/unit.sel" -LISTMODS="" -if [ "$UNIT_GATED" = 1 ] && [ -s "$WORK/unitflows" ]; then - # canonical module names (as typed in FLIPPED) for the list rows; warn on a missing reason - for m in $FLIPPED; do - awk -F'\t' -v m="$m" 'tolower(substr($1, 1, index($1, ".") - 1)) == tolower(m) { c = 1 } END { exit !c }' "$WORK/unitflows" && LISTMODS="$LISTMODS $m" +canon_mod() { awk -v m="$1" 'tolower($0) == tolower(m) { print; f = 1; exit } END { if (!f) print m }' "$WORK/known" 2>/dev/null || echo "$1"; } +# list_sel : the tests of that module's listed flows at the current scope; warns on an owed test +list_sel() { + case "$U" in none) return ;; module) grep -qixF "$1" "$WORK/modules" || return ;; esac + awk -F'\t' -v m="$1" 'tolower(substr($1, 1, index($1, ".") - 1)) == tolower(m) { print $1 "\t" $2 }' "$WORK/unitflows" | + while IFS=' ' read -r _fl _why; do + if [ "$U" = affected ]; then grep -qixF "$_fl" "$WORK/touched" || grep -qixF "$1" "$WORK/datamods" || continue; fi + _hit=0 + while IFS= read -r f; do + grep -q -i -w -F "$_fl" "$f" 2>/dev/null && { _hit=1; printf '%s\tnames listed flow %s%s\n' "$f" "$_fl" "${_why:+ ($_why)}"; } + done < "$WORK/tests" + [ "$_hit" = 1 ] || echo "warning: Unit flows lists $_fl, which no .test.mdl names; its unit test is owed" >&2 done +} +names_for_unit | sort -u > "$WORK/touched" +modules_for_unit > "$WORK/datamods" +LISTMODS="" +if [ -s "$WORK/unitflows" ] && [ "$MOMENT" != deploy ]; then + for m in $(cut -f1 "$WORK/unitflows" | sed 's/\..*//' | sort -u); do LISTMODS="$LISTMODS +$(canon_mod "$m")"; done + LISTMODS="$(printf '%s\n' "$LISTMODS" | sed '/^$/d' | sort -u | tr '\n' ' ' | sed 's/ $//')" awk -F'\t' '$2 == "" { print "warning: Unit flows lists " $1 " without a reason; one line why it needs a unit test (logic or integration)" > "/dev/stderr" }' "$WORK/unitflows" - # unknown flow names: best effort, one batched SHOW MICROFLOWS IN (skipped without a model) + # unknown flow names: best effort, SHOW MICROFLOWS IN (skipped without a model) if [ "$NO_MODEL" != 1 ]; then _mx="${MXCLI:-$(find_project_mxcli 2>/dev/null || true)}"; _mpr="$(find_mpr 2>/dev/null || true)" if [ -n "$_mx" ] && [ -x "$_mx" ] && [ -n "$_mpr" ]; then - _q="$(cut -f1 "$WORK/unitflows" | sed 's/\..*//' | sort -u | sed 's/.*/SHOW MICROFLOWS IN &/' | awk '{ printf "%s%s", (NR > 1 ? "; " : ""), $0 }')" - if "$_mx" -p "$_mpr" -c "$_q" > "$WORK/mf.out" 2>/dev/null && grep -q '^| ' "$WORK/mf.out"; then + _lm="$(cut -f1 "$WORK/unitflows" | sed 's/\..*//' | sort -u)" + _q="$(printf '%s\n' "$_lm" | sed 's/.*/SHOW MICROFLOWS IN &/' | awk '{ printf "%s%s", (NR > 1 ? "; " : ""), $0 }')" + : > "$WORK/mf.missing" + if ! "$_mx" -p "$_mpr" -c "$_q" > "$WORK/mf.out" 2>/dev/null; then + # one call per module: missing = its error says so, or it fails while another module answers + : > "$WORK/mf.out"; : > "$WORK/mf.fail"; _okn=0 + for _m in $_lm; do + if "$_mx" -p "$_mpr" -c "SHOW MICROFLOWS IN $_m" >> "$WORK/mf.out" 2> "$WORK/mf.one"; then _okn=$((_okn+1)) + elif grep -qiE 'not found|does not exist|no such module|unknown module' "$WORK/mf.one"; then echo "$_m" >> "$WORK/mf.missing" + else echo "$_m" >> "$WORK/mf.fail"; fi + done + [ "$_okn" -gt 0 ] && cat "$WORK/mf.fail" >> "$WORK/mf.missing" + while IFS= read -r _m; do echo "warning: module $_m not found in the model (Unit flows lists it)" >&2; done < "$WORK/mf.missing" + fi + if grep -q '^| ' "$WORK/mf.out"; then awk -F'|' '/^\| *[A-Za-z_][A-Za-z0-9_]*\.[A-Za-z_][A-Za-z0-9_]* *\|/ { n = $2; gsub(/ /, "", n); print n }' "$WORK/mf.out" | sort -u > "$WORK/mf.names" - cut -f1 "$WORK/unitflows" | while IFS= read -r _f; do - grep -qixF "$_f" "$WORK/mf.names" || echo "warning: Unit flows names '$_f', which is not a microflow in the model" >&2 + cut -f1 "$WORK/unitflows" | sort -u | while IFS= read -r _f; do + grep -qixF "${_f%%.*}" "$WORK/mf.missing" && continue + grep -qxF "$_f" "$WORK/mf.names" && continue + _real="$(grep -ixF "$_f" "$WORK/mf.names" | head -1)" + if [ -n "$_real" ]; then echo "warning: Unit flows lists '$_f' but the model has '$_real' (casing differs)" >&2 + else echo "warning: Unit flows names '$_f', which is not a microflow in the model" >&2; fi done fi fi @@ -352,26 +403,22 @@ if [ "$UNIT_GATED" = 1 ] && [ -s "$WORK/unitflows" ]; then fi if [ "$UNIT_GATED" = 1 ]; then # default no, some modules flipped: only those modules' tests : > "$WORK/unit.gated" - names_for_unit | sort -u > "$WORK/touched" - modules_for_unit > "$WORK/datamods" for m in $FLIPPED; do case " $LISTMODS " in - *" $m "*) # per flow: the tests naming a listed flow (touched ones at scope affected) - awk -F'\t' -v m="$m" 'tolower(substr($1, 1, index($1, ".") - 1)) == tolower(m) { print $1 "\t" $2 }' "$WORK/unitflows" | - while IFS=' ' read -r _fl _why; do - case "$U" in - none) continue ;; - affected) grep -qixF "$_fl" "$WORK/touched" || grep -qixF "$m" "$WORK/datamods" || continue ;; - esac - while IFS= read -r f; do - grep -q -w -F "$_fl" "$f" 2>/dev/null && printf '%s\tnames listed flow %s%s\n' "$f" "$_fl" "${_why:+ ($_why)}" - done < "$WORK/tests" - done >> "$WORK/unit.gated" ;; + *" $m "*) list_sel "$m" >> "$WORK/unit.gated" ;; # per flow: the listed flows' tests only *) while IFS=' ' read -r f w; do grep -qE "(^|[^A-Za-z0-9_])$m\." "$f" 2>/dev/null && printf '%s\t%s\n' "$f" "$w"; done < "$WORK/unit.sel" >> "$WORK/unit.gated" ;; esac done awk -F'\t' '!seen[$1]++' "$WORK/unit.gated" > "$WORK/unit.sel" +elif [ -n "$LISTMODS" ]; then # default yes: a module with a list is narrowed to it, the rest stay on + : > "$WORK/unit.ls"; for m in $LISTMODS; do list_sel "$m" >> "$WORK/unit.ls"; done + while IFS=' ' read -r f w; do + _gov=0; for m in $LISTMODS; do grep -qE "(^|[^A-Za-z0-9_])$m\." "$f" 2>/dev/null && _gov=1; done + [ "$_gov" = 0 ] && printf '%s\t%s\n' "$f" "$w" + done < "$WORK/unit.sel" > "$WORK/unit.new" + cat "$WORK/unit.ls" >> "$WORK/unit.new" + awk -F'\t' '!seen[$1]++' "$WORK/unit.new" > "$WORK/unit.sel" fi select_journeys "$J" > "$WORK/journey.sel" if [ -n "$UNIT_OFF_WHY" ]; then out "unit off $UNIT_OFF_WHY" diff --git a/skills/checkpoints/checkpoint-build.md b/skills/checkpoints/checkpoint-build.md index 74b23e8a..08d4b40b 100644 --- a/skills/checkpoints/checkpoint-build.md +++ b/skills/checkpoints/checkpoint-build.md @@ -118,8 +118,9 @@ developer may change any cell in the same answer: > module's own (`--moment module --module `); `full` = everything once; `none` = skip. > Build complete = you say the app is ready for UAT or production; I do not declare it alone. > Say 'run everything' any time. -> Unit tests: only flows you name, a reason each (calculation, money/date, mapping, -> REST/Java logic, a fixed bug), Default none; journeys catch the rest. Change a cell, name flows, or say 'defaults'." +> Unit tests: only flows you name, written while the logic is fresh (writing is not running), a +> reason each (calculation, money/date, mapping, REST/Java logic, a fixed bug). Default: off. +> Change a cell, name flows, or say 'defaults'." **Record as:** `PROJECT.md` → `## Test strategy`, the table plus a `Status` column (`CONFIRMED`, or `ASSUMED` in auto mode); each named flow goes in its module brief as `Unit flows:` (`- ACT_X: reason`). Unit cells are scopes. diff --git a/skills/testing-shape.md b/skills/testing-shape.md index bf233550..77c0f0f8 100644 --- a/skills/testing-shape.md +++ b/skills/testing-shape.md @@ -157,7 +157,7 @@ recorded. A reviewer who doubts an artifact says which claim it fails to prove; not a second run. **Each project picks what runs when** at CAC-5: PROJECT.md `## Test strategy` (`checkpoint-build.md`; -defaults `affected`, `module`, `full`). Unit tests are off unless a module is named (`Unit modules:`). +defaults `affected`, `module`, `full`). Unit is off unless named (`Unit modules:`/`Unit flows:`). Never at batch close, after a fix, or because a change "felt wide". | Event | What happens | diff --git a/tests/wave2/test-affected-tests.sh b/tests/wave2/test-affected-tests.sh index a6047c40..b6aa090b 100644 --- a/tests/wave2/test-affected-tests.sh +++ b/tests/wave2/test-affected-tests.sh @@ -64,6 +64,7 @@ printf '%s\n' "$*" >> "$D/.stub-log" [ "${STUB_NOPROJ:-}" = 1 ] && { cat "$FX/noproj.out"; cat "$FX/noproj.err" >&2; exit "$(cat "$FX/noproj.exit")"; } q=""; while [ $# -gt 0 ]; do [ "$1" = -c ] && q="$2"; shift; done case "$q" in "SHOW MICROFLOWS IN"*) # synthetic listing, the real table shape + [ -n "${STUB_BADMOD:-}" ] && case "$q" in *"IN $STUB_BADMOD"*) echo "Error: module not found: $STUB_BADMOD" >&2; exit 1 ;; esac echo "| Qualified Name | Module | Name | Excluded |"; echo "|---|---|---|---|" for n in ${STUB_MF:-Shop.SUB_Order_Total Shop.VAL_Order Shop.ACT_Order_Submit Admin.ACT_Reset}; do echo "| $n | ${n%%.*} | ${n#*.} | false |"; done; exit 0 ;; esac cat "$FX/callers.err" >&2 @@ -379,6 +380,43 @@ printf '# Project\n\nUnit tests default: no\n## Test strategy\n| Moment | Unit t fl 'Unit flows:\n- SUB_Order_Total: money calculation' run L10q "$TOTAL" >/dev/null 2>&1 has L10q "Total.test.mdl" && ok "L10q row none + a Unit flows list: the listed flow's test runs (like L5c)" || bad "L10q" "$(cat "$WORK/out.L10q")" +# L10r..x: review fixes (module scope, parser forms, default yes, owed tests, casing, missing module) +am() { mkdir -p "$BRIEF_DIR/Admin"; printf '# Brief\n\n%b\n' "$1" > "$BRIEF_DIR/Admin/module-brief.md"; } +fl 'Unit flows:\n- SUB_Order_Total: money calculation'; am 'Unit flows:\n- ACT_Reset: reset logic' +run L10r --moment module --module Shop >/dev/null 2>&1 +has L10r "Total.test.mdl" && lacks L10r "Other.test.mdl" && ok "L10r --moment module --module Shop: Shop's listed flows only, not Admin's" || bad "L10r" "$(cat "$WORK/out.L10r")" +run L10r2 --moment module --module Shop --module Admin >/dev/null 2>&1 +has L10r2 "Total.test.mdl" && has L10r2 "Other.test.mdl" && ok "L10r2 both modules named: both lists run" || bad "L10r2" "$(cat "$WORK/out.L10r2")" +rm -rf "$BRIEF_DIR/Admin" +fl 'Unit flows:\n- SUB_Order_Total: money calculation\n\n- ACT_Order_Submit: REST call\n\n\n- VAL_Order: rule' +run L10s --moment module --module Shop >/dev/null 2>&1 +has L10s "Total.test.mdl" && has L10s "Submit.test.mdl" && has L10s "Validate.test.mdl" && ok "L10s blank lines between bullets do not end the list" || bad "L10s" "$(cat "$WORK/out.L10s")" +fl 'Unit flows:\n- SUB_Order_Total (money): calculation\n- Shop.ACT_Order_Submit: REST call\n\n## Next heading\n- VAL_Order: not part of the list' +run L10t --moment module --module Shop >/dev/null 2>&1 +has L10t "Total.test.mdl" && has L10t "Submit.test.mdl" && lacks L10t "Validate.test.mdl" && [ ! -s "$WORK/err.L10t" ] && ok "L10t 'ACT (note): reason' and 'Module.ACT: reason' are read; a heading ends the list" || bad "L10t" "$(cat "$WORK/out.L10t" "$WORK/err.L10t")" +fl 'Unit flows: SUB_Order_Total, ACT_Order_Submit' +run L10u --moment module --module Shop >/dev/null 2>&1 +has L10u "Total.test.mdl" && has L10u "Submit.test.mdl" && grep -q "Shop.SUB_Order_Total without a reason" "$WORK/err.L10u" && grep -q "Shop.ACT_Order_Submit without a reason" "$WORK/err.L10u" && ok "L10u inline 'Unit flows: A, B' is read; items without a reason warn" || bad "L10u" "$(cat "$WORK/out.L10u" "$WORK/err.L10u")" +fl 'Unit flows:\n- SUB_Order_Total: ok\n- some words, not a flow\n- ACT_Order_Submit ?? why' +run L10v --moment module --module Shop >/dev/null 2>&1 +grep -q 'cannot read "some words, not a flow"' "$WORK/err.L10v" && grep -q 'cannot read "ACT_Order_Submit ?? why"' "$WORK/err.L10v" && has L10v "Total.test.mdl" && ok "L10v an unreadable line warns with its text and does not drop the rest" || bad "L10v" "$(cat "$WORK/out.L10v" "$WORK/err.L10v")" +printf '# Project\n\nUnit tests default: yes\n' > "$P/PROJECT.md" +fl 'Unit flows:\n- SUB_Order_Total: money calculation'; am 'Unit tests: yes' +run L10w --moment module --module Shop --module Admin >/dev/null 2>&1 +has L10w "Total.test.mdl" && lacks L10w "Validate.test.mdl" && lacks L10w "Submit.test.mdl" && has L10w "Other.test.mdl" && ok "L10w default yes: Shop narrowed to its list, Admin (no list) stays on" || bad "L10w" "$(cat "$WORK/out.L10w")" +run L10w2 "$TOTAL" >/dev/null 2>&1 +has L10w2 "Total.test.mdl" && lacks L10w2 "Validate.test.mdl" && ok "L10w2 default yes, change moment: the unlisted touched caller's test is skipped" || bad "L10w2" "$(cat "$WORK/out.L10w2")" +rm -rf "$BRIEF_DIR/Admin" +printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md" +fl 'Unit flows:\n- SUB_Order_Total: money\n- NoTestYet: owed' +run L10x --moment module --module Shop >/dev/null 2>&1 +grep -q "Shop.NoTestYet, which no .test.mdl names" "$WORK/err.L10x" && ! grep -q "SUB_Order_Total, which no" "$WORK/err.L10x" && has L10x "Total.test.mdl" && ok "L10x a listed flow with no test warns (owed), a tested one does not" || bad "L10x" "$(cat "$WORK/err.L10x")" +fl 'Unit flows:\n- sub_order_total: money' +run L10y --moment module --module Shop >/dev/null 2>&1 +has L10y "Total.test.mdl" && grep -q "model has 'Shop.SUB_Order_Total' (casing differs)" "$WORK/err.L10y" && ok "L10y list casing differs: test still found (case-insensitive), casing warned" || bad "L10y" "$(cat "$WORK/out.L10y" "$WORK/err.L10y")" +fl 'Unit flows:\n- SUB_Order_Total: money\n- Nope.ACT_X: invented' +STUB_BADMOD=Nope run L10z --moment module --module Shop >/dev/null 2>&1 +grep -q "module Nope not found in the model" "$WORK/err.L10z" && ! grep -q "Unit modules names" "$WORK/err.L10z" && has L10z "Total.test.mdl" && ok "L10z a listed module the model lacks warns 'not found in the model', non-fatal" || bad "L10z" "$(cat "$WORK/out.L10z" "$WORK/err.L10z")" rm -rf "$P/architecture" printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md" [ ! -e "$P/.mxcli" ] && ok "wrote nothing into the project" || bad "project written" From 153d9781b76641ecc7cfd4b7a1d661f2159317da Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 20:27:34 +0800 Subject: [PATCH 3/4] fix(testing): per-flow unit tests r2 (list forms, mixed-module tests, CRLF) Numbered, + bullets and a '## Unit flows' heading are read; inline 'A: x, B: y' yields both; a Unit flows block with no flow warns; default yes drops a test only when every module it names has a list excluding it; CR stripped; CHANGELOG baseline 75,194. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 +- project-bin/affected-tests.sh | 31 ++++++++++++++++++++---------- tests/wave2/test-affected-tests.sh | 22 +++++++++++++++++++++ 3 files changed, 44 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e47a618d..81c0a22a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased -- process(testing): **unit tests are chosen per flow, not per module: a module brief lists `Unit flows:` (one `- ACT_X: reason` line each) and `affected-tests.sh` runs only the listed flows the change touched.** Follow-up to #417 on the maintainer's policy that unit-testing everything is overkill (logic and integrations only, UI journeys catch the rest). Module done runs the `--module` modules' listed flows only; a list wins for its module even under `Unit tests default: yes`; bullets may be separated by blank lines, carry `(note)` or `Module.` forms, or sit inline (`Unit flows: A, B`), and a line that cannot be read warns with its text; `Unit tests: yes` or `Unit modules:` with no list keeps the module-level selection; a flow listed without a reason, with no test, with a different casing than the model, or not in the model (`SHOW MICROFLOWS IN`), and a module the model lacks, warn. Test: wave2 `test-affected-tests.sh` L10 (26 checks). Baseline unchanged at 75,197 words (budget 80,000) — MendixMau +- process(testing): **unit tests are chosen per flow, not per module: a module brief lists `Unit flows:` (one `- ACT_X: reason` line each) and `affected-tests.sh` runs only the listed flows the change touched.** Follow-up to #417 on the maintainer's policy that unit-testing everything is overkill (logic and integrations only, UI journeys catch the rest). Module done runs the `--module` modules' listed flows only; a list wins for its module even under `Unit tests default: yes`; bullets (`-`, `*`, `+`, `1.`, `1)`) may be separated by blank lines, carry `(note)` or `Module.` forms, or sit inline (`Unit flows: A: money, B: rest`), a `## Unit flows` heading works with or without a colon, a line that cannot be read warns with its text, and a Unit flows block that yields no flow warns; under default yes a test is dropped only when every module it names has a list that excludes it; CRLF briefs are read; `Unit tests: yes` or `Unit modules:` with no list keeps the module-level selection; a flow listed without a reason, with no test, with a different casing than the model, or not in the model (`SHOW MICROFLOWS IN`), and a module the model lacks, warn. Test: wave2 `test-affected-tests.sh` L10 (35 checks). Baseline unchanged at 75,194 words (budget 80,000) — MendixMau - fix(wire-agents): **a sync re-wire no longer wipes `.claude/settings.json`.** `mxcli init` overwrites that file; wire-agents now snapshots it and merges after init: the project's entries win and keep their order (permission profile, user entries, the `mxcli exec` deny rule, hooks), and entries that only mxcli's new file has are appended. Merged rather than added to PRESERVE because PRESERVE restores old bytes and would also drop anything a newer mxcli adds; an unparsable snapshot is restored byte for byte. Test: `tests/wave2/test-wire-agents-settings.sh` (4 of 6 checks fail on master). — review of #414 (#419) - fix(sync): **the exec-approval item-3 rewrite now works on BSD awk (stock macOS) and `sync-project.sh` no longer prints "Rewrote" for a rewrite that failed.** `_ea_rewrite_item3` passed the multi-line replacement through `awk -v`, which BSD awk rejects ("newline in string", exit 2); the file stayed unchanged and sync still reported the rewrite on every run. The replacement is now streamed from a temp file (as the Q9 repair does), the helper returns 2 when awk fails, and the caller prints a `WARN:` instead of "Rewrote" and does not count a change. `test-exec-approval.sh` gains 3 assertions (failing-awk shim: no "Rewrote", a WARN, file unchanged). No other multi-line `awk -v` call site exists (the rest are single-line values) ([#426](https://github.com/MendixMau/mxcli-project-toolkit/issues/426)) — MendixMau - process(testing): **unit tests are now off by default and `affected-tests.sh` enforces it; each project picks its test strategy with the developer at CAC-5.** Fast-tracked by the maintainer on 2026-10-16 because users report tests firing on every change (the baseline-text A/B is waived; E3 follows as a check). Moments: every change = gate plus what it touched (unit only for affected modules with unit on); module done = that module's journeys (+ its unit tests if on); build complete and before every later deploy = everything once; "run everything" any time. The CAC-5 Offer records it in PROJECT.md `## Test strategy` (Status `ASSUMED` in auto mode). `affected-tests.sh` reads `Unit tests default:` (`no` or absent means unit off, one `unit off` line, never a `.test.mdl` run row); turn unit on per project with `Unit tests default: yes`, per module with `Unit modules: Shop, Admin` in PROJECT.md or `Unit tests: yes` in the module brief; the strategy row's Unit cell is a scope only and never turns unit on. A grant on a microflow or page the same script changes is part of that subject and no longer selects the owning module's tests. `module-review.md` PROVE now runs `--moment module --module `. `testing-shape.md`, `checkpoint-build.md`, `iterative-build-loop.md`, `conversion-runbook.md` and the exec.sh logic lane text updated. `tests/wave2/test-test-strategy-docs.sh` and the extended `test-affected-tests.sh` pin it. Baseline words stay at or below master (75,219 vs 75,225). Split out of #409; found by the field test of the release candidate — the maintainer's decision, 2026-10-16 diff --git a/project-bin/affected-tests.sh b/project-bin/affected-tests.sh index e7e8e8b6..e72fa797 100755 --- a/project-bin/affected-tests.sh +++ b/project-bin/affected-tests.sh @@ -69,7 +69,7 @@ # `SHOW MICROFLOWS IN ` does not list, a different casing, or a module the model lacks warns # (best effort: skipped under --no-model or when mxcli cannot run). `Unit tests: yes` or # `Unit modules:` with NO list keeps the module-level selection. Under `Unit tests default: yes` -# a module with a list is still narrowed to it (the list wins for its module); the others stay on. +# a module with a list is narrowed to it, but a test is dropped only when EVERY module it names has a list that excludes it; a test naming any module without a list stays (that module is on). # Journeys are not affected by any of this. # # HOW THE SELECTION IS MADE (judgement-free facts only; skills-over-scripts.md): @@ -176,14 +176,16 @@ for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md if (k ~ /^[ \t]*unit[ \t]*\(/ && v ~ /^yes/) f = 1 } END { exit !f }' "$_b"; then FLIPPED="$FLIPPED $_m"; fi # Unit flows: a bullet list `- [Module.]Flow[ (note)][: reason]` under a `Unit flows:` line, or inline - # `Unit flows: A, B`. Blank lines between bullets are fine; a heading or any other line ends the block. + # `Unit flows: A, B` (comma-split, each may carry `: reason`), `-`/`*`/`+`/numbered bullets, or a `## Unit flows` + # heading. Blank lines between bullets are fine; a heading or any other line ends the block. # A line in the block that cannot be read warns with its text, never dropped silently. awk -v m="$_m" -v b="$_b" ' function trim(t) { gsub(/^[ \t]+|[ \t]+$/, "", t); return t } function warn(t) { print "warning: Unit flows in " b ": cannot read \"" t "\"; write `- ACT_Name: reason`" > "/dev/stderr" } function item(l, nm, rest, par, r, i) { gsub(/`|\*\*/, "", l); l = trim(l) - if (l == "" || substr(l, 1, 1) == "<") return + if (l == "") return + if (substr(l, 1, 1) == "<") { seen++; return } if (!match(l, /^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)?/)) { warn(l); return } nm = substr(l, 1, RLENGTH); rest = trim(substr(l, RLENGTH + 1)); par = "" if (substr(rest, 1, 1) == "(") { i = index(rest, ")"); if (i) { par = substr(rest, 2, i - 2); rest = trim(substr(rest, i + 1)) } } @@ -191,14 +193,17 @@ for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md else if (substr(rest, 1, 1) == ":") { r = trim(substr(rest, 2)); if (r == "") r = par } else if (rest ~ /^-[ \t]/) r = trim(substr(rest, 2)) else { warn(l); return } - print (index(nm, ".") ? nm : m "." nm) "\t" r + seen++; print (index(nm, ".") ? nm : m "." nm) "\t" r } - tolower($0) ~ /^[ \t*>-]*unit flows:/ { on = 1; v = substr($0, index(tolower($0), "unit flows:") + 11); gsub(/`|\*\*/, "", v); v = trim(v) - if (v != "") { if (index(v, ":")) item(v); else { k = split(v, a, ","); for (j = 1; j <= k; j++) item(a[j]) } } + { sub(/\r$/, "") } + tolower($0) ~ /^[ \t*>-]*unit flows:/ { on = 1; hdr = 1; v = substr($0, index(tolower($0), "unit flows:") + 11); gsub(/`|\*\*/, "", v); v = trim(v) + if (v != "") { k = split(v, a, ","); for (j = 1; j <= k; j++) item(a[j]) } next } + tolower($0) ~ /^[ \t]*(#+[ \t]+)?[*]*unit flows[ \t*:]*$/ { on = 1; hdr = 1; next } on && /^[ \t]*$/ { next } - on && /^[ \t]*[-*][ \t]/ { l = $0; sub(/^[ \t]*[-*][ \t]+/, "", l); item(l); next } - on { on = 0 }' "$_b" >> "$WORK/unitflows" + on && /^[ \t]*([-*+]|[0-9]+[.)])[ \t]/ { l = $0; sub(/^[ \t]*([-*+]|[0-9]+[.)])[ \t]+/, "", l); item(l); next } + on { on = 0 } + END { if (hdr && !seen) print "warning: Unit flows in " b " looks like a Unit flows block but lists no flow; write `- ACT_Name: reason` lines under `Unit flows:`" > "/dev/stderr" }' "$_b" >> "$WORK/unitflows" done # a module with a Unit flows list is flipped by it for _lm in $(cut -f1 "$WORK/unitflows" | sed 's/\..*//' | sort -u); do FLIPPED="$FLIPPED $_lm"; done @@ -414,8 +419,14 @@ if [ "$UNIT_GATED" = 1 ]; then # default no, some modules flipped: only those elif [ -n "$LISTMODS" ]; then # default yes: a module with a list is narrowed to it, the rest stay on : > "$WORK/unit.ls"; for m in $LISTMODS; do list_sel "$m" >> "$WORK/unit.ls"; done while IFS=' ' read -r f w; do - _gov=0; for m in $LISTMODS; do grep -qE "(^|[^A-Za-z0-9_])$m\." "$f" 2>/dev/null && _gov=1; done - [ "$_gov" = 0 ] && printf '%s\t%s\n' "$f" "$w" + # governed by a list only when EVERY module the test names is a listed module; any other module is on, keep the test + _gov="$(awk -v lm="$LISTMODS" 'BEGIN { n = split(tolower(lm), L, " "); for (i = 1; i <= n; i++) isl[L[i]] = 1 } + { s = $0; while (match(s, /(^|[^A-Za-z0-9_.$])[A-Za-z_][A-Za-z0-9_]*\.[A-Za-z_]/)) { + q = substr(s, RSTART, RLENGTH); sub(/^[^A-Za-z_]/, "", q); sub(/\..*/, "", q); q = tolower(q) + if (q in isl) hl = 1; else if (q != "system") ho = 1 + s = substr(s, RSTART + RLENGTH) } } + END { print (hl && !ho) ? 1 : 0 }' "$f" 2>/dev/null)" + [ "$_gov" = 1 ] || printf '%s\t%s\n' "$f" "$w" done < "$WORK/unit.sel" > "$WORK/unit.new" cat "$WORK/unit.ls" >> "$WORK/unit.new" awk -F'\t' '!seen[$1]++' "$WORK/unit.new" > "$WORK/unit.sel" diff --git a/tests/wave2/test-affected-tests.sh b/tests/wave2/test-affected-tests.sh index b6aa090b..0db56f45 100644 --- a/tests/wave2/test-affected-tests.sh +++ b/tests/wave2/test-affected-tests.sh @@ -417,6 +417,28 @@ has L10y "Total.test.mdl" && grep -q "model has 'Shop.SUB_Order_Total' (casing d fl 'Unit flows:\n- SUB_Order_Total: money\n- Nope.ACT_X: invented' STUB_BADMOD=Nope run L10z --moment module --module Shop >/dev/null 2>&1 grep -q "module Nope not found in the model" "$WORK/err.L10z" && ! grep -q "Unit modules names" "$WORK/err.L10z" && has L10z "Total.test.mdl" && ok "L10z a listed module the model lacks warns 'not found in the model', non-fatal" || bad "L10z" "$(cat "$WORK/out.L10z" "$WORK/err.L10z")" +# L10aa..: review r2 (parser forms, mixed-module tests, CRLF) +printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md" +for _f in 'Unit flows:\n1. SUB_Order_Total: money\n2) ACT_Order_Submit: REST' 'Unit flows:\n+ SUB_Order_Total: money\n+ ACT_Order_Submit: REST' '## Unit flows\n\n- SUB_Order_Total: money\n- ACT_Order_Submit: REST' '### Unit flows:\n- SUB_Order_Total: money\n- ACT_Order_Submit: REST' 'Unit flows: SUB_Order_Total: money, ACT_Order_Submit: REST'; do + fl "$_f"; run L10aa --moment module --module Shop >/dev/null 2>&1 + has L10aa "Total.test.mdl" && has L10aa "Submit.test.mdl" && lacks L10aa "Validate.test.mdl" && [ ! -s "$WORK/err.L10aa" ] && ok "L10aa list form read, both flows: $(printf '%s' "$_f" | head -c 28)" || bad "L10aa $_f" "$(cat "$WORK/out.L10aa" "$WORK/err.L10aa")" +done +fl '## Unit flows\n\nTBD, to be decided\n\n## Other' +run L10ab --moment module --module Shop >/dev/null 2>&1 +grep -q "looks like a Unit flows block but lists no flow" "$WORK/err.L10ab" && ok "L10ab a Unit flows block that yields no flow warns" || bad "L10ab" "$(cat "$WORK/err.L10ab")" +printf '# Brief\r\n\r\nUnit flows:\r\n- SUB_Order_Total: money\r\n\r\n- ACT_Order_Submit: REST\r\n' > "$BRIEF_DIR/Shop/module-brief.md" +run L10ac --moment module --module Shop >/dev/null 2>&1 +has L10ac "Total.test.mdl" && has L10ac "Submit.test.mdl" && [ ! -s "$WORK/err.L10ac" ] && ok "L10ac CRLF brief: flows read, no 'cannot read' warning" || bad "L10ac" "$(cat "$WORK/out.L10ac" "$WORK/err.L10ac")" +# mixed-module test: Admin has no list, so it is on +printf '/**\n * @test Mixed\n */\ncall microflow Admin.ACT_Reset();\ncall microflow Shop.VAL_Order();\n/\n' > "$P/tests/unit/Mixed.test.mdl" +fl 'Unit flows:\n- SUB_Order_Total: money' +printf '# Project\n\nUnit tests default: yes\n' > "$P/PROJECT.md" +run L10ad "$TOTAL" >/dev/null 2>&1 +has L10ad "Mixed.test.mdl" && lacks L10ad "Validate.test.mdl" && has L10ad "Total.test.mdl" && ok "L10ad default yes: a test naming a listed AND an unlisted module is kept; a Shop-only unlisted test is dropped" || bad "L10ad" "$(cat "$WORK/out.L10ad")" +printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md"; am 'Unit tests: yes' +run L10ae "$TOTAL" >/dev/null 2>&1 +has L10ae "Mixed.test.mdl" && lacks L10ae "Validate.test.mdl" && ok "L10ae default no, Admin flipped: the same mixed test is selected via Admin" || bad "L10ae" "$(cat "$WORK/out.L10ae")" +rm -f "$P/tests/unit/Mixed.test.mdl"; rm -rf "$BRIEF_DIR/Admin" rm -rf "$P/architecture" printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md" [ ! -e "$P/.mxcli" ] && ok "wrote nothing into the project" || bad "project written" From 81ca391919d5ddade444afd6f2374d2616ebb244 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 20:39:00 +0800 Subject: [PATCH 4/4] fix(testing): Unit flows block parsed by rule (continuations, (note) headers, field-ended block) Co-Authored-By: Claude Opus 5.5 --- project-bin/affected-tests.sh | 23 ++++++++++++++++------- tests/wave2/test-affected-tests.sh | 15 +++++++++++++++ 2 files changed, 31 insertions(+), 7 deletions(-) diff --git a/project-bin/affected-tests.sh b/project-bin/affected-tests.sh index e72fa797..5be5c851 100755 --- a/project-bin/affected-tests.sh +++ b/project-bin/affected-tests.sh @@ -177,7 +177,8 @@ for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md END { exit !f }' "$_b"; then FLIPPED="$FLIPPED $_m"; fi # Unit flows: a bullet list `- [Module.]Flow[ (note)][: reason]` under a `Unit flows:` line, or inline # `Unit flows: A, B` (comma-split, each may carry `: reason`), `-`/`*`/`+`/numbered bullets, or a `## Unit flows` - # heading. Blank lines between bullets are fine; a heading or any other line ends the block. + # heading, optionally `(note)`. Blank lines are skipped; an indented non-bullet line continues the previous + # reason; the block ends at the next heading or an unindented `Word words:` field; any other line warns. # A line in the block that cannot be read warns with its text, never dropped silently. awk -v m="$_m" -v b="$_b" ' function trim(t) { gsub(/^[ \t]+|[ \t]+$/, "", t); return t } @@ -195,15 +196,23 @@ for _b in architecture/modules/*/module-brief.md architecture/modules/*-brief.md else { warn(l); return } seen++; print (index(nm, ".") ? nm : m "." nm) "\t" r } + function flush() { if (pend != "") item(pend); pend = "" } { sub(/\r$/, "") } - tolower($0) ~ /^[ \t*>-]*unit flows:/ { on = 1; hdr = 1; v = substr($0, index(tolower($0), "unit flows:") + 11); gsub(/`|\*\*/, "", v); v = trim(v) + # header: `Unit flows` (any heading level or plain), optional `(note)`, optional `:` and inline items + tolower($0) ~ /^[ \t*>#-]*unit flows[ \t*]*(\([^)]*\))?[ \t*]*(:.*)?$/ { + flush(); on = 1; hdr = 1; v = $0; sub(/^[^:]*(\([^)]*\))?[^:]*:?/, "", v) + if (index($0, ":") == 0 || $0 ~ /^[^:]*\([^)]*:/) v = "" + gsub(/`|\*\*/, "", v); v = trim(v) if (v != "") { k = split(v, a, ","); for (j = 1; j <= k; j++) item(a[j]) } next } - tolower($0) ~ /^[ \t]*(#+[ \t]+)?[*]*unit flows[ \t*:]*$/ { on = 1; hdr = 1; next } - on && /^[ \t]*$/ { next } - on && /^[ \t]*([-*+]|[0-9]+[.)])[ \t]/ { l = $0; sub(/^[ \t]*([-*+]|[0-9]+[.)])[ \t]+/, "", l); item(l); next } - on { on = 0 } - END { if (hdr && !seen) print "warning: Unit flows in " b " looks like a Unit flows block but lists no flow; write `- ACT_Name: reason` lines under `Unit flows:`" > "/dev/stderr" }' "$_b" >> "$WORK/unitflows" + !on { next } + /^[ \t]*$/ { next } + /^[ \t]*([-*+]|[0-9]+[.)])[ \t]/ { flush(); l = $0; sub(/^[ \t]*([-*+]|[0-9]+[.)])[ \t]+/, "", l); pend = l; next } + /^[ \t]+[^ \t]/ && !/^[ \t]*#/ { if (pend != "") pend = pend " " trim($0); next } + /^[ \t]*#/ { flush(); on = 0; next } + /^[A-Za-z][A-Za-z]*([ \t]+[A-Za-z]+)*[ \t]*:/ { flush(); on = 0; next } + { flush(); warn(trim($0)); next } + END { flush(); if (hdr && !seen) print "warning: Unit flows in " b " looks like a Unit flows block but lists no flow; write `- ACT_Name: reason` lines under `Unit flows:`" > "/dev/stderr" }' "$_b" >> "$WORK/unitflows" done # a module with a Unit flows list is flipped by it for _lm in $(cut -f1 "$WORK/unitflows" | sed 's/\..*//' | sort -u); do FLIPPED="$FLIPPED $_lm"; done diff --git a/tests/wave2/test-affected-tests.sh b/tests/wave2/test-affected-tests.sh index 0db56f45..c5ce5377 100644 --- a/tests/wave2/test-affected-tests.sh +++ b/tests/wave2/test-affected-tests.sh @@ -429,6 +429,21 @@ grep -q "looks like a Unit flows block but lists no flow" "$WORK/err.L10ab" && o printf '# Brief\r\n\r\nUnit flows:\r\n- SUB_Order_Total: money\r\n\r\n- ACT_Order_Submit: REST\r\n' > "$BRIEF_DIR/Shop/module-brief.md" run L10ac --moment module --module Shop >/dev/null 2>&1 has L10ac "Total.test.mdl" && has L10ac "Submit.test.mdl" && [ ! -s "$WORK/err.L10ac" ] && ok "L10ac CRLF brief: flows read, no 'cannot read' warning" || bad "L10ac" "$(cat "$WORK/out.L10ac" "$WORK/err.L10ac")" +# L10af..: review r3 (rule-based block parser) +printf '# Project\n\nUnit tests default: no\n' > "$P/PROJECT.md" +fl 'Unit flows:\n- SUB_Order_Total: money, rounded\n to cents per line\n- ACT_Order_Submit: REST' +run L10af --moment module --module Shop >/dev/null 2>&1 +has L10af "Total.test.mdl" && has L10af "Submit.test.mdl" && [ ! -s "$WORK/err.L10af" ] && ok "L10af wrapped reason line then another bullet: both flows kept" || bad "L10af" "$(cat "$WORK/out.L10af" "$WORK/err.L10af")" +for _f in '## Unit flows (logic only)\n- SUB_Order_Total: money\n- ACT_Order_Submit: REST' 'Unit flows (logic only):\n- SUB_Order_Total: money\n- ACT_Order_Submit: REST' 'Unit flows (logic only): SUB_Order_Total: money, ACT_Order_Submit: REST'; do + fl "$_f"; run L10ag --moment module --module Shop >/dev/null 2>&1 + has L10ag "Total.test.mdl" && has L10ag "Submit.test.mdl" && [ ! -s "$WORK/err.L10ag" ] && ok "L10ag header with (note) read: $(printf '%s' "$_f" | head -c 30)" || bad "L10ag $_f" "$(cat "$WORK/out.L10ag" "$WORK/err.L10ag")" +done +fl 'Unit flows:\n- SUB_Order_Total: money\n\nJourneys:\n- ACT_Order_Submit: REST' +run L10ah --moment module --module Shop >/dev/null 2>&1 +has L10ah "Total.test.mdl" && lacks L10ah "Submit.test.mdl" && [ ! -s "$WORK/err.L10ah" ] && ok "L10ah a following 'Journeys:' field ends the block, silently" || bad "L10ah" "$(cat "$WORK/out.L10ah" "$WORK/err.L10ah")" +fl 'Unit flows:\n- SUB_Order_Total: money\nstray prose here\n- ACT_Order_Submit: REST' +run L10ai --moment module --module Shop >/dev/null 2>&1 +has L10ai "Submit.test.mdl" && grep -q 'cannot read "stray prose here"' "$WORK/err.L10ai" && ok "L10ai other unindented line warns with its text, block continues" || bad "L10ai" "$(cat "$WORK/out.L10ai" "$WORK/err.L10ai")" # mixed-module test: Admin has no list, so it is on printf '/**\n * @test Mixed\n */\ncall microflow Admin.ACT_Reset();\ncall microflow Shop.VAL_Order();\n/\n' > "$P/tests/unit/Mixed.test.mdl" fl 'Unit flows:\n- SUB_Order_Total: money'