From 88f2410859f1964ffdb565c4fb9a052ad407affa Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 18:34:03 +0800 Subject: [PATCH] learn(unit-test): write and run a microflow unit test that can fail New on-demand skill that gathers the how of .test.mdl tests from testing-shape.md, rest-between-mendix-apps.md and the bug log into one file a user can ask for outside the pipeline: what is worth a test, equality-only @expect (BUG-LOCAL-30), no file-level header (BUG-LOCAL-29), --list before a run, test-scratch.sh so the live app survives (#278), BUG-157, a positive control, a four-line report. Whether and when unit tests run stays with the project's test strategy. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + README.md | 1 + ROUTING.md | 1 + agents/mdl-agent.md | 1 + agents/review-agent.md | 1 + agents/test-agent.md | 1 + bin/gate-check.sh | 4 +- bin/lib/skill-routing.tsv | 1 + skills/testing-shape.md | 1 + skills/unit-test.md | 106 ++++++++++++++++++++++++++++++++++++++ 10 files changed, 116 insertions(+), 2 deletions(-) create mode 100644 skills/unit-test.md diff --git a/CHANGELOG.md b/CHANGELOG.md index d76e227f..c57ffa33 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- learn(unit-test): **new on-demand skill for writing and running a microflow unit test (`.test.mdl`, `mxcli test`) whose assertion is live: what is worth a test and what is not, equality-only `@expect` (BUG-LOCAL-30), no file-level header (BUG-LOCAL-29), `--list` count against `/` separators before any run, `test-scratch.sh` so the live app survives (#278), the BUG-157 after-startup trap, a positive control that must fail, and a four-line report.** It gathers the how that was spread over `testing-shape.md`, `rest-between-mendix-apps.md` and the bug log into one file a user can ask for outside the pipeline; whether and when unit tests run stays with the project's test strategy. Routed ondemand for lead, mdl, test and review; one row in `testing-shape.md`'s owner table. Field evidence: projects on mxcli v0.19 to v0.27, Mendix 11.13 to 11.15, 2026-08 to 2026-10 — MendixMau - learn(skills): **five lab findings land as compact rows, each paid for by trimming prose in the same files, so always-loaded words do not grow (baseline 75,225 to 75,204, budget 80,000).** Enum value `New` and `AutoCreatedDate`/`AutoOwner` (detection gaps), `[%CurrentDateTime%]` not `currentDateTime()`, expression return types (Decimal from date diffs and `div`, absolute `secondsBetween`, `formatDecimal` on pages), CE2729 from the default home page and page microflow grants, Atlas selector carriers for list views, page body and timelines, `-small` spacing suffixes do not exist, `display: contents` rows paint nothing, CSS pseudo text is invisible to `innerText` — lab field runs, Refs #276 (TK-020, TK-021, TK-022, TK-024) - fix(design-check): **a design system whose palette lives only in scoped `:root.` blocks no longer reads as "0 of N tokens carry the design's value".** Pass 2b compared every token against a bare `:root` of the built sheet, so a themed-only palette always warned, a false warning that teaches agents to ignore the check. A token the design declares only in scoped blocks is now compared block for block against the same-named block in the built sheet, and a wrong value is still named with its block (knob binding stays bare-only: a knob bound inside a theme the app is not wearing is not bound). Synthetic fixtures `ds-themed.css`, `built-themed.css`, `built-themed-wrong.css`; 5 new cases in `test-design-reaches-app.sh` (50 total) — TK-015 part (b) - fix(tests): **`test-exec-hash-verify.sh` case K passes on macOS.** `$TMPDIR` ends in `/`, so the bare `mktemp` path held `//` and the exact `MXPATCH_BACKUP_DIR` path assert could not match; `WORK` is now `cd`-ed into and read back with `pwd -P`. Test only, no script change. 42 pass / 1 fail before, 43 / 0 after — #384 review diff --git a/README.md b/README.md index 05c99b68..697cc60e 100644 --- a/README.md +++ b/README.md @@ -670,6 +670,7 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini | Proving a module's user journey end-to-end — the deep form of step 3 PROVE; use whenever an instrument reports green and you cannot say what would have made it red | `skills/journey-proof.md` | | Writing or running e2e cases beyond the happy journeys (blank/whitespace required fields, over-length, duplicates, cancel, double submit, wrong state, wrong role, backend down, stale write, empty search) — every case cites a requirement first, UNSPECIFIED is re-scored against house rules, GAP counts carry the denominator; runs after journey-proof and before monkey-test | `skills/unhappy-path-testing.md` | | Running the fuzz/crash net on a module whose journeys are already green — and reading the result, which is NOT evidence the module works | `skills/monkey-test.md` | +| Writing or running a microflow unit test (.test.mdl, mxcli test, @expect) once the project or the user has decided a flow gets one — what is worth a test, equality-only asserts, --list before any run, a scratch copy so the live app survives, and a positive control that must fail | `skills/unit-test.md` | | UX audit and screenshot-loop discipline | `skills/learned-skill-ux-audit.md` | | Tracking scope delta between the BRD and the built state | `skills/learned-skill-scope-delta.md` | | Writing or reading docs/report.json — the append-only contract every instrument writes to and every renderer reads; open BEFORE building a new instrument or a second renderer | `skills/report-schema.md` | diff --git a/ROUTING.md b/ROUTING.md index fbe25354..fee9ce0d 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -222,6 +222,7 @@ picks the row up. That is the whole procedure — there is no second list to rem | Proving a module's user journey end-to-end — the deep form of step 3 PROVE; use whenever an instrument reports green and you cannot say what would have made it red | `skills/journey-proof.md` | test,review | 5,6 | ondemand | | Writing or running e2e cases beyond the happy journeys (blank/whitespace required fields, over-length, duplicates, cancel, double submit, wrong state, wrong role, backend down, stale write, empty search) — every case cites a requirement first, UNSPECIFIED is re-scored against house rules, GAP counts carry the denominator; runs after journey-proof and before monkey-test | `skills/unhappy-path-testing.md` | test,review | 5,6 | ondemand | | Running the fuzz/crash net on a module whose journeys are already green — and reading the result, which is NOT evidence the module works | `skills/monkey-test.md` | test,review | 5,6 | ondemand | +| Writing or running a microflow unit test (.test.mdl, mxcli test, @expect) once the project or the user has decided a flow gets one — what is worth a test, equality-only asserts, --list before any run, a scratch copy so the live app survives, and a positive control that must fail | `skills/unit-test.md` | lead,mdl,test,review | 5,6 | ondemand | | UX audit and screenshot-loop discipline | `skills/learned-skill-ux-audit.md` | review | 6 | ondemand | | Tracking scope delta between the BRD and the built state | `skills/learned-skill-scope-delta.md` | review | 6 | ondemand | | Writing or reading docs/report.json — the append-only contract every instrument writes to and every renderer reads; open BEFORE building a new instrument or a second renderer | `skills/report-schema.md` | test,review,gate | 5,6 | ondemand | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index cb1bb928..da8e2530 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -80,6 +80,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/mendix-agents.md` | Building a Mendix AI agent | | `skills/mendix-agent-ui.md` | Embedding a copilot chat panel | | `skills/mendix-agent-setup.md` | Standing up a project's GenAI agents in any environment | +| `skills/unit-test.md` | Writing or running a microflow unit test (.test.mdl, mxcli test, @expect) once the project or the user has decided a flow gets one | | `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN | | `skills/close-the-loop.md` | Cutover and retrospective | | `skills/measured-claims.md` | Before citing ANY behavioural claim about the harness, the Mendix runtime or a test tool as evidence, or counting tokens from transcripts | diff --git a/agents/review-agent.md b/agents/review-agent.md index f8590697..8f884a96 100644 --- a/agents/review-agent.md +++ b/agents/review-agent.md @@ -70,6 +70,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/journey-proof.md` | Proving a module's user journey end-to-end | | `skills/unhappy-path-testing.md` | Writing or running e2e cases beyond the happy journeys (blank/whitespace required fields, over-length, duplicates, cancel, double submit, wrong state, wrong role, backend down, stale write, empty search) | | `skills/monkey-test.md` | Running the fuzz/crash net on a module whose journeys are already green | +| `skills/unit-test.md` | Writing or running a microflow unit test (.test.mdl, mxcli test, @expect) once the project or the user has decided a flow gets one | | `skills/e2e-locators-that-lie.md` | An e2e rung reports a UI defect (widget never appeared, click did nothing, screen contradicts the data, fixture count wrong) | | `skills/learned-skill-ux-audit.md` | UX audit and screenshot-loop discipline | | `skills/learned-skill-scope-delta.md` | Tracking scope delta between the BRD and the built state | diff --git a/agents/test-agent.md b/agents/test-agent.md index f2d30d89..e5ed79a9 100644 --- a/agents/test-agent.md +++ b/agents/test-agent.md @@ -55,6 +55,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/journey-proof.md` | Proving a module's user journey end-to-end | | `skills/unhappy-path-testing.md` | Writing or running e2e cases beyond the happy journeys (blank/whitespace required fields, over-length, duplicates, cancel, double submit, wrong state, wrong role, backend down, stale write, empty search) | | `skills/monkey-test.md` | Running the fuzz/crash net on a module whose journeys are already green | +| `skills/unit-test.md` | Writing or running a microflow unit test (.test.mdl, mxcli test, @expect) once the project or the user has decided a flow gets one | | `skills/e2e-locators-that-lie.md` | An e2e rung reports a UI defect (widget never appeared, click did nothing, screen contradicts the data, fixture count wrong) | | `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN | | `skills/close-the-loop.md` | Cutover and retrospective | diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 4885ae39..311016ba 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -1771,8 +1771,8 @@ stage_protocol_paths() { 2) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-brd.md skills/checkpoints/checkpoint-architecture.md skills/dispatching.md skills/image-transcription.md skills/small-project-tier.md skills/kb-generation.md skills/brd-generation.md skills/brd-validation.md" ;; 3) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-design.md skills/dispatching.md skills/small-project-tier.md skills/mendix-best-practices-index.md skills/decompose-app.md skills/mendix-service-seam.md skills/layering-review.md skills/architecture-blueprint.md skills/modularize-domain.md skills/design-artifacts.md skills/interaction-model.md skills/brd-to-build-plan.md skills/workflow-structure-rules.md skills/rest-between-mendix-apps.md skills/learned-mdl-cannot-express.md" ;; 4) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-build.md skills/agent-roles.md skills/dispatching.md skills/small-project-tier.md skills/module-brief.md skills/module-folder-convention.md skills/brd-to-build-plan.md skills/coverage-ledger.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/rest-between-mendix-apps.md skills/learned-constants-and-secrets.md" ;; - 5|build-ready) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/agent-roles.md skills/dispatching.md skills/module-brief.md skills/learned-mdl-preflight.md skills/module-folder-convention.md skills/learned-microflow-patterns.md skills/microflow-preflight.md skills/mendix-best-practices-index.md skills/ui-preflight-pages.md skills/design-spacing.md skills/learned-stylegallery.md skills/ui-loop.md skills/module-review.md skills/testing-shape.md skills/decompose-app.md skills/mendix-service-seam.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/iterative-build-loop.md skills/mdl-cookbook-microflows.md skills/build/mdl/oneshot-mdl-method.md skills/learned-page-patterns.md skills/oneshot-page-structure-patterns.md skills/mendix-agents.md skills/mendix-agent-ui.md skills/mendix-agent-setup.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/e2e-locators-that-lie.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/learned-workflow-patterns.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/learned-sidebar-collapse-icons.md skills/learned-popup-navigation.md skills/learned-datagrid-customcontent-binding.md skills/learned-popup-feedback-pattern.md skills/learned-mdl-cannot-express.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-dg2-patterns.md skills/learned-file-upload-widget.md skills/security-is-not-a-later-script.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/preview-over-hub-tunnel.md skills/walking-skeleton.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; - 6) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/dispatching.md skills/mendix-best-practices-index.md skills/module-review.md skills/testing-shape.md skills/existing-app-assurance.md skills/decompose-app.md skills/app-analysis.md skills/module-dependency-review.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/qa-loop-goal-pattern.md skills/mendix-agent-setup.md skills/e2e-harness-base.md skills/learned-db-assertions.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/e2e-locators-that-lie.md skills/learned-skill-ux-audit.md skills/learned-skill-scope-delta.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/e2e-evidence-report.md skills/record-demo-video.md skills/demo-recording-polish.md skills/share-demo-package.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/workflow-structure-rules.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/anonymize-client-app-for-demo.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/handoff-to-studio-pro.md skills/preview-over-hub-tunnel.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; + 5|build-ready) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/agent-roles.md skills/dispatching.md skills/module-brief.md skills/learned-mdl-preflight.md skills/module-folder-convention.md skills/learned-microflow-patterns.md skills/microflow-preflight.md skills/mendix-best-practices-index.md skills/ui-preflight-pages.md skills/design-spacing.md skills/learned-stylegallery.md skills/ui-loop.md skills/module-review.md skills/testing-shape.md skills/decompose-app.md skills/mendix-service-seam.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/iterative-build-loop.md skills/mdl-cookbook-microflows.md skills/build/mdl/oneshot-mdl-method.md skills/learned-page-patterns.md skills/oneshot-page-structure-patterns.md skills/mendix-agents.md skills/mendix-agent-ui.md skills/mendix-agent-setup.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/unit-test.md skills/e2e-locators-that-lie.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/learned-workflow-patterns.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/learned-sidebar-collapse-icons.md skills/learned-popup-navigation.md skills/learned-datagrid-customcontent-binding.md skills/learned-popup-feedback-pattern.md skills/learned-mdl-cannot-express.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-dg2-patterns.md skills/learned-file-upload-widget.md skills/security-is-not-a-later-script.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/preview-over-hub-tunnel.md skills/walking-skeleton.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; + 6) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/dispatching.md skills/mendix-best-practices-index.md skills/module-review.md skills/testing-shape.md skills/existing-app-assurance.md skills/decompose-app.md skills/app-analysis.md skills/module-dependency-review.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/qa-loop-goal-pattern.md skills/mendix-agent-setup.md skills/e2e-harness-base.md skills/learned-db-assertions.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/unit-test.md skills/e2e-locators-that-lie.md skills/learned-skill-ux-audit.md skills/learned-skill-scope-delta.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/e2e-evidence-report.md skills/record-demo-video.md skills/demo-recording-polish.md skills/share-demo-package.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/workflow-structure-rules.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/anonymize-client-app-for-demo.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/handoff-to-studio-pro.md skills/preview-over-hub-tunnel.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; 7) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/close-the-loop.md skills/share-demo-package.md skills/handoff-to-studio-pro.md skills/platform-link.md skills/teamserver-alignment.md skills/deploy-to-sandbox.md" ;; *) echo "" ;; # diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index 94ade35c..acc980ff 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -173,6 +173,7 @@ fixture-seeding skills/fixture-seeding.md Seed data, test data, demo data or fix journey-proof skills/journey-proof.md Proving a module's user journey end-to-end — the deep form of step 3 PROVE; use whenever an instrument reports green and you cannot say what would have made it red test,review 5,6 ondemand verify unhappy-path-testing skills/unhappy-path-testing.md Writing or running e2e cases beyond the happy journeys (blank/whitespace required fields, over-length, duplicates, cancel, double submit, wrong state, wrong role, backend down, stale write, empty search) — every case cites a requirement first, UNSPECIFIED is re-scored against house rules, GAP counts carry the denominator; runs after journey-proof and before monkey-test test,review 5,6 ondemand verify monkey-test skills/monkey-test.md Running the fuzz/crash net on a module whose journeys are already green — and reading the result, which is NOT evidence the module works test,review 5,6 ondemand verify +unit-test skills/unit-test.md Writing or running a microflow unit test (.test.mdl, mxcli test, @expect) once the project or the user has decided a flow gets one — what is worth a test, equality-only asserts, --list before any run, a scratch copy so the live app survives, and a positive control that must fail lead,mdl,test,review 5,6 ondemand verify e2e-locators-that-lie skills/e2e-locators-that-lie.md An e2e rung reports a UI defect (widget never appeared, click did nothing, screen contradicts the data, fixture count wrong) — check the locator before blaming the app: :visible, click order, poll, derived fixtures test,review 5,6 ondemand diagnose learned-skill-ux-audit skills/learned-skill-ux-audit.md UX audit and screenshot-loop discipline review 6 ondemand verify learned-skill-scope-delta skills/learned-skill-scope-delta.md Tracking scope delta between the BRD and the built state review 6 ondemand verify diff --git a/skills/testing-shape.md b/skills/testing-shape.md index b9defd0f..b5f8b004 100644 --- a/skills/testing-shape.md +++ b/skills/testing-shape.md @@ -12,6 +12,7 @@ and the guard for each. Job 2 is why this skill exists; if you shorten it, do no | Playwright harness setup, widget discovery, suite structure | `e2e-harness-base.md` (shared toolkit) | | DB/OQL assertion mechanics | `learned-db-assertions.md` (shared toolkit) | | The run → fix → re-run loop shape | `qa-loop-goal-pattern.md` (shared toolkit) | +| Writing and running a unit test: what is worth one, live asserts, a safe run | `unit-test.md` (shared toolkit) | | Unit-test syntax and the runner | `.ai-context/skills/test-microflows.md` — ⚠ mxcli-bundled, **regenerated on upgrade**. Read for mechanics; never edit it, never link doctrine at it. | | Non-pipeline app, à-la-carte assurance | `existing-app-assurance.md` (shared toolkit) | | Why a green UI report can still be lying | **§4a below** — merged in, no longer a separate file | diff --git a/skills/unit-test.md b/skills/unit-test.md new file mode 100644 index 00000000..60436d97 --- /dev/null +++ b/skills/unit-test.md @@ -0,0 +1,106 @@ +# Skill: unit-test — write and run a microflow unit test that can fail + +**Applies to:** any mxcli project with a runnable app, pipeline or not. +**Purpose:** the how of `.test.mdl` tests: what is worth a test, how to write one whose +assertion is live, how to run it without breaking the live app, and how to report it. +**Source:** field projects on mxcli v0.19 to v0.27, Mendix 11.13 to 11.15, 2026-08 to 2026-10; +the false-green rows are `testing-shape.md` §4. + +**Whether** a module gets unit tests, and **when** they run, is the project's test strategy +(`testing-shape.md` §1: project default no, flipped per module). This file does not +decide that. When the user asks for unit tests outside a pipeline, they have decided. + +--- + +## 1. Pick the flow, or say why not + +| Worth a unit test | Not worth one | +|---|---| +| calculation, money, dates, rounding | a page or layout change: that owes a journey and a look | +| mapping (import/export, JSON shaping) | a flow that only retrieves and shows | +| a branchy rule or tool microflow (each branch, each refusal message) | wiring: a test cannot see that no button calls the flow | +| REST or Java action logic behind a stable contract | anything needing the user's session or module security (the runner switches security off) | +| a flow where a bug was fixed: the test pins the fix | | + +Measured the other way round: a 143-minute microflow run on one project found stale strings and +not one logic bug. Spend unit tests where logic is, not everywhere. + +## 2. Write it so the assertion is live + +``` +/** + * @test Discount: order total of 1000.00 gets 5 percent + * @expect $Discount = 50.00 + * @cleanup rollback + */ +$Discount = call microflow Orders.CALC_Discount(Total = 1000.00); +/ +``` + +- One block per test, closed by a `/` line. Seed data inline (`create` + `commit`) before the + call; `@cleanup rollback` undoes it. +- **Equality only in `@expect`:** `$Var = value` or `$Entity/Attr = value`. `length(...)`, + `!= empty` and `> 0` parse to **no assertion at all** (8 of 14 tests in one file, BUG-LOCAL-30). + Compute the number in the block and compare with `=`. +- **No file-level `/** */` header.** The runner silently drops the first test (BUG-LOCAL-29). +- `@verify ` and `@setup` really run from mxcli v0.19.0. Before that they were parsed and + ignored, so an old suite re-run on a new binary can surface real failures. That is a finding, + not a regression in the test. +- Test helpers (`TEST_` microflows a test calls) live in the real module and ship with it. + Lint flags them (QUAL004, CUSTOM002): baseline with a reason (`lint-that-actually-runs.md`), + and keep them read-only or seed-only. Hypothesis, not yet observed: a helper that writes data + is a production entry point nobody secured. + +## 3. Check before you run: free, no model write + +``` +mxcli test tests/unit/Orders.test.mdl -p App.mpr --list +``` + +Pass only when **both** hold: the listed count equals the number of `/` separators in the file, +and **every** test shows its `@expect`. A plausible count with a missing test is BUG-LOCAL-29. + +## 4. Run it without breaking anything + +`mxcli test` writes to the model: it injects a TestRunner, sets security off, builds into +`deployment/`, starts a runtime and restores. In the live project that broke a running app +(`NoClassDefFoundError`, field run 2026-10-12, #278). So: + +| Situation | Command | +|---|---| +| Default | `./bin/test-scratch.sh -- tests/unit/Orders.test.mdl --local` (scratch copy, port lock, live app untouched) | +| Fast loop against an already warm app started with `--test-endpoint` | `mxcli test -p App.mpr --attach` (seconds, not ~30 s; it writes into the live app, and the catalog goes stale until `REFRESH CATALOG FULL`) | +| No scratch helper in the project | close Studio Pro, snapshot the `.mpr`, run, then **verify the restore**: `git status` clean, after-startup not `MxTest.*`, security back on | + +A run stopped by a signal leaves after-startup on `MxTest.*`; the next run recurses +(BUG-157). `test-scratch.sh` refuses that state and prints the repair. Only one +`mxcli test --local` per machine at a time: the test ports are machine-wide. + +Cost to plan for: the model injection dominates. One file took 9 min 21 s for 848 ms of +tests; one module's suite took ~76 s (`rest-between-mendix-apps.md`). + +## 5. Prove it can fail, once + +Change one `@expect` to a wrong value and run. It must fail, naming expected and actual. Field +example: a control copy of a 6-test file failed 1 of 6 (expected `Bay Z`, actual `Bay A`). An +assertion never seen failing is a claim about your test code, not about the app +(`testing-shape.md` §6). Revert the control; do not commit it. + +## 6. Report + +``` +unit · · · mxcli + listed = separators · every test has @expect: yes +

pass / fail · positive control: failed as expected () + Read as: these flows return these values on this seed. Not: the feature is wired or secured. +``` + +A failing test is a finding for the user, not a cue to change the `@expect` until it passes. +Same rule as every test in this toolkit: diagnose, report, no unapproved fix +(`e2e-harness-base.md`). + +## Related + +- `skills/testing-shape.md`: the module shape, the strategy, §4 false greens, §6 controls +- `skills/learned-db-assertions.md`: asserting the row landed, from the outside +- `skills/journey-proof.md`: what finds wiring defects a unit test cannot see