From 247677dd377ac8fc38bf6ee24f9a6d1c9395ec58 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 13:49:41 +0000 Subject: [PATCH 1/2] new(mdl-scan): refuse scannable STOP-row patterns before the apply (#349) bin/mdl-scan.sh greps an MDL script for the STOP rows of learned-mdl-preflight.md that a text pattern identifies and that have a captured failure in bug-logs/: rows 1 (BUG-18), 12 (BUG-142), 16 (CE0161 CurrentUser path) and 26 (dg2 quoted association path). Rules are data in bin/lib/mdl-scan-rules.tsv. exec.sh runs it first, before the lock, the commit, mxcli check and the snapshot; a hit refuses the exec. The only override is MXTK_SCAN_OVERRIDE="", printed and logged (#348 will harden it). Fixtures: tests/wave2/test-mdl-scan.sh (43 checks, keys and CE text asserted verbatim in the ledger), tests/wave2/bench-mdl-scan.sh (4/9 captured failures caught, 0/17 clean files flagged). Field evidence: captured bug-log output only; no live mxcli/mxbuild run. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 1 + bin/lib/mdl-scan-rules.tsv | 18 ++ bin/mdl-scan.sh | 127 +++++++++++ project-bin/exec.sh | 48 +++- skills/learned-mdl-preflight.md | 10 +- tests/wave2/bench-mdl-scan.sh | 76 +++++++ tests/wave2/fixtures/mdl-scan/STOP-1.bad.mdl | 18 ++ tests/wave2/fixtures/mdl-scan/STOP-1.good.mdl | 17 ++ tests/wave2/fixtures/mdl-scan/STOP-12.bad.mdl | 8 + .../wave2/fixtures/mdl-scan/STOP-12.good.mdl | 9 + tests/wave2/fixtures/mdl-scan/STOP-16.bad.mdl | 5 + .../wave2/fixtures/mdl-scan/STOP-16.good.mdl | 4 + tests/wave2/fixtures/mdl-scan/STOP-26.bad.mdl | 8 + .../wave2/fixtures/mdl-scan/STOP-26.good.mdl | 8 + .../uncovered/BUG-75.dynamic-class.mdl | 7 + .../uncovered/BUG-77.quoted-member.mdl | 7 + .../uncovered/BUG-78.refset-literal.mdl | 9 + .../uncovered/BUG-86.device-type-nanoflow.mdl | 9 + .../mdl-scan/uncovered/BUG-94.hasrole.mdl | 6 + tests/wave2/test-mdl-scan.sh | 212 ++++++++++++++++++ 20 files changed, 601 insertions(+), 6 deletions(-) create mode 100644 bin/lib/mdl-scan-rules.tsv create mode 100755 bin/mdl-scan.sh create mode 100755 tests/wave2/bench-mdl-scan.sh create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-1.bad.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-1.good.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-12.bad.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-12.good.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-16.bad.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-16.good.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-26.bad.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/STOP-26.good.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/uncovered/BUG-75.dynamic-class.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/uncovered/BUG-77.quoted-member.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/uncovered/BUG-78.refset-literal.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/uncovered/BUG-86.device-type-nanoflow.mdl create mode 100644 tests/wave2/fixtures/mdl-scan/uncovered/BUG-94.hasrole.mdl create mode 100755 tests/wave2/test-mdl-scan.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index ff60ea83..c2150708 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- new(mdl-scan): **`bin/mdl-scan.sh` refuses a script that carries a scannable STOP-row pattern, inside `exec.sh` before the lock, the snapshot and the apply** ([#349](https://github.com/MendixMau/mxcli-project-toolkit/issues/349)). Four of the fourteen live STOP rows are text patterns with a captured failure, and each is one row of `bin/lib/mdl-scan-rules.tsv`: 1 (conditional Visible/Editable inside a customContent column, BUG-18 StorageLoadException), 12 (an expression in `ContentParams`, BUG-142 CE1613), 16 (a path inside `[%CurrentUser%]`, CE0161), 26 (a quoted association path in `Attribute:`, CE1613). Output is `STOP- : β†’ `; exit 0/1/2. The other rows need model state or judgement and stay prose; 1b, 4, 18 and 24 are scannable but have no captured failure in `bug-logs/`, so they are listed as proposed, not shipped. A hit refuses the exec and logs a blocked BUILD-LOG row; `FORCE_EXEC`/`SKIP_CHECK` do not skip it, and the only override is `MXTK_SCAN_OVERRIDE=""`, for the user, printed and logged (#348 hardens it). `learned-mdl-preflight.md` marks the enforced rows with πŸ”Ž (+6 baseline words). Fixture `tests/wave2/test-mdl-scan.sh` (43 checks: each rule fires on the bad MDL captured in its bug entry and is silent on the corrected form, the keys and CE text are asserted verbatim in the ledger, StyleGallery clean, exec refusal/override; 8 of them fail against master's exec.sh). `tests/wave2/bench-mdl-scan.sh`: 4/9 captured failures caught, 0/17 clean files flagged, about 10 ms per 1,000 lines, 88–260 s of applies saved over the corpus. Field evidence: captured bug-log output only, no live run. β€” external reviewer feedback (2026-10-14) - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` β†’ Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. β€” MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History β†’ Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). β€” field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. β€” MendixMau diff --git a/bin/lib/mdl-scan-rules.tsv b/bin/lib/mdl-scan-rules.tsv new file mode 100644 index 00000000..81960a4a --- /dev/null +++ b/bin/lib/mdl-scan-rules.tsv @@ -0,0 +1,18 @@ +# mdl-scan-rules.tsv β€” one row per STOP row of skills/learned-mdl-preflight.md that a text pattern +# identifies. Read by bin/mdl-scan.sh. Columns, TAB-separated: +# id STOP-, the preflight row number +# kind line ERE on the comment-stripped, lowercased line (strings kept) +# customcontent the same ERE, but only on lines inside a datagrid column body +# whose header says ShowContentAs: customContent +# contentparams built-in: a ContentParams value that is not a bare attribute, +# an unquoted association path or a quoted literal (pattern unused) +# pattern lowercase ERE (POSIX awk; no \b, no lookaround) +# message what the hit is +# fix the rewrite +# evidence the bug-log entry holding the captured failure; its fixture is +# tests/wave2/fixtures/mdl-scan/.{bad,good}.mdl + .evidence.txt +# A rule with no captured failure in bug-logs/ does not get a row (issue #349). +STOP-1 customcontent (visible|editable)[[:space:]]*:[[:space:]]*\[ conditional Visible/Editable inside a datagrid customContent column (StorageLoadException on Studio Pro open) set it in Studio Pro by hand, or bind dynamictext (Attribute: "State") BUG-18 +STOP-12 contentparams - ContentParams value is an expression; it is stored as an attribute path (CE1613 at mxbuild) bind a bare attribute ({1} = Attr) or a quoted literal; make a computed label a String attribute BUG-142 +STOP-16 line \[%currentuser/ path inside the [%CurrentUser%] token in an XPath constraint (CE0161) keep the token bare and walk the whole path on the left: [Mod.A_B/Mod.B/Mod.B_Account = '[%CurrentUser%]'] bug-logs/mxcli-bugs.md "Any GRANT/REVOKE exec in a module fails once ANY entity there has an Account_Supplier-style cross-module XPath" +STOP-26 line attribute[[:space:]]*:[[:space:]]*"[^"]*/[^"]*" quoted association path in Attribute: is stored as one attribute named "Assoc/Attr" (CE1613) write the path unquoted: Attribute: Assoc_A_B/Name BUG-DRAFT-dg2-association-path-bindings diff --git a/bin/mdl-scan.sh b/bin/mdl-scan.sh new file mode 100755 index 00000000..f4a4c52a --- /dev/null +++ b/bin/mdl-scan.sh @@ -0,0 +1,127 @@ +#!/usr/bin/env bash +# mdl-scan.sh β€” refuse an MDL script that contains a STOP-row pattern, before anyone applies it. +# +# bin/mdl-scan.sh [-p ] +# +# One rule per row of skills/learned-mdl-preflight.md's STOP table that a text pattern in the +# script identifies (issue #349). The rules are data: bin/lib/mdl-scan-rules.tsv, whose header +# documents the columns and the three rule kinds. Rows that need model state or judgement are +# not here; they stay prose in the skill, unmarked. +# +# Output: one line per hit, `STOP- : β†’ `. +# Exit: 0 clean Β· 1 at least one hit Β· 2 usage error (no script, unreadable file, bad flag, +# rules file missing). +# +# -p is accepted and checked, so a caller can pass the model the same way it passes it to +# `mxcli check`. No current rule reads it: every row that needs model state stayed prose +# (the PR for #349 lists them). A future rule that uses it must stay read-only. +# +# WHY. The STOP table is prose an agent reads, or skips. Every rule here was a script that +# passed `mxcli check`, cost a full apply, and failed only at mxbuild or on Studio Pro open +# (bug ids in the rules file). A text scan costs milliseconds; a failed apply costs 22-65 s +# plus the restore (iterative-build-loop.md, "Gate cost"). +# +# HOW IT READS A SCRIPT. Comments (`--`, `//`, `/* */`) and `$$ ... $$` bodies are removed, +# string literals are kept (STOP-16 lives inside one), and the line is lowercased. Bracket +# depth is counted outside strings, so a customContent column's body is known by depth, not +# by indentation. It is a line scanner, not a parser: a one-line column whose body opens and +# holds the conditional on the header line itself is not seen as inside the column. +# +# Golden input: the bad MDL captured in bug-logs/mxcli-bugs.md for each rule, and the +# StyleGallery example (applied, 0 errors in mx check) as the clean corpus β€” +# tests/wave2/test-mdl-scan.sh, tests/wave2/bench-mdl-scan.sh. Plain bash 3.2 + POSIX awk. +set -u + +usage() { echo "usage: bin/mdl-scan.sh [-p ]" >&2; exit 2; } + +SCRIPT=""; MPR="" +while [ $# -gt 0 ]; do + case "$1" in + -p) [ $# -ge 2 ] || usage; MPR="$2"; shift 2 ;; + -h|--help) sed -n '2,13p' "$0"; exit 0 ;; + -*) echo "mdl-scan: unknown option '$1'" >&2; usage ;; + *) [ -z "$SCRIPT" ] || usage; SCRIPT="$1"; shift ;; + esac +done +[ -n "$SCRIPT" ] || usage +[ -f "$SCRIPT" ] && [ -r "$SCRIPT" ] || { echo "mdl-scan: cannot read script: $SCRIPT" >&2; exit 2; } +if [ -n "$MPR" ] && [ ! -f "$MPR" ]; then echo "mdl-scan: -p: no such model: $MPR" >&2; exit 2; fi + +RULES="${MDL_SCAN_RULES:-$(cd "$(dirname "$0")" && pwd)/lib/mdl-scan-rules.tsv}" +[ -r "$RULES" ] || { echo "mdl-scan: rules file missing: $RULES" >&2; exit 2; } + +awk -F'\t' -v file="$SCRIPT" ' + FNR == NR { # the rules file + if ($0 ~ /^#/ || NF < 5) next + n++; rid[n] = $1; rkind[n] = $2; rpat[n] = $3; rmsg[n] = $4; rfix[n] = $5 + next + } + function hit(i, ln) { printf "%s %s:%d %s β†’ %s\n", rid[i], file, ln, rmsg[i], rfix[i]; hits++ } + # strip(line): sets code (comments and $$-bodies removed, strings kept) and bare (strings + # blanked too). inblock / indollar carry across lines. + function strip(line, m, i, c, c2, instr) { + code = ""; bare = ""; m = length(line); i = 1; instr = 0 + while (i <= m) { + c = substr(line, i, 1); c2 = substr(line, i, 2) + if (inblock) { if (c2 == "*/") { inblock = 0; i += 2 } else i++; continue } + if (indollar) { if (c2 == "$$") { indollar = 0; i += 2 } else i++; continue } + if (instr) { + code = code c; bare = bare " " + if (c == "\047") instr = 0 + i++; continue + } + if (c2 == "/*") { inblock = 1; i += 2; continue } + if (c2 == "$$") { indollar = 1; i += 2; continue } + if (c2 == "--" || c2 == "//") break + if (c == "\047") instr = 1 + code = code c; bare = bare (instr ? " " : c) + i++ + } + code = tolower(code); bare = tolower(bare) + } + # A ContentParams value is fine when it is a quoted literal, or a bare (optionally + # double-quoted) identifier, or an unquoted identifier path. Anything else is an expression. + function bad_value(v) { + sub(/^[[:space:]]+/, "", v); sub(/[[:space:],]+$/, "", v) + if (v ~ /^\047([^\047]|\047\047)*\047$/) return 0 + if (v ~ /^"?[a-z_][a-z0-9_]*"?$/) return 0 + if (v ~ /^[a-z_][a-z0-9_.]*(\/[a-z_][a-z0-9_.]*)*$/) return 0 + return 1 + } + function check_params(text, ln, i, rest, p, q, v) { + rest = text + while (match(rest, /\{[0-9]+\}[[:space:]]*=/)) { + rest = substr(rest, RSTART + RLENGTH) + if (match(rest, /,[[:space:]]*\{[0-9]+\}[[:space:]]*=/)) v = substr(rest, 1, RSTART - 1) + else { v = rest; sub(/\][^]]*$/, "", v) } + if (bad_value(v)) { hit(i, ln); return } + } + } + { + strip($0) + incc = (ccn > 0) # inside a customContent column body? + for (i = 1; i <= n; i++) { + if (rkind[i] == "line" && code ~ rpat[i]) hit(i, FNR) + else if (rkind[i] == "customcontent" && incc && code ~ rpat[i]) hit(i, FNR) + else if (rkind[i] == "contentparams") { + if (cp_open) { # a ContentParams list still open from above + cp_text = cp_text " " code + if (index(bare, "]")) { check_params(cp_text, cp_line, i); cp_open = 0 } + } else if (match(code, /contentparams[[:space:]]*:[[:space:]]*\[/)) { + cp_text = substr(code, RSTART + RLENGTH); cp_line = FNR + if (index(substr(bare, RSTART + RLENGTH), "]")) check_params(cp_text, cp_line, i) + else cp_open = 1 + } + } + } + if (bare ~ /^[[:space:]]*column[[:space:]]/) pending = 0 # a new column: forget a body-less one + if (bare ~ /showcontentas[[:space:]]*:[[:space:]]*customcontent/) pending = 1 + m = length(bare) + for (j = 1; j <= m; j++) { + c = substr(bare, j, 1) + if (c == "{") { depth++; if (pending) { cc[++ccn] = depth; pending = 0 } } + else if (c == "}") { if (ccn > 0 && cc[ccn] == depth) ccn--; depth-- } + } + } + END { exit (hits > 0) } +' "$RULES" "$SCRIPT" diff --git a/project-bin/exec.sh b/project-bin/exec.sh index b85dde08..6e9f001b 100755 --- a/project-bin/exec.sh +++ b/project-bin/exec.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # exec.sh β€” the guard chain around a model write. # -# concurrent-writer guard β†’ module-brief advisory β†’ mxcli check β†’ snapshot β†’ baseline β†’ exec +# STOP-row scan β†’ concurrent-writer guard β†’ module-brief advisory β†’ mxcli check β†’ snapshot β†’ baseline β†’ exec # β†’ mxbuild delta gate β†’ auto-restore only on a NEW error β†’ lint ratchet β†’ SP reopen # # The mxbuild gate is a DELTA gate, not an absolute "0 errors" one: a write is kept @@ -72,6 +72,7 @@ # ALLOW_UNVERIFIED=1 (write even though the mxbuild gate cannot run β€” # the write is then refused by default, see "Gate must be able to run"), # MXTK_NO_INSTALL=1 (never download mxbuild; offline CI) +# MXTK_SCAN_OVERRIDE="" (the user only: run despite a STOP-row scan hit, #349) set -e _T0=$(date +%s) @@ -436,7 +437,7 @@ HDR2 # The gate cell is read from GATE_STATE rather than passed in, so no caller # can omit it β€” a blank cell in this table reads as "fine", which is the exact # false-green the gate exists to prevent. - _detail="$2${CHANGE_CLASS:+ Β· change: $CHANGE_CLASS}${CR_ID:+ Β· cr: $CR_ID}$_EXEC_APPROVAL_SUFFIX" + _detail="$2${CHANGE_CLASS:+ Β· change: $CHANGE_CLASS}${CR_ID:+ Β· cr: $CR_ID}${SCAN_OVERRIDE_NOTE:+ Β· $SCAN_OVERRIDE_NOTE}$_EXEC_APPROVAL_SUFFIX" printf '| %s | `%s` | %s | %s | %s |\n' \ "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$LOG_NAME" \ "${GATE_STATE:-not-run}" "$1" "$_detail" >> "$BUILD_LOG" @@ -474,6 +475,49 @@ take_score() { fi } +# ── STOP-row scan (added 2026-10-14, #349) ─────────────────────────────────── +# The toolkit's bin/mdl-scan.sh greps the script for the STOP rows of learned-mdl-preflight.md +# that a text pattern identifies β€” each one a script that passed `mxcli check`, cost a full +# apply and failed at mxbuild or on Studio Pro open. It runs first, before the lock, the +# commit, `mxcli check` and the snapshot: a hit costs milliseconds here and 22-65 s plus a +# restore after the apply. A hit refuses the exec and prints the scan. FORCE_EXEC and +# SKIP_CHECK do not skip it. The only override is MXTK_SCAN_OVERRIDE="", for the human: +# it is printed and lands in the BUILD-LOG row. Hardening that override against an agent +# setting it itself is #348's job. Found like exec-approval.sh: $MXTK_ROOT, else the +# "Toolkit root" row of CLAUDE.local.md. Not found, or the scan itself errors (exit 2) β†’ one +# warning, and the exec goes on: mxcli check and the mxbuild gate still stand behind it. +SCAN_OVERRIDE_NOTE="" +if [ "$PATCH_MODE" != "1" ]; then + _SCAN="" + for _s in "${MXTK_ROOT:-}/bin/mdl-scan.sh" \ + "$(sed -nE 's/^\| *Toolkit root *\| *`([^`]+)`.*/\1/p' "$PROJECT_ROOT/CLAUDE.local.md" 2>/dev/null | head -1)/bin/mdl-scan.sh"; do + [ -f "$_s" ] && { _SCAN="$_s"; break; } + done + if [ -z "$_SCAN" ]; then + echo "⚠ STOP-row scan skipped: bin/mdl-scan.sh not found (set MXTK_ROOT to the toolkit)." >&2 + else + _scan_rc=0; _scan_out="$(bash "$_SCAN" "$SCRIPT" 2>&1)" || _scan_rc=$? # set -e: keep the rc + if [ "$_scan_rc" -ne 0 ] && [ "$_scan_rc" -ne 1 ]; then # 2 = the scan itself could not run + echo "⚠ STOP-row scan could not run (exit $_scan_rc): $_scan_out" >&2 + elif [ "$_scan_rc" -eq 1 ]; then + echo "$_scan_out" + case "${MXTK_SCAN_OVERRIDE:-}" in + ""|1|true|yes) + echo "" + echo "βœ— STOP-row scan: refusing exec. NOTHING was locked, snapshotted or written." + echo " Each line above names its learned-mdl-preflight.md row and the rewrite. Fix the script." + echo " Only the user may override, with a reason: MXTK_SCAN_OVERRIDE=\"\" ./bin/exec.sh $SCRIPT" + log_build "🚫 blocked" "STOP-row scan: $(printf '%s\n' "$_scan_out" | grep -c '^STOP-') hit(s), $(printf '%s\n' "$_scan_out" | sed -n 's/^\(STOP-[0-9a-z]*\) .*/\1/p' | sort -u | tr '\n' ' ')β€” nothing written" + exit 1 ;; + *) + echo "⚠ STOP-row scan OVERRIDDEN by MXTK_SCAN_OVERRIDE: ${MXTK_SCAN_OVERRIDE}" >&2 + echo " Only the user sets this. An agent that set it has broken the preflight rule." >&2 + SCAN_OVERRIDE_NOTE="scan overridden: $(printf '%s' "$MXTK_SCAN_OVERRIDE" | tr '|' '/')" ;; + esac + fi + fi +fi + # ── Concurrent-writer guard ────────────────────────────────────────────────── # The .mpr must have exactly ONE writer. Two mxcli/SP writers on the same file # silently clobber each other and have caused near-total module loss. diff --git a/skills/learned-mdl-preflight.md b/skills/learned-mdl-preflight.md index 8acf7c76..31a3ff86 100644 --- a/skills/learned-mdl-preflight.md +++ b/skills/learned-mdl-preflight.md @@ -30,9 +30,11 @@ Re-probed on mxcli v0.24.0 / Mendix 11.13.0 (2026-09-30); rules that now work or ## The checklist β€” run before writing a single line of MDL +πŸ”Ž = `bin/mdl-scan.sh` refuses it in `exec.sh`. + | # | If your script will… | Then… | Root cause | |---|---|---|---| -| 1 | Use `visible:` or `editable:` **conditional expressions** on containers **inside a `datagrid customContent` column** | **STOP β†’ Studio Pro by hand** | mxcli writes blank `AttributeIdentifier` β†’ `StorageLoadException` on SP open. **BUG-18, still open on v0.13.0** (found 2026-07-03 after v0.13.0 release). **Primary real-world trigger: per-value badge pattern** β€” multiple containers with `Visible: [State = ENUM.Value]` inside a column. mxbuild passes clean; SP load fails. *Re-probe 2026-09-30 (v0.24.0): `mx check` 0 errors and the BSON is identical to the plain-container case; Studio Pro load could not be tested, so the STOP stays.* Safe CLI alternative: `dynamictext (Attribute: "State")` renders enum caption natively. Do NOT use `ContentParams: [{1} = toString("State")]` inside customContent β€” resolves as attribute path β†’ CE1613. Per-value colours: set them in Studio Pro. | +| 1 πŸ”Ž | Use `visible:` or `editable:` **conditional expressions** on containers **inside a `datagrid customContent` column** | **STOP β†’ Studio Pro by hand** | mxcli writes blank `AttributeIdentifier` β†’ `StorageLoadException` on SP open. **BUG-18, still open on v0.13.0** (found 2026-07-03 after v0.13.0 release). **Primary real-world trigger: per-value badge pattern** β€” multiple containers with `Visible: [State = ENUM.Value]` inside a column. mxbuild passes clean; SP load fails. *Re-probe 2026-09-30 (v0.24.0): `mx check` 0 errors and the BSON is identical to the plain-container case; Studio Pro load could not be tested, so the STOP stays.* Safe CLI alternative: `dynamictext (Attribute: "State")` renders enum caption natively. Do NOT use `ContentParams: [{1} = toString("State")]` inside customContent β€” resolves as attribute path β†’ CE1613. Per-value colours: set them in Studio Pro. | | 1b | Put `Editable:` on an input widget in `CREATE PAGE` | **Write it as an expression, `Editable: []` β€” never the quoted string `Editable: ''`.** (`Visible: []` and `ALTER PAGE … SET Visible/Editable = []` work.) | v0.24.0: the quoted form is dropped with no message β€” `describe` shows no `Editable` β€” so the field is silently always editable. `check` and `exec` are both silent; read the page back with `describe page`. | | 1c | *(retired 2026-09-30 β€” ALTER PAGE SET Visible/Editable stores correctly, `mx check` 0 errors)* | | | | 2 | *(retired 2026-09-30 β€” settings and security-level writes work in the CLI)* | | | @@ -46,19 +48,19 @@ Re-probed on mxcli v0.24.0 / Mendix 11.13.0 (2026-09-30); rules that now work or | 9 | *(retired 2026-09-30 β€” inline association-set in CHANGE/CREATE works, cross-module too β€” set it from the owner side, see `learned-microflow-patterns.md`)* | | | | 10 | *(retired 2026-09-30 β€” `mxcli check` refuses `count()` (MDL044) and exec will not run it; write `length($list)`)* | | | | 11 | *(retired 2026-09-30 β€” an association path as XPath comparand works)* | | | -| 12 | Put anything but a bare attribute or a quoted literal in a `dynamictext`'s `ContentParams` β€” `toString(Attr)`, `$currentObject/Attr`, `if … then … else`, any function | **Bind a bare attribute (`{1} = Attr`) or a quoted literal, nothing else.** A computed label becomes data: a stored or derived String attribute, bound bare. Tone variants go in `DynamicClasses`, which takes a real expression (and there *needs* the prefix: `'if $currentObject/State = Enum.Value then ...'`). | v0.24.0 (BUG-142): every unquoted value is stored as an attribute path, so an expression is CE1613 at mxbuild ("The selected attribute '.' no longer exists"). `check` now refuses the `toString`, `concat` and `$currentObject` forms, but **`if … then … else` passes `check` and `exec`** and fails only at `mx check`; `mxcli lint` does not flag it. A bare Boolean renders "Yes"/"No", not `true`/`false`. `Attribute: Attr` is the documented shorthand for the bare form (`learned-datagrid-customcontent-binding.md`). | +| 12 πŸ”Ž | Put anything but a bare attribute or a quoted literal in a `dynamictext`'s `ContentParams` β€” `toString(Attr)`, `$currentObject/Attr`, `if … then … else`, any function | **Bind a bare attribute (`{1} = Attr`) or a quoted literal, nothing else.** A computed label becomes data: a stored or derived String attribute, bound bare. Tone variants go in `DynamicClasses`, which takes a real expression (and there *needs* the prefix: `'if $currentObject/State = Enum.Value then ...'`). | v0.24.0 (BUG-142): every unquoted value is stored as an attribute path, so an expression is CE1613 at mxbuild ("The selected attribute '.' no longer exists"). `check` now refuses the `toString`, `concat` and `$currentObject` forms, but **`if … then … else` passes `check` and `exec`** and fails only at `mx check`; `mxcli lint` does not flag it. A bare Boolean renders "Yes"/"No", not `true`/`false`. `Attribute: Attr` is the documented shorthand for the bare form (`learned-datagrid-customcontent-binding.md`). | | 13 | *(retired 2026-09-30 β€” `check` refuses an unqualified association name in a retrieve XPath (CE0161) and exec refuses; write `Module.Assoc`)* | | | | 15 | *(retired 2026-09-30 β€” quoted attribute names in CHANGE work; `describe` normalises them)* | | | | 14 | Narrow an existing entity grant (fewer attributes or rights than the rule already has) | **`revoke` first, then `grant` the narrower set.** A lone narrowing `grant` merges into the existing rule instead of replacing it (BUG-145). | v0.24.0: revoke + re-grant no longer drops another role's grant (`mx check` 0 errors); the merge on a lone `grant` remains. | | 17 | *(retired 2026-09-30 β€” uppercase `AND`/`OR` with `!=` builds at 0 errors)* | | | | 18 | Write **any workflow MDL** β€” `CREATE`/`ALTER WORKFLOW`, a `CALL MICROFLOW` step, a `JUMP TO`, user-task outcomes | **STOP β†’ read [[learned-workflow-patterns]] first, then script. Create every callee microflow in an EARLIER script than the workflow that calls it.** Read the BRD lifecycle + wireframe before writing a single outcome. | v0.24.0: with the callee already in the model, `check` and `exec` refuse a call-microflow with an unmapped parameter, a missing callee, and a `JUMP TO` that is not a real activity name (MDL-WF05). With the callee created **in the same script**, `check --references` skips the call and `exec` auto-fills the `WITH` mapping β€” a non-context parameter gets `(Name = '$WorkflowContext')` β€” and `mx check` fails with CE0117 at the call. A context-typed parameter auto-maps fine. `mxcli lint` does not flag it. Earlier incidents: 2026-07-24 Γ—2, 2026-08-21, 2026-08-31 (`learned-workflow-patterns.md` Β§7, Β§13, Β§14, Β§21). | -| 16 | Write an access-rule XPath constraint with a **path expression inside the `[%CurrentUser/...%]` substitution token** β€” `[Assoc = '[%CurrentUser/Module.OtherAssoc%]']`, especially crossing module boundaries | **Never put a path inside the CurrentUser substitution. Keep it bare (`'[%CurrentUser%]'`) and do the ENTIRE multi-hop/cross-module traversal on the left-hand side instead**, qualifying the entity name at every module crossing (same rule as the "cross-module association expression paths" MDL gotcha, extended here to access-rule XPath constraints, not just microflow expressions). Was: `[FeasibilityDecision_Supplier = '[%CurrentUser/PLM.Account_Supplier%]']`. **Fix (confirmed working):** `[PLM.FeasibilityDecision_Supplier/PLM.Supplier/Administration.Account_Supplier = '[%CurrentUser%]']` β€” walk Supplierβ†’Account fully on the left, compare directly to the (already-an-Account) CurrentUser token. | This is genuinely invalid Mendix XPath, not an mxcli-specific bug β€” `CE0161` "Error(s) in XPath constraint" is Mendix correctly rejecting it every time it's checked, which is why it kept resurfacing across three separate incidents on a PLM parts-flow project (2026-07-22/23, `bug-logs/mxcli-bugs.md`) that were originally misdiagnosed as mxcli corrupting the domain model on unrelated grant rewrites. In fact the "unrelated entity broke too" symptom was just two independently-broken rules (both using the same invalid CurrentUser-path form) surfacing together once a full `mx check` ran β€” not one rule corrupting the other. Once rewritten to the correct bare-CurrentUser/full-LHS-chain form, the earlier "Studio Pro GUI only, module-wide" restriction no longer applies β€” **confirmed mxcli-safe 2026-07-23** via `test-05-currentuser-xpath-chain.mdl` (rebuilt all 4 role grants on the affected entity through mxcli exec, reproducing the corrected XPath verbatim, `mx check: 0 errors`). Use mxcli directly for this pattern; no GUI fallback needed. | +| 16 πŸ”Ž | Write an access-rule XPath constraint with a **path expression inside the `[%CurrentUser/...%]` substitution token** β€” `[Assoc = '[%CurrentUser/Module.OtherAssoc%]']`, especially crossing module boundaries | **Never put a path inside the CurrentUser substitution. Keep it bare (`'[%CurrentUser%]'`) and do the ENTIRE multi-hop/cross-module traversal on the left-hand side instead**, qualifying the entity name at every module crossing (same rule as the "cross-module association expression paths" MDL gotcha, extended here to access-rule XPath constraints, not just microflow expressions). Was: `[Decision_Partner = '[%CurrentUser/PLM.Account_Partner%]']`. **Fix (confirmed working):** `[PLM.Decision_Partner/PLM.Partner/Administration.Account_Partner = '[%CurrentUser%]']` β€” walk Partnerβ†’Account fully on the left, compare directly to the (already-an-Account) CurrentUser token. | This is genuinely invalid Mendix XPath, not an mxcli-specific bug β€” `CE0161` "Error(s) in XPath constraint" is Mendix correctly rejecting it every time it's checked, which is why it kept resurfacing across three separate incidents on a PLM parts-flow project (2026-07-22/23, `bug-logs/mxcli-bugs.md`) that were originally misdiagnosed as mxcli corrupting the domain model on unrelated grant rewrites. In fact the "unrelated entity broke too" symptom was just two independently-broken rules (both using the same invalid CurrentUser-path form) surfacing together once a full `mx check` ran β€” not one rule corrupting the other. Once rewritten to the correct bare-CurrentUser/full-LHS-chain form, the earlier "Studio Pro GUI only, module-wide" restriction no longer applies β€” **confirmed mxcli-safe 2026-07-23** via `test-05-currentuser-xpath-chain.mdl` (rebuilt all 4 role grants on the affected entity through mxcli exec, reproducing the corrected XPath verbatim, `mx check: 0 errors`). Use mxcli directly for this pattern; no GUI fallback needed. | | 21 | `DROP` + `CREATE` (or `CREATE OR REPLACE`) any document **another script also edits** β€” an aggregator microflow, an after-startup chain, a navigation profile | **STOP β†’ `describe` the live document first, diff your restatement against that dump, and name in the script header which call sites you preserved.** mxcli has no `ALTER MICROFLOW` for an existing body, and `ALTER NAVIGATION` **does not exist at all** (the parser rejects it: `no viable alternative at input 'alternavigation'`). Whole-document restatement is therefore the only write mxcli offers, which makes losing a sibling script's edit the *default* outcome, not an unlikely one. | Confirmed 2026-08-26 on a field run: two scripts each did DROP+CREATE on one shared seed aggregator. The second restated it as it had been *before* the first ran, and a demo-data seed call silently vanished. Every instrument was green β€” `mx check` reported **0 errors** because an uncalled microflow is perfectly legal, and `mxcli lint`'s orphan rule (QUAL004, `orphaned_elements.star`) exempts the `ACT_` prefix as a presumed UI entry point, so an `ACT_` microflow with no callers at all reports clean. It was found by opening a page and reading "0 to 0 of 0". | | 22 | Write a script containing ANY non-idempotent statement β€” `alter entity … add attribute`, bare `CREATE ASSOCIATION`, bare `CREATE MICROFLOW`/`CREATE PAGE` on a name that may already exist | **STOP β†’ use the `create or modify` form for every statement, and give every document exactly ONE owning script.** mxcli is **not transactional across statements**: the first failure stops the script and every statement below it is not run. The property to hold is that `mdlsource/` replays cleanly against an EMPTY app; that is one command and it is the only thing that proves it. | v0.24.0: `check` passes; `exec` now fails loudly (`Error: attribute 'Note' already exists on entity Mod.Order`, exit 1) and a later `CREATE MICROFLOW` in the script is **not created** β€” the earlier statements stay written. `mx check` reports 0 errors on the partial model, so the only signal is the exit code: never ignore it. Field case (2026-08-31, mxcli v0.20.0): script `09b` re-ran after its attributes existed, died on statement 1, and two microflows at the end were never applied; `ACT_Version_Delete` quietly lost a call, found by `DESCRIBE MICROFLOW` on a hunch. Compounded by rule 21 one level up: six scripts held a grant on the same three entities, so whichever ran last decided their security. The fix for both is the same: one owner per document, `create or modify` throughout, and a from-empty replay in CI or at wrap-up. | | 23 | Name any document or entity that **might already exist** β€” `create or modify` on an entity, or any `create or replace` on a page, layout or microflow | **STOP β†’ run `mxcli diff -p .mpr