From a5b0b348eaa77d850de0ff0ae53ae3b8374d621a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 14:20:00 +0000 Subject: [PATCH 1/7] mxpatch: shared patch core + fix hashes verify around exec.sh --patch (#294, #295) project-bin/mxpatch.py finds a unit (byte pre-filter, then the codec) and an element by Name, gets/sets a value by key, and writes the unit together with its .mpr ContentsHash; dry run, already-applied and refuse-if-drifted harness. Both wf-*.py patchers write through it (master's left a stale hash, captured as a real mxcli v0.26.0 MISMATCH). mxunit_bson.py reads BSON 0x09. exec.sh --patch brackets the script with `mxcli fix hashes` (v0.25.0+) and restores on a new issue, before mxbuild. Fixtures replay captured mxcli v0.25.0/v0.26.0 output on a captured public model. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 1 + bin/lib/install-manifest.sh | 5 +- project-bin/exec.sh | 110 ++++ project-bin/mxpatch.py | 537 ++++++++++++++++++ project-bin/mxunit_bson.py | 18 +- project-bin/wf-add-path-terminators.py | 59 +- project-bin/wf-set-call-captions.py | 66 ++- skills/learned-mdl-preflight.md | 2 +- tests/wave2/fixtures/mxpatch/CAPTURE.txt | 81 +++ .../mxpatch/after-master-patchers-v0.26.0.err | 1 + .../after-master-patchers-v0.26.0.exit | 1 + .../mxpatch/after-master-patchers-v0.26.0.out | 6 + .../mxpatch/after-mxpatch-set-v0.26.0.err | 1 + .../mxpatch/after-mxpatch-set-v0.26.0.exit | 1 + .../mxpatch/after-mxpatch-set-v0.26.0.out | 4 + .../mxpatch/after-ported-patchers-v0.26.0.err | 1 + .../after-ported-patchers-v0.26.0.exit | 1 + .../mxpatch/after-ported-patchers-v0.26.0.out | 4 + tests/wave2/fixtures/mxpatch/capture-se.mdl | 8 + tests/wave2/fixtures/mxpatch/capture-wf.mdl | 25 + .../mxpatch/fix-hashes-help-v0.24.0.exit | 1 + .../mxpatch/fix-hashes-help-v0.24.0.txt | 21 + .../mxpatch/fix-hashes-help-v0.25.0.txt | 36 ++ .../mxpatch/fix-hashes-help-v0.26.0.txt | 36 ++ .../fixtures/mxpatch/model/TestApp.mpr.sql | 48 ++ ...56b39ea-2e14-4d12-8ae1-87fa4b3c038a.mxunit | Bin 0 -> 3883 bytes ...6747dc6-b6b7-42ae-aabf-255dca2aeeaf.mxunit | Bin 0 -> 236 bytes ...fcc2823-c303-5a27-b6c7-78b04c96c88f.mxunit | Bin 0 -> 14224 bytes ...5d3e460-c53d-419e-a2a9-e1428f40b14f.mxunit | Bin 0 -> 2585 bytes ...4f92135-5c65-40c1-9af9-166186ecf501.mxunit | Bin 0 -> 1137 bytes ...f64baa2-b85c-4999-9a3c-3cc38382eb54.mxunit | Bin 0 -> 189 bytes ...a8475b3-14cb-4100-8963-58f2fd3a0be8.mxunit | Bin 0 -> 1817 bytes ...e70063d-53e1-477a-ab92-6b763330c293.mxunit | Bin 0 -> 3035 bytes ...586af73-672c-4a18-9ce4-8a9c0f092ef3.mxunit | Bin 0 -> 1793 bytes ...7d2ecb5-6c2f-4ffa-a0b3-a37a40da3adf.mxunit | Bin 0 -> 57322 bytes ...b276f2e-eed5-4fa9-903b-3483792cbea7.mxunit | Bin 0 -> 5638 bytes ...c6805fc-cadf-4d72-a5ef-f25da7cf4c97.mxunit | Bin 0 -> 4934 bytes ...52106e9-3f38-46c7-bcc4-2b4e4ad39160.mxunit | Bin 0 -> 168159 bytes ...c1ceba7-34e0-4d4c-b6ae-c54b487a3562.mxunit | Bin 0 -> 2382 bytes ...e16f1f7-1207-4d70-8847-c369a7dd360a.mxunit | Bin 0 -> 2066 bytes ...844f052-e1ef-4335-a833-4fe39c4b63a5.mxunit | Bin 0 -> 77 bytes ...a6d6436-a0be-4115-9b12-abdb05cfe1cc.mxunit | Bin 0 -> 1429 bytes ...d06ff15-6fef-4887-a15a-4230d851a87d.mxunit | Bin 0 -> 2601 bytes ...4776610-13b2-48cb-a265-d5e984d63129.mxunit | Bin 0 -> 4167 bytes ...8d29079-1e07-4135-bb38-598e72eb0a0d.mxunit | Bin 0 -> 1786 bytes ...62986ab-1ec9-5c84-9d33-1f0829ffce9b.mxunit | Bin 0 -> 9165 bytes ...d6f65bd-2393-539d-9388-4684a54d0646.mxunit | Bin 0 -> 1864 bytes ...4dd08b7-2dc0-4cc9-823c-3e612eb3a281.mxunit | Bin 0 -> 1786 bytes ...54d8347-a8ec-40ba-b88e-873bfec0a493.mxunit | Bin 0 -> 433 bytes ...121ee37-f102-56ae-b644-160f1d340296.mxunit | Bin 0 -> 402 bytes ...1e8b130-26ba-43bd-b2e7-0f6b61a61008.mxunit | Bin 0 -> 4727 bytes ...928bb10-1832-49c8-9415-7e24495ec970.mxunit | Bin 0 -> 239 bytes .../mxpatch/model/mprcontents/mprname | 1 + .../fixtures/mxpatch/verify-clean-v0.25.0.err | 1 + .../mxpatch/verify-clean-v0.25.0.exit | 1 + .../fixtures/mxpatch/verify-clean-v0.25.0.out | 4 + .../fixtures/mxpatch/verify-clean-v0.26.0.err | 1 + .../mxpatch/verify-clean-v0.26.0.exit | 1 + .../fixtures/mxpatch/verify-clean-v0.26.0.out | 4 + .../mxpatch/verify-notfound-v0.26.0.err | 2 + .../mxpatch/verify-notfound-v0.26.0.exit | 1 + .../mxpatch/verify-notfound-v0.26.0.out | 0 .../mxpatch/verify-orphan-missing-v0.26.0.err | 1 + .../verify-orphan-missing-v0.26.0.exit | 1 + .../mxpatch/verify-orphan-missing-v0.26.0.out | 7 + .../fixtures/mxpatch/verify-stale-v0.25.0.err | 1 + .../mxpatch/verify-stale-v0.25.0.exit | 1 + .../fixtures/mxpatch/verify-stale-v0.25.0.out | 6 + .../fixtures/mxpatch/verify-stale-v0.26.0.err | 1 + .../mxpatch/verify-stale-v0.26.0.exit | 1 + .../fixtures/mxpatch/verify-stale-v0.26.0.out | 6 + .../fixtures/mxpatch/verify-v1-v0.26.0.err | 1 + .../fixtures/mxpatch/verify-v1-v0.26.0.exit | 1 + .../fixtures/mxpatch/verify-v1-v0.26.0.out | 1 + tests/wave2/test-exec-hash-verify.sh | 155 +++++ tests/wave2/test-mxpatch.sh | 150 +++++ 76 files changed, 1348 insertions(+), 75 deletions(-) create mode 100755 project-bin/mxpatch.py create mode 100644 tests/wave2/fixtures/mxpatch/CAPTURE.txt create mode 100644 tests/wave2/fixtures/mxpatch/after-master-patchers-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/after-master-patchers-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/after-master-patchers-v0.26.0.out create mode 100644 tests/wave2/fixtures/mxpatch/after-mxpatch-set-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/after-mxpatch-set-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/after-mxpatch-set-v0.26.0.out create mode 100644 tests/wave2/fixtures/mxpatch/after-ported-patchers-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/after-ported-patchers-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/after-ported-patchers-v0.26.0.out create mode 100644 tests/wave2/fixtures/mxpatch/capture-se.mdl create mode 100644 tests/wave2/fixtures/mxpatch/capture-wf.mdl create mode 100644 tests/wave2/fixtures/mxpatch/fix-hashes-help-v0.24.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/fix-hashes-help-v0.24.0.txt create mode 100644 tests/wave2/fixtures/mxpatch/fix-hashes-help-v0.25.0.txt create mode 100644 tests/wave2/fixtures/mxpatch/fix-hashes-help-v0.26.0.txt create mode 100644 tests/wave2/fixtures/mxpatch/model/TestApp.mpr.sql create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/05/6b/056b39ea-2e14-4d12-8ae1-87fa4b3c038a.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/16/74/16747dc6-b6b7-42ae-aabf-255dca2aeeaf.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/1f/cc/1fcc2823-c303-5a27-b6c7-78b04c96c88f.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/25/d3/25d3e460-c53d-419e-a2a9-e1428f40b14f.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/34/f9/34f92135-5c65-40c1-9af9-166186ecf501.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/3f/64/3f64baa2-b85c-4999-9a3c-3cc38382eb54.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/4a/84/4a8475b3-14cb-4100-8963-58f2fd3a0be8.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/4e/70/4e70063d-53e1-477a-ab92-6b763330c293.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/55/86/5586af73-672c-4a18-9ce4-8a9c0f092ef3.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/57/d2/57d2ecb5-6c2f-4ffa-a0b3-a37a40da3adf.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/5b/27/5b276f2e-eed5-4fa9-903b-3483792cbea7.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/5c/68/5c6805fc-cadf-4d72-a5ef-f25da7cf4c97.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/65/21/652106e9-3f38-46c7-bcc4-2b4e4ad39160.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/6c/1c/6c1ceba7-34e0-4d4c-b6ae-c54b487a3562.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/6e/16/6e16f1f7-1207-4d70-8847-c369a7dd360a.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/78/44/7844f052-e1ef-4335-a833-4fe39c4b63a5.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/7a/6d/7a6d6436-a0be-4115-9b12-abdb05cfe1cc.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/7d/06/7d06ff15-6fef-4887-a15a-4230d851a87d.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/84/77/84776610-13b2-48cb-a265-d5e984d63129.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/98/d2/98d29079-1e07-4135-bb38-598e72eb0a0d.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/b6/29/b62986ab-1ec9-5c84-9d33-1f0829ffce9b.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/bd/6f/bd6f65bd-2393-539d-9388-4684a54d0646.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/c4/dd/c4dd08b7-2dc0-4cc9-823c-3e612eb3a281.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/c5/4d/c54d8347-a8ec-40ba-b88e-873bfec0a493.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/d1/21/d121ee37-f102-56ae-b644-160f1d340296.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/e1/e8/e1e8b130-26ba-43bd-b2e7-0f6b61a61008.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/f9/28/f928bb10-1832-49c8-9415-7e24495ec970.mxunit create mode 100644 tests/wave2/fixtures/mxpatch/model/mprcontents/mprname create mode 100644 tests/wave2/fixtures/mxpatch/verify-clean-v0.25.0.err create mode 100644 tests/wave2/fixtures/mxpatch/verify-clean-v0.25.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/verify-clean-v0.25.0.out create mode 100644 tests/wave2/fixtures/mxpatch/verify-clean-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/verify-clean-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/verify-clean-v0.26.0.out create mode 100644 tests/wave2/fixtures/mxpatch/verify-notfound-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/verify-notfound-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/verify-notfound-v0.26.0.out create mode 100644 tests/wave2/fixtures/mxpatch/verify-orphan-missing-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/verify-orphan-missing-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/verify-orphan-missing-v0.26.0.out create mode 100644 tests/wave2/fixtures/mxpatch/verify-stale-v0.25.0.err create mode 100644 tests/wave2/fixtures/mxpatch/verify-stale-v0.25.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/verify-stale-v0.25.0.out create mode 100644 tests/wave2/fixtures/mxpatch/verify-stale-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/verify-stale-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/verify-stale-v0.26.0.out create mode 100644 tests/wave2/fixtures/mxpatch/verify-v1-v0.26.0.err create mode 100644 tests/wave2/fixtures/mxpatch/verify-v1-v0.26.0.exit create mode 100644 tests/wave2/fixtures/mxpatch/verify-v1-v0.26.0.out create mode 100755 tests/wave2/test-exec-hash-verify.sh create mode 100755 tests/wave2/test-mxpatch.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index ff60ea83..567b4ac4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- new(mxpatch, exec): **`project-bin/mxpatch.py`, the shared patch core, and a `mxcli fix hashes` check around every `exec.sh --patch` run: the two shipped workflow patchers no longer leave a stale ContentsHash.** mxpatch finds a unit by type + name (+ module) with a byte pre-filter, finds an element by Name, gets/sets a value by key, and writes the unit and its `.mpr` ContentsHash (base64 SHA-256 of the file) in one step; dry run by default, `already applied` on a re-run, and a refusal (exit 3) when the current value is not the expected old one. `wf-set-call-captions.py` and `wf-add-path-terminators.py` now write through it and also take the `exec.sh --patch` form (` Module.Workflow`). `mxunit_bson.py` reads BSON 0x09 (UTC datetime). `exec.sh --patch` runs `mxcli fix hashes` (v0.25.0+, probed via `fix --help`) before and after the script and restores the snapshot on any issue the baseline did not have, before mxbuild, which cannot see it. Field evidence: captured mxcli v0.25.0/v0.26.0 runs on a public model (`tests/wave2/fixtures/mxpatch/CAPTURE.txt`): master's patchers leave a MISMATCH, the ported ones verify clean; the hash routine agrees with mxcli on 67 of 67 mismatched and 302 matching units of a second model. Widget-shape copying and pluggable-widget properties wait on a captured page unit. — jopterhorst ([#294](https://github.com/MendixMau/mxcli-project-toolkit/issues/294), [#295](https://github.com/MendixMau/mxcli-project-toolkit/issues/295)) - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index f33a52b0..cbf3af3a 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -97,7 +97,8 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # none; the trend view lived only as a grep in the skill. # wf-add-path-terminators.py + mxunit_bson.py are the BUG-121 repair, and they install as a # PAIR — the patcher imports the codec from its own directory, so shipping one without the -# other is an ImportError on first use. They are here rather than in bin/ because a PROJECT +# other is an ImportError on first use. Since 2026-10-14 (#294) it is a TRIO: both wf-*.py +# patchers import mxpatch.py (unit lookup, write + ContentsHash), which imports the codec. They are here rather than in bin/ because a PROJECT # runs them, against that project's own model unit, after every MDL script that rewrites a # workflow carrying a PARALLEL SPLIT. Until MDL has a branch-ending keyword there is no other # way to make a scripted split fan out, and every gate is blind to its absence. @@ -114,7 +115,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # may run itself (stops only the pid run --local recorded for THIS project), and `mxcli test` on a # scratch copy under the test-port lock. exec.sh names app-restart.sh in its watch advisory, so # it has to be installed beside it. -MXTK_PROJECT_BIN="app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh" +MXTK_PROJECT_BIN="app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh" # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either diff --git a/project-bin/exec.sh b/project-bin/exec.sh index b85dde08..ef523306 100755 --- a/project-bin/exec.sh +++ b/project-bin/exec.sh @@ -37,6 +37,9 @@ # without spending an mxbuild on it. A gate that fails, cannot run, or reports `unverified` is # restored too, and ALLOW_UNVERIFIED does not apply. A dirty model with the same errors as # before is kept, same as for MDL (the delta gate). +# Since 2026-10-14 (#295) a --patch run also brackets the script with `mxcli fix hashes` +# (v0.25.0+, read-only): a ContentsHash the script left stale is a restore, before mxbuild, +# which cannot see it. See "ContentsHash verify" below; bin/mxpatch.py writes units with it. # What it does NOT guard: anything the script touches besides the model. `./bin/exec.sh` is on # the project's allow-list (install-claude-permissions.sh), so in auto mode Claude Code's # classifier does not review the script. Read what a patch does outside the .mpr before running it. @@ -1230,6 +1233,73 @@ if [ "${SKIP_BASELINE:-0}" != "1" ] && [ -x "$MXBUILD" ] && [ -x "$JAVA_EXE" ]; fi fi +# ── ContentsHash verify (--patch only; #295, 2026-10-14) ───────────────────── +# An MPR v2 .mpr indexes every mprcontents/**/*.mxunit by base64(SHA-256(file)). A patch that +# rewrites a unit and not its index row leaves Studio Pro deciding "what changed" from a stale +# index, and NOTHING else in this chain can see it — mxcli's own help: "'mx check' does not +# notice". Both patchers the toolkit shipped did exactly that until mxpatch.py (#294). So +# around a --patch run: `mxcli fix hashes` (read-only without --repair) once before the +# script as a baseline, once after it exits 0, before the mxbuild gate. Only an issue the +# BASELINE did not have fails the run (a stale index from an earlier `git checkout` is not +# this patch's doing — it is reported with its remedy and left alone). +# +# The contract is captured, not assumed (tests/wave2/fixtures/mxpatch/, v0.25.0 and v0.26.0 +# verbatim): exit 0 + "The ContentsHash index matches every unit file." (or "is an MPR v1 +# project") = clean; exit 1 + the "N mismatch(es), N missing file(s), N orphan file(s)." +# summary = issues, one " MISMATCH|MISSING|ORPHAN" row each; exit 1 WITHOUT the summary is an +# error ("project not found") — never read the exit code alone. Support is read from +# `mxcli fix --help` listing `hashes`, the way doctor.sh probes property-sets: on v0.24.0 +# `mxcli fix hashes --help` exits 0 with the parent's help, so it cannot be the probe. +# No mxcli, or one older than v0.25.0: a loud NOT RUN, and the patch goes on to the gate — +# blocking every patch on an old binary would block the remedy (guard rule 7). +HV_ON=0; HV_BASE=""; HV_POST=""; HV_STATE=""; HV_RC=0 +hv_supported() { # 0 listed, 1 absent, 2 the help itself failed + _hv_f=$(mktemp "${TMPDIR:-/tmp}/exec-hv.XXXXXX"); _hv_r=0 + "$PMXCLI" fix --help > "$_hv_f" 2>&1 < /dev/null || _hv_r=2 + if [ "$_hv_r" -eq 0 ]; then grep -qE '^[[:space:]]+hashes[[:space:]]' "$_hv_f" || _hv_r=1; fi + rm -f "$_hv_f"; return "$_hv_r" +} +hv_run() { # — sets HV_STATE clean|issues|error and HV_RC + HV_RC=0 + "$PMXCLI" fix hashes -p "$MPR" > "$1" 2>&1 < /dev/null || HV_RC=$? + if [ "$HV_RC" -eq 0 ] && grep -qE 'ContentsHash index matches every unit file|is an MPR v1 project' "$1"; then + HV_STATE=clean + elif [ "$HV_RC" -eq 1 ] && grep -qE '^[0-9]+ mismatch\(es\), [0-9]+ missing file\(s\), [0-9]+ orphan file\(s\)\.' "$1"; then + HV_STATE=issues + else + HV_STATE=error + fi +} +hv_keys() { grep -E '^ (MISMATCH|MISSING|ORPHAN) ' "$1" 2>/dev/null | awk '{print $1, $2}' | LC_ALL=C sort -u; } +hv_first() { grep -v -e '^[[:space:]]*$' -e '^WARNING: This is a' "$1" | head -1; } +if [ "$PATCH_MODE" = "1" ]; then + echo "→ ContentsHash baseline: mxcli fix hashes (read-only)..." + _hv_s=0 + if [ -z "$PMXCLI" ]; then + echo " ⚠ NOT RUN — no project mxcli. A stale ContentsHash this patch leaves will go unseen:" + echo " mx check and mxbuild do not read the index." + else + hv_supported || _hv_s=$? + if [ "$_hv_s" -eq 1 ]; then + echo " ⚠ NOT RUN — this mxcli has no 'fix hashes' (arrived in v0.25.0). Upgrade it: a stale" + echo " ContentsHash this patch leaves is invisible to mx check and to the mxbuild gate." + elif [ "$_hv_s" -eq 2 ]; then + echo " ⚠ NOT RUN — 'mxcli fix --help' itself failed, so support is unknown." + else + HV_BASE=$(mktemp "${TMPDIR:-/tmp}/exec-hv-base.XXXXXX") + hv_run "$HV_BASE" + case "$HV_STATE" in + clean) HV_ON=1; echo " ✓ index matches every unit file" ;; + issues) HV_ON=1 + echo " ⚠ the index is ALREADY out of step before this patch ($(grep -E '^[0-9]+ mismatch' "$HV_BASE"))." + echo " Not this patch's doing, and not held against it. Repair it on its own (rewrites the" + echo " index from the files, never the files): ./mxcli fix hashes -p $MPR_BASE --repair" ;; + *) echo " ⚠ NOT RUN — the baseline could not be classified (exit $HV_RC: $(hv_first "$HV_BASE"))." ;; + esac + fi + fi +fi + # ── Execute ────────────────────────────────────────────────────────────────── # The exit status is CAPTURED, not allowed to trip `set -e`. # @@ -1261,8 +1331,48 @@ if [ "$PATCH_MODE" = "1" ]; then else log_build "❌ patch failed" "script exit $EXEC_STATUS — NOT rolled back (restore did not run)" fi + rm -f "$HV_BASE" exit "$EXEC_STATUS" fi + if [ "$HV_ON" = "1" ]; then + echo "→ ContentsHash verify: mxcli fix hashes (read-only)..." + HV_POST=$(mktemp "${TMPDIR:-/tmp}/exec-hv-post.XXXXXX") + hv_run "$HV_POST" + _hv_new="" + [ "$HV_STATE" = issues ] && _hv_new="$(hv_keys "$HV_POST" | LC_ALL=C comm -13 <(hv_keys "$HV_BASE") - )" + if [ "$HV_STATE" = clean ]; then + echo " ✓ index matches every unit file" + elif [ "$HV_STATE" = issues ] && [ -z "$_hv_new" ]; then + echo " ✓ no new ContentsHash issue (the baseline's are still there — see above)" + else + echo "" + if [ "$HV_STATE" = issues ]; then + _hv_n=$(printf '%s\n' "$_hv_new" | grep -c . || true) + echo " ✗ the patch left the .mpr's ContentsHash index out of step with the unit files: $_hv_n new issue(s)." + printf '%s\n' "$_hv_new" | while read -r _k _id; do + grep -A1 -E "^ $_k +$_id " "$HV_POST" | sed 's/^/ /' + done + echo " Write units through bin/mxpatch.py (write_unit sets the hash with the file), or end the" + echo " script with: ./mxcli fix hashes -p \"\$1\" --repair" + _hv_detail="ContentsHash: $_hv_n new issue(s) after the patch" + else + echo " ✗ could not verify the ContentsHash index after the patch (exit $HV_RC: $(hv_first "$HV_POST"))." + echo " The baseline ran, so this is new. A patch is kept only on a verified model." + _hv_detail="ContentsHash verify could not run after the patch (exit $HV_RC)" + fi + echo " → restoring the snapshot (the mxbuild gate cannot see this, so it is not run)." + restore_snapshot + if [ -x ./bin/model-stamp.sh ]; then ./bin/model-stamp.sh clear >/dev/null 2>&1 || true; fi + rm -f "$HV_BASE" "$HV_POST" + if [ "$RESTORED" -eq 1 ]; then + log_build "❌ patch refused" "$_hv_detail — rolled back, gate not run" + else + log_build "❌ patch refused" "$_hv_detail — NOT rolled back (restore did not run)" + fi + exit 1 + fi + rm -f "$HV_BASE" "$HV_POST" + fi else "${PMXCLI:-./mxcli}" exec "$SCRIPT" -p "$MPR" || EXEC_STATUS=$? fi diff --git a/project-bin/mxpatch.py b/project-bin/mxpatch.py new file mode 100755 index 00000000..9ee9ab22 --- /dev/null +++ b/project-bin/mxpatch.py @@ -0,0 +1,537 @@ +#!/usr/bin/env python3 +"""mxpatch.py -- the shared core of a targeted .mxunit patch (#294). + +WHY THIS EXISTS. A patch for something MDL cannot express (a widget inside a pluggable +container, a workflow activity's Caption, a node mxcli never emits) needs the same layer +every time: find the unit, find the element, read or set one value, write the unit back, +and keep the .mpr's ContentsHash index in step. Field report #294: 9 of 12 Python change +scripts on one production app re-implemented that layer, and two different helper +libraries grew in one project. The toolkit's own two patchers wrote the unit and never +touched the index -- every --apply left a stale ContentsHash for Studio Pro to trip on, +which `mx check` and mxbuild cannot see (mxcli `fix hashes --help`, v0.26.0: "'mx check' +does not notice; Studio Pro uses the index to decide what changed"). + +THE HASH. An MPR v2 .mpr indexes each mprcontents/**/.mxunit by +base64(SHA-256(file bytes)) in Unit.ContentsHash, keyed by the UnitID blob in .NET GUID +byte order. Verified, not assumed: identical to the stored value on all 27 units of the +captured fixture model (tests/wave2/fixtures/mxpatch/) and all 369 of mxcli's pedapp test +model, and on mxcli's expr-checker model it flags exactly the 67 units `mxcli fix hashes` +v0.26.0 flags as MISMATCH. write_unit() updates the row in the same transaction as the +file swap, so a patch never leaves the index behind. `bin/exec.sh --patch` then runs +`mxcli fix hashes` (v0.25.0+) as an independent check after the script exits 0. + +LIBRARY (what a patch script imports -- it lives beside this file): + + sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + import mxpatch + m = mxpatch.Model(sys.argv[1]) # .mpr, its directory, or a unit file in it + u = m.find_unit("Workflows$Workflow", "Approval", module="MyFirstModule") + act = mxpatch.find_element(u.doc, "ACT_Approval_Review", "Workflows$CallMicroflowActivity") + mxpatch.set_value(act, "Caption", "Review", expect="ACT_Approval_Review") + m.write_unit(u, apply=args.apply) # dry run unless apply=True + + set_value() returns "changed" or "already" (the new value is already there: a re-run is a + no-op) and raises Drift when the current value is neither the expected old one nor the new + one -- the model is not the one the patch was written against, so it refuses. + write_unit() returns "written", "dry-run" or "already" (encoded bytes identical to disk). + +CLI (dry run unless --apply; exit 0 ok/already/dry-run, 2 usage, 3 drift, 4 not found or +ambiguous, 5 hash index problem): + + mxpatch.py find --type T [--name N] [--module M] + mxpatch.py get --type T --name N [--module M] [--element E [--element-type ET]] --key K + mxpatch.py set --type T --name N [--module M] [--element E [--element-type ET]] \\ + --key K --value V [--expect OLD] [--apply] + mxpatch.py verify # every unit file against its ContentsHash (no mxcli needed) + mxpatch.py rehash ... [--apply] # re-index units another tool wrote + +Run a write under `./bin/exec.sh --patch