From 1649f3188f749c814c3f05cab531d38fd779303c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 14:32:17 +0000 Subject: [PATCH 1/2] fix(app-facts): open the catalog through an escaped read-only URI "file:%s?mode=ro" % DB cut the path at '#' or '?' in a folder name, dropped mode=ro, and SQLite created and opened an empty file at the truncated path. New mxtk_sqlite_ro_uri in _common.sh (the pathname2url escaping mxtk_model_version already used, which now calls it); step 2 refuses a 0-table catalog. Fixture test-app-facts-db-uri.sh: 8/8, 6 fail on master. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 1 + project-bin/_common.sh | 22 ++++- project-bin/app-facts.sh | 19 +++- tests/wave2/test-app-facts-db-uri.sh | 137 +++++++++++++++++++++++++++ 4 files changed, 174 insertions(+), 5 deletions(-) create mode 100755 tests/wave2/test-app-facts-db-uri.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index ff60ea83..7741ade0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- fix(app-facts): **the catalog is opened through an escaped read-only URI, so a model folder with `#` or `?` in its name no longer breaks step 2 or leaves a stray file beside the project.** Step 2 built `"file:%s?mode=ro" % DB` from the raw path; `#` and `?` end the path in a URI, so `mode=ro` was dropped and SQLite created an empty database at the truncated path (a zero-byte `a ` beside `a #2/`) and opened that. On master that ends in a Python traceback (`no such table: modules_data`, exit 2) after the shell's catalog check had passed. The URI now comes from new `mxtk_sqlite_ro_uri` in `_common.sh`, the `pathname2url` escaping `mxtk_model_version` already did, which now calls the same helper. Step 2 also refuses with a FAULT line when the catalog it opened has 0 tables, rather than writing facts from it, and app-facts refuses outright with a `_common.sh` that predates the helper. Fixture `tests/wave2/test-app-facts-db-uri.sh` (8 cases: folders `a #2`, `b?c` and a plain control, the URI shape, the 0-table refusal); 6 fail on master. Windows (MSYS path conversion of the env var) is unchanged and unverified. Field evidence: none yet. — r-eng-backlog item 3, MendixMau - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau diff --git a/project-bin/_common.sh b/project-bin/_common.sh index 48113307..40d8cd1f 100755 --- a/project-bin/_common.sh +++ b/project-bin/_common.sh @@ -361,6 +361,22 @@ find_mxcli_cache() { fi } +# --------------------------------------------------------------------------- +# mxtk_sqlite_ro_uri — print a read-only SQLite URI for : absolute, URI-escaped, +# ending in ?mode=ro. Hand it to Python as sqlite3.connect(uri, uri=True). +# Escaping is the point: a plain "file:%s?mode=ro" cuts the path at a '#' or '?' in any folder +# name ("a #2", "b?c"), drops mode=ro with it, and SQLite then CREATES an empty file at the +# truncated path (a zero-byte "a " beside the model folder) and opens that instead. +# Uses $PY when set, else resolve_py. Returns 1 when there is no Python. The caller decides +# whether to pass the path through native_path first; this does no platform conversion. +# --------------------------------------------------------------------------- +mxtk_sqlite_ro_uri() { + local py="${PY:-$(resolve_py 2>/dev/null || true)}" + [ -n "$py" ] || return 1 + "$py" -c "import os,sys,urllib.request as u +print('file:'+u.pathname2url(os.path.abspath(sys.argv[1]))+'?mode=ro')" "$1" | tr -d '\r' +} + # --------------------------------------------------------------------------- # mxtk_model_version [mpr] — the model's Mendix version (e.g. 11.12.2), read from the # .mpr's own SQLite _MetaData._ProductVersion. v1 and v2 models both carry it (v2's .mpr @@ -379,9 +395,9 @@ mxtk_model_version() { fi if [ -z "$v" ]; then py="${PY:-$(resolve_py 2>/dev/null || true)}" - [ -n "$py" ] && v=$("$py" -c "import sqlite3,sys,os,urllib.request as u -c=sqlite3.connect('file:'+u.pathname2url(os.path.abspath(sys.argv[1]))+'?mode=ro',uri=True) -print(c.execute('select _ProductVersion from _MetaData').fetchone()[0])" "$(native_path "$mpr")" 2>/dev/null | head -1 | tr -d '\r') + [ -n "$py" ] && v=$("$py" -c "import sqlite3,sys +c=sqlite3.connect(sys.argv[1],uri=True) +print(c.execute('select _ProductVersion from _MetaData').fetchone()[0])" "$(PY="$py" mxtk_sqlite_ro_uri "$(native_path "$mpr")")" 2>/dev/null | head -1 | tr -d '\r') fi case "$v" in [0-9]*.[0-9]*) printf '%s\n' "$v"; return 0 ;; diff --git a/project-bin/app-facts.sh b/project-bin/app-facts.sh index 534d3263..ec955919 100755 --- a/project-bin/app-facts.sh +++ b/project-bin/app-facts.sh @@ -284,11 +284,26 @@ log "catalog ok: build_mode=${BUILD_MODE:-unset} modules=$N_MODULES refs=$N_REFS # --------------------------------------------------------------------------------------------- log "step 2: inventory + dependency facts" T2=$(date +%s) -DB="$DB" FACTS_DIR="$FACTS_DIR" MX_VERSION="$MX_VERSION" MPR_BASE="$MPR_BASE" MAX_LOOPS="$MAX_LOOPS" "$PY" - <<'PY' || exit 2 +# The catalog is opened through mxtk_sqlite_ro_uri (_common.sh): a bare "file:%s?mode=ro" cut the +# path at a '#' or '?' in a folder name, lost mode=ro, and SQLite created and opened an empty file +# beside the model folder. Without the helper (a _common.sh older than this script) refuse rather +# than open an unescaped path. +type mxtk_sqlite_ro_uri >/dev/null 2>&1 || { + echo "FAULT: project-bin/_common.sh has no mxtk_sqlite_ro_uri (older than app-facts.sh). Run bin/sync-project.sh." >&2 + exit 2 +} +DB_URI="$(mxtk_sqlite_ro_uri "$DB")" && [ -n "$DB_URI" ] || { echo "FAULT: could not build a read-only URI for $DB" >&2; exit 2; } +DB="$DB" DB_URI="$DB_URI" FACTS_DIR="$FACTS_DIR" MX_VERSION="$MX_VERSION" MPR_BASE="$MPR_BASE" MAX_LOOPS="$MAX_LOOPS" "$PY" - <<'PY' || exit 2 import json, os, sqlite3, sys, collections -db = sqlite3.connect("file:%s?mode=ro" % os.environ["DB"], uri=True) +db = sqlite3.connect(os.environ["DB_URI"], uri=True) db.row_factory = sqlite3.Row +# The shell checked the catalog with the sqlite3 CLI on the plain path; this is a second open. If +# it reached a different file, an empty graph must not be written as the app's facts. +if db.execute("SELECT count(*) FROM sqlite_master WHERE type='table'").fetchone()[0] == 0: + sys.stderr.write("FAULT: %s opened with 0 tables (URI %s), though the catalog check passed. Refusing to write facts.\n" + % (os.environ["DB"], os.environ["DB_URI"])) + sys.exit(2) F = os.environ["FACTS_DIR"] def rows(sql, *a): return [dict(r) for r in db.execute(sql, a)] def one(sql, *a): return db.execute(sql, a).fetchone()[0] diff --git a/tests/wave2/test-app-facts-db-uri.sh b/tests/wave2/test-app-facts-db-uri.sh new file mode 100755 index 00000000..4d92a764 --- /dev/null +++ b/tests/wave2/test-app-facts-db-uri.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# Usage: bash test-app-facts-db-uri.sh [path-to-app-facts.sh] +# +# Fixture for the catalog URI in project-bin/app-facts.sh step 2. +# +# The defect: step 2 opened the catalog as "file:%s?mode=ro" % DB with the path pasted in raw. +# In a URI, '#' starts a fragment and '?' starts the query, so a model folder named "a #2" or +# "b?c" cut the path short and took mode=ro with it. SQLite then CREATED an empty database at +# the truncated path (a zero-byte "a " or "b" beside the model folder) and opened that. The +# shell's catalog check had already passed, because it reads the plain path with the sqlite3 +# CLI, so the two opens disagreed. Measured on master d16f578 with this fixture: both folders +# end in a Python traceback ("no such table: modules_data", exit 2), no inventory.json, and a +# stray zero-byte file left outside the project. +# +# The fix: the URI comes from mxtk_sqlite_ro_uri in _common.sh (pathname2url, the escaping +# mxtk_model_version already used), and step 2 refuses when the catalog it opened has 0 tables. +# +# Cases, for each folder name ("a #2", "b?c", and a plain control): +# 1. app-facts exits 0 and inventory.json names the catalog's two modules +# 2. no stray file is created beside the model folder +# Plus: +# 3. mxtk_sqlite_ro_uri escapes '#', '?' and ' ' and ends in ?mode=ro +# 4. a catalog that is empty at the second open is refused with exit 2 and a FAULT line, +# never written as facts +# +# The catalog database is SYNTHETIC: tables and columns are exactly the ones app-facts.sh's +# step 1 guard and step 2 queries name. The mxcli is a stub that copies the prepared catalog +# into place, as in test-catalog-edge-guard.sh. No real mxcli runs. + +set -uo pipefail + +AF="${1:-}" +[ -z "$AF" ] && AF="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-bin/app-facts.sh" +if [ ! -f "$AF" ]; then + echo "SKIP: subject not found at $AF"; echo "SCORE: 0/0"; exit 0 +fi +AF="$(cd "$(dirname "$AF")" && pwd)/$(basename "$AF")" +command -v sqlite3 >/dev/null || { echo "SKIP: sqlite3 not on PATH"; echo "SCORE: 0/0"; exit 0; } + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/af-dburi.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT + +PREP="$WORK/prep.db" +{ + echo "CREATE TABLE catalog_meta (Key TEXT, Value TEXT); INSERT INTO catalog_meta VALUES ('build_mode','full');" + echo "CREATE TABLE projects (ProjectName TEXT, MendixVersion TEXT); INSERT INTO projects VALUES ('Shop','11.12.2');" + echo "CREATE TABLE modules_data (Name TEXT, Source TEXT, AppStoreVersion TEXT, AppStoreGuid TEXT);" + echo "INSERT INTO modules_data VALUES ('Orders','','',''),('Billing','','','');" + echo "CREATE TABLE entities_data (QualifiedName TEXT, ModuleName TEXT, EntityType TEXT, AttributeCount INTEGER);" + echo "INSERT INTO entities_data VALUES ('Orders.Order','Orders','PERSISTENT',3);" + echo "CREATE TABLE microflows_data (QualifiedName TEXT, ModuleName TEXT, Name TEXT, ActivityCount INTEGER, MicroflowType TEXT);" + echo "INSERT INTO microflows_data VALUES ('Orders.ACT_Save','Orders','ACT_Save',3,'MICROFLOW'),('Billing.ACT_Send','Billing','ACT_Send',2,'MICROFLOW');" + echo "CREATE TABLE activities_data (MicroflowQualifiedName TEXT, ModuleName TEXT, ActivityType TEXT);" + echo "INSERT INTO activities_data VALUES ('Orders.ACT_Save','Orders','ActionActivity');" + echo "CREATE TABLE pages_data (QualifiedName TEXT, ModuleName TEXT);" + echo "CREATE TABLE scheduled_events_data (QualifiedName TEXT, ModuleName TEXT, Microflow TEXT, Enabled INTEGER, IntervalSeconds INTEGER, RepeatDescription TEXT);" + echo "CREATE TABLE objects (QualifiedName TEXT, ModuleName TEXT, ObjectType TEXT);" + echo "INSERT INTO objects VALUES ('Orders.ACT_Save','Orders','MICROFLOW'),('Billing.ACT_Send','Billing','MICROFLOW');" + echo "CREATE TABLE refs (SourceType TEXT, SourceName TEXT, TargetName TEXT, RefKind TEXT, ModuleName TEXT);" + echo "INSERT INTO refs VALUES ('MICROFLOW','Billing.ACT_Send','Orders.ACT_Save','call','Billing');" + echo "CREATE TABLE graph_dead_assets (QualifiedName TEXT, ObjectType TEXT, ModuleName TEXT);" +} | sqlite3 "$PREP" + +# stub mxcli: copies $1 (a prepared catalog) into .mxcli/catalog.db, writes the report file +mkstub() { + cat > "$WORK/mxcli-stub" < "\$o" +EOF + chmod +x "$WORK/mxcli-stub" +} + +run_af() { # -> rc; output in $WORK/out + ( cd "$1" && PROJECT_ROOT="$1" MXCLI="$WORK/mxcli-stub" bash "$AF" --skip-loops --facts-dir "$1/facts" ) > "$WORK/out" 2>&1 + echo $? +} +out() { tr '\n' ' ' < "$WORK/out" | cut -c1-400; } + +mkstub "$PREP" +for name in 'a #2' 'b?c' 'plain'; do + echo "== folder '$name'" + PARENT="$WORK/p-$RANDOM"; P="$PARENT/$name" + mkdir -p "$P"; : > "$P/Shop.mpr" + rc="$(run_af "$P")" + if [ "$rc" = 0 ] && [ -f "$P/facts/inventory.json" ] \ + && grep -q '"Orders"' "$P/facts/inventory.json" && grep -q '"Billing"' "$P/facts/inventory.json"; then + ok "[$name] facts written from the real catalog (rc 0, Orders and Billing in inventory.json)" + else + bad "[$name] no real facts" "rc=$rc $(out)" + fi + stray="$(cd "$PARENT" && ls -A | grep -vxF -- "$name")" + if [ -z "$stray" ]; then + ok "[$name] nothing created beside the model folder" + else + bad "[$name] stray file(s) beside the model folder (a truncated URI was opened read-write)" "$stray" + fi +done + +echo "== mxtk_sqlite_ro_uri" +uri="$( . "$(dirname "$AF")/_common.sh" >/dev/null 2>&1; mxtk_sqlite_ro_uri "/x/a #2/b?c/catalog.db" )" +case "$uri" in + 'file:/x/a%20%232/b%3Fc/catalog.db?mode=ro') ok "escapes ' ', '#', '?' and keeps ?mode=ro ($uri)" ;; + *) bad "unexpected URI" "$uri" ;; +esac + +echo "== a catalog that is empty at the second open is refused" +# The state the truncated URI produced, made directly: step 1 checks a good catalog, then a +# python wrapper ($PYTHON, which resolve_py tries first) replaces it with an empty database just +# before step 2's program starts reading it from stdin. +P="$WORK/empty-$RANDOM/m"; mkdir -p "$P"; : > "$P/Shop.mpr" +mkdir -p "$WORK/bin" +REAL_PY="$(command -v python3 || command -v python)" +cat > "$WORK/bin/python3" < "$WORK/out" 2>&1; echo $? )" +if [ "$rc" = 2 ] && grep -q "opened with 0 tables" "$WORK/out" && [ ! -f "$P/facts/inventory.json" ]; then + ok "0-table catalog at step 2: exit 2, FAULT line, no inventory.json" +else + bad "0-table catalog not refused" "rc=$rc $(out)" +fi + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From b97c9f6a77db484ef0fd2207e2883c3e52046f99 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 15:08:16 +0000 Subject: [PATCH 2/2] fix(app-facts): fall back to an inline read-only URI when _common.sh lacks the helper Review finding on #377: the hard refusal when _common.sh had no mxtk_sqlite_ro_uri broke app-facts' own design (it already falls back for require_py and find_mpr), and its message named project-bin/_common.sh (bin/_common.sh in a consuming project) and a sync command with no project root. Step 2 now builds the same pathname2url file:...?mode=ro URI inline when the helper is missing, and passes the catalog path through native_path when that exists, as mxtk_model_version does. Fixture test-app-facts-db-uri.sh gains a case: app-facts beside a _common.sh with the helper stripped, model folder "a #2" -> facts written, nothing created beside the folder (fails on 1649f31, which refused). The 0-table case now takes its interpreter from require_py and a $PYTHON wrapper instead of hard-coded python3 names (4 check-portability violations). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 2 +- project-bin/app-facts.sh | 19 +++++++----- tests/wave2/test-app-facts-db-uri.sh | 44 ++++++++++++++++++++++++---- 3 files changed, 52 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7741ade0..7fb56aa0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased -- fix(app-facts): **the catalog is opened through an escaped read-only URI, so a model folder with `#` or `?` in its name no longer breaks step 2 or leaves a stray file beside the project.** Step 2 built `"file:%s?mode=ro" % DB` from the raw path; `#` and `?` end the path in a URI, so `mode=ro` was dropped and SQLite created an empty database at the truncated path (a zero-byte `a ` beside `a #2/`) and opened that. On master that ends in a Python traceback (`no such table: modules_data`, exit 2) after the shell's catalog check had passed. The URI now comes from new `mxtk_sqlite_ro_uri` in `_common.sh`, the `pathname2url` escaping `mxtk_model_version` already did, which now calls the same helper. Step 2 also refuses with a FAULT line when the catalog it opened has 0 tables, rather than writing facts from it, and app-facts refuses outright with a `_common.sh` that predates the helper. Fixture `tests/wave2/test-app-facts-db-uri.sh` (8 cases: folders `a #2`, `b?c` and a plain control, the URI shape, the 0-table refusal); 6 fail on master. Windows (MSYS path conversion of the env var) is unchanged and unverified. Field evidence: none yet. — r-eng-backlog item 3, MendixMau +- fix(app-facts): **the catalog is opened through an escaped read-only URI, so a model folder with `#` or `?` in its name no longer breaks step 2 or leaves a stray file beside the project.** Step 2 built `"file:%s?mode=ro" % DB` from the raw path; `#` and `?` end the path in a URI, so `mode=ro` was dropped and SQLite created an empty database at the truncated path (a zero-byte `a ` beside `a #2/`) and opened that. On master that ends in a Python traceback (`no such table: modules_data`, exit 2) after the shell's catalog check had passed. The URI now comes from new `mxtk_sqlite_ro_uri` in `_common.sh`, the `pathname2url` escaping `mxtk_model_version` already did, which now calls the same helper. Step 2 also refuses with a FAULT line when the catalog it opened has 0 tables, rather than writing facts from it, and with a `_common.sh` that predates the helper (or none) app-facts builds the same `pathname2url` URI inline, as its `require_py`/`find_mpr` fallbacks already do, passing the path through `native_path` when that exists. Fixture `tests/wave2/test-app-facts-db-uri.sh` (10 cases: folders `a #2`, `b?c` and a plain control, the URI shape, the 0-table refusal, and `a #2` beside an older `_common.sh` without the helper); 8 fail on master, and the older-`_common.sh` case fails on the previous revision of this fix, which refused. Windows (MSYS path conversion of the env var) is unchanged and unverified. Field evidence: none yet. — r-eng-backlog item 3, MendixMau - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau diff --git a/project-bin/app-facts.sh b/project-bin/app-facts.sh index ec955919..608af066 100755 --- a/project-bin/app-facts.sh +++ b/project-bin/app-facts.sh @@ -286,13 +286,18 @@ log "step 2: inventory + dependency facts" T2=$(date +%s) # The catalog is opened through mxtk_sqlite_ro_uri (_common.sh): a bare "file:%s?mode=ro" cut the # path at a '#' or '?' in a folder name, lost mode=ro, and SQLite created and opened an empty file -# beside the model folder. Without the helper (a _common.sh older than this script) refuse rather -# than open an unescaped path. -type mxtk_sqlite_ro_uri >/dev/null 2>&1 || { - echo "FAULT: project-bin/_common.sh has no mxtk_sqlite_ro_uri (older than app-facts.sh). Run bin/sync-project.sh." >&2 - exit 2 -} -DB_URI="$(mxtk_sqlite_ro_uri "$DB")" && [ -n "$DB_URI" ] || { echo "FAULT: could not build a read-only URI for $DB" >&2; exit 2; } +# beside the model folder. A _common.sh older than this script (or none) lacks the helper: build +# the same URI inline, as require_py and find_mpr fall back above. native_path (when present) +# turns an MSYS /c/... path into C:/... first, as mxtk_model_version does. +DB_NATIVE="$DB" +if type native_path >/dev/null 2>&1; then DB_NATIVE="$(native_path "$DB")"; fi +if type mxtk_sqlite_ro_uri >/dev/null 2>&1; then + DB_URI="$(mxtk_sqlite_ro_uri "$DB_NATIVE")" +else + DB_URI="$("$PY" -c "import os,sys,urllib.request as u +print('file:'+u.pathname2url(os.path.abspath(sys.argv[1]))+'?mode=ro')" "$DB_NATIVE" | tr -d '\r')" +fi +[ -n "$DB_URI" ] || { echo "FAULT: could not build a read-only URI for $DB" >&2; exit 2; } DB="$DB" DB_URI="$DB_URI" FACTS_DIR="$FACTS_DIR" MX_VERSION="$MX_VERSION" MPR_BASE="$MPR_BASE" MAX_LOOPS="$MAX_LOOPS" "$PY" - <<'PY' || exit 2 import json, os, sqlite3, sys, collections diff --git a/tests/wave2/test-app-facts-db-uri.sh b/tests/wave2/test-app-facts-db-uri.sh index 4d92a764..333421cc 100755 --- a/tests/wave2/test-app-facts-db-uri.sh +++ b/tests/wave2/test-app-facts-db-uri.sh @@ -22,6 +22,8 @@ # 3. mxtk_sqlite_ro_uri escapes '#', '?' and ' ' and ends in ?mode=ro # 4. a catalog that is empty at the second open is refused with exit 2 and a FAULT line, # never written as facts +# 5. beside an OLDER _common.sh with no mxtk_sqlite_ro_uri, app-facts falls back to its inline +# URI (no refusal) and a folder with a '#' still opens read-only: facts written, no stray file # # The catalog database is SYNTHETIC: tables and columns are exactly the ones app-facts.sh's # step 1 guard and step 2 queries name. The mxcli is a stub that copies the prepared catalog @@ -111,12 +113,13 @@ esac echo "== a catalog that is empty at the second open is refused" # The state the truncated URI produced, made directly: step 1 checks a good catalog, then a -# python wrapper ($PYTHON, which resolve_py tries first) replaces it with an empty database just +# python wrapper ($PYTHON, which _common.sh's resolve_py tries first) replaces it with an empty database just # before step 2's program starts reading it from stdin. P="$WORK/empty-$RANDOM/m"; mkdir -p "$P"; : > "$P/Shop.mpr" mkdir -p "$WORK/bin" -REAL_PY="$(command -v python3 || command -v python)" -cat > "$WORK/bin/python3" < "$WORK/bin/py-wrap" < "$WORK/out" 2>&1; echo $? )" +chmod +x "$WORK/bin/py-wrap" +rc="$( ( cd "$P" && PY="$WORK/bin/py-wrap" PYTHON="$WORK/bin/py-wrap" PROJECT_ROOT="$P" MXCLI="$WORK/mxcli-stub" bash "$AF" --skip-loops --facts-dir "$P/facts" ) > "$WORK/out" 2>&1; echo $? )" if [ "$rc" = 2 ] && grep -q "opened with 0 tables" "$WORK/out" && [ ! -f "$P/facts/inventory.json" ]; then ok "0-table catalog at step 2: exit 2, FAULT line, no inventory.json" else bad "0-table catalog not refused" "rc=$rc $(out)" fi +echo "== older _common.sh without mxtk_sqlite_ro_uri: inline fallback" +# A consuming project whose bin/_common.sh predates the helper: app-facts.sh copied beside a +# _common.sh with that one function removed. Everything else (find_mpr, require_py, native_path) +# is still there, as in a real older copy. +OLD="$WORK/old-bin"; mkdir -p "$OLD" +cp "$AF" "$OLD/app-facts.sh" +"$REAL_PY" - "$(dirname "$AF")/_common.sh" "$OLD/_common.sh" <<'PYEOF' +import re, sys +s = open(sys.argv[1]).read() +s2 = re.sub(r'(?ms)^mxtk_sqlite_ro_uri\(\) \{.*?^\}\n', '', s, count=1) +assert s2 != s, "mxtk_sqlite_ro_uri not found in _common.sh" +open(sys.argv[2], "w").write(s2) +PYEOF +if ( . "$OLD/_common.sh" >/dev/null 2>&1; type mxtk_sqlite_ro_uri >/dev/null 2>&1 ); then + bad "test setup: stripped _common.sh still defines mxtk_sqlite_ro_uri" +else + PARENT="$WORK/old-$RANDOM"; P="$PARENT/a #2"; mkdir -p "$P"; : > "$P/Shop.mpr" + rc="$( ( cd "$P" && PROJECT_ROOT="$P" MXCLI="$WORK/mxcli-stub" bash "$OLD/app-facts.sh" --skip-loops --facts-dir "$P/facts" ) > "$WORK/out" 2>&1; echo $? )" + if [ "$rc" = 0 ] && grep -q '"Orders"' "$P/facts/inventory.json" 2>/dev/null && grep -q '"Billing"' "$P/facts/inventory.json"; then + ok "[older _common.sh, 'a #2'] no refusal; facts written from the real catalog" + else + bad "[older _common.sh, 'a #2'] fallback did not produce facts" "rc=$rc $(out)" + fi + stray="$(cd "$PARENT" && ls -A | grep -vxF -- 'a #2')" + if [ -z "$stray" ]; then + ok "[older _common.sh, 'a #2'] opened read-only: nothing created beside the model folder" + else + bad "[older _common.sh, 'a #2'] stray file(s) beside the model folder" "$stray" + fi +fi + echo "" echo "PASS=$PASS FAIL=$FAIL" [ "$FAIL" -eq 0 ]