new(app-facts): incremental loop sweep + app-map freshness in status (#297)

[MendixMau]

Direct lane

What changed and why (one paragraph):
project-bin/app-facts.sh re-described every loop-containing microflow on every run. The field report in #297 measured 2 h 45 at 6 workers for 1,917 describes on a 17,500-unit app (about 31 s each), while about 80 units had changed between rounds. manifest.json now stores a {UnitID: ContentsHash} snapshot of the .mpr Unit table, with its date in units.snapshot_taken, and an mdl_index recording which unit state each mdl/ file was described from. A refresh describes only candidates that are new, changed (hash), moved (module or folder path), or without output. It keeps the rest and deletes outputs for units that were removed or no longer loop. The sweep is full on the first run, with --full, after an mxcli version change, or when the Unit table cannot be read (MPR v1); sections.loops.sweep says which and why. UnitID-to-name mapping is a new units subcommand of project-bin/mx-metadata-note.py. Its Rev walk was refactored into a shared Model base with a working-tree Tree subclass, so the mapping is not duplicated. bin/status.sh prints app map: N units changed since <date> or app map: fresh (built <date>) when a map exists (app-facts.sh --freshness, no mxcli needed); with no map it prints nothing. skills/app-analysis.md now gives the cost measured in #297 by app size, and names --full as the remedy after edits made outside Studio Pro.

Hash source: the hash is computed from the .mxunit bytes in ContentsHash format rather than read from the column. Nothing guarantees that a row is rewritten in the same moment as its file, and a lagging row would hide a change. The column is the fallback for a unit with no file, and disagreements are counted in units.reason (fixture case T3b).

Not in scope / NEEDS-FIELD-DATA: replacing describe with catalog queries (mxcli v0.25 activities with loop depth). That needs a real v0.25 catalog capture to check parity with the parser. Follow-up: skip re-hashing unchanged .mxunit files by mtime + size. Not built: reading the files is already cheap, and an mtime-preserving copy could hide a change.

Field evidence — no field run; checked on synthetic and captured fixtures only.

Review (MERGE-AFTER-FIX) — addressed in e52e778

1. Blocker, env-var size. --freshness passed the full hash list to Python in NOW_TSV. Linux caps one env string at 128 KB, about 1,600 units, so the call failed with "Argument list too long" and status.sh silently dropped the line. The list now goes through a mktemp file, passed as an argument and removed on exit. No other unit list is passed through env or argv: the sweep already used files (.units.tsv, .sweep-plan.json, and the describe list on xargs' stdin). New case T13 runs --freshness and status.sh --brief on 2,101 units: 3 checks fail on 6550815 and pass now. The bench also calls --freshness: fresh, then 20 changed, then fresh.
2. Module rename. New case T14: a renamed module (same UnitIDs and microflow bytes) re-describes all 5 candidates under the new name and leaves no outputs under the old one. The skill now has one line on --full after edits made outside Studio Pro.
3. --skip-loops freshness. A --skip-loops run now keeps the last loop sweep's snapshot and its date, so mdl/ that is stale never reads fresh. New case T15: it fails on 6550815 (both layouts) and passes now.
4. Optional mtime + size skip. Listed as a follow-up (above).

• No client data anywhere in the diff
• Size cap: 9 files; non-test lines within the cap
• Test tier: T1 scoped fixtures (T2 not reached)
  • tests/wave2/test-app-facts-incremental.sh project-bin/app-facts.sh: 108/108, both layouts (78 checks fail against master's app-facts.sh)
  • test-app-report.sh bin/app-report.sh 109/0 · test-catalog-edge-guard.sh project-bin/graph-sweep.sh 20/0 · test-status-existing-app.sh bin/status.sh 27/27 · test-mx-metadata-note.sh 40/0 (re-run before e52e778; that commit does not change mx-metadata-note.py)
  • tests/wave2/bench-app-sweep.sh (2,000 units, 1 % changed): first map 2,000 describes, no-change refresh 0, incremental 20, --full 2,000; freshness reads fresh / 20 changed / fresh
• Golden input: the module and MyFirstLogic units are real captured units from fixtures/mx-metadata-note (Mendix 11.15.0); the rest is synthetic (fixtures/app-facts-incremental/mkmodel.py). Both layouts are tested; both platforms were considered (stdlib Python, forward-slash sqlite URI, mktemp with TMPDIR, no sed -i).
• New skill: n/a
• CHANGELOG.md line, credited "jopterhorst ([field 2026-10-13] App map: incremental loop sweep (2 h 45 on a 17.5k-unit model), map freshness in status #297)"
• Bug entries: n/a

Checks: render-routing --check in sync, baseline 75,028 words before and after (budget 80,000); check-scripts, check-portability, the leak guard, check-no-private-citations and check-pr-discipline are clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw

[MendixMau]

Landed on master in #385 (squash 868b3cb), after its own independent review on the night integration branch. Closing this PR.

---

Generated by Claude Code