Skip to content

Fix graph-sweep/app-facts catalog guard and per-BRD conformance ledgers (#259, #260) - #307

Closed
MendixMau wants to merge 4 commits into
masterfrom
fix/graph-sweep-conformance
Closed

Fix graph-sweep/app-facts catalog guard and per-BRD conformance ledgers (#259, #260)#307
MendixMau wants to merge 4 commits into
masterfrom
fix/graph-sweep-conformance

Conversation

@MendixMau

@MendixMau MendixMau commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Two field reports from AlvinnYeo (2026-10-11, mxcli v0.24.0, Mendix 11.14.0), plus a follow-up to #313. Commits 2 and 3 address two rounds of independent review.

#259: graph-sweep.sh / app-facts.sh refuse build_mode='source'

Bug. Both scripts FAULTed unless catalog_meta.build_mode was the literal full. mxcli v0.24.0 writes source even after REFRESH CATALOG FULL, on a complete catalog (5,368 refs rows in the report):
FAULT: catalog build_mode is 'source'; reference edges require 'full'.

Fix. Both scripts stop checking the label and check what they read instead.

  • When it FAULTs. Only when the refs table is empty, or when its rows carry no RefKind. The message names the mode and the edge count per kind.
  • Models with no calls still run. A model with no microflow-to-microflow calls still sweeps, as it did on master.
  • Orphan warning. graph-sweep.sh warns when none of the kinds its orphan query reads is present. That kind list is one shared variable used by both the check and the query.
  • Manifest. app-facts.sh records the real mode in manifest.json.
  • Agent note. agents/review-agent.md now describes the new FAULT conditions.

Evidence. tests/wave2/test-catalog-edge-guard.sh: 20 checks on both layouts, 16 of them fail on master. They include the verbatim golden FAULT line, the action-only case that must pass, the case where refs have no RefKind, and the warning branch with and without an action edge.

Assumption. The catalogs are synthetic because no real catalog.db is captured in the repo. They use only the tables and columns the scripts query.

#260: conformance-check.sh reports "0 measurable rows" on per-BRD ledgers

Bug. The script read only <M>/coverage-ledger.md, which is an index in the per-BRD layout:
FAULT: 0 measurable rows extracted from 1 ledger(s).

Fix.

  • conformance-check.sh
    • Measures every <M>/coverage-ledger/*.md.
    • Writes one report per BRD, report-<date>-<BRD>.tsv (a BRD id with a space stays one BRD).
    • Baseline pointers are keyed <BRD>:<pointer>.
    • LEDGER_FILE can be a file or a directory and is documented in --help; a missing one now FAULTs.
    • An index file is reported as an index, with the files its rows are in.
  • review-module.sh
    • Reads every report of the run: the reused base name plus its -<BRD> files, or every report written since the run started.
    • Accepts a module that has only the per-BRD directory.
    • The summary and log field name the report files actually read.
  • verify-module.sh. Its ledger lookup (resolve_ledger_hint) accepts a directory LEDGER_FILE and a module that has only the per-BRD directory, instead of falling back to architecture/coverage-ledger.md.
  • Docs. skills/coverage-ledger.md is updated.

Not in this PR. Per-BRD reports reach docs/report.json through #310. Its report-normalize.js filter reads only report-<date>.tsv today and is being extended there.

Evidence. All new cases fail on the code they fix:

Fixture Checks Layouts
test-conformance-per-brd-ledger.sh 19 both
test-review-conformance-per-brd.sh 6 both
test-verify-ledger-hint.sh 4 —

Upgrade notes (in CHANGELOG).

  • A missing LEDGER_FILE now FAULTs.
  • If a project has both a filled index and a per-BRD directory, its baseline rows are re-keyed. The first run after the upgrade re-seeds them and reports no regressions, so run once, then check again.

#313 follow-up: test-stack-up exit 4 graded as FINDING

verify-module.sh now grades exit 4 on the stack rung (port held by another process) as FAULT, not FINDING. It does this on the stack rung only, because exit 4 means NOT APPLICABLE from conformance-check. test-verify-stack-blocked.sh (6 checks) fails 2 on master.

Checks

  • All new fixtures pass.
  • Existing fixtures for the touched files pass:
    • test-conformance-baseline-scope.sh: 7/7
    • test-app-report.sh: 109/109
  • No full suites were run.
  • bin/check-scripts.sh, bin/render-routing.sh --check, bin/check-portability.sh, bin/check-no-private-citations.sh, LEAKGUARD_ALLOW_NO_DENYLIST=1 bin/check-no-client-data.sh and bin/check-pr-discipline.sh origin/master are all clean.
  • origin/master is merged at b9cf37c, keeping all CHANGELOG lines.

Field run (rule 4)

Still owed for all three fixes. This session had no real v0.24 catalog, no per-BRD project and no held port, and mxcli was not run. The closest field evidence is the reporter's own workarounds: the label patch swept one module (70 elements, 1 inbound / 38 outbound edges), and LEDGER_FILE on one per-BRD file measured 128 rows (66 measured, all OK).

Closes #259
Closes #260

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw

claude added 3 commits October 13, 2026 10:06
@claude
Fix graph-sweep/app-facts catalog guard and per-BRD conformance ledge…
5412e82 
…rs (#259, #260)

graph-sweep.sh and app-facts.sh refused any catalog whose build_mode label
was not 'full'; mxcli v0.24+ writes 'source' for a complete catalog. Both now
check what they read: a non-empty refs table with at least one RefKind='call'
edge, and FAULT naming the mode and per-kind edge counts otherwise.

conformance-check.sh read only <Module>/coverage-ledger.md, which is an index
in the per-BRD layout. It now measures every <Module>/coverage-ledger/*.md,
writes one report per BRD, keys baseline pointers by BRD, documents
LEDGER_FILE (file or directory), FAULTs on a missing LEDGER_FILE, and names
an index as an index with the files its rows are in.

Fixtures (both layouts, fail on master's scripts):
tests/wave2/test-catalog-call-edges.sh (12 checks, 10 fail on master),
tests/wave2/test-conformance-per-brd-ledger.sh (17 checks, 17 fail on master).
Synthetic catalogs/ledgers; field runs on a v0.24 / per-BRD project are owed.

Reported by AlvinnYeo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
Merge origin/master into fix/graph-sweep-conformance
a8df6ad 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
Address review on #307: no-call models sweep, per-BRD reuse, stack ex…
59e7c16 
…it 4

1. graph-sweep.sh / app-facts.sh: the first cut required a 'call' edge and
   refused a model with no microflow-to-microflow calls (reached only from
   pages/buttons), which master swept. Now FAULT only on an empty or
   kind-less refs table, report per-kind counts, and graph-sweep warns when
   none of its orphan-read kinds is present. The per-kind query is fixed
   (aggregate in GROUP BY). Fixture renamed test-catalog-edge-guard.sh,
   adds the action-only and kind-less cases (16 checks).
2. review-module.sh reads every report of the run: --reuse-conformance
   <report-date.tsv> also reads its report-<date>-<BRD>.tsv siblings, and
   its own run slices every report newer than the run start, so a per-BRD
   module no longer FAULTs verify-module's rung 13 or drops BRDs.
   verify-module.sh documents the contract. test-review-conformance-per-brd.sh.
3. conformance-check.sh: per-BRD reports via while-read, so a BRD id with
   a space stays one BRD (fixture case 9).
4. CHANGELOG: missing LEDGER_FILE now FAULTs; per-BRD baseline rows are
   re-keyed <BRD>:<pointer>, so run once after upgrade and re-check.
5. #313 follow-up: verify-module.sh grades test-stack-up exit 4 (port held
   by another process) as FAULT on the stack rung only, not FINDING.
   test-verify-stack-blocked.sh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 13, 2026
@claude
report: read per-BRD conformance reports (report-<date>-<BRD>.tsv)
57bf81b 
conformance-check now writes one report per BRD (#307), but
report-normalize.js kept only report-<date>.tsv, so per-BRD modules never
reached report.json and a stale base file could read as measured. It now
takes the newest date among report-<date>[-<BRD>].tsv and concatenates
every file of that date; single-file projects are unchanged.
pickConformanceFiles is exported; --selftest cases and case 3 of
test-report-normalize-two-tree.sh (fails on master: reads the stale file).

Also: the new fixtures use here-strings instead of `printf | grep -q`,
which under pipefail can SIGPIPE and flip a check (seen once under load).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau MendixMau mentioned this pull request Oct 13, 2026
Closed
@claude
Address re-review on #307: directory ledgers, log field, warning fixture
8fc7c6a 
- agents/review-agent.md: the sweep faults on empty refs or refs with no
  RefKind, not on missing call edges.
- verify-module.sh: resolve_ledger_hint accepts a directory LEDGER_FILE and
  a module with only the per-BRD directory, instead of falling back to
  architecture/coverage-ledger.md. test-verify-ledger-hint.sh (4).
- review-module.sh: a module with only <M>/coverage-ledger/*.md is a ledger
  (conformance and coverage rungs, BRD names read from the per-BRD files);
  the reuse log/summary field names the report files actually read.
  test-review-conformance-per-brd.sh case 1b (6 checks).
- graph-sweep.sh: fixture for the "no orphan-read kinds" WARNING branch and
  its absence when an action edge exists (test-catalog-edge-guard.sh, 20).
- CHANGELOG: per-BRD reports reach docs/report.json via #310.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 13, 2026
@claude
Merge #307: graph-sweep/app-facts edge guard; per-BRD conformance led…
dffaf6f 
…gers (closes #259 #260; #313 follow-up)

Review: MERGE (two fix rounds + confirmation) — no-call models sweep again; per-BRD reports read by review/verify; BRD ids with spaces; verify stack rung exit 4 = FAULT.
MendixMau pushed a commit that referenced this pull request Oct 13, 2026
@claude
report: per-BRD conformance — any suffix, BRD-prefixed pointers, capt…
84dc6b6 
…ured golden

- pickConformanceFiles accepts report-<date>-<any suffix>.tsv: the
  narrower class dropped "report-<date>-F003 shipping.tsv" (a BRD id with
  a space, which #307's own fixture writes) and fell back to an older
  base file — the stale-report bug again.
- collectConformance strips a leading "<BRD>:" from the pointer, so a
  per-BRD row keeps requirement "/x" (was null for every per-BRD row);
  the BRD stays in the check id, so one pointer in two BRDs stays two.
- Golden: tests/wave2/fixtures/conformance-per-brd/ is conformance-check.sh's
  real output (PR #307, 8fc7c6a) on that PR's per-BRD fixture layout;
  case 3 of test-report-normalize-two-tree.sh reads it (fails on master
  and on the previous commit). report-schema.md: any suffix of the newest
  date is joined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 13, 2026
@claude
Merge #310: OQL rows by alias; liveness for negative-only spans; APP-…
a5235d6 
…DOWN; #302 credential fix; aborted reports; per-BRD conformance in report.json (closes #154 #150, refs #285 #286)

Review: MERGE (initial review + fix round + 3 confirmation rounds) — project-set DEFAULT_PASS runs again (urgent #302 regression fixed); normal report.json shape unchanged except 'aborted': null; per-BRD golden reproduced byte-for-byte from #307's conformance-check.
@MendixMau MendixMau mentioned this pull request Oct 13, 2026
Merged
3 of 4 tasks
MendixMau added a commit that referenced this pull request Oct 14, 2026
@MendixMau @claude
Integration 2026-10-14: 19 independently reviewed PRs + follow-ups (#329
31e90ae 
)

Every constituent PR was reviewed by a fresh agent that did not write it (CLAUDE.md "Merging a PR"); only MERGE verdicts joined. CI green on each constituent head and on this head (40814fc); full suites run on the branch.

Review lines (per merged PR):
- #315 context trim — MERGE; 360/385 removed lines found verbatim in their new homes, rest read by hand; baseline 79,997 -> 76,259.
- #316 independent-review + word-budget rules — MERGE-AFTER-FIX (3 rounds), fixes applied as specified.
- #318 page-fidelity displayed text — MERGE after fix round + confirmation.
- #317 interaction-model skill (#291 part 1) — MERGE after fix round + confirmation; no client names leaked.
- #321 PROOF-OF-LOOK per section; final-look not due before derived Stage 6 — MERGE after fix round + confirmation.
- #307 graph-sweep guard; per-BRD conformance ledgers — MERGE after two fix rounds + confirmation.
- #325 extraction-quality section; token-burn fixes — MERGE after fix round + confirmation.
- #310 e2e OQL by alias, negative-span liveness, APP-DOWN, #302 credential regression fix — MERGE after fix round + 3 confirmations.
- #323 inbox promotions — MERGE after fix round + confirmation.
- #308 lint gate BLIND on reverted stock rules — MERGE (FIRST-SEEN reverted) after two fix rounds + confirmation.
- #328 pipefail + grep -q flake — MERGE; 0/30 flakes vs 15/100 on master. Follow-up: run-all resolves bin/lib/ subjects (reviewed, MERGE).
- #319 test-stack-up command-line ownership proof — MERGE after fix round + 2 confirmations.
- #327 change-intake core + on-demand detail; CRLF-safe change log — MERGE after fix round + confirmation.
- #320 follow-ups to #290/#292; Dispatching split out — MERGE after fix round + confirmation.
- #326 status delivery rounds (existing-app mode) — MERGE after fix round + confirmation.
- #312 lint batch 4: SEC014, toolkit CONV009 — MERGE after review + 2 fix rounds + confirmation; field run on mxcli 0.26.
- #324 exec.sh change classes, shared stale-lock takeover, SEC014 note echo — MERGE (joint review); follow-up owed for a takeover race after a mid-takeover crash.
- #322 MxTest after-startup guard, mxcli-run pid check — MERGE (joint review).
- #314 doctor property-set exit code; sync announces its bound — MERGE (joint review).
- #330 review follow-ups (exit 4/5 docs, pipefail, NUL strip) — MERGE-AFTER-FIX, fixes applied; takeover re-check dropped.

Field runs are still owed for most instruments in this batch (stated in each PR).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau

MendixMau commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Landed on master as part of the integration squash #329 (commit 31e90ae), after independent review — its Review: line is in that commit message. Closing as superseded.

Generated by Claude Code

@MendixMau MendixMau closed this Oct 14, 2026
This was referenced Oct 14, 2026
Closed
Closed
@MendixMau
MendixMau deleted the fix/graph-sweep-conformance branch October 15, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

2 participants

@MendixMau @claude