Skip to content

process(testing): three check tiers — every exec, module close, release gate - #364

Closed
MendixMau wants to merge 2 commits into
masterfrom
process/test-tiers
Closed

MendixMau wants to merge 2 commits into
masterfrom
process/test-tiers

Conversation

@MendixMau

@MendixMau MendixMau commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner

Direct lane

What changed and why (one paragraph):
This follows a maintainer decision made after external reviewer feedback: only fast checks run during development, and heavy checks wait for module close or the release gate. skills/testing-shape.md §3 now holds a three-row check moments table, and it is the single source. The term was "tiers" until review, because §1 says "No tiers". The three moments are:

  • Every exec, about 1 minute. mxbuild dominates it at 22–65 s per script (iterative-build-loop.md → Gate cost).
  • Module close, minutes.
  • Release gate, as needed.

The every-exec row lists the chain that project-bin/exec.sh actually runs, checked against its header: lock → module-brief advisory → mxcli check → snapshot → apply → mxbuild delta gate → auto-restore on a new error → lint ratchet. The existing re-run event row "Project close / cutover" is renamed "Release gate (before UAT/production)". Its rule is unchanged (the project-wide suite, once), so the release gate has one name and one timing. The lanes and the false-green register are unchanged. Every check that is not built yet is marked planned: #N: #349, #351 and #348 for every exec; #354 and #352 for module close; #353, #352 and #354 (multi-language apps only) for the release gate. The runbook's Stage 6 paragraph gets one pointer line. iterative-build-loop.md already points at testing-shape.md §3, so it is unchanged.

Word budget: baseline 75,028 → 75,225 words (+197; budget 80,000). Per role:

  • Stage 5: gate 34,629 → 34,826, test 33,689 → 33,886, review 44,818 → 45,015.
  • Stage 6: gate 33,880 → 34,077, test 33,689 → 33,886, review 39,723 → 39,920.
  • Every other role grows +13, from the runbook line.

The addition fits inline.

Field evidence: none yet. This is a process/doc change and not an instrument.

  • No client data anywhere in the diff
  • Size cap: 3 files, well under 400 lines
  • Test tier reached: T0 (doc change). These checks pass: render-routing.sh --check, check-scripts, check-portability, the leak guard, private-citations and pr-discipline. Scoped fixtures pass: test-bug03-gates.sh 31/31, test-routing-tier.sh 12/12, test-baseline-pack.sh 16/16. The first revision also passed test-surface-column.sh 13/13, test-triage-report.sh 66/66 and test-guide-reopen.sh 10/10.
  • CHANGELOG.md line appended, credited "MendixMau (from external reviewer feedback)"

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw

…se gate

Maintainer decision: only fast checks run during development; heavy checks
run at module close or at the release gate. testing-shape.md §3 now holds
the tier table as the single source, beside the unchanged lanes and re-run
events. Unbuilt checks are marked (planned: #N): #348, #349 (PR #355), #351,
#352, #353, #354. The runbook's Stage 6 paragraph points at it in one line.

Baseline words 75,028 -> 75,261 (budget 80,000).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
…one release gate

- Rename "tiers" to "check moments" so it does not collide with §1's "No tiers".
- Every-exec row lists the real exec.sh chain (lock, module-brief advisory,
  mxcli check, snapshot, apply, mxbuild delta gate, auto-restore on a new
  error, lint ratchet), verified against project-bin/exec.sh's header; budget
  ~1 minute (mxbuild 22-65 s per script, iterative-build-loop.md Gate cost).
- The re-run event row "Project close / cutover" is renamed "Release gate
  (before UAT/production)", so the project-wide suite has one name and runs
  once; the moment row points at it.
- Drop the closing sentence that repeated the "felt wide" rule; shorter
  planned markers.

Baseline words 75,261 -> 75,225.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
…close, release gate

Review: independent review MERGE-AFTER-FIX — every-exec row incomplete and "seconds" wrong (mxbuild 22–65 s); release-gate row duplicated the cutover row (suite twice); "tiers" clashed with §1; one repeated sentence → chain listed per exec.sh with ~1-minute budget, event row renamed so the suite runs once, "check moments", trims, correction comments on #351–#354, in 4933b1c; confirmation MERGE: chain order verified against exec.sh line by line, rename breaks no reference. Baseline 75,028 → 75,225 (+197). From external reviewer feedback; maintainer-approved tiering.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau added a commit that referenced this pull request Oct 15, 2026
…journey controls, fixes) (#385)

Lands #340 #363 #362 #355 #370 #368 #364 #365 #366 #372 #367 #371 #376 #369 #373 #377 #378 #380.

Review: each constituent PR independently reviewed by a fresh agent that did not write it; merged into integration/2026-10-15 only on a MERGE verdict with green CI (per-PR Review lines in the integration merge commits). After-fix rounds: #373 (env-var cap, rename/skip-loops freshness), #377 (inline URI fallback), #378 (full-pack double count, slice content). Stub diet (#369/#378/#380) behaviour A/B, 5 reps x 2 drafting tasks + 5 retrieval probes x 3 reps: trimmed context no worse than master on every scored metric (planted STOP refused 20/20, parse failures page 2/5 vs 3/5, probes 15/15); mdl Stage 5 56,236 -> 30,647 words, baseline 75,028 -> 73,233. Full suites on c9bc073: run-tests 33/0, wave2 111 fixtures 0 failing. Field evidence: none yet for the instruments (#355 warn-only, #373/#377 read-only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau

Copy link
Copy Markdown
Owner Author

Landed on master in #385 (squash 868b3cb), after its own independent review on the night integration branch. Closing this PR.


Generated by Claude Code

@MendixMau MendixMau closed this Oct 15, 2026
@MendixMau
MendixMau deleted the process/test-tiers branch October 15, 2026 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants