diff --git a/CHANGELOG.md b/CHANGELOG.md index ff60ea83..3856221e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- new(journey-proof): **every journey names its claim and ships a control; `named-controls.js` proves the claim can fail** ([#159](https://github.com/MendixMau/mxcli-project-toolkit/issues/159)). `--positive-control` proves the harness can fail, not that the journey's own claim can: one journey passed every rung while its positive control proved 7 of 7, yet its claim held only by accident. New `project-tests/e2e/named-controls.js` (installed via `install-manifest.sh`) runs control files, named `*.control.json` so no real walk, `report-normalize.js` or `fixture-manifest.sh` picks them up as journeys (`"control": {"of", "mustFail"}`), through the runner and reads PROVEN only when the runner exits 1, every check named by `mustFail` is FAIL (at least one), and nothing else is FAIL or INVALID; a `persona` other than the run user is a FAULT and is not run. `journey-runner.js` gains `--out=` (creates its directory; control findings never overwrite the walk's) and a warn-only `WARN named control:` line for a journey with no `claim` or no `*.control.json` beside it (or in `controls/`), also written as `namedControlGaps`; no row, no exit-code change. `journey-proof.md` gains the rule and a checklist line. Fixture `tests/wave2/test-named-controls.sh` (24 cases, over the captured card-disbursement walk). Field evidence: the issue's card-disbursement build (5 of 5 named controls PROVEN after the fix, with the project's own copy); no field run of this toolkit port yet. — MendixMau (card-disbursement requirements-driven build) - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index f33a52b0..17a02a98 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -190,7 +190,7 @@ MXTK_LINT_RULES_NOINSTALL="README.md STOCK-HASHES.txt" # # project.config.template.js is listed separately because it is the one file a project is # EXPECTED to edit — installing it over an edited copy would silently revert the port. -MXTK_PROJECT_TESTS="config.js helpers.js settle.js otel.js journey-runner.js journey-runner.selftest.js journey-rung4-scope.test.js monkey.js monkey.selftest.js page-audit.js page-audit-rules.js design-audit.js full-app-walkthrough.js report-normalize.js report-render.js review-report.js example.journey.json" +MXTK_PROJECT_TESTS="config.js helpers.js settle.js otel.js journey-runner.js named-controls.js journey-runner.selftest.js journey-rung4-scope.test.js monkey.js monkey.selftest.js page-audit.js page-audit-rules.js design-audit.js full-app-walkthrough.js report-normalize.js report-render.js review-report.js example.journey.json" MXTK_PROJECT_TESTS_TEMPLATE="project.config.template.js" # Deliberately not installed. Same contract as the lists above. diff --git a/project-tests/e2e/journey-runner.js b/project-tests/e2e/journey-runner.js index a9eb64f3..39428d7d 100644 --- a/project-tests/e2e/journey-runner.js +++ b/project-tests/e2e/journey-runner.js @@ -51,11 +51,14 @@ const PROTECT_STATE = process.argv.includes('--protect-state'); // --assert-landing (or any journey with "assertLanding": true): the landing check after login. const ASSERT_LANDING = process.argv.includes('--assert-landing'); const files = process.argv.slice(2).filter(a => !a.startsWith('--')); +// --out=: write findings there instead of the default slot. named-controls.js uses it +// so a named control never overwrites the real walk's (or the positive control's) findings (#159). +const OUT_ARG = (process.argv.find(a => a.startsWith('--out=')) || '').slice(6); // `require.main === module` guard: tests/e2e/journey-rung4-scope.test.js requires this // file to exercise the rung-4 SQL builders directly. Without the guard, requiring it // would parse the TEST runner's argv, find no journey files and exit(3). if (require.main === module && !files.length) { - console.error('usage: node tests/e2e/journey-runner.js [--positive-control]'); + console.error('usage: node tests/e2e/journey-runner.js [--positive-control] [--out=]'); process.exit(3); } @@ -1337,11 +1340,12 @@ function flagAppDown(rows, detail) { // fixture rows without a running app. See tests/e2e/journey-rung4-scope.test.js. // runExitCode is exported for tests/wave2/test-journey-control-exit.sh, personaMismatch // for tests/wave2/test-journey-persona.sh, the preflight/ordering set for -// tests/wave2/test-journey-preflight.sh. +// tests/wave2/test-journey-preflight.sh, namedControlGaps/findControls for +// tests/wave2/test-named-controls.sh. module.exports = { sqlRowCount, sqlWatermark, whereScoped, scopeLiteral, sqlAssocTotal, sqlAssocLinked, sqlMustPointAt, captureScope, scopeEvidence, - runExitCode, personaMismatch, + runExitCode, personaMismatch, namedControlGaps, findControls, classifyAdminProbe, journeyNeedsAdmin, missingEnv, preflight, orderJourneys, stateTracker, judgeLanding, traceRung, controlMutants, forbiddenSample, probeApp, appGate, flagAppDown, LIVENESS_SKEW_MS, @@ -1371,7 +1375,7 @@ function armRelease(page, li) { // as this run — the same stale-slot defect the preflight abort already guards against. function writeAbortedFindings(reason, extra = {}) { try { - fs.mkdirSync(cfg.artifactsDir, { recursive: true }); + fs.mkdirSync(path.dirname(findingsFile()), { recursive: true }); // --out= may name a new dir fs.writeFileSync(findingsFile(), JSON.stringify({ capturedAt: new Date().toISOString(), aborted: reason, configuredUser: cfg.user, actualUser: extra.actualUser ?? null, usedFallback: !!extra.usedFallback, @@ -1390,9 +1394,49 @@ function armSignals() { } } -const findingsFile = () => path.join(cfg.artifactsDir, +const findingsFile = () => OUT_ARG ? path.resolve(OUT_ARG) : path.join(cfg.artifactsDir, POSITIVE_CONTROL ? 'journey-findings-control.json' : 'journey-findings.json'); +// ── Named controls: every journey names its claim and ships a control (#159) ─ +// The positive control proves the harness can fail; it cannot prove the claim a journey exists +// for. Field case (card-disbursement build, 2026-09-26): a journey passed every rung, its positive +// control proved every rung, and its claim ("a case waits while an upstream system is offline") +// was true only by accident; its named control read NOT PROVEN. So each journey declares +// `"claim"` and has a control file named `*.control.json` (`"control": {"of": , "mustFail": ...}`) +// beside it or in controls/ beside it, run by named-controls.js. The suffix is required: every real +// walk discovers journeys as `*.journey.json` (verify-module, report-normalize, fixture-manifest), +// so a control saved under that suffix would be walked, normalized and seeded as a real journey. WARN-ONLY until a field run: printed and written +// to the findings as `namedControlGaps`, never a row, never the exit code. A control file is not +// itself checked. +function namedControlGaps(journeys, controlOf) { + const gaps = []; + for (const j of journeys) { + if (j.control) continue; + if (!j.claim) gaps.push(`${j.id}: no "claim" — name the business claim this journey exists to prove`); + if (!controlOf.has(j.id)) gaps.push(`${j.id}: no named control — no *.control.json beside it (or in controls/) ` + + `with control.of = "${j.id}"; write one and run named-controls.js`); + } + return gaps; +} + +// The journey ids that have a `*.control.json` file in the given journey files' directories (and +// their controls/ subdirectories). Other names are never read as controls; unreadable JSON is +// skipped: this is a warning scan. +function findControls(journeyFiles) { + const of = new Set(); + const dirs = new Set(); + for (const f of journeyFiles) { const d = path.dirname(path.resolve(f)); dirs.add(d); dirs.add(path.join(d, 'controls')); } + for (const d of dirs) { + let names = []; + try { names = fs.readdirSync(d).filter(n => n.endsWith('.control.json')); } catch (_) { continue; } + for (const n of names) { + try { const c = JSON.parse(fs.readFileSync(path.join(d, n), 'utf8')).control; if (c && c.of) of.add(c.of); } catch (_) { /* not a journey */ } + } + } + return of; +} +let CONTROL_GAPS = []; + (async () => { const loaded = files.map(f => JSON.parse(fs.readFileSync(f, 'utf8'))); // Declared order first (requires/leaves/consumes); without headers this is file order. @@ -1407,6 +1451,8 @@ const findingsFile = () => path.join(cfg.artifactsDir, console.log('\n=== PREFLIGHT ==='); const pf = preflight(journeys); for (const l of pf.lines) console.log(` ${l}`); + CONTROL_GAPS = namedControlGaps(journeys, findControls(files)); + for (const g of CONTROL_GAPS) console.log(` WARN named control: ${g}`); results.push(...pf.rows); if (pf.abort !== null) { console.log('\n=== SUMMARY ==='); @@ -1605,11 +1651,12 @@ const findingsFile = () => path.join(cfg.artifactsDir, // These two files answer different questions ("does the app work" / "can this // harness tell when it doesn't") and must never share storage. const out = findingsFile(); + fs.mkdirSync(path.dirname(out), { recursive: true }); // --out= may name a new directory fs.writeFileSync(out, JSON.stringify({ capturedAt: new Date().toISOString(), configuredUser: cfg.user, actualUser: li.user, usedFallback: !!li.usedFallback, baseUrl: cfg.baseUrl, positiveControl: POSITIVE_CONTROL, mutants, results, walks, - appDown, appDiedIn, + appDown, appDiedIn, namedControlGaps: CONTROL_GAPS, }, null, 2)); console.log(` findings → ${path.relative(cfg.root, out)}`); diff --git a/project-tests/e2e/named-controls.js b/project-tests/e2e/named-controls.js new file mode 100644 index 00000000..65adee61 --- /dev/null +++ b/project-tests/e2e/named-controls.js @@ -0,0 +1,58 @@ +'use strict'; +// named-controls.js — prove each journey's OWN claim can fail (#159) +// node tests/e2e/named-controls.js journeys/*.control.json journeys/controls/*.control.json +// +// --positive-control proves the HARNESS can fail: it breaks generic rungs. It cannot prove the +// claim a journey exists for. A control file, named *.control.json (never *.journey.json, which +// real walks pick up), is a copy of its journey with ONE claim the app must not satisfy, plus +// "control": { "of": "J-01", "mustFail": "" }. +// PROVEN only when all three hold: (1) the runner exits 1; (2) every check whose name contains +// mustFail is FAIL, and there is at least one; (3) nothing else is FAIL or INVALID. Rule 3 is the +// point: a control that goes red somewhere else (a login fault, a moved landing) proves nothing +// about the named claim. The control's persona must equal the run's user (TEST_USER); a mismatch +// is a FAULT and the control does not run. Findings go to artifacts/control-.json via the +// runner's --out=, never over the real walk's file. Exit 0 only when every control is PROVEN. +// Field case (card-disbursement build, 2026-09-26): a journey passed every rung and its positive +// control proved every rung, yet its claim was true by accident; this control read NOT PROVEN. +const fs = require('fs'); +const path = require('path'); +const { spawnSync } = require('child_process'); + +function judge(exitCode, results, mustFail) { + if (exitCode !== 1) return { verdict: 'NOT PROVEN', why: `runner exited ${exitCode}, not 1` }; + const named = results.filter(r => String(r.name || '').includes(mustFail)); + if (!named.length) return { verdict: 'NOT PROVEN', why: `no check name contains "${mustFail}"` }; + const held = named.filter(r => r.verdict !== 'FAIL'); + if (held.length) return { verdict: 'NOT PROVEN', why: `named check not FAIL: ${held.map(r => `${r.name} = ${r.verdict}`).join('; ')}` }; + const other = results.filter(r => !named.includes(r) && (r.verdict === 'FAIL' || r.verdict === 'INVALID')); + if (other.length) return { verdict: 'NOT PROVEN', why: `red elsewhere: ${other.map(r => `${r.name} = ${r.verdict}`).join('; ')}` }; + return { verdict: 'PROVEN', why: `${named.length} named check(s) FAIL, nothing else red` }; +} + +function runControl(file, { user, artifactsDir, runner = path.join(__dirname, 'journey-runner.js') }) { + if (!file.endsWith('.control.json')) return { id: path.basename(file), verdict: 'FAULT', why: 'not named *.control.json; not run' }; + const j = JSON.parse(fs.readFileSync(file, 'utf8')); + const c = j.control || {}; + if (!c.of || !c.mustFail) return { id: j.id, verdict: 'FAULT', why: 'no control.of / control.mustFail' }; + if (j.persona !== user) return { id: j.id, of: c.of, verdict: 'FAULT', why: `persona "${j.persona}" is not the run user "${user}"; not run` }; + const out = path.join(artifactsDir, `control-${j.id}.json`); + fs.rmSync(out, { force: true }); // a stale file from an earlier run must not be read + const r = spawnSync(process.execPath, [runner, file, `--out=${out}`], { stdio: 'inherit' }); + if (!fs.existsSync(out)) return { id: j.id, of: c.of, verdict: 'NOT PROVEN', why: `runner exited ${r.status}; wrote no findings` }; + return { id: j.id, of: c.of, ...judge(r.status, JSON.parse(fs.readFileSync(out, 'utf8')).results || [], c.mustFail) }; +} + +module.exports = { judge, runControl }; + +if (require.main === module) { + const files = process.argv.slice(2).filter(a => !a.startsWith('--')); + if (!files.length) { console.error('usage: node tests/e2e/named-controls.js '); process.exit(3); } + const { cfg } = require(__dirname + '/config.js'); + fs.mkdirSync(cfg.artifactsDir, { recursive: true }); + const rows = files.map(f => runControl(f, { user: cfg.user, artifactsDir: cfg.artifactsDir })); + console.log('\n=== NAMED CONTROLS ==='); + for (const r of rows) console.log(` ${r.verdict.padEnd(10)} ${r.id}${r.of ? ` (of ${r.of})` : ''} — ${r.why}`); + const n = rows.filter(r => r.verdict === 'PROVEN').length; + console.log(` ${n} of ${rows.length} named controls PROVEN`); + process.exit(n === rows.length ? 0 : 1); +} diff --git a/skills/journey-proof.md b/skills/journey-proof.md index c15ce2f3..282c773f 100644 --- a/skills/journey-proof.md +++ b/skills/journey-proof.md @@ -243,6 +243,23 @@ the same vacuity bug one level up. (pilot: one transactional table went 50 → 57 rows). Never run controls against anything you are also using as a demo baseline, and never against production. +### Every journey names its claim and ships a control + +**Trigger:** you write or change a journey. Declare `"claim"` (the one business sentence the journey +exists to prove), then copy the journey to `.control.json` beside it, or in `controls/` beside +it, with one claim the app must not satisfy and `"control": { "of": "", "mustFail": +"" }`. Run `tests/e2e/named-controls.js `. A control is PROVEN only +when the runner exits 1, every check whose name contains `mustFail` is FAIL (at least one), and +nothing else is FAIL or INVALID. Its `persona` must equal `TEST_USER`, or it is a FAULT and does not +run. Never name a control `*.journey.json`: real walks, `report-normalize.js` and +`fixture-manifest.sh` discover journeys by that suffix, and would walk the control as a journey. +The seven mutants above prove the harness. Only a named control proves the claim. + +*Failure it prevents (2026-09-26, #159):* a journey claimed "a case waits while an upstream system is +offline". It passed every rung, and its positive control proved 7 of 7. Its named control read +`NOT PROVEN`, because the case was waiting for a different reason. `journey-runner.js` prints +`WARN named control:` for a journey with no `claim` or no control file (warn-only, no field run yet). + ### Control output goes in its own file `journey-findings.json` is a single slot. A control run used to overwrite the real walk in it — @@ -337,6 +354,7 @@ across runs, so a diff in the report means a diff in the project. - [ ] All five rungs asserted; none quietly absent. - [ ] `--positive-control` run, **all seven mutants proven** (every one of rungs 1–5 covered), results in their own file. +- [ ] Every journey has a `claim` and a named control, and `named-controls.js` reads PROVEN for each. - [ ] Zero `INVALID` in the real run — or each one explained as an instrument fault, not a feature. - [ ] Walk renders: every step has actions, a shot (or an honest placeholder), and its checks. - [ ] Ledger rows the journey exercises are marked, and modules with a ledger and no journey are diff --git a/tests/wave2/test-named-controls.sh b/tests/wave2/test-named-controls.sh new file mode 100755 index 00000000..08d04a6b --- /dev/null +++ b/tests/wave2/test-named-controls.sh @@ -0,0 +1,183 @@ +#!/usr/bin/env bash +# Usage: bash test-named-controls.sh [path-to-named-controls.js] +# +# Fixture for project-tests/e2e/named-controls.js and journey-runner.js's named-control +# warning and --out= (#159). The runner beside the subject is the one exercised. +# +# The defect (card-disbursement requirements-driven build, 2026-09-26): a journey passed every +# rung and its --positive-control proved every rung, while its claim was true only by accident. +# Its named control read NOT PROVEN. A named control is PROVEN only when the runner exits 1, +# every check whose name contains mustFail is FAIL (at least one), and nothing else is red. +# +# Golden input is CAPTURED: fixtures/journey-control/walk-run.json is that build's real walk +# (79 rows, all PASS), trimmed to rung/name/verdict. Each case derives a control run from it by +# editing single rows, the shape each control outcome takes in the field. No field run of +# named-controls.js against a live app has been made from the toolkit copy yet. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/named-controls.js" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FIX="$HERE/fixtures/journey-control" +[ -f "$FIX/walk-run.json" ] || { echo "FAIL — golden capture missing under $FIX"; exit 1; } +E2E="$(cd "$(dirname "$SUT")" && pwd)" +for f in journey-runner.js helpers.js settle.js otel.js config.js; do + [ -f "$E2E/$f" ] || { echo "FAIL — $f not beside $SUT"; exit 1; } +done +CFG="$E2E/project.config.template.js" +[ -f "$CFG" ] || CFG="$HERE/../../project-tests/e2e/project.config.template.js" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +# The runner and config resolve a project at require time: an .mpr two levels up. +WORK="$(mktemp -d "${TMPDIR:-/tmp}/named-controls.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT +mkdir -p "$WORK/tests/e2e" "$WORK/journeys/controls" "$WORK/stub"; : > "$WORK/Fixture.mpr" +cp "$SUT" "$E2E/journey-runner.js" "$E2E/helpers.js" "$E2E/settle.js" "$E2E/otel.js" "$E2E/config.js" "$WORK/tests/e2e/" +cp "$CFG" "$WORK/tests/e2e/project.config.js" + +echo "== judge(): the three PROVEN rules over the captured walk ==" +IFS= read -r -d '' JS <<'EOF' || true +const fix = process.argv[2]; +const N = require('./named-controls.js'); +const walk = require(fix + '/walk-run.json'); +const clone = () => JSON.parse(JSON.stringify(walk.results)); +const name = walk.results[walk.results.length - 1].name; // the claim this control targets +const say = (k, code, rows, mf = name) => { const v = N.judge(code, rows, mf); console.log(`${k} ${v.verdict} | ${v.why}`); }; +let r = clone(); r[r.length - 1].verdict = 'FAIL'; say('PROVEN', 1, r); +say('EXIT0', 0, clone()); // claim held: app satisfied it +r = clone(); r[r.length - 1].verdict = 'FAIL'; say('EXIT2', 2, r); +r = clone(); r[r.length - 1].verdict = 'FAIL'; say('NOMATCH', 1, r, 'no check is named this'); +r = clone(); r[r.length - 1].verdict = 'INVALID'; say('NAMEDINV', 1, r); +r = clone(); r[r.length - 1].verdict = 'FAIL'; r[0].verdict = 'FAIL'; say('ELSEFAIL', 1, r); +r = clone(); r[r.length - 1].verdict = 'FAIL'; r[0].verdict = 'INVALID'; say('ELSEINV', 1, r); +EOF +OUT="$(cd "$WORK/tests/e2e" && printf '%s' "$JS" | APP_PORT=1 node - "$FIX" 2>&1)" +line() { printf '%s\n' "$OUT" | sed -n "s/^$1 //p"; } +[ -n "$(line PROVEN)" ] || { echo "FAIL — harness could not require $SUT"; echo "$OUT" | tail -5; exit 1; } +case "$(line PROVEN)" in 'PROVEN |'*) ok "exit 1 + named check FAIL + nothing else red → PROVEN" ;; *) bad "proven control not PROVEN" "$(line PROVEN)" ;; esac +case "$(line EXIT0)" in 'NOT PROVEN |'*'exited 0'*) ok "runner exit 0 (app satisfied the claim) → NOT PROVEN" ;; *) bad "exit 0 accepted" "$(line EXIT0)" ;; esac +case "$(line EXIT2)" in 'NOT PROVEN |'*'exited 2'*) ok "runner exit 2 (preflight abort) → NOT PROVEN" ;; *) bad "exit 2 accepted" "$(line EXIT2)" ;; esac +case "$(line NOMATCH)" in 'NOT PROVEN |'*'no check name contains'*) ok "mustFail matching no check → NOT PROVEN" ;; *) bad "zero named checks accepted" "$(line NOMATCH)" ;; esac +case "$(line NAMEDINV)" in 'NOT PROVEN |'*'INVALID'*) ok "named check INVALID, not FAIL → NOT PROVEN" ;; *) bad "INVALID named check accepted" "$(line NAMEDINV)" ;; esac +case "$(line ELSEFAIL)" in 'NOT PROVEN |'*'red elsewhere'*) ok "rule 3: a FAIL elsewhere → NOT PROVEN" ;; *) bad "red elsewhere accepted" "$(line ELSEFAIL)" ;; esac +case "$(line ELSEINV)" in 'NOT PROVEN |'*'red elsewhere'*) ok "rule 3: an INVALID elsewhere → NOT PROVEN" ;; *) bad "INVALID elsewhere accepted" "$(line ELSEINV)" ;; esac + +echo "== runControl(): persona gate, --out= handoff, stale file ==" +# A stub runner stands in for journey-runner.js: it writes the captured walk with the last row +# FAILed to the --out= path, and exits 1. STUB_MODE=nofile exits 1 and writes nothing. +cat > "$WORK/stub/runner.js" <<'EOF' +const fs = require('fs'); +const out = (process.argv.find(a => a.startsWith('--out=')) || '').slice(6); +fs.appendFileSync(process.env.STUB_ARGV, process.argv.slice(2).join(' ') + '\n'); +if (process.env.STUB_MODE !== 'nofile') { + const w = JSON.parse(fs.readFileSync(process.env.WALK, 'utf8')); + w.results[w.results.length - 1].verdict = 'FAIL'; + fs.writeFileSync(out, JSON.stringify(w)); +} +process.exit(1); +EOF +LAST="$(node -e 'const w=require(process.argv[1]); console.log(w.results[w.results.length-1].name)' "$FIX/walk-run.json")" +mk() { node -e 'const [f,id,p,mf]=process.argv.slice(1); const j={id,title:"control",steps:[{name:"s"}],control:{of:"J-01",mustFail:mf}}; if(p!=="-") j.persona=p; require("fs").writeFileSync(f,JSON.stringify(j))' "$@"; } +mk "$WORK/journeys/controls/C-ok.control.json" C-ok demo_user "$LAST" +mk "$WORK/journeys/controls/C-who.control.json" C-who Reviewer "$LAST" +# Misnamed controls: a control saved as *.journey.json beside its journey, and a bare .json in +# controls/. Neither may count as J-02's / J-03's control, and named-controls.js must refuse it. +mk "$WORK/journeys/C-bad.journey.json" C-bad demo_user "$LAST"; node -e 'const f=process.argv[1],j=require(f);j.control.of="J-02";require("fs").writeFileSync(f,JSON.stringify(j))' "$WORK/journeys/C-bad.journey.json" +mk "$WORK/journeys/controls/C-bare.json" C-bare demo_user "$LAST"; node -e 'const f=process.argv[1],j=require(f);j.control.of="J-03";require("fs").writeFileSync(f,JSON.stringify(j))' "$WORK/journeys/controls/C-bare.json" +IFS= read -r -d '' JS2 <<'EOF' || true +const [runner, art, dir] = process.argv.slice(2); +const N = require('./named-controls.js'); +const fs = require('fs'); +const show = (k, r) => console.log(`${k} ${r.verdict} | ${r.why}`); +show('RUN', N.runControl(dir + '/C-ok.control.json', { user: 'demo_user', artifactsDir: art, runner })); +console.log('OUTFILE ' + fs.existsSync(art + '/control-C-ok.json')); +show('WHO', N.runControl(dir + '/C-who.control.json', { user: 'demo_user', artifactsDir: art, runner })); +fs.writeFileSync(art + '/control-C-ok.json', JSON.stringify({ results: [] })); // stale file +process.env.STUB_MODE = 'nofile'; +show('STALE', N.runControl(dir + '/C-ok.control.json', { user: 'demo_user', artifactsDir: art, runner })); +delete process.env.STUB_MODE; +show('NAME', N.runControl(dir + '/../C-bad.journey.json', { user: 'demo_user', artifactsDir: art, runner })); +EOF +mkdir -p "$WORK/art"; : > "$WORK/argv" +OUT="$(cd "$WORK/tests/e2e" && printf '%s' "$JS2" | APP_PORT=1 WALK="$FIX/walk-run.json" STUB_ARGV="$WORK/argv" \ + node - "$WORK/stub/runner.js" "$WORK/art" "$WORK/journeys/controls" 2>&1)" +case "$(line RUN)" in 'PROVEN |'*) ok "spawned runner exit 1 + findings at --out= → PROVEN" ;; *) bad "runControl did not prove a proven control" "$OUT" ;; esac +grep -q -- "--out=$WORK/art/control-C-ok.json" "$WORK/argv" && ok "runner is passed --out=/control-.json" \ + || bad "--out= not passed as control-.json" "$(cat "$WORK/argv")" +case "$(line WHO)" in 'FAULT |'*'not run'*) ok "persona ≠ run user → FAULT, runner not spawned" ;; *) bad "persona mismatch not a FAULT" "$(line WHO)" ;; esac +[ "$(grep -c 'C-who' "$WORK/argv")" -eq 0 ] || bad "runner was spawned for a persona-mismatched control" +case "$(line STALE)" in 'NOT PROVEN |'*'wrote no findings'*) ok "a stale findings file from an earlier run is not read" ;; *) bad "stale findings file read" "$(line STALE)" ;; esac +case "$(line NAME)" in 'FAULT |'*'not named *.control.json'*) ok "a control not named *.control.json → FAULT, not run" ;; *) bad "misnamed control was run" "$(line NAME)" ;; esac +[ "$(grep -c 'C-bad' "$WORK/argv")" -eq 0 ] || bad "runner was spawned for a misnamed control" + +echo "== journey-runner.js: named-control gaps (warn-only) ==" +IFS= read -r -d '' JS3 <<'EOF' || true +const dir = process.argv[2]; +const R = require('./journey-runner.js'); +if (typeof R.namedControlGaps !== 'function') { console.log('NOFN'); process.exit(0); } +const g = R.namedControlGaps([ + { id: 'J-01', claim: 'a case waits while upstream is offline' }, + { id: 'J-02' }, + { id: 'J-03', claim: 'x' }, + { id: 'C-ok', control: { of: 'J-01', mustFail: 'x' } }, +], R.findControls([dir + '/J-01.journey.json'])); +console.log('GAPS ' + JSON.stringify(g)); +EOF +OUT="$(cd "$WORK/tests/e2e" && printf '%s' "$JS3" | APP_PORT=1 node - "$WORK/journeys" 2>&1)" +if printf '%s\n' "$OUT" | grep -qx NOFN; then bad "journey-runner.js exports no namedControlGaps"; else + G="$(line GAPS)" + case "$G" in *'J-01'*) bad "J-01 has a claim and a control in controls/, still flagged" "$G" ;; *) ok "claim + control in controls/ → no gap" ;; esac + case "$G" in *'J-02: no \"claim\"'*'J-02: no named control'*) ok "no claim and no control → both named" ;; *) bad "J-02 gaps missing" "$G" ;; esac + case "$G" in *'J-03: no \"claim\"'*) bad "J-03 has a claim, flagged anyway" "$G" ;; *'J-03: no named control'*) ok "claim without control → control gap only" ;; *) bad "J-03 control gap missing" "$G" ;; esac + case "$G" in *'C-ok'*) bad "a control file was itself checked" "$G" ;; *) ok "control files are not themselves checked" ;; esac + case "$G" in *'J-02: no named control'*) ok "a control saved as *.journey.json does not count as one" ;; *) bad "misnamed *.journey.json control counted" "$G" ;; esac + case "$G" in *'J-03: no named control'*) ok "a bare .json in controls/ does not count as one" ;; *) bad "bare .json in controls/ counted" "$G" ;; esac +fi + +echo "== a *.control.json is NOT discovered as a real journey ==" +# Every real-walk discovery point selects journeys by the *.journey.json suffix. Assert each one +# still does (by inspection of the shipped source), then apply those exact patterns to a journeys/ +# dir holding J-01.journey.json and J-01.control.json: only the journey may be selected. +D="$WORK/disc"; mkdir -p "$D/journeys" +node -e 'require("fs").writeFileSync(process.argv[1], JSON.stringify({id:"J-01",steps:[{name:"s"}]}))' "$D/journeys/J-01.journey.json" +mk "$D/journeys/J-01.control.json" C-01 demo_user "$LAST" +TK="$(cd "$E2E/../.." && pwd)" +RN="$E2E/report-normalize.js"; FM="$TK/project-bin/fixture-manifest.sh"; VM="$TK/project-bin/verify-module.sh" +if [ -f "$RN" ] && [ -f "$FM" ] && [ -f "$VM" ]; then + grep -qF "f.endsWith('.journey.json')" "$RN" && grep -qF '"*.journey.json"' "$FM" && grep -qF '$MODULE.journey.json' "$VM" \ + && ok "report-normalize, fixture-manifest and verify-module select journeys by *.journey.json" \ + || bad "a discovery point no longer selects by *.journey.json — re-check control naming" + NODEPICK="$(node -e 'console.log(require("fs").readdirSync(process.argv[1]).filter(f => f.endsWith(".journey.json")).sort().join(" "))' "$D/journeys")" + GLOBPICK="$(cd "$D/journeys" && ls -1 *.journey.json | paste -sd' ' -)" + [ "$NODEPICK" = "J-01.journey.json" ] && [ "$GLOBPICK" = "J-01.journey.json" ] \ + && ok "J-01.control.json beside J-01.journey.json is not picked up as a journey" \ + || bad "control file discovered as a journey" "node: $NODEPICK / glob: $GLOBPICK" +else + echo " skip — discovery sources not found under $TK (subject copied out of the toolkit)" +fi + +echo "== real runner: --out= and the WARN line, through a preflight abort ==" +# TEST_PASS unset aborts in preflight before any login or probe (#278) and writes aborted +# findings, so this exercises --out= and the warning without an app. +node -e 'require("fs").writeFileSync(process.argv[1], JSON.stringify({id:"J-02",steps:[{name:"s"}]}))' "$WORK/journeys/J-02.journey.json" +E2EOUT="$(cd "$WORK/tests/e2e" && env -u TEST_PASS -u APP_PASS -u ALLOW_CONFIG_CREDENTIALS APP_PORT=1 \ + node journey-runner.js "$WORK/journeys/J-02.journey.json" "--out=$WORK/art/new/deeper/custom.json" 2>&1)"; RC=$? +[ "$RC" -eq 2 ] && ok "preflight abort exit 2 (fixture precondition)" || bad "expected preflight exit 2" "rc=$RC $(printf '%s' "$E2EOUT" | tail -3)" +[ -f "$WORK/art/new/deeper/custom.json" ] && ok "findings written to an --out= path whose directories did not exist" || bad "no findings at --out= path (parent dir not created?)" "$(printf '%s' "$E2EOUT" | grep findings)" +[ ! -f "$WORK/tests/e2e/artifacts/journey-findings.json" ] && ok "default slot untouched under --out=" || bad "default slot written under --out=" +[ "$(printf '%s\n' "$E2EOUT" | grep -c 'WARN named control: J-02')" -eq 2 ] && ok "two WARN lines for J-02 (no claim, no control)" \ + || bad "WARN lines missing" "$(printf '%s\n' "$E2EOUT" | grep -i 'named control')" + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ]