diff --git a/CHANGELOG.md b/CHANGELOG.md index 21d46a3c..59bca1b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,10 +13,13 @@ days the maintainer cuts a release with `bin/cut-release.sh`: the `## Unreleased top section is always "what changed since the last release", a project reports the release it is on (`bin/sync-project.sh` prints `Toolkit release: v2026.09.22` — or `v2026.09.22-3-g` when three commits past it), and a bug report can name a release instead of a sha nobody can place. -Sections dated before 2026-09-19 predate the cycle and stay as they are. +Sections dated before 2026-09-19 predate the cycle and stay as they are. Named milestones +(1.0, 1.1, 2.0) sit on top of this train; what a milestone promises is defined in +`docs/RELEASE-PROCESS.md` §8. ## Unreleased +- process(docs): **1.0 release notes, a versioning rule, and a README for adopters.** New `docs/releases/v1.0.md`: the first named milestone, covering 2026-09-30 to 2026-10-14 by theme (independent review before merge, baseline words 79,997 → 76,259, exec.sh lock and restart verdicts, lint batches 1–4 with SEC014 and the stock-revert guard, journey/e2e fixes, the Stage 3 interaction model, delivery rounds), with upgrade steps, known gaps and credits. `docs/RELEASE-PROCESS.md` §8 is the single source for versioning: a milestone (1.0, 1.1, 2.0) is a train release the maintainer also names, with curated notes, a GitHub Release and a CHANGELOG pointer; major = a consuming project breaks without action (needs a migration section and a sync path), minor = no action needed; deprecations are announced in one milestone and removed no earlier than the next major; the `vYYYY.MM.DD` train and `bin/cut-release.sh` are unchanged; plus how a consumer checks, upgrades and pins a version. The header above now defers to it. README rewritten for newcomers: what it is, how it works, ways to use it, quick start, Make it yours (company brain, CLAUDE.local.md, lint opt-outs, waivers and --adopt, interview mode, exec approval and permission profiles, agent stubs, other harnesses), testing, a call for input, releases. The earlier material sits under Reference with the generated routing blocks untouched. Corrected: the Windows `pgrep` hazard (replaced by the per-project exec lock in #304), the hand-edit steps in "How to add a new skill" (tables are generated), and the stale Stage 6 and `examples/` notes. — MendixMau - fix(interview-mode): **`attended` and `unattended` are now read as aliases for `steering` and `auto` — a project that opted into unattended as documented was silently run as steering.** CLAUDE.md, the runbook and intake Q9 taught `Interview mode: unattended`, but `bin/interview-mode.sh` knew only `steering | assist | auto` and fell back to steering. The aliases are accepted in every source (env, `.interview-mode`, `PROJECT.md`, `--set`) under the same parse rules, with one stderr note. The `PROJECT.md` value is read after stripping bold/backticks: a bare mode or alias (`**auto**`, `` `auto` ``, `auto.`) resolves, a note may follow `steering`, but `assist`/`auto` with any trailing text falls back to steering with a "must stand alone on the line" message — so an annotated or hedged line never resolves quieter than before (two hedge word lists were tried in review and both leaked); env, session file and `--set` stay exact. Output stays canonical so `interview-gate.sh` is unchanged, and any other value still falls back to steering. Docs now name the canonical values (vocabulary in `interview-protocol.md`); intake Q9's body offers the three modes with steering the default (heading kept: it is sync's locator), and `sync-project.sh` reads an earlier unanswered Q9 body as unanswered rather than "edited". `test-interview-mode.sh` +57 checks (19 → 76) — MendixMau (found in the 1.0 README review) - fix(dispatching, sync-project, obligation-check): **review follow-ups for the 2026-10-14 integration batch.** `dispatching.md` names app-restart exits 4 and 5 beside 3 and 1. `test-test-scratch.sh` drops `ls | grep -q` under pipefail for here-strings; sync step 4d no longer aborts on empty doctor output; `_ob_has_denominator` strips NUL bytes so bash stops warning. — integration 2026-10-14 reviews - feat(lint): **Lint Batch 4 — SEC014 external role can write state, audit or ownership members; CONV009 reads its `maxObjects` option.** SEC014 ([#268](https://github.com/MendixMau/mxcli-project-toolkit/issues/268)): a partner role with `create, read *, write *` on self-owned entities passed every stock rule, so it could set its own record's status through the client API. Lint showed only CONV006 +4, which was re-baselined, and a manual review found it five days later. The new rule flags an external role (named in `lint-config.yaml` as `external_roles: "A, B"`, or `"none"`) that writes all members, a `Status`/`State`/`Result`/`Stage`/`Phase` enumeration, an `…At`/`…By` attribute, or the association its own XPath uses for ownership. It is a warning in the security tier. A `_rule` finding marks the rule blind when the option is blank or given as a YAML list (mxcli passes a list as `None`), when a role name does not exist, or when the catalog is not FULL. Each message is under 160 characters and names the place to fix it, because lint-gate prints only that much. An **absent** option is not a `_rule` finding, so it does not block every exec (the #229 pattern). Instead `bin/sync-project.sh` prints a NOTE on every sync through new `bin/lib/sec014-notice.sh` until the option is set. It is a NOTE rather than a counted warning, so a pristine scaffold still syncs clean under `--strict` (fixture `tests/wave2/test-sec014-notice.sh`, 14 cases; `test-bug12-sync.sh` 61/61). `"none"` switches a security rule off, so it is set only on the user's answer in chat and recorded in `PROJECT.md`. Right after configuring, re-baseline with `lint-gate.sh --update-baseline --reason "SEC014 configured" --approved-in-chat`, because the 0 → N jump would otherwise be blamed on the next script. CONV009 ([#267](https://github.com/MendixMau/mxcli-project-toolkit/issues/267)): the stock rule hard-codes 15, so the `maxObjects: 20` example in the config `mxcli init` writes did nothing. The toolkit copy reads `maxObjects` and `countLoopBodies`, with the **default kept at 15** (raising it is a maintainer call). It replaces the stock file through `STOCK-HASHES.txt`, which gains conv009 hashes for v0.17.0/0.24.0/0.25.0 and a full v0.26.0 set, captured by `mxcli init` from the official release assets; `test-stock-hashes.sh` T4 passes live on all four. `lint-that-actually-runs.md` now says `options:` reaches only rules that call `get_option()`. 32 cases in `tests/lint-rules/test-batch4-rules.sh`; the harness gained `get_option`, `permissions_for`, `attributes_for`, `user_roles` and `role_mappings`. Field run (official mxcli 0.26.0, two model copies): stock CONV009 stayed at 5 with `maxObjects: 20`, while the toolkit copy gave 5 at 15, 10 at 10 and 6 at 20 with loop bodies counted. SEC014 gave 1 true finding with the model's external role configured, and 7 with every role configured, matched by a SQL replay. Both rules load on 0.17.0 and 0.24.0 too. CONV009's "every flow reports 0 objects" self-check now needs `activities_for()` to find a real object in a zero-count flow, so an early greenfield model with empty microflows is not reported blind. `lint-rules/README.md` and the TXT002 header now give the supported way to turn a toolkit rule off. ISO dates in particular are a toolkit opinion, so a project with a non-ISO display standard sets `rules: TXT002: enabled: false` in `lint-config.yaml` and records that in `PROJECT.md`. It must not delete the file, which sync would re-install. On mxcli 0.26.0, a rule with 1 finding gave 0 once disabled. After #308, there is one opt-out story, always the user's decision (asked in chat, recorded in `PROJECT.md`), and it never deletes a `.star` file. `enabled: false` in `lint-config.yaml` stops a rule reporting; this is preferred. `.claude/lint-rules-disabled.txt` with a `# reason:` stops sync installing a toolkit rule, and refuses stock-seeded rules, a list that now includes conv009. `rules: SEC014: enabled: false` is the same decision as `external_roles: "none"`. While SEC014 is undecided, `project-bin/lint-gate.sh` also prints one uncounted `note: SEC014 not configured …` line on stderr on every run (no exit-code change, no lint row). Both it and the sync NOTE now count `external_roles` only when it sits under the `SEC014` key. `test-sec014-notice.sh` has 24 cases. — MendixMau diff --git a/README.md b/README.md index 5ebc1e73..b4f44cbd 100644 --- a/README.md +++ b/README.md @@ -2,99 +2,192 @@ > **Agents: this README is orientation, not the spec.** For any pipeline work, the executable authority is `skills/conversion-runbook.md` (stages, gates, interview protocol) plus each stage's owning skill — load those before producing anything. Stage summaries here and in `toolkit-guide.html` are routing, not deliverable lists. -Shared skills, prompt templates, and learnings for **Mendix migration and development projects**. +## What it is + +A shared process layer for building and changing **Mendix apps with an AI coding agent** and +[mxcli](https://github.com/mendixlabs/mxcli). You clone it once; each of your projects points at +it. It gives the agent a staged pipeline, a single place to record decisions, skills it loads +only when a task calls for them, and mechanical checks that report what is actually in your +project rather than what the agent believes is there. + +It is for Mendix developers and leads who already use (or want to use) an agent such as Claude +Code to migrate a legacy app, build from requirements, or change a live app, and who want the +result to be reviewable: every decision written down, every stage gated, every rule backed by a +real incident. + +## How it works + +- **Stages and gates.** Work moves through the same stages for every project: kickoff (P), + triage (0), analysis (1), requirements (2), architecture and design (3), build plan (4), build + (5), test (6), and cutover (7, migrations only). Your entry mode decides which of them run + (next section). Hard gates stop until you answer. Detail: `skills/conversion-runbook.md`. +- **One decision register.** Every gate question is asked in chat, and the answer lands in your + project's `PROJECT.md` as `CONFIRMED` or `ASSUMED`. `ASSUMED` means you said "you decide", never + that the question was skipped (`skills/interview-protocol.md`). +- **Skills on demand.** `skills/*.md` hold the method: how to triage a source, write a BRD, draw a + module boundary, build a page, review a module. The agent loads a skill when its trigger + happens, not all at once; the routing tables under [Reference](#reference) say which trigger + loads which file. A small always-on set is kept under a word budget + (`bin/render-routing.sh --check`). +- **Mechanical gates.** `bin/gate-check.sh ` checks the artifacts a stage owes, + the per-module passes nobody may skip (LOOK, wiring sweep, journeys), and that every source + file was used. It reports `PASS` / `PENDING` / `FAIL` / `WAIVED` / `MANUAL` and regenerates the + project dashboard (`index.html`). `bin/status.sh` gives the one next action. +- **Instruments in your project.** `bin/init-project.sh` copies `project-bin/` into your project + as `bin/`: `exec.sh` (every model write: snapshot, mxbuild gate, lint gate, restore on failure), + `verify-module.sh`, `page-fidelity.js`, `lint-gate.sh` and the rest. `bin/sync-project.sh` + keeps the copies you did not edit current. -Serves five ways in — four pipeline entry modes that share the same stages, plus one route with no pipeline at all (see `skills/conversion-runbook.md` "Entry Modes"; the mode is a **confirmed Stage-P decision**, never silently inferred — if source code exists it gets analyzed, if specs exist stages 2–4 run, and greenfield is only for starting from a conversation): +``` +P kickoff → 0 triage ✋ → 1 analysis → 2 requirements → 3 architecture & design ✋ + → 4 build plan ✋ → 5 build (module by module, gated) → 6 test → 7 cutover ✋ +``` -**Which scripts run at each step of each walk:** `docs/pipeline-walks.html` — one process diagram per entry mode (plus the shared spine and the Stage 5 module loop), each with a stage-by-stage table of the commands executed. A browser page, like the guide. +`✋` is a hard gate. Nothing before Stage 5 touches the model: a wrong module boundary is cheaper +to fix in a diagram than after 40 MDL scripts assume it. Each entry mode's walk, with the +commands that run at every step: `docs/pipeline-walks.html`. -- **Migrations** (legacy source code) — all stages. -- **Requirements-driven builds** (specs/BRDs/SME input, no legacy code) — stages 1–6; document discovery replaces source triage, extraction Path B/C replaces code extractors. -- **Greenfield mxcli builds** — Stage 5 onward; the standard Mendix build discipline is not migration-specific. -- **Changing an existing app** (a live `.mpr`, a feature or flow being added or altered) — stages P, 0–6 per change; the knowledge base is queried from the model itself (Path D), a regression net goes under the app first, and Stage 7 is N/A because the app never stops being live. `skills/existing-app-change.md`. -- **Existing apps — à la carte, no pipeline** — audit, lint, or put a regression/e2e test net under a Mendix app you already have. No intake, no stages, no gates: start at `skills/existing-app-assurance.md` and grab only the tools you need. +## Ways to use it -Used across all mxcli-powered projects — OS migrations, Java/Angular migrations, Node/Express+React migrations, and other client integration work. +Four pipeline entry modes share the same stages, plus one route with no pipeline at all. The +agent proposes the mode at kickoff with evidence and you confirm it; it is never silently +inferred (`skills/conversion-runbook.md` → "Entry Modes"). ---- +| Mode | Pick it when | Stages | Start from | +|---|---|---|---| +| **Migration** | Legacy source code exists (OutSystems, Java/Angular, Node/React, or another stack) | P, 0–7 | `bin/init-project.sh`, then `skills/conversion-runbook.md`; extractors in `pipelines/` | +| **Requirements-driven** | Specs, BRDs, workshop output or wireframes, no legacy code | P, 0–6 | `bin/init-project.sh`; document discovery replaces code extraction | +| **Greenfield** | Starting from a conversation or a plan you already have | P (light), 0 (scope), 5–6 | `bin/init-project.sh` | +| **Change an existing app** | A live `.mpr` you are adding to or altering, the rest stays | P, 0–6 per change | `bin/init-project.sh`, then `skills/existing-app-change.md` | +| **À la carte assurance** | Audit, lint, or put a regression/e2e net under an app you have, no pipeline | none | `skills/existing-app-assurance.md` | -## Quickstart +## Quick start ```bash git clone https://github.com/MendixMau/mxcli-project-toolkit.git ~/Mendix/mxcli-project-toolkit -~/Mendix/mxcli-project-toolkit/bin/doctor.sh # <- run this first, on any platform -``` - -`doctor.sh` probes the machine once and says in plain language what is missing — bash, a Python 3, -sqlite3, line endings, Studio Pro — and exits non-zero if something will break a pipeline stage. -It takes a second and it replaces every "the toolkit is broken" that is really a missing -prerequisite. **Windows and Linux users: do not skip it.** See *Platform support* below. - -When the mxbuild toolchain is what's missing, doctor can also fix it instead of only reporting it: - -```bash -~/Mendix/mxcli-project-toolkit/bin/doctor.sh --install +~/Mendix/mxcli-project-toolkit/bin/doctor.sh # what is missing on this machine +~/Mendix/mxcli-project-toolkit/bin/init-project.sh # one command per project ``` -first fetches a missing project `./mxcli` (right OS/arch build from the mendixlabs/mxcli -releases; `MXCLI_VERSION=vX.Y.Z` pins it), then runs `./mxcli setup mxbuild` — the same download -the headless container build uses — caching the version-matched mxbuild under `~/.mxcli/mxbuild/`. -Discovery (and therefore `exec.sh`'s mxbuild gate) finds that cache automatically, so a machine -without Studio Pro still verifies every model write. - -It never downloads silently: it prints the plan first — what, from where, how big (~90 MB + -~800 MB one-time), why, and the OS/arch it detected — then asks `[y/N]` at the terminal. -Unattended runs (agents, CI) must pass `--yes` explicitly; without it nothing is downloaded and -the report simply shows what is missing. Declining leaves you the URLs to fetch by hand — the -right move on a wrong OS/arch guess or when company policy routes binaries through an approved -channel. - -This clone stays clean — project output never lands inside it. **Everything else lives inside one project folder** (usually one git repo — it is the session root, the workspace root, and the mxcli target all at once): +1. **`doctor.sh`** names any missing prerequisite (bash, Python 3, sqlite3, line endings, mxcli, + mxbuild) and exits non-zero if a stage would break. `doctor.sh --install ` fetches + mxcli and mxbuild after showing the plan and asking. Windows and Linux: read + [Platform support](#platform-support) first. +2. **`init-project.sh`** scaffolds the project folder: `intake.md`, `PROJECT.md`, + `CLAUDE.local.md` (the wiring every session reads), the six agent stubs, `bin/` instruments + and the `index.html` dashboard. It is idempotent and never overwrites. Optional on Claude + Code: copy `commands/toolkit-init.md` to `~/.claude/commands/` and type `/toolkit-init` in any + project instead. +3. **The visual guide.** `toolkit-guide.html` walks the whole journey in a browser. The scaffold + opens it once per project; agents follow the first-touch rule in `CLAUDE.md`. +4. **Open your agent in the project folder and say where you are starting from**, for example: + *"Start the toolkit pipeline. I have the legacy source in `source/`."* or *"I want to add an + approval flow to this app."* The agent reads the runbook, proposes an entry mode and starts + the kickoff interview. In later sessions, `bin/status.sh` is the first command. +5. **Upgrade** with `git -C ~/Mendix/mxcli-project-toolkit pull`, then + `bin/sync-project.sh ` for each project. Skills update with the pull; sync + refreshes what was copied into the project and prints the `Toolkit release:` you are on once + release tags are published (from 1.0 on), or `none tagged yet` before that. + +Everything a project produces lives inside its own folder; this clone stays clean +([Consuming this toolkit](#consuming-this-toolkit)). + +## Make it yours + +The toolkit is opinionated, and every opinion has a supported way to adjust it that survives +`git pull` + `sync-project.sh`. Use these instead of editing toolkit files in place, which is the +drift a fork gets. + +| What you want | How | Where it lives | +|---|---|---| +| **House conventions, naming, lint rules, MDL snippets, approved components** for every project in your company | A private *company brain* repo: `bin/init-company-brain.sh ` creates it from the template; `bin/init-project.sh --company ` (or `bin/wire-company-brain.sh`) wires a project to it. The public toolkit never cites it | `templates/company-brain/`, `skills/company-brain.md` | +| **Project-specific instructions** | Add your own sections to the project's `CLAUDE.local.md`. Sync rewrites only its generated blocks (baseline routing, session ritual) and keeps the rest | `/CLAUDE.local.md` | +| **A lint rule tuned or off** | Rule options and `enabled: false` under `rules:` in the project's lint config; for a toolkit rule you never want installed, a line in `.claude/lint-rules-disabled.txt` with a `# reason:`. Either is your decision, recorded in `PROJECT.md`; never delete the `.star` file (sync reinstalls it) | `/.claude/lint-config.yaml`, `lint-rules/README.md` | +| **Skip a stage or pass on purpose** | Record it instead of leaving it red: `bin/gate-check.sh --adopt --reason "..."` for a project that joined mid-flight, `--waive ` or `--waive look/` for one deliberate skip. Each lands as a register line | `PROJECT.md`; `skills/conversion-runbook.md` §2 | +| **Fewer questions, or more** | Interview mode `steering` (default: every consequential question is asked), `assist` (batched at gates, low-stakes ones assumed and reported) or `auto` (nothing blocks, every position recorded). Set an `Interview mode:` line in `PROJECT.md`, or `bin/interview-mode.sh --set ` (kept in `.claude/.interview-mode` until `--clear`); `$CLAUDE_INTERVIEW_MODE` overrides for one session | `skills/interview-protocol.md` | +| **No prompt before every model write** | `bin/exec-approval.sh` decides whether the agent asks before an exec; `bin/permission-profile.sh` (`wrappers` / `project` / `full`, picked at intake) decides what the harness allows without prompting | intake Q12; `bin/doctor.sh` shows the current profile | +| **Your own team roles** | The six agent stubs (ba, architect, mdl, gate, test, review) are yours to complete and edit; sync refreshes only stubs you never touched | `/.claude/agents/`, `skills/agent-roles.md` | +| **A different agent** | Claude Code is the primary harness. `bin/wire-agents.sh` writes pointer files for Copilot, Cursor, Continue, Windsurf and Aider (and Kiro with `--with-kiro`) so every agent starts from the same wiring; `bin/install-harness-permissions.sh` sets each one's command allow-list where it has one | `toolkit-guide.html` → section 9 | +| **A source stack with no extractor** | Build one with the scored quality loop and copy a worked pipeline | `skills/extractor-quality-loop.md`, `pipelines/` | + +If an opinion is wrong for everyone, not just you, change it upstream: see +[We need your input](#we-need-your-input). + +## Testing and quality + +**What the toolkit tests about itself.** + +- `tests/run-tests.sh`: the guard scripts, each proven to fail on a case it must catch. +- `tests/wave2/`: about a hundred fixtures, one per instrument or bug, each taking its subject as + an argument (read its `usage:` line); `tests/lint-rules/` covers every toolkit lint rule. +- CI (`.github/workflows/checks.yml`) runs on every PR: script parsing, routing render and word + budget, portability, the client-data and private-citation guards, PR discipline, and both + fixture suites. +- **Field proof before merge.** A new or changed instrument needs captured (never hand-written) + input, both checkout layouts, both platforms and one cited run on a real project (`CLAUDE.md` → + "Shipping an instrument"). +- **Independent review.** Every PR is reviewed by someone who did not write it before it merges. + Test tiers and the triage rubric: `docs/RELEASE-PROCESS.md` §2–§3. + +**What it tests in your app.** + +- **Gates:** `bin/gate-check.sh` per stage, plus the obligation, artifact and source-ledger checks. +- **Every model write:** `bin/exec.sh` snapshots, runs the mxbuild gate and the lint gate, and + restores on failure. +- **Lint:** mxcli's built-in rules plus the toolkit's Starlark rules (`lint-rules/`), against a + baseline so a rise is blamed on the script that caused it. +- **Pages:** `bin/check-page-shell.sh` and `bin/page-fidelity.js` score built pages against their + wireframes; the LOOK pass opens every page. +- **Behaviour:** Playwright journeys plus OQL database assertions (`skills/testing-shape.md`, + `skills/journey-proof.md`), and `bin/verify-module.sh` to run every instrument at module close. + +## We need your input + +Every rule in this toolkit traces to something that went wrong on a real project: a corrupted +model, a gate that went green over a broken feature, a question nobody asked. That is also its +limit. It knows the walls its users have hit so far, and it learns about yours only if you tell +it. Field feedback is how it improves. + +Ways in, from least to most effort ([CONTRIBUTING.md](CONTRIBUTING.md) has the detail): + +1. **Drop a file in `contrib/inbox/`** (copy `contrib/inbox/TEMPLATE.md`) and open a PR. No + quality bar: a bug-log entry, a diff, three sentences, raw terminal output. +2. **Run `bin/harvest-learnings.sh ` at project wrap-up.** It drafts inbox files + from your project's bug logs, decision-register promotions and locally patched toolkit + scripts. Review them, then PR. +3. **Open a direct PR** into `skills/`, `bin/` or `bug-logs/`, held to the field-proof bar. +4. **Open a GitHub issue.** For a bug: the exact command, its verbatim output, and the + `Toolkit release:` line `bin/sync-project.sh` prints. For an idea: the failure it would + prevent. + +**No client data**, anywhere: no client, partner or person names, codenames, internal hosts or +real paths. Genericize first; the leak guard runs in CI but cannot know your client. + +Merged contributions get a `CHANGELOG.md` line crediting you or your project by name, and the +release notes name contributors too. + +## Releases + +Merged changes ship on a release train tagged `vYYYY.MM.DD` (the cycle is defined at the top of +`CHANGELOG.md`). Tags are published from 1.0 on; earlier dated CHANGELOG sections were never +pushed as tags. On top of the train, named milestones get curated notes: start +with [**1.0**](docs/releases/v1.0.md). What a milestone promises, major versus minor, deprecation, +and how to check, upgrade or pin the version a project uses: `docs/RELEASE-PROCESS.md` §8. -``` -/ ← ONE folder per project; open your agent here - PROJECT.md · intake.md · index.html ← decision register + dashboard (bin/init-project.sh) - .claude/agents/ ← agent stubs (bin/init-agents.sh) - source/ ← legacy source or requirements docs, read-only - analysis// - knowledge-base/ ← extraction JSON + BRDs (config.json knowledgeBaseDir) - (single-source projects may flatten to analysis/knowledge-base/) - architecture/ · design/ ← stage 3/4 artifacts - mdlsource/ ← MDL scripts - .mpr ← the target app -``` +--- -Do **not** create `analysis//` as a sibling of the project — analysis output belongs *inside* the project folder. The only exception is when licence/security constraints (intake Q2) forbid storing the client source alongside the target app; that split-workspace variant is described in `migration-pipeline.md` → "Project Workspace Convention". +## Reference -**New here? Open `toolkit-guide.html` in a browser first** — the whole journey as a visual page: entry modes, the 9 stages, what each gate asks of you, and the don't-panic section. *Agents:* open it for the user only when `/.claude/.guide-shown` is absent, then `touch` it — see the first-touch rule in `CLAUDE.md`. Never once per session. +The rest of this page is lookup material: platform notes, the stage-by-stage flow, write modes, +the skill routing tables and how projects consume the toolkit. -**First session on a harness other than Claude Code?** `toolkit-guide.html` → section 9 has the one setting per harness (Copilot, Aider, Cursor, Windsurf) that stops it prompting on every `bin/exec.sh` call — `bin/install-harness-permissions.sh` sets what it can automatically; the table covers what's left. +### Platform support **Where does this run? Wherever you started the chat.** `doctor.sh` detects the lane — Claude Code on the web (cloud container), a devcontainer, or your own machine with Studio Pro — and the agent records it; nobody is asked. Every stage runs headless in all three; what each lane changes is in `CONVERSION-RUNBOOK.md` → *Where you run this*. Do not assume "no Studio Pro = no build". -**Install is one command per project:** - -```bash -~/Mendix/mxcli-project-toolkit/bin/init-project.sh -``` - -It scaffolds everything — `intake.md`, `PROJECT.md`, `CLAUDE.local.md` (runbook-first wiring + baseline routing, auto-loaded every session), all six agent stubs (inert until completed per `skills/agent-roles.md`), the `index.html` dashboard — and opens the visual guide. Idempotent: re-running never overwrites. Then just follow `skills/conversion-runbook.md` — it interviews you through each stage below. - -**Even easier — install the slash command once**, and every project is a `/toolkit-init` away: - -```bash -mkdir -p ~/.claude/commands && cp ~/Mendix/mxcli-project-toolkit/commands/toolkit-init.md ~/.claude/commands/ -``` - -Typing `/toolkit-init` in any Claude Code session then runs the install, reads the runbook, and starts the Stage-P interview (or runs `sync-project.sh` if the project is already wired). Each stage's "done" checklist runs `bin/gate-check.sh `, which fails loudly when an artifact is *there and wrong* and regenerates `index.html` from the project's real state. An artifact that simply isn't there yet reports `PENDING`, not red — and a project that joined the toolkit mid-flight records that once (`--adopt --reason "..."`) so the stages it will never run report `WAIVED` instead of failing forever. Alongside the stage gates it runs the **obligation check** (`bin/lib/obligations.tsv`): the per-module *passes* that owe a mark — the LOOK, the wiring sweep, journeys, the coherence pass — each named with the artifact that proves it happened and the denominator that artifact must state. A pass nobody performed reports `PENDING` (or `FAULT`, if the artifact is there but silent on its denominator); it never renders green by being absent. `--waive look/ --reason "..."` records one deliberately not performed, and any obligation below an `--adopt` point reports `ADOPTED`. See `skills/conversion-runbook.md` §2 → "What a gate verdict means". - ---- - -## Platform support - The toolkit is bash plus Python 3. Most of it is platform neutral; the exceptions are listed here rather than discovered halfway through a stage. @@ -157,9 +250,8 @@ they would have saved, restarted or reopened it. Everything either side of that > export MXBUILD_PATH="/c/Program Files/Mendix//modeler/mxbuild.exe" > ``` > -> Same file, second gap: the guard that refuses two concurrent writes uses `pgrep`, which Git Bash -> does not have, so it silently never fires. Do not run two write sessions against one model on -> Windows. Both are tracked for a proper fix. +> `bin/doctor.sh --install ` closes the gap without Studio Pro: it fetches a +> version-matched mxbuild that `exec.sh` then finds on its own. **Existing clone from before 2026-08-18?** You predate `.gitattributes`, so if git checked the scripts out with CRLF they will not run. Fix once: @@ -172,8 +264,7 @@ bin/install-claude-hooks.sh --full # re-install: the installed copies are th Then re-run `bin/sync-project.sh ` per project to refresh its copied scripts. - -## How a migration flows through this toolkit +### How a migration flows through this toolkit Every migration moves through the same stages, regardless of source stack. Each stage has one skill that owns it, one agent responsible for running it, and hands a concrete artifact + a recorded decision to the next. The full stage-by-stage detail — what you're asked, what gate stops the pipeline, who owns it — lives in `skills/conversion-runbook.md`; this is the summary: @@ -221,7 +312,7 @@ P. KICKOFF source folder, constraints, SME availability → workspa **Stage 0 (Triage) is a gate, not a formality.** It decides whether this app is even big enough to justify an extraction pipeline (small apps: skip straight to manual `assess-migration.md` + hand-written BRD), whether existing extractors/mappers cover this source stack or a new one needs building, and — for large sources — recommends a bounded scope subset (**an ordering, not an exclusion**) rather than processing everything at once. It also flags (without deciding) whether the app is large enough to raise a multiple-Mendix-apps question, resolved before Stage 3's module-boundary work. Stage 2 (BRD generation) does not start until this is signed off. -### How `assess-migration` and the extraction pipeline complement each other +#### How `assess-migration` and the extraction pipeline complement each other These are two tools for the same stage — they work together, not instead of each other: @@ -252,43 +343,15 @@ stages 1–6 proceed **Nothing in stages 0–4 touches mxcli.** MDL scripting only starts at stage 5, against a plan that's already been reviewed. This is deliberate — it's cheaper to fix a wrong module boundary in a diagram (or a wrong scope decision before any extraction ran) than to fix it after 40 MDL scripts assume it. -*(A worked end-to-end example used to live in `examples/`. It was removed on 2026-08-03 because it was a client architecture document rather than a synthetic sample. A replacement built from invented data is on the backlog.)* - ---- - -## Decision flow: query the model, then read the source, then ask the human - -Before asking the user anything, or writing anything: **query the model → read the source → ask the human, in that order.** Never skip to the last one. Full source-of-truth table (which class of question answers from which source, and why) in `skills/query-the-model.md`. The two rules that are already load-bearing and easy to skip under pressure: +### Which tools do what — and when -- **`SHOW ASSOCIATIONS` before every `CREATE ASSOCIATION`** — MDL has no `IF NOT EXISTS`; re-running a CREATE silently duplicates it. -- **`SHOW ENTITIES IN ` before referencing a marketplace module** — `mxcli check --references` can't validate a module that isn't imported yet. - -Reads are always safe and free; writes go through the STOP table below. - -## Which tools do what — and when - -### Stages 0–4: pure LLM, no mxcli needed - -Triage, analysis, BRD generation, architecture, and design are entirely model-driven. The LLM reads source code, documents, and SME input; produces markdown, JSON, and diagrams; and hands a reviewed, signed-off plan to stage 5. No mxcli command runs, no `.mpr` is touched. This is deliberate — it is far cheaper to fix a wrong module boundary in a diagram than after 40 MDL scripts assume it. - -### Stage 5+: write modes +#### Stage 5+: write modes Once you have a reviewed build plan, the default way to write the `.mpr` is the **CLI** (`./bin/exec.sh script.mdl`, Studio Pro closed). On mxcli v0.24 it writes everything MDL can express — domain model, microflows (inline association sets included), pages (association-mode comboboxes and dropdown filters, cross-module datasources), access rules, security level and project settings, navigation, workflows. It is also the only mode that exists in a cloud container or devcontainer (no Studio Pro there — `CONVERSION-RUNBOOK.md` → *Where you run this*). You write a readable, version-controlled MDL script; `exec.sh` snapshots the model first, runs the mxbuild gate after, and restores on failure. **Studio Pro by hand** covers only what MDL cannot express (`skills/learned-mdl-cannot-express.md`, and the STOP rows in `skills/learned-mdl-preflight.md`) — a human step, sequenced by the main session: save and close Studio Pro before the next CLI write. -### Stage 6: testing - -Testing runs after a gate-agent pass and uses two independent layers: - -- **Playwright** (via `test-agent`) — walks the running app as a real user: login flows, form submission, navigation, happy-path and edge cases per BRD use case. Driven by the same use-case list from `migration/knowledge-base/brd/`. -- **DB assertions** (`mxcli -p ... -c "SELECT ..."` OQL queries) — cross-checks what the UI shows against what's actually in the database. UI alone can't confirm a create/update/delete landed correctly; OQL can. Patterns in `learned-db-assertions.md`. - -These two layers catch different things — Playwright catches broken flows; OQL catches silent data corruption. Both run before any scenario is marked passing. - -**Screenshot discipline.** `mxcli exec` writes the model file but the browser serves a JS bundle compiled by Studio Pro — not the raw model. Screenshots before SP recompiles are worthless. Protocol: exec → user closes and reopens SP manually → wait for confirmation → `curl` port 200 → only then screenshot or run UI assertions. Never auto-kill/relaunch SP from a script. Add this rule to each project's `CLAUDE.md` at setup. - -### Per-operation reference table +#### Per-operation reference table Use this before every write. Full per-rule detail (root causes, bug IDs, retest stamps) is in `skills/learned-mdl-preflight.md`. @@ -303,7 +366,7 @@ Use this before every write. Full per-rule detail (root causes, bug IDs, retest **The crash net.** An MPR is two parts: `Project.mpr` (SQLite index) and `mprcontents/` (BSON units). `bin/exec.sh` snapshots both before every batch; 5 rotate; `bin/restore-mpr.sh` rolls back both together (either alone is useless). Git commits at phase gates are the real history. Ad-hoc `.mpr.backup` copies are banned. By design, `exec.sh` refuses to run at all while the model has uncommitted changes — its snapshot would not cover them, so a later auto-restore could silently lose that work; commit the model first (`FORCE_EXEC=1` overrides, at your own risk). -### Something went wrong? Don't panic. +#### Something went wrong? Don't panic. Two things go wrong regularly on active projects. Both are recoverable. @@ -335,7 +398,7 @@ You don't need to restart your machine or reinstall anything. The stale process --- -## Context cost — read the finding before you install anything +### Context cost — read the finding before you install anything A month's token allowance, 20% of it gone in three days: about **2x the sustainable rate**. The cause was not carelessness. Measured across 10,855 main-loop calls on one project: @@ -365,7 +428,7 @@ Two things people reach for that do not fix it: paid ~300 times. That is what `RESUME.md` and `checkpoint.sh` exist to prevent; see `skills/close-the-loop.md`. -### The hooks, and why they are tiered +#### The hooks, and why they are tiered ```bash bin/install-claude-hooks.sh # prints the tier table, installs nothing @@ -408,87 +471,36 @@ with the finding, not the install command. --- -## What's in here +### What's in here ``` mxcli-project-toolkit/ - CONVERSION-RUNBOOK.md ← thin front door: how to start, entry modes - toolkit-guide.html ← visual onboarding page + shared CSS shell for stage HTMLs - bin/ - init-project.sh ← Stage P scaffold: intake.md, PROJECT.md, index.html (opens the guide) - init-agents.sh ← scaffold all six agent stubs into a project's .claude/agents/ - gate-check.sh ← mechanical stage gates (P–7) + self-regenerating dashboard - PASS / PENDING / FAIL / WAIVED / MANUAL; --adopt, --waive - + the obligation check: which per-module PASSES owe a mark - lib/obligations.tsv ← the obligation manifest: pass, performer, scope, proof artifact, - whether it must state a denominator, and the stage it arms at - lib/obligation-check.sh ← forward check (every owed mark is there, or waived in the register) - + reverse check (a verify-group skill no obligation names → warn) - sync-project.sh ← after toolkit git pull: refresh the artifacts copied into a project - split-claude-md.sh ← move MDL/lint reference out of CLAUDE.md into load-on-demand files - install-claude-hooks.sh ← tiered context-cost hooks → ~/.claude (see "Context cost" above) - install-claude-permissions.sh ← allow-lists the safe wrappers (bin/exec.sh & friends, mx) - in /.claude/settings.json + settings.local.json - (absolute-path entries only) so Manual mode stops prompting - install-harness-permissions.sh ← ONE entry point: calls install-claude-permissions.sh, plus - Copilot (.vscode/settings.json), Aider (.aider.conf.yml); - run at scaffold time, --check'd on sync — see toolkit-guide.html §9 - permission-profile.sh ← which permission profile a project runs: wrappers | project - (default) | full (per machine) — intake Q12; doctor.sh shows it - install-hooks.sh ← unrelated: the git pre-commit client-data guard for THIS repo - claude-hooks/ ← sources for the above: hooks/ (5) + bin/ (checkpoint, close-task) - agents/ ← the six agent stub templates (ba/architect/mdl/gate/test/review) - skills/ - conversion-runbook.md ← [any project] The spine: stage matrix + interview protocol + entry modes + gates - checkpoints/ ← CAC checkpoint scripts (scope/extraction-scope/BRD/architecture/design/build/cutover) - query-the-model.md ← [any project] Query-before-ask source-of-truth ordering - interview-protocol.md ← [any project] How a question is put to the user: chat not files, options + recommendation, batch per gate, record the answer - existing-app-assurance.md ← [any project] Audit / regression-test an existing app — no pipeline - agent-roles.md ← [any project] Generate ba/architect/mdl/gate/test/review subagents with scoped tool rights - bootstrap-project.md ← [any project] Generate a new project's CLAUDE.md: Baseline routing + project-specific facts - extractor-quality-loop.md ← [migration] Scored quality loop for building/validating extractors - ui-preflight-pages.md ← [any project] Mandatory wireframe→tokens→StyleGallery cross-check before any page MDL - learned-stylegallery.md ← [any project] Turn ds.css into a live in-app StyleGallery module - migration-pipeline.md ← [migration] Full pipeline phase guide (XML → KB → BRD → MDL) - source-triage.md ← [migration] Gate before extraction: coverage check, reuse-vs-build-new call, bounded scope - modularize-domain.md ← [migration] Deciding Mendix module boundaries (Stage 3): criteria, sign-off, HTML rationale - architecture-blueprint.md ← [migration] Target-architecture blueprint: diagrams, module defs, wiring, fit-gap, marketplace, security, NFRs, integrations - design-artifacts.md ← [migration] UI/brand layer: versioned design system + annotated wireframes + branding interview - brd-to-build-plan.md ← [migration] Plan definition: BRD + architecture → dependency-ordered, numbered build plan - module-brief.md ← [any project] Per-module brief: ba-agent synthesizes BRD/wireframe/access into the mdl-agent's single entry point - iterative-build-loop.md ← [any project] Per-module build discipline: gate loop, coverage checklist, CE triage, Studio Pro handoffs - module-review.md ← [any project] The one per-module closing pass: build, gate, prove (UI+Data+wiring-sweep+monkey), LOOK (every page, not just tested ones), confirm with denominator, diagnostic-only - brd-generation.md ← [migration] BRD JSON prompt templates + validation checklist - brd-validation.md ← [migration] Validating BRDs against code + doc KB - document-discovery.md ← [migration] Scanning/classifying an unstructured document folder - kb-generation.md ← [migration] Document extraction (Excel/Word/PDF → KB markdown) - source-os11.md ← [migration] OutSystems 11 XML schema reference - os-xml-schema.md ← [migration] OS module XML structure details - source-node-express-react.md ← [migration] Node/Express+React extraction layout + known gaps - mdl-cookbook-microflows.md ← [any project] MDL scripting patterns for microflows - qa-loop-goal-pattern.md ← [any project] Iterative /goal-driven pipeline validation technique - e2e-harness-base.md ← [any project] End-to-end test harness base - assess-migration.md ← [migration] Up-front migration assessment - migrate-general.md ← [migration] Source-agnostic migration guidance - migrate-outsystems.md ← [migration] OutSystems-specific migration guide - learned-*.md ← [any project] Validated learnings from live projects - pipelines/ ← Source-specific extraction tooling (code; node_modules gitignored) - outsystems/ ← OS XML → KB → BRD (extraction tooling; output is gitignored) - java-angular/ ← Java + Angular/Spring Boot → KB → BRD - node-express-react/ ← Node/Express + React → KB → BRD — regex-based, proven on one source shape only; read its README first - bug-logs/ - mxcli-bugs.md ← Known mxcli CLI bugs and workarounds (shared) - process/ - process-learnings.md ← Cross-project process improvements - roadmap.md ← Future scope, parked deliberately (RM-NN entries, one per goal) - kiro-*.html ← Figures cited by roadmap.md RM-01 — read from there, not standalone + CONVERSION-RUNBOOK.md ← thin front door: how to start, entry modes, where you run this + toolkit-guide.html ← visual onboarding page + shared CSS shell for stage HTMLs + skills/ ← the method: runbook, checkpoints/, stage skills, learned-*.md from live projects + bin/ ← toolkit-side tools: init/sync/doctor, gate-check, status, render-routing, guards + lib/ ← manifests: skill-routing.tsv, obligations.tsv, artifact-manifest.tsv, install manifest + project-bin/ ← instruments copied into each project as bin/ (exec.sh, verify-module.sh, lint-gate.sh, …) + project-tests/e2e/ ← journey runner and Playwright/OQL harness copied into each project + agents/ ← the six agent stub templates (ba/architect/mdl/gate/test/review) + lint-rules/ ← toolkit Starlark lint rules + README (options, how to turn one off) + pipelines/ ← source-specific extractors (OutSystems, Java/Angular, Node/React) + templates/ ← templates/company-brain/ (private-tier template), manual-steps.md + claude-hooks/ ← sources for the tiered context-cost hooks (see "Context cost") + commands/ ← /toolkit-init slash command + bug-logs/mxcli-bugs.md ← known mxcli bugs and workarounds (look up with bin/bug-lookup.sh) + contrib/inbox/ ← contribution drop (unreviewed, nothing loads it) + process/ ← process learnings, roadmap, incident history + docs/ ← release process, release notes, pipeline walks + tests/ ← the toolkit's own fixtures (run-tests.sh, wave2/, lint-rules/) ``` -`[any project]` vs `[migration]` above mirrors each skill's own `Applies to:` header line — greenfield mxcli builds only need the `[any project]` set, starting at Stage 5. The stage 1–4 skills (document discovery, KB/BRD generation and validation, modularization, architecture, design, build plan) also apply to **requirements-driven builds** with no legacy source — their headers say so explicitly. +Each skill's own `Applies to:` header says whether it is for any project or migrations only; +the routing tables below say when each one loads. --- -## Division of labor: this toolkit vs bundled mxcli skills +### Division of labor: this toolkit vs bundled mxcli skills Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli init`, refreshed with each release) containing syntax references, widget patterns, CRUD templates, and how-to guides. **This toolkit does not duplicate those.** The two sets are complementary: @@ -501,7 +513,7 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini --- -## When to use which skill +### When to use which skill @@ -702,7 +714,7 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini --- -## How to add a new skill +### How to add a new skill 1. Create a new `.md` file in `skills/` with this header: ```markdown @@ -712,28 +724,30 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini **Source:** which project or session this came from ``` 2. Structure it as a step-by-step guide with prompt templates where applicable -3. Add it to the "When to use which skill" table above -4. **If it applies on every MDL-writing session regardless of task** (not situational — e.g. a new universal MDL gotcha, not a phase-specific procedure), also add it to "Baseline routing" below. Situational skills stay out of that table; it's deliberately short. +3. Add its routing row to `bin/lib/skill-routing.tsv` and run `LC_ALL=C bin/render-routing.sh` — the tables in this README are generated from it, never hand-edited +4. **If it applies on every MDL-writing session regardless of task**, mark its row baseline so it lands in "Baseline routing" below; the always-on tier has a word budget that `render-routing.sh --check` enforces. Authoring rules for behaviour-shaping skills: `CLAUDE.md` → "Adding new skills" 5. Commit and push — available to all projects on next `git pull` --- -## How to add a project-specific learning - -For validated patterns from a live project, add a file `skills/learned-{topic}.md`. These get loaded by Claude when relevant and accumulate into cross-project knowledge. If the pattern is universal enough to belong in "Baseline routing" (most `learned-microflow-patterns.md`-style discipline is), add it there too — don't leave it purely situational. - -For bugs, append to `bug-logs/mxcli-bugs.md` or create a project-specific log. +### Consuming this toolkit -**Lower-friction path — use it liberally:** drop what you have (a bug-log entry, a diff, three -sentences) into `contrib/inbox/` and open a PR — no quality bar, triage does the polishing. Or -let it be drafted for you: `bin/harvest-learnings.sh ` scans a project's bug -logs, register promotion tables and locally-patched toolkit scripts and writes ready-to-PR -inbox files. Full picture, lanes and review process: `CONTRIBUTING.md`. Merged contributions -get a `CHANGELOG.md` credit line naming you or your project. +This clone stays clean — project output never lands inside it. **Everything else lives inside one project folder** (usually one git repo — it is the session root, the workspace root, and the mxcli target all at once): ---- +``` +/ ← ONE folder per project; open your agent here + PROJECT.md · intake.md · index.html ← decision register + dashboard (bin/init-project.sh) + .claude/agents/ ← agent stubs (bin/init-agents.sh) + source/ ← legacy source or requirements docs, read-only + analysis// + knowledge-base/ ← extraction JSON + BRDs (config.json knowledgeBaseDir) + (single-source projects may flatten to analysis/knowledge-base/) + architecture/ · design/ ← stage 3/4 artifacts + mdlsource/ ← MDL scripts + .mpr ← the target app +``` -## Consuming this toolkit +Do **not** create `analysis//` as a sibling of the project — analysis output belongs *inside* the project folder. The only exception is when licence/security constraints (intake Q2) forbid storing the client source alongside the target app; that split-workspace variant is described in `migration-pipeline.md` → "Project Workspace Convention". **Reference model (default):** clone once, point projects at it — no copies, no drift. ``` @@ -742,9 +756,9 @@ git clone https://github.com/MendixMau/mxcli-project-toolkit.git ~/Mendix/mxcli- Each project's `CLAUDE.local.md` references `~/Mendix/mxcli-project-toolkit`. Pull updates with `git pull` — **everything referenced (skills, runbook, checkpoints, gate-check) updates instantly for all projects.** The artifacts that were *copied* into a project (intake.md, agent stubs, and — only on a project with no `CLAUDE.local.md` — the baseline-routing table in its `CLAUDE.md`) don't: run `bin/sync-project.sh ` after a pull — it appends new intake questions, refreshes untouched agent stubs (never completed ones), refreshes the Baseline and situational tables in `CLAUDE.local.md` when one exists, and otherwise flags a stale baseline routing table in `CLAUDE.md` for hand update. It also warns (never fetches) when the toolkit clone itself is off `master` or behind `origin/master`; `MXTK_SYNC_SKIP_CLONE_CHECK=1` skips that one check, for the toolkit's own test suite only — real users leave it unset. Then tell any already-running session to re-read the runbook. For a self-contained handoff, add it as a git submodule instead. Per pipeline, run `npm install` inside `pipelines//pipeline` (node_modules is gitignored). -**Company brain.** A company running several projects keeps its own private tier — own skills, conventions, lint rules, MDL snippets, approved MPKs — in a repo shaped like this one: `bin/init-company-brain.sh ` instantiates `templates/company-brain/`, and `bin/init-project.sh --company ` (or `bin/wire-company-brain.sh`) points a project at it with one on-demand block. Rules and the three-tier split: `skills/company-brain.md`. +Company-specific material goes to a private company brain, never into this repo: [Make it yours](#make-it-yours). -### Baseline routing — every consuming project needs this table +#### Baseline routing — every consuming project needs this table The "When to use which skill" table above is *situational* — load a skill when a specific task calls for it. A few skills apply on **every** MDL-writing session regardless of task, and situational discovery quietly misses them, because nothing mid-task prompts loading them. Every consuming project needs this table somewhere an agent reads it before writing MDL — but "somewhere" is not "copy it into `CLAUDE.md`": a project scaffolded with `bin/init-project.sh` already has it in `CLAUDE.local.md`, refreshed by every `bin/sync-project.sh` run, and `CLAUDE.md` there should carry only a short pointer to it (`skills/bootstrap-project.md` Step 2 has the exact block). Paste the table itself into `CLAUDE.md` only on a project with no `CLAUDE.local.md` convention (a non-Claude harness, or one bootstrapped before `init-project.sh` existed): @@ -806,9 +820,8 @@ Then filter by Role(s). A helper dispatched with a role stub (ba, architect, mdl **Your build plan, `PROJECT.md`, and session notes live in your own project, not here.** This repo holds reusable tools + skills + small curated examples only. A project's architecture blueprint, numbered build plan, decision register, and running session diary belong in that project's own repo (e.g. `architecture/build-plan.md`, `PROJECT.md`, `SESSION-NOTES.md` at the project root) — never committed back into the toolkit. If a pattern from that plan turns out to be reusable across projects, promote it into a `skills/learned-*.md` file here instead of leaving the whole plan in place. -## Used by +### Pipelines - `pipelines/outsystems/` — OutSystems 11 → Mendix pipeline (was the standalone `os-migration-pipeline` repo) - `pipelines/java-angular/` — Java + Angular/Spring Boot → Mendix pipeline - `pipelines/node-express-react/` — Node/Express + React → Mendix pipeline (regex-based, proven on one source shape — see its README) -- Several other client integration and migration projects diff --git a/docs/RELEASE-PROCESS.md b/docs/RELEASE-PROCESS.md index 9a4af3c4..ab9d6fd6 100644 --- a/docs/RELEASE-PROCESS.md +++ b/docs/RELEASE-PROCESS.md @@ -5,8 +5,8 @@ that release is cut. `CONTRIBUTING.md` already covers the three ways to *send* a (inbox drop, harvest script, direct PR) and the two rules that apply in every lane — read that first if you're contributing, not maintaining. This doc adds the fourth intake channel (GitHub issues), the triage rubric that turns an arrived item into a verdict, the test tiers a change -must reach before it can merge, and the release train that turns merged commits into a tagged -version. Reconciling the two documents into one (folding this rubric into `CONTRIBUTING.md`) is +must reach before it can merge, the release train that turns merged commits into a tagged +version, and (§8) what a version name promises: milestones, major versus minor, deprecation. Reconciling the two documents into one (folding this rubric into `CONTRIBUTING.md`) is a follow-up, not done here — see "Not yet built" at the end. ## 1. Intake lanes @@ -154,6 +154,66 @@ the bottleneck. - **One purpose per PR.** A PR that mixes skill prose, an instrument change, and an inbox drop is split before grading — the rubric in §2 grades one artifact at a time. +## 8. Versioning: milestones on top of the train + +This section is the single source for what a version name promises. The train itself (how +`## Unreleased` becomes a dated section and a tag) is defined at the top of `CHANGELOG.md` and +implemented by `bin/cut-release.sh` (§6); nothing here changes either. + +**The train stays.** Every cut is tagged `vYYYY.MM.DD`. `bin/sync-project.sh` prints that tag as +`Toolkit release:`, bug reports name it, and projects pin to it, so the tag format and the script +do not change for milestones. Tags start at 1.0: earlier dated CHANGELOG sections were never +pushed, and until the first tag is published sync prints `Toolkit release: none tagged yet`. + +**A milestone is a train release the maintainer also names.** Milestones are `1.0`, `1.1`, `2.0`. +Naming one adds three things to an ordinary cut, and no new tag scheme: + +1. curated notes in `docs/releases/v..md`, written for someone deciding whether to + adopt or upgrade (themes, what each means for them, upgrade steps, known gaps, credits), where + every claim traces to a `CHANGELOG.md` line or a commit; +2. a GitHub Release on the train tag of that cut, titled with the milestone name and using the + notes file as its body; +3. one `CHANGELOG.md` line in that cut's section pointing to the notes file. + +**Major or minor.** + +| | Major (`2.0`) | Minor (`1.1`) | +|---|---|---| +| When | Anything that breaks a consuming project unless someone acts: a changed `PROJECT.md` / register format, a renamed or removed script, gate semantics that turn a passing project red | New capability that needs no action: a skill, a lint rule that only warns, an instrument, a fix | +| Notes must add | a **Migration** section with the exact steps, and the `bin/sync-project.sh` path that performs or detects them | an **Upgrading** section (pull + sync is enough) | + +A change that would be major waits for the next major milestone, or ships behind a sync step that +migrates the project itself (then it no longer breaks anyone and counts as minor). A `fix(...)` +line that changes an exit code or output format a project may parse is named in the notes even +when it is minor (`CHANGELOG.md` already marks these as "Upgrade notes" or "Behaviour change"). + +**Cadence.** The train's cadence is defined at the top of `CHANGELOG.md`. A milestone is named +when a theme has landed, roughly monthly; there is no fixed date and no milestone without notes. + +**Deprecation.** Announce it in one milestone's notes (what goes, what replaces it, the sync +behaviour in the meantime). Remove it no earlier than the next **major** milestone. Until removal, +the old name keeps working or prints a pointer to the new one. + +**What "stable" means for consumers.** A train tag never moves (§6: never force-push, never delete +a tag; a bad release is reverted forward and gets a hotfix tag). A project that wants stability +pins its toolkit clone to a tag and upgrades on its own schedule with pull + sync. A milestone is +the recommended pin: its notes say what to check. + +### For consumers: which version am I on, how to upgrade, how to pin + +- **Which version:** `bin/sync-project.sh ` prints `Toolkit release: vYYYY.MM.DD`, or + `vYYYY.MM.DD-3-g` when the clone is three commits past it. `git -C describe + --tags` gives the same. Map a train tag to a milestone through the GitHub Releases page or + `docs/releases/`. +- **Upgrade:** `git -C pull`, then `bin/sync-project.sh ` for each project. + Read the milestone notes first when you cross one; for a major, follow its Migration section. +- **Pin:** `git -C checkout vYYYY.MM.DD` (a detached checkout of the tag; sync's + stale-clone check stays quiet for a detached tag and warns only on a named non-`master` branch + or a `master` behind `origin/master`). The session-start ritual in `CLAUDE.local.md` runs + `git pull --ff-only` and warns that the clone is not on `master`; while pinned, that warning is + expected and the clone stays on the tag. To unpin: + `git -C checkout master && git -C pull`. + ## Not yet built (follow-ups, not implemented in this change) Kept here rather than silently dropped, per the source proposal's own `next`/`later` staging: diff --git a/docs/releases/v1.0.md b/docs/releases/v1.0.md new file mode 100644 index 00000000..46b7f75d --- /dev/null +++ b/docs/releases/v1.0.md @@ -0,0 +1,183 @@ +# mxcli-project-toolkit 1.0 + +**What it is.** A shared process layer for building and changing Mendix apps with an AI coding +agent and [mxcli](https://github.com/mendixlabs/mxcli). It gives the agent a staged pipeline +with recorded decisions (`PROJECT.md`), skills it loads only when a task calls for them, and +mechanical gates (`bin/gate-check.sh`, lint, mxbuild, end-to-end journeys) that report what is +actually in your project. It runs for migrations, requirements-driven builds, greenfield builds +and changes to a live app, or à la carte to audit and test an app you already have. + +1.0 is the first named milestone on top of the release train. It is not a rewrite: these notes +pick out what changed in the two weeks before it (2026-09-30 to 2026-10-14). Every line traces to +an entry in [`CHANGELOG.md`](../../CHANGELOG.md), which has the full detail and field evidence. +How milestones relate to the `vYYYY.MM.DD` tags: [`docs/RELEASE-PROCESS.md`](../RELEASE-PROCESS.md) +§8. + +1.0 is the first release-train tag published to GitHub, cut with `bin/cut-release.sh` when this +release is announced. Earlier `vYYYY.MM.DD` headings in CHANGELOG were never pushed as tags. + +Most PRs linked below landed together through +[#329](https://github.com/MendixMau/mxcli-project-toolkit/pull/329) and show as closed on GitHub; their +changes are on master. + +## Highlights + +### Every merge is reviewed by someone who did not write it + +A PR now gets an independent review (MERGE / MERGE-AFTER-FIX / HOLD, on value, regression risk +after sync, correctness and context cost) before it lands; green CI and the author's own report no +longer count as one. Nine PRs had merged on 2026-10-13 without one +([#316](https://github.com/MendixMau/mxcli-project-toolkit/pull/316)). Several fixes in this +release came out of those reviews (the exec lock race, the final-LOOK timing, the +`test-stack-up` exit codes). + +**For you:** what reaches your project on `git pull` has been read by a second reviewer, not just +tested by its author. + +### Less text in every session + +The always-loaded skill tier had reached 79,997 words against an 80,000-word cap. Incident stories +moved behind pointers into `process/baseline-incident-history.md` and the tier dropped to 76,259 +words with no rule removed ([#315](https://github.com/MendixMau/mxcli-project-toolkit/pull/315)). +New rule: room is made by moving text, never by deleting rules. `change-intake.md` became a +331-word core with an on-demand detail file +([#327](https://github.com/MendixMau/mxcli-project-toolkit/pull/327)), dispatch guidance moved to +its own small file ([#320](https://github.com/MendixMau/mxcli-project-toolkit/pull/320)), a +scaffolded `CLAUDE.local.md` is 17% shorter +([#292](https://github.com/MendixMau/mxcli-project-toolkit/pull/292)), routing in +`CLAUDE.local.md` carries triggers only +([#226](https://github.com/MendixMau/mxcli-project-toolkit/pull/226)), helper stubs read about 18% +less per dispatch ([#223](https://github.com/MendixMau/mxcli-project-toolkit/pull/223)), and a +build helper can get one context-pack file per build step instead of a reading list +([#198](https://github.com/MendixMau/mxcli-project-toolkit/pull/198)). + +**For you:** run `bin/sync-project.sh` and existing projects pick up the shorter wiring; sessions +start with less to read. + +### `exec.sh`: per-project lock, restart verdicts, safer restarts + +In a 20-task field run, 57 of 148 `exec.sh` runs applied nothing, and a machine-wide guard blocked +writes because a *different* project was writing. `exec.sh` now locks per project with a stale-lock +takeover, offers an opt-in `--commit-first`, and ends every applied script with a verdict: restart +needed, watch can apply this, or unknown +([#304](https://github.com/MendixMau/mxcli-project-toolkit/pull/304), +[#324](https://github.com/MendixMau/mxcli-project-toolkit/pull/324)). The review of #304 found the +takeover could let up to three contenders hold the lock; it is now serialised, and a fixture runs +10 rounds of 12 contenders and asserts at most one holder. New `bin/app-restart.sh` (#304) restarts only +this project's own `mxcli run --local`; since +[#322](https://github.com/MendixMau/mxcli-project-toolkit/pull/322) it also refuses over a leftover `mxcli test` injection. + +**For you:** two projects on one machine no longer block each other, and the agent is told when a +change needs an app restart instead of guessing. + +### Lint you can trust and tune + +UX001/002 plus Batches 1–4 landed: UX001/UX002, then ERR001/LOOP001, PERF001/002/004, +TXT001/TXT002/UI001, and Batch 4 with **SEC014** (an external role that can write status, audit or ownership members) and a +CONV009 that reads its `maxObjects` option +([#238](https://github.com/MendixMau/mxcli-project-toolkit/pull/238), +[#248](https://github.com/MendixMau/mxcli-project-toolkit/pull/248), +[#270](https://github.com/MendixMau/mxcli-project-toolkit/pull/270), +[#305](https://github.com/MendixMau/mxcli-project-toolkit/pull/305), +[#312](https://github.com/MendixMau/mxcli-project-toolkit/pull/312)). When `mxcli init` reverts +toolkit rules to stock, the lint gate now stops as **BLIND** instead of blaming the next script +for the jump, and sync restores them +([#308](https://github.com/MendixMau/mxcli-project-toolkit/pull/308)). Turning a rule off has one +supported route (`enabled: false` in `lint-config.yaml`, recorded in `PROJECT.md`). A metrics +ledger records outcome, speed, lint total and `mxcli report` score for every exec +([#265](https://github.com/MendixMau/mxcli-project-toolkit/pull/265)). + +**For you:** SEC014 asks one question (which roles are external) and prints a NOTE until you +answer it. + +### End-to-end proof that fails for the right reason + +The journey runner preflights the suite (a refused admin credential aborts before the first +journey) and journeys can declare `requires` / `leaves` / `consumes` +([#302](https://github.com/MendixMau/mxcli-project-toolkit/pull/302)). OQL rows are read by SELECT +alias, an app that dies mid-suite reads APP-DOWN rather than FAIL, and negative-only claims get a +liveness reference ([#310](https://github.com/MendixMau/mxcli-project-toolkit/pull/310)). +`test-stack-up.sh` identifies this project's own `run --local` by its command line +([#319](https://github.com/MendixMau/mxcli-project-toolkit/pull/319)). Re-runs are scoped to what +a change touched, UI-only changes get a light lane, and unit tests are off by default +([#290](https://github.com/MendixMau/mxcli-project-toolkit/pull/290)). The page-fidelity scorer +matches what a page displays, on whole words +([#318](https://github.com/MendixMau/mxcli-project-toolkit/pull/318)). + +**For you:** fewer false reds from the harness, and less time re-proving changes nobody touched. +**Check after upgrading:** `oqlScalar` now returns the first SELECT alias. + +### Stage 3 designs how users reach each page + +Stage 3 now writes `design/interaction-model.md` before the first wireframe: a home per role, an +inbox per user task, and for each page how it is reached and what it receives. Every wireframe +opens with a contract block ([#317](https://github.com/MendixMau/mxcli-project-toolkit/pull/317), +part 1 of [#291](https://github.com/MendixMau/mxcli-project-toolkit/issues/291)). Three apps had +passed every gate while one rooted 8 pages on a parameterless retrieve and gave all 6 roles one +home page. + +**For you:** navigation and page-parameter defects surface at design time, when they are cheap. + +### Changing an existing app: delivery rounds and mid-build changes + +In change-an-existing-app mode, `bin/status.sh` reports the delivery round from a `## Rounds` +table in `PROJECT.md`, drops noise that does not apply there, and counts targeted manual Studio +Pro steps written from `templates/manual-steps.md` +([#326](https://github.com/MendixMau/mxcli-project-toolkit/pull/326)). In every mode, feedback +typed mid-build is logged as a change-log row in `PROJECT.md` before any edit, and the +requirement or design is changed before the MDL +([#306](https://github.com/MendixMau/mxcli-project-toolkit/pull/306)). A final whole-app LOOK is +now owed at Stage 6 ([#303](https://github.com/MendixMau/mxcli-project-toolkit/pull/303), +[#321](https://github.com/MendixMau/mxcli-project-toolkit/pull/321)). + +**For you:** "where are we" has an answer in rounds, not a stage number that never moves. + +## Upgrading + +```bash +git -C ~/Mendix/mxcli-project-toolkit pull +~/Mendix/mxcli-project-toolkit/bin/sync-project.sh +``` + +Skills update with the pull; sync refreshes what was copied into the project (intake questions, +untouched agent stubs, `bin/` and `tests/e2e/` copies nobody edited, lint rules). Once release tags +are published (from 1.0 on) it prints the `Toolkit release:` line you are on; before that it +prints `Toolkit release: none tagged yet`. Then: + +- Answer SEC014's question (which roles are external), then re-baseline lint once, as the sync + NOTE says. +- On mxcli v0.26 or later, sync runs a read-only property-set check (models written by mxcli up + to v0.25 can break Studio Pro's merge). If it reports a repair is needed, run + `bin/doctor.sh --fix-property-sets ` + ([#314](https://github.com/MendixMau/mxcli-project-toolkit/pull/314)). +- Projects with per-BRD coverage ledgers: run conformance once after the upgrade; old rows are + re-keyed ([#307](https://github.com/MendixMau/mxcli-project-toolkit/pull/307)). + +## Known gaps + +- **Field runs still owed.** Fixtures pass, but no real-project run is cited yet for: the + property-set check, the page-fidelity rescoring, PROOF-OF-LOOK per section, per-BRD conformance + ledgers, the graph-sweep catalog guard on mxcli v0.24, and the metrics ledger inside a live + `exec.sh`. +- **[#291](https://github.com/MendixMau/mxcli-project-toolkit/issues/291) parts 2 and 3** (gate + facts for the interaction model, and the model-side check) are not built. +- The exec stale-lock takeover can still race if a holder is killed mid-takeover. It is rare; a + designed fix and a regression test are owed. +- `mxcli fix hashes` as an `exec.sh --patch` post-step is pending + ([#295](https://github.com/MendixMau/mxcli-project-toolkit/issues/295)). +- No shipped pipeline writes an extraction-quality score file yet; the Stage 1 report shows that + section as "manual". +- On Windows and Linux without Studio Pro, `exec.sh` skips its mxbuild gate unless mxbuild is + found; run `bin/doctor.sh --install ` or set `MXBUILD_PATH`. + +## Credits + +Thanks to everyone whose field reports, fixes and reviews fed this release: +**MendixMau** (maintainer), **AlvinnYeo** (the lint stock-revert guard, the catalog edge guard, +per-BRD conformance ledgers, the port-holder report, a microflow preflight STOP row, a leftover +test-injection case), and **jopterhorst** (`mxcli fix property-sets` in the upgrade path, +delivery rounds in change-an-existing-app mode, two UI detection gaps, token-burn rates and +windows). Independent reviewers caught several of the defects fixed here before they shipped. + +Found something? See [CONTRIBUTING.md](../../CONTRIBUTING.md). The bar is "it happened and you +wrote it down".