From 58fda4d73612f43ce7970ad380d98b054e6ca88e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 15:02:35 +0000 Subject: [PATCH 1/3] test-stack-up: runtime command-line ownership proof, own runtime is not a port thief, tracked stale-process sweep (#160, #278, refs #262) - #160: a scanned Mendix port whose listener runs runtimelauncher.jar on this project's deployment dir (/deployment or /app/deployment) is APP_OWNERSHIP=verified, APP_SOURCE=runtime; freshness stays unknown. Linux /proc, macOS ps, Git Bash PowerShell; unreadable stays unverified. - port_holder / holder_text / in-tree walk moved into _common.sh (additive) and sourced by both test-stack-up.sh and app-restart.sh; an older _common.sh stops with exit 2 and the sync remedy. - #262 follow-up to #313: a port held by this project's own process (live recorded pid or descendant, or a runtime on this deployment dir, incl. Studio Pro running this project) is NOT READY, exit 1, not INSTRUMENT BLOCKED. - #278: --track [label] records a background test process; every run lists tracked ones older than MXTK_STALE_HOURS; --reap stops them (pid record only, start-time checked). test-result-audit.md names the two outside causes of INVALID. Fixtures: test-stack-up-local.sh T8-T14 (16 assertions fail on master), test-app-restart.sh old-_common.sh guard case. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 3 + project-bin/_common.sh | 125 ++++++++++++++++ project-bin/app-restart.sh | 62 ++------ project-bin/test-stack-up.sh | 226 ++++++++++++++++++++++------- skills/test-result-audit.md | 1 + tests/wave2/test-app-restart.sh | 11 ++ tests/wave2/test-stack-up-local.sh | 139 +++++++++++++++++- 7 files changed, 459 insertions(+), 108 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 20342404..4e89ae4a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,9 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(test-stack-up): **an app with no run --local handshake is `verified` when its listener runs `runtimelauncher.jar` on this project's own deployment dir** ([#160](https://github.com/MendixMau/mxcli-project-toolkit/issues/160)). Studio Pro's Run Locally, or a run --local without the handshake, was published `APP_OWNERSHIP=unverified`, so every journey run in a requirements-driven build needed `APP_PORT` asserted by hand, though the runtime's command line named the project's deployment dir all along. Both layouts (`/deployment`, `/app/deployment`); /proc on Linux, `ps` on macOS, PowerShell on Git Bash; unreadable stays unverified. The port/process helpers app-restart.sh and test-stack-up.sh each carried now live once in `_common.sh` (additive; an older `_common.sh` stops with the sync remedy, exit 2) — MendixMau +- fix(test-stack-up): **this project's own booting runtime on its port reads NOT READY (exit 1), no longer "INSTRUMENT BLOCKED" (exit 4)** (refs [#262](https://github.com/MendixMau/mxcli-project-toolkit/issues/262), follow-up to #313). A runtime binds its ports before it serves a page, and the #262 port check named it as a thief; the same happened with `--no-docker` while Studio Pro booted this project. Ours = the live recorded pid or its descendant, or a command line on this project's deployment dir; any other holder still blocks — independent review of #313 +- new(test-stack-up, test-result-audit): **`--track ` records a background test process this project started; every run lists the recorded ones older than `MXTK_STALE_HOURS` (4) and `--reap` stops them — by pid record only, a reused pid (start time changed) is never signalled** ([#278](https://github.com/MendixMau/mxcli-project-toolkit/issues/278)). Hung journey-runner shells from earlier retries filled a trial licence's session cap and made a later run INVALID; test-result-audit.md now names the two outside causes of INVALID (account lockout, licence session cap) — MendixMau - learn(testing-shape, test-result-audit): **three test-strength rules, each with a denominator: refusals probed on the server, every undeclared 4xx a failure, one clean-database final run** ([#172](https://github.com/MendixMau/mxcli-project-toolkit/issues/172)). A field comparison of two builds against one frozen spec found the toolkit build checked permission refusals only in the client (a hidden menu item), logged unexpected 400/422 responses as information, and reached 20/20 by stitching three runs. `testing-shape.md` §7 gains three report lines (`N of N restricted pairs probed server-side`, `undeclared 4xx/5xx: 0 of M requests — each fails its step`, `one clean-DB run : N of N scenarios incl. timers, not stitched`); `test-result-audit.md` gains check 2e, which carries the reasons and the field evidence and asks for those numbers. Both baseline skills stay inside the 80,000-word baseline budget (79,997): the reasons live in the on-demand audit skill, and two redundant sentences in `testing-shape.md` §4's verdict paragraphs were folded (the empty-set corollary is stated in the Trace row, `harness-architecture.md` and `report-schema.md`). The backing inbox note is trimmed to its unpromoted "Keep" items. — MendixMau - learn(learned-detection-gaps, module-review): **two UI defects every model gate passes: a lazy-loaded combo box filter hides its preset, and a theme that pins row heights by position lets the next row cover an inserted widget** ([#299](https://github.com/MendixMau/mxcli-project-toolkit/issues/299)). Both passed check, mxbuild, lint and the scripted checks on a 127-module production app and were found only in the running app. Two register rows with the first oracle and the pre-build check (pluggable properties that change *when* data loads; grep the theme for fixed `height` on positional row classes before an insert), and two LOOK lines in `module-review.md` §4c with denominators (`N of N filters show their preset on first load`, `N of N widgets inserted into an existing container fully visible`). — jopterhorst - fix(app-restart, test-stack-up): **a port held by another process before `run --local` is named and reported as "instrument blocked" (exit 4), never killed** ([#262](https://github.com/MendixMau/mxcli-project-toolkit/issues/262)). Studio Pro had started its own runtime on the app/admin ports a restart needed; `mxcli run --local` then failed with "Authentication failed" (the admin API answering was Studio Pro's), the next journey ran against Studio Pro's runtime, and four more runs were NOT RUN. `app-restart.sh` now checks the app port and the admin port (record's `adminPort`, else app + 10) before stopping anything — a holder outside the recorded process tree exits 4 with its name, pid and parent — and again after the stop, where anything still holding a port is by definition not ours. `test-stack-up.sh` runs the same check wherever it would tell you to start `run --local` (`--no-docker`, no Docker daemon, `--check` with a leftover record), on the record's ports, `RUN_LOCAL_APP_PORT`/`RUN_LOCAL_ADMIN_PORT`, or mxcli's defaults 8080/8090. Holder lookup: lsof, then ss, then netstat + tasklist on Windows; unnamed holders still block when nothing of ours is running. Fixtures: `test-app-restart.sh` +2 cases (14 assertions, all fail on the old script), `test-stack-up-local.sh` T7 (4 of 6 fail on the old script). — AlvinnYeo diff --git a/project-bin/_common.sh b/project-bin/_common.sh index 3a58af7a..1a1414c3 100755 --- a/project-bin/_common.sh +++ b/project-bin/_common.sh @@ -787,3 +787,128 @@ mxtk_sha256() { else openssl dgst -sha256 | sed 's/^.*= //' fi } + +# --------------------------------------------------------------------------- +# Process identity — who holds a port, whose process it is. Shared by test-stack-up.sh and +# app-restart.sh (moved here from both, #160; they had carried identical copies since #262). +# All read-only: nothing below stops, signals or starts a process. +# --------------------------------------------------------------------------- + +# mxtk_port_holder — who LISTENs on it: prints "|" and returns 0, or +# returns 1 when the port is free. "?|an unidentified process" when something accepts connections +# but no tool here can name it (lsof without root cannot see another user's sockets). +# Per platform: lsof (macOS, Linux), then ss (Linux without lsof), netstat -ano + tasklist (Windows). +mxtk_port_holder() { + local p="$1" out pid="" name="" + case "$(mxtk_platform)" in + windows) + pid="$(netstat -ano 2>/dev/null | tr -d '\r' | awk -v p=":$p" '$1=="TCP" && $4=="LISTENING" && substr($2, length($2)-length(p)+1)==p { print $5; exit }')" + [ -n "$pid" ] && name="$(tasklist //FI "PID eq $pid" //FO CSV //NH 2>/dev/null | tr -d '\r' | head -1 | cut -d'"' -f2)" ;; + *) + if command -v lsof >/dev/null 2>&1; then + out="$(lsof -nP -iTCP:"$p" -sTCP:LISTEN -Fpc 2>/dev/null)" + pid="$(printf '%s\n' "$out" | sed -n 's/^p//p' | head -1)" + name="$(printf '%s\n' "$out" | sed -n 's/^c//p' | head -1)" + fi + if [ -z "$pid" ] && command -v ss >/dev/null 2>&1; then + out="$(ss -ltnpH "sport = :$p" 2>/dev/null | head -1)" + pid="$(printf '%s' "$out" | sed -n 's/.*pid=\([0-9][0-9]*\).*/\1/p')" + name="$(printf '%s' "$out" | sed -n 's/.*users:(("\([^"]*\)".*/\1/p')" + fi ;; + esac + if [ -n "$pid" ]; then printf '%s|%s\n' "$pid" "${name:-unknown}"; return 0; fi + if (exec 3<>"/dev/tcp/127.0.0.1/$p") 2>/dev/null; then printf '?|an unidentified process\n'; return 0; fi + return 1 +} + +# mxtk_holder_text "|" — "studiopro pid 4711" plus its parent where ps can show one, +# because a runtime Studio Pro started shows up as plain "java"; the parent names the owner. +mxtk_holder_text() { + local pid="${1%%|*}" name="${1#*|}" pp pn + [ "$pid" = "?" ] && { printf '%s (pid unknown — no lsof/ss/netstat could name it)\n' "$name"; return; } + printf '%s pid %s' "$name" "$pid" + if [ "$(mxtk_platform)" != windows ]; then + pp="$(ps -o ppid= -p "$pid" 2>/dev/null | tr -d ' ')" + pn="$(ps -o comm= -p "${pp:-0}" 2>/dev/null | sed 's#.*/##')" + [ -n "$pp" ] && [ "$pp" != 1 ] && [ -n "$pn" ] && printf ' (parent: %s pid %s)' "$pn" "$pp" + fi + printf '\n' +} + +# mxtk_pid_in_tree — 0 when is or a descendant of it (walked up +# by parent pid, at most 10 levels). Git Bash's ps shows no parent of a Windows pid: returns 1 there +# unless the two pids are equal, so a caller must treat "not in tree" on Windows as "unknown". +mxtk_pid_in_tree() { + local p="$1" root="$2" i=0 + [ -n "$p" ] && [ -n "$root" ] || return 1 + while [ -n "$p" ] && [ "$p" != 0 ] && [ "$p" != 1 ] && [ "$i" -lt 10 ]; do + [ "$p" = "$root" ] && return 0 + p="$(ps -o ppid= -p "$p" 2>/dev/null | tr -d ' ')"; i=$((i + 1)) + done + return 1 +} + +# mxtk_pid_args — the process's command line. Linux: argv one per line from /proc (exact, a +# space inside an argument survives). macOS: `ps -o args=`, ONE line (arguments joined by spaces). +# Windows (Git Bash): Win32_Process.CommandLine via PowerShell, one line — Git Bash's ps cannot +# show a Windows process's arguments. Returns 1 when nothing could be read. +mxtk_pid_args() { + local pid="$1" out="" + case "$pid" in ''|*[!0-9]*) return 1 ;; esac + case "$(mxtk_platform)" in + linux) + if [ -r "/proc/$pid/cmdline" ]; then out="$(tr '\0' '\n' < "/proc/$pid/cmdline" 2>/dev/null)" + else out="$(ps -o args= -p "$pid" 2>/dev/null)"; fi ;; + macos) out="$(ps -o args= -p "$pid" 2>/dev/null)" ;; + windows) + command -v powershell.exe >/dev/null 2>&1 \ + && out="$(powershell.exe -NoProfile -NonInteractive -Command "(Get-CimInstance Win32_Process -Filter 'ProcessId=$pid').CommandLine" 2>/dev/null | tr -d '\r')" ;; + esac + [ -n "$out" ] || return 1 + printf '%s\n' "$out" +} + +# mxtk_pid_runs_deployment ... — 0 when is a Mendix runtime started on +# one of the named deployment directories: its command line runs runtimelauncher.jar and the +# argument after the jar resolves to one of them. That is how both `mxcli run --local` and Studio +# Pro's Run Locally start the runtime (field capture, run --local, Linux, 2026-09-25: +# `java … -jar …/runtimelauncher.jar /app/deployment`). A deployment directory belongs to +# exactly one project checkout, so this is an ownership claim, not a "looks like Mendix" guess. +# Linux compares exact argv after resolving symlinks (and a relative argument against the process's +# cwd); macOS and Windows have only the joined line, so the directory is matched as text in both +# its logical and physical spellings (Windows: drive form, case-insensitive). Unreadable → 1. +mxtk_pid_runs_deployment() { + local pid="$1" args line next="" arg="" cwd d cand want + shift + args="$(mxtk_pid_args "$pid")" || return 1 + case "$args" in *runtimelauncher.jar*) ;; *) return 1 ;; esac + if [ "$(mxtk_platform)" = linux ] && [ -r "/proc/$pid/cmdline" ]; then + while IFS= read -r line; do + [ -n "$next" ] && { arg="$line"; break; } + case "$line" in *runtimelauncher.jar) next=1 ;; esac + done </dev/null)" || return 1; arg="$cwd/$arg" ;; esac + arg="$(cd "$arg" 2>/dev/null && pwd -P)" || return 1 + for d in "$@"; do + want="$(cd "$d" 2>/dev/null && pwd -P)" || continue + [ "$arg" = "$want" ] && return 0 + done + return 1 + fi + # One joined line: match "runtimelauncher.jar " followed by the end, a slash or a space. + line="$(printf '%s' "$args" | head -1 | tr -d '"')" + [ "$(mxtk_platform)" = windows ] && line="$(printf '%s' "$line" | tr '\\' '/' | tr '[:upper:]' '[:lower:]')" + for d in "$@"; do + for cand in "${d%/}" "$(cd "$d" 2>/dev/null && pwd -P)"; do + [ -n "$cand" ] || continue + [ "$(mxtk_platform)" = windows ] && cand="$(native_path "$cand" | tr '\\' '/' | tr '[:upper:]' '[:lower:]')" + case "$line" in + *"runtimelauncher.jar $cand"|*"runtimelauncher.jar $cand/"|*"runtimelauncher.jar $cand "*|*"runtimelauncher.jar $cand/ "*) return 0 ;; + esac + done + done + return 1 +} diff --git a/project-bin/app-restart.sh b/project-bin/app-restart.sh index b18ffe19..c7c70f8b 100755 --- a/project-bin/app-restart.sh +++ b/project-bin/app-restart.sh @@ -129,48 +129,14 @@ mask() { # print argv joined, with the value after any password/secret/token/ke "s/(-[-A-Za-z_]*([Pp][Aa][Ss][Ss]|[Ss][Ee][Cc][Rr][Ee][Tt]|[Tt][Oo][Kk][Ee][Nn]|[Kk][Ee][Yy])[-A-Za-z_]*)(=| +)[^ ']+/\1\3****/g" } -# port_holder — who LISTENs on it: prints "|" and returns 0, or returns 1 -# when the port is free. "?|an unidentified process" when something accepts connections but no -# tool here can name it (lsof without root cannot see another user's sockets). Never kills anything. -# Per platform: lsof (macOS, Linux), then ss (Linux without lsof), netstat -ano + tasklist (Windows). -port_holder() { - local p="$1" out pid="" name="" - case "$(mxtk_platform)" in - windows) - pid="$(netstat -ano 2>/dev/null | tr -d '\r' | awk -v p=":$p" '$1=="TCP" && $4=="LISTENING" && substr($2, length($2)-length(p)+1)==p { print $5; exit }')" - [ -n "$pid" ] && name="$(tasklist //FI "PID eq $pid" //FO CSV //NH 2>/dev/null | tr -d '\r' | head -1 | cut -d'"' -f2)" ;; - *) - if command -v lsof >/dev/null 2>&1; then - out="$(lsof -nP -iTCP:"$p" -sTCP:LISTEN -Fpc 2>/dev/null)" - pid="$(printf '%s\n' "$out" | sed -n 's/^p//p' | head -1)" - name="$(printf '%s\n' "$out" | sed -n 's/^c//p' | head -1)" - fi - if [ -z "$pid" ] && command -v ss >/dev/null 2>&1; then - out="$(ss -ltnpH "sport = :$p" 2>/dev/null | head -1)" - pid="$(printf '%s' "$out" | sed -n 's/.*pid=\([0-9][0-9]*\).*/\1/p')" - name="$(printf '%s' "$out" | sed -n 's/.*users:(("\([^"]*\)".*/\1/p')" - fi ;; - esac - if [ -n "$pid" ]; then printf '%s|%s\n' "$pid" "${name:-unknown}"; return 0; fi - if (exec 3<>"/dev/tcp/127.0.0.1/$p") 2>/dev/null; then printf '?|an unidentified process\n'; return 0; fi - return 1 -} -# holder_text "|" — "studiopro pid 4711" plus its parent where ps can show one, because a -# runtime Studio Pro started shows up as plain "java"; the parent is what names the owner. -holder_text() { - local pid="${1%%|*}" name="${1#*|}" pp pn - [ "$pid" = "?" ] && { printf '%s (pid unknown — no lsof/ss/netstat could name it)\n' "$name"; return; } - printf '%s pid %s' "$name" "$pid" - if [ "$(mxtk_platform)" != windows ]; then - pp="$(ps -o ppid= -p "$pid" 2>/dev/null | tr -d ' ')" - pn="$(ps -o comm= -p "${pp:-0}" 2>/dev/null | sed 's#.*/##')" - [ -n "$pp" ] && [ "$pp" != 1 ] && [ -n "$pn" ] && printf ' (parent: %s pid %s)' "$pn" "$pp" - fi - printf '\n' -} +# Port and process helpers (mxtk_port_holder, mxtk_holder_text, mxtk_pid_in_tree) live in +# _common.sh, shared with test-stack-up.sh. A project whose bin/_common.sh predates them stops here +# with the remedy, instead of dying mid-restart on "command not found". +command -v mxtk_port_holder >/dev/null 2>&1 || { + echo "app-restart: bin/_common.sh is older than this script (no mxtk_port_holder) — run the toolkit's bin/sync-project.sh" >&2; exit 2; } # blocked "|" — exit 4, the holder named, nothing of it touched. blocked() { - echo "✗ app-restart: instrument blocked — port :$1 is held by $(holder_text "$2"), not this project's run --local." >&2 + echo "✗ app-restart: instrument blocked — port :$1 is held by $(mxtk_holder_text "$2"), not this project's run --local." >&2 echo " $3. This script never stops a process it did not record, so it is not touching that one." >&2 echo " Starting now would collide: mxcli reports 'Authentication failed' against the other runtime's admin API." >&2 echo " → Stop that process yourself (Studio Pro: stop its run), or start on free ports:" >&2 @@ -179,15 +145,7 @@ blocked() { exit 4 } # in_our_tree — the recorded pid itself, or a descendant of it (the runtime mxcli started). -in_our_tree() { - local p="$1" i=0 - [ "$ALIVE" = 1 ] || return 1 - while [ -n "$p" ] && [ "$p" != 0 ] && [ "$p" != 1 ] && [ "$i" -lt 10 ]; do - [ "$p" = "$PID" ] && return 0 - p="$(ps -o ppid= -p "$p" 2>/dev/null | tr -d ' ')"; i=$((i + 1)) - done - return 1 -} +in_our_tree() { [ "$ALIVE" = 1 ] && mxtk_pid_in_tree "$1" "$PID"; } # ── 1. The record ─────────────────────────────────────────────────────────── [ -n "$HS" ] || refuse "no run --local record for this project (.mxcli/run-local.json beside $(basename "$MPR"))." \ @@ -254,7 +212,7 @@ echo " ports: app :$PORT, admin :$ADMIN_PORT" # pids), it is left to the post-stop check below, which is exact: after our tree is gone, whatever # still holds the port is not ours. for _p in "$PORT" "$ADMIN_PORT"; do - _h="$(port_holder "$_p")" || continue + _h="$(mxtk_port_holder "$_p")" || continue if [ "$ALIVE" = 1 ]; then case "${_h%%|*}" in '?') continue ;; esac [ "$(mxtk_platform)" = windows ] && continue @@ -282,12 +240,12 @@ fi _busy="" for _i in $(seq 1 30); do _busy="" - for _p in "$PORT" "$ADMIN_PORT"; do port_holder "$_p" >/dev/null && { _busy="$_p"; break; }; done + for _p in "$PORT" "$ADMIN_PORT"; do mxtk_port_holder "$_p" >/dev/null && { _busy="$_p"; break; }; done [ -z "$_busy" ] && ! port_answers "$PORT" && break sleep 1 done if [ -n "$_busy" ]; then - blocked "$_busy" "$(port_holder "$_busy" || echo '?|an unidentified process')" "The recorded app was stopped; nothing was started" + blocked "$_busy" "$(mxtk_port_holder "$_busy" || echo '?|an unidentified process')" "The recorded app was stopped; nothing was started" elif port_answers "$PORT"; then blocked "$PORT" "?|an unidentified process" "The recorded app was stopped; nothing was started" fi diff --git a/project-bin/test-stack-up.sh b/project-bin/test-stack-up.sh index 17af56ad..dfc0e141 100755 --- a/project-bin/test-stack-up.sh +++ b/project-bin/test-stack-up.sh @@ -28,6 +28,8 @@ # bin/test-stack-up.sh # report, then bring up what is missing (may run a Docker build) # bin/test-stack-up.sh --no-docker # bring up auxiliaries only; never build. "SP is already running it". # bin/test-stack-up.sh --restart # skip the hot reload; rebuild and restart the container (old path) +# bin/test-stack-up.sh --track [label] # record a background test process THIS project started +# bin/test-stack-up.sh --reap # stop recorded processes older than MXTK_STALE_HOURS; nothing else # # WARM LOOP. After a model change the old loop was: exec.sh gate (mxbuild) -> `docker run` (mx check + # mxbuild again) -> container restart -> boot wait. Now: the app's model is recorded in @@ -70,6 +72,34 @@ # RUN_LOCAL_ADMIN_PORT (stack.conf), else mxcli's defaults 8080 / app port + 10 — and exits 4 # naming the holder (lsof / ss; netstat on Windows) instead of advising a start that cannot work. # +# RUNTIME PROOF WITHOUT A HANDSHAKE (#160, field run 2026-09-25, requirements-driven build). An app +# with no run-local.json — Studio Pro's Run Locally, or a run --local older than the handshake — was +# found by scan and published APP_OWNERSHIP=unverified, so every journey run needed APP_PORT +# asserted by hand. The proof was there all along: the runtime's command line runs +# `runtimelauncher.jar /app/deployment`, and a deployment directory belongs to one checkout. +# So a scanned Mendix port whose listener (lsof / ss / netstat) runs runtimelauncher.jar on THIS +# project's deployment dir (/deployment, /app/deployment, or beside the real .mpr) is +# `verified` with APP_SOURCE=runtime. Freshness stays `unknown` — this script did not build it. +# Command line: /proc on Linux, `ps -o args=` on macOS, PowerShell CIM on Git Bash (_common.sh's +# mxtk_pid_runs_deployment). Unreadable → `unverified`, exactly as before. +# +# OWN PROCESS ON THE PORT IS NOT A THIEF (follow-up to #262). The port check above must not name +# this project's own runtime: while it boots it binds its ports before it serves a page, and the +# check said "INSTRUMENT BLOCKED" (exit 4) about it. A holder is ours — reported NOT READY, exit 1 +# — when it is the live recorded run --local pid or a descendant of it, or when its command line +# runs this project's deployment dir (that includes Studio Pro running THIS project). Anything else +# still blocks. +# +# STALE TEST PROCESSES (#278, field run 2026-10-12). A night of retries left journey-runner and +# monitor shells running for hours; a later run then hit the trial licence's session cap and was +# INVALID. Every run of this script lists the recorded processes older than MXTK_STALE_HOURS +# (default 4) and prints the one command that stops them; --reap does it. It acts ONLY on +# .claude/loop/bg-pids.tsv — lines ` TAB TAB TAB