test-stack-up: runtime command-line ownership proof, own runtime is not a port thief, tracked stale-process sweep (#160, #278)

CHANGELOG.md

@@ -17,6 +17,9 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are.
 
 ## Unreleased
 
+- fix(test-stack-up): **an app with no run --local handshake is `verified` when its listener runs `runtimelauncher.jar` on this project's own deployment dir** ([#160](https://github.com/MendixMau/mxcli-project-toolkit/issues/160)). Studio Pro's Run Locally, or a run --local without the handshake, was published `APP_OWNERSHIP=unverified`, so every journey run in a requirements-driven build needed `APP_PORT` asserted by hand, though the runtime's command line named the project's deployment dir all along. Both layouts (`<root>/deployment`, `<root>/app/deployment`); /proc on Linux, `ps` on macOS, PowerShell on Git Bash; unreadable stays unverified. The port/process helpers app-restart.sh and test-stack-up.sh each carried now live once in `_common.sh` (additive; an older `_common.sh` stops with the sync remedy, exit 2) — MendixMau
+- fix(test-stack-up): **this project's own booting runtime on its port reads NOT READY (exit 1), no longer "INSTRUMENT BLOCKED" (exit 4)** (refs [#262](https://github.com/MendixMau/mxcli-project-toolkit/issues/262), follow-up to #313). A runtime binds its ports before it serves a page, and the #262 port check named it as a thief; the same happened with `--no-docker` while Studio Pro booted this project. Ours = the live recorded pid or its descendant, or a command line on this project's deployment dir; any other holder still blocks — independent review of #313
+- new(test-stack-up, test-result-audit): **`--track <pid>` records a background test process this project started; every run lists the recorded ones older than `MXTK_STALE_HOURS` (4) and `--reap` stops them — by pid record only, a reused pid (start time changed) is never signalled** ([#278](https://github.com/MendixMau/mxcli-project-toolkit/issues/278)). Hung journey-runner shells from earlier retries filled a trial licence's session cap and made a later run INVALID; test-result-audit.md now names the two outside causes of INVALID (account lockout, licence session cap) — MendixMau
 - learn(testing-shape, test-result-audit): **three test-strength rules, each with a denominator: refusals probed on the server, every undeclared 4xx a failure, one clean-database final run** ([#172](https://github.com/MendixMau/mxcli-project-toolkit/issues/172)). A field comparison of two builds against one frozen spec found the toolkit build checked permission refusals only in the client (a hidden menu item), logged unexpected 400/422 responses as information, and reached 20/20 by stitching three runs. `testing-shape.md` §7 gains three report lines (`N of N restricted pairs probed server-side`, `undeclared 4xx/5xx: 0 of M requests — each fails its step`, `one clean-DB run <id>: N of N scenarios incl. timers, not stitched`); `test-result-audit.md` gains check 2e, which carries the reasons and the field evidence and asks for those numbers. Both baseline skills stay inside the 80,000-word baseline budget (79,997): the reasons live in the on-demand audit skill, and two redundant sentences in `testing-shape.md` §4's verdict paragraphs were folded (the empty-set corollary is stated in the Trace row, `harness-architecture.md` and `report-schema.md`). The backing inbox note is trimmed to its unpromoted "Keep" items. — MendixMau
 - learn(learned-detection-gaps, module-review): **two UI defects every model gate passes: a lazy-loaded combo box filter hides its preset, and a theme that pins row heights by position lets the next row cover an inserted widget** ([#299](https://github.com/MendixMau/mxcli-project-toolkit/issues/299)). Both passed check, mxbuild, lint and the scripted checks on a 127-module production app and were found only in the running app. Two register rows with the first oracle and the pre-build check (pluggable properties that change *when* data loads; grep the theme for fixed `height` on positional row classes before an insert), and two LOOK lines in `module-review.md` §4c with denominators (`N of N filters show their preset on first load`, `N of N widgets inserted into an existing container fully visible`). — jopterhorst
 - fix(app-restart, test-stack-up): **a port held by another process before `run --local` is named and reported as "instrument blocked" (exit 4), never killed** ([#262](https://github.com/MendixMau/mxcli-project-toolkit/issues/262)). Studio Pro had started its own runtime on the app/admin ports a restart needed; `mxcli run --local` then failed with "Authentication failed" (the admin API answering was Studio Pro's), the next journey ran against Studio Pro's runtime, and four more runs were NOT RUN. `app-restart.sh` now checks the app port and the admin port (record's `adminPort`, else app + 10) before stopping anything — a holder outside the recorded process tree exits 4 with its name, pid and parent — and again after the stop, where anything still holding a port is by definition not ours. `test-stack-up.sh` runs the same check wherever it would tell you to start `run --local` (`--no-docker`, no Docker daemon, `--check` with a leftover record), on the record's ports, `RUN_LOCAL_APP_PORT`/`RUN_LOCAL_ADMIN_PORT`, or mxcli's defaults 8080/8090. Holder lookup: lsof, then ss, then netstat + tasklist on Windows; unnamed holders still block when nothing of ours is running. Fixtures: `test-app-restart.sh` +2 cases (14 assertions, all fail on the old script), `test-stack-up-local.sh` T7 (4 of 6 fail on the old script). — AlvinnYeo

project-bin/_common.sh

@@ -787,3 +787,131 @@ mxtk_sha256() {
   else openssl dgst -sha256 | sed 's/^.*= //'
   fi
 }
+
+# ---------------------------------------------------------------------------
+# Process identity — who holds a port, whose process it is. Shared by test-stack-up.sh and
+# app-restart.sh (moved here from both, #160; they had carried identical copies since #262).
+# All read-only: nothing below stops, signals or starts a process.
+# ---------------------------------------------------------------------------
+
+# mxtk_port_holder <port> — who LISTENs on it: prints "<pid>|<process name>" and returns 0, or
+# returns 1 when the port is free. "?|an unidentified process" when something accepts connections
+# but no tool here can name it (lsof without root cannot see another user's sockets).
+# Per platform: lsof (macOS, Linux), then ss (Linux without lsof), netstat -ano + tasklist (Windows).
+mxtk_port_holder() {
+  local p="$1" out pid="" name=""
+  case "$(mxtk_platform)" in
+    windows)
+      pid="$(netstat -ano 2>/dev/null | tr -d '\r' | awk -v p=":$p" '$1=="TCP" && $4=="LISTENING" && substr($2, length($2)-length(p)+1)==p { print $5; exit }')"
+      [ -n "$pid" ] && name="$(tasklist //FI "PID eq $pid" //FO CSV //NH 2>/dev/null | tr -d '\r' | head -1 | cut -d'"' -f2)" ;;
+    *)
+      if command -v lsof >/dev/null 2>&1; then
+        out="$(lsof -nP -iTCP:"$p" -sTCP:LISTEN -Fpc 2>/dev/null)"
+        pid="$(printf '%s\n' "$out" | sed -n 's/^p//p' | head -1)"
+        name="$(printf '%s\n' "$out" | sed -n 's/^c//p' | head -1)"
+      fi
+      if [ -z "$pid" ] && command -v ss >/dev/null 2>&1; then
+        out="$(ss -ltnpH "sport = :$p" 2>/dev/null | head -1)"
+        pid="$(printf '%s' "$out" | sed -n 's/.*pid=\([0-9][0-9]*\).*/\1/p')"
+        name="$(printf '%s' "$out" | sed -n 's/.*users:(("\([^"]*\)".*/\1/p')"
+      fi ;;
+  esac
+  if [ -n "$pid" ]; then printf '%s|%s\n' "$pid" "${name:-unknown}"; return 0; fi
+  if (exec 3<>"/dev/tcp/127.0.0.1/$p") 2>/dev/null; then printf '?|an unidentified process\n'; return 0; fi
+  return 1
+}
+
+# mxtk_holder_text "<pid>|<name>" — "studiopro pid 4711" plus its parent where ps can show one,
+# because a runtime Studio Pro started shows up as plain "java"; the parent names the owner.
+mxtk_holder_text() {
+  local pid="${1%%|*}" name="${1#*|}" pp pn
+  [ "$pid" = "?" ] && { printf '%s (pid unknown — no lsof/ss/netstat could name it)\n' "$name"; return; }
+  printf '%s pid %s' "$name" "$pid"
+  if [ "$(mxtk_platform)" != windows ]; then
+    pp="$(ps -o ppid= -p "$pid" 2>/dev/null | tr -d ' ')"
+    pn="$(ps -o comm= -p "${pp:-0}" 2>/dev/null | sed 's#.*/##')"
+    [ -n "$pp" ] && [ "$pp" != 1 ] && [ -n "$pn" ] && printf ' (parent: %s pid %s)' "$pn" "$pp"
+  fi
+  printf '\n'
+}
+
+# mxtk_pid_in_tree <pid> <root pid> — 0 when <pid> is <root pid> or a descendant of it (walked up
+# by parent pid, at most 10 levels). Git Bash's ps shows no parent of a Windows pid: returns 1 there
+# unless the two pids are equal, so a caller must treat "not in tree" on Windows as "unknown".
+mxtk_pid_in_tree() {
+  local p="$1" root="$2" i=0
+  [ -n "$p" ] && [ -n "$root" ] || return 1
+  while [ -n "$p" ] && [ "$p" != 0 ] && [ "$p" != 1 ] && [ "$i" -lt 10 ]; do
+    [ "$p" = "$root" ] && return 0
+    p="$(ps -o ppid= -p "$p" 2>/dev/null | tr -d ' ')"; i=$((i + 1))
+  done
+  return 1
+}
+
+# mxtk_pid_args <pid> — the process's command line. Linux: argv one per line from /proc (exact, a
+# space inside an argument survives). macOS: `ps -o args=`, ONE line (arguments joined by spaces).
+# Windows (Git Bash): Win32_Process.CommandLine via PowerShell, one line — Git Bash's ps cannot
+# show a Windows process's arguments. Returns 1 when nothing could be read.
+mxtk_pid_args() {
+  local pid="$1" out=""
+  case "$pid" in ''|*[!0-9]*) return 1 ;; esac
+  case "$(mxtk_platform)" in
+    linux)
+      if [ -r "/proc/$pid/cmdline" ]; then out="$(tr '\0' '\n' < "/proc/$pid/cmdline" 2>/dev/null)"
+      else out="$(ps -o args= -p "$pid" 2>/dev/null)"; fi ;;
+    macos) out="$(ps -o args= -p "$pid" 2>/dev/null)" ;;
+    windows)
+      command -v powershell.exe >/dev/null 2>&1 \
+        && out="$(powershell.exe -NoProfile -NonInteractive -Command "(Get-CimInstance Win32_Process -Filter 'ProcessId=$pid').CommandLine" 2>/dev/null | tr -d '\r')" ;;
+  esac
+  [ -n "$out" ] || return 1
+  printf '%s\n' "$out"
+}
+
+# mxtk_pid_runs_deployment <pid> <deployment dir>... — 0 when <pid> is a Mendix runtime started on
+# one of the named deployment directories: its command line runs runtimelauncher.jar and the
+# argument after the jar resolves to one of them. That is how both `mxcli run --local` and Studio
+# Pro's Run Locally start the runtime (field capture, run --local, Linux, 2026-09-25:
+# `java … -jar …/runtimelauncher.jar <project>/app/deployment`). A deployment directory belongs to
+# exactly one project checkout, so this is an ownership claim, not a "looks like Mendix" guess.
+# Linux compares exact argv after resolving symlinks (and a relative argument against the process's
+# cwd); macOS and Windows have only the joined line, so the directory is matched as text in both
+# its logical and physical spellings (Windows: drive form, case-insensitive). Unreadable → 1.
+mxtk_pid_runs_deployment() {
+  local pid="$1" args line next="" arg="" cwd d cand want
+  shift
+  args="$(mxtk_pid_args "$pid")" || return 1
+  case "$args" in *runtimelauncher.jar*) ;; *) return 1 ;; esac
+  if [ "$(mxtk_platform)" = linux ] && [ -r "/proc/$pid/cmdline" ]; then
+    while IFS= read -r line; do
+      [ -n "$next" ] && { arg="$line"; break; }
+      case "$line" in *runtimelauncher.jar) next=1 ;; esac
+    done <<EOF_ARGS
+$args
+EOF_ARGS
+    [ -n "$arg" ] || return 1
+    case "$arg" in /*) ;; *) cwd="$(readlink "/proc/$pid/cwd" 2>/dev/null)" || return 1; arg="$cwd/$arg" ;; esac
+    arg="$(cd "$arg" 2>/dev/null && pwd -P)" || return 1
+    for d in "$@"; do
+      want="$(cd "$d" 2>/dev/null && pwd -P)" || continue
+      [ "$arg" = "$want" ] && return 0
+    done
+    return 1
+  fi
+  # One joined line: arguments are not delimited, so a directory is matched WHOLE — followed by the
+  # end of the line, a lone slash, or a space and a flag (" -"). A space and anything else could be
+  # the rest of a longer, spaced path: "/work/a" must not match "…runtimelauncher.jar /work/a b/x"
+  # (a foreign checkout read as ours, PR #319 review). Each <dir> arrives as ONE argument.
+  line="$(printf '%s' "$args" | head -1 | tr -d '"')"
+  [ "$(mxtk_platform)" = windows ] && line="$(printf '%s' "$line" | tr '\\' '/' | tr '[:upper:]' '[:lower:]')"
+  for d in "$@"; do
+    for cand in "${d%/}" "$(cd "$d" 2>/dev/null && pwd -P)"; do
+      [ -n "$cand" ] || continue
+      [ "$(mxtk_platform)" = windows ] && cand="$(native_path "$cand" | tr '\\' '/' | tr '[:upper:]' '[:lower:]')"
+      case "$line" in
+        *"runtimelauncher.jar $cand"|*"runtimelauncher.jar $cand/"|*"runtimelauncher.jar $cand -"*|*"runtimelauncher.jar $cand/ -"*) return 0 ;;
+      esac
+    done
+  done
+  return 1
+}

project-bin/app-restart.sh

@@ -129,48 +129,15 @@ mask() {  # print argv joined, with the value after any password/secret/token/ke
     "s/(-[-A-Za-z_]*([Pp][Aa][Ss][Ss]|[Ss][Ee][Cc][Rr][Ee][Tt]|[Tt][Oo][Kk][Ee][Nn]|[Kk][Ee][Yy])[-A-Za-z_]*)(=| +)[^ ']+/\1\3****/g"
 }
 
-# port_holder <port> — who LISTENs on it: prints "<pid>|<process name>" and returns 0, or returns 1
-# when the port is free. "?|an unidentified process" when something accepts connections but no
-# tool here can name it (lsof without root cannot see another user's sockets). Never kills anything.
-# Per platform: lsof (macOS, Linux), then ss (Linux without lsof), netstat -ano + tasklist (Windows).
-port_holder() {
-  local p="$1" out pid="" name=""
-  case "$(mxtk_platform)" in
-    windows)
-      pid="$(netstat -ano 2>/dev/null | tr -d '\r' | awk -v p=":$p" '$1=="TCP" && $4=="LISTENING" && substr($2, length($2)-length(p)+1)==p { print $5; exit }')"
-      [ -n "$pid" ] && name="$(tasklist //FI "PID eq $pid" //FO CSV //NH 2>/dev/null | tr -d '\r' | head -1 | cut -d'"' -f2)" ;;
-    *)
-      if command -v lsof >/dev/null 2>&1; then
-        out="$(lsof -nP -iTCP:"$p" -sTCP:LISTEN -Fpc 2>/dev/null)"
-        pid="$(printf '%s\n' "$out" | sed -n 's/^p//p' | head -1)"
-        name="$(printf '%s\n' "$out" | sed -n 's/^c//p' | head -1)"
-      fi
-      if [ -z "$pid" ] && command -v ss >/dev/null 2>&1; then
-        out="$(ss -ltnpH "sport = :$p" 2>/dev/null | head -1)"
-        pid="$(printf '%s' "$out" | sed -n 's/.*pid=\([0-9][0-9]*\).*/\1/p')"
-        name="$(printf '%s' "$out" | sed -n 's/.*users:(("\([^"]*\)".*/\1/p')"
-      fi ;;
-  esac
-  if [ -n "$pid" ]; then printf '%s|%s\n' "$pid" "${name:-unknown}"; return 0; fi
-  if (exec 3<>"/dev/tcp/127.0.0.1/$p") 2>/dev/null; then printf '?|an unidentified process\n'; return 0; fi
-  return 1
-}
-# holder_text "<pid>|<name>" — "studiopro pid 4711" plus its parent where ps can show one, because a
-# runtime Studio Pro started shows up as plain "java"; the parent is what names the owner.
-holder_text() {
-  local pid="${1%%|*}" name="${1#*|}" pp pn
-  [ "$pid" = "?" ] && { printf '%s (pid unknown — no lsof/ss/netstat could name it)\n' "$name"; return; }
-  printf '%s pid %s' "$name" "$pid"
-  if [ "$(mxtk_platform)" != windows ]; then
-    pp="$(ps -o ppid= -p "$pid" 2>/dev/null | tr -d ' ')"
-    pn="$(ps -o comm= -p "${pp:-0}" 2>/dev/null | sed 's#.*/##')"
-    [ -n "$pp" ] && [ "$pp" != 1 ] && [ -n "$pn" ] && printf ' (parent: %s pid %s)' "$pn" "$pp"
-  fi
-  printf '\n'
-}
+# Port and process helpers (mxtk_port_holder, mxtk_holder_text, mxtk_pid_in_tree) live in
+# _common.sh, shared with test-stack-up.sh. A project whose bin/_common.sh predates them stops here
+# with the remedy, instead of dying mid-restart on "command not found".
+command -v mxtk_port_holder >/dev/null 2>&1 || {
+  echo "app-restart: bin/_common.sh is older than this script (no mxtk_port_holder)." >&2
+  echo "  → <toolkit>/bin/sync-project.sh $PROJECT_ROOT --upgrade-bin _common.sh   (backs up a hand-edited copy first)" >&2; exit 2; }
 # blocked <port> "<pid>|<name>" <when> — exit 4, the holder named, nothing of it touched.
 blocked() {
-  echo "✗ app-restart: instrument blocked — port :$1 is held by $(holder_text "$2"), not this project's run --local." >&2
+  echo "✗ app-restart: instrument blocked — port :$1 is held by $(mxtk_holder_text "$2"), not this project's run --local." >&2
   echo "  $3. This script never stops a process it did not record, so it is not touching that one." >&2
   echo "  Starting now would collide: mxcli reports 'Authentication failed' against the other runtime's admin API." >&2
   echo "  → Stop that process yourself (Studio Pro: stop its run), or start on free ports:" >&2
@@ -179,15 +146,7 @@ blocked() {
   exit 4
 }
 # in_our_tree <pid> — the recorded pid itself, or a descendant of it (the runtime mxcli started).
-in_our_tree() {
-  local p="$1" i=0
-  [ "$ALIVE" = 1 ] || return 1
-  while [ -n "$p" ] && [ "$p" != 0 ] && [ "$p" != 1 ] && [ "$i" -lt 10 ]; do
-    [ "$p" = "$PID" ] && return 0
-    p="$(ps -o ppid= -p "$p" 2>/dev/null | tr -d ' ')"; i=$((i + 1))
-  done
-  return 1
-}
+in_our_tree() { [ "$ALIVE" = 1 ] && mxtk_pid_in_tree "$1" "$PID"; }
 
 # ── 1. The record ───────────────────────────────────────────────────────────
 [ -n "$HS" ] || refuse "no run --local record for this project (.mxcli/run-local.json beside $(basename "$MPR"))." \
@@ -254,7 +213,7 @@ echo "  ports: app :$PORT, admin :$ADMIN_PORT"
 # pids), it is left to the post-stop check below, which is exact: after our tree is gone, whatever
 # still holds the port is not ours.
 for _p in "$PORT" "$ADMIN_PORT"; do
-  _h="$(port_holder "$_p")" || continue
+  _h="$(mxtk_port_holder "$_p")" || continue
   if [ "$ALIVE" = 1 ]; then
     case "${_h%%|*}" in '?') continue ;; esac
     [ "$(mxtk_platform)" = windows ] && continue
@@ -282,12 +241,12 @@ fi
 _busy=""
 for _i in $(seq 1 30); do
   _busy=""
-  for _p in "$PORT" "$ADMIN_PORT"; do port_holder "$_p" >/dev/null && { _busy="$_p"; break; }; done
+  for _p in "$PORT" "$ADMIN_PORT"; do mxtk_port_holder "$_p" >/dev/null && { _busy="$_p"; break; }; done
   [ -z "$_busy" ] && ! port_answers "$PORT" && break
   sleep 1
 done
 if [ -n "$_busy" ]; then
-  blocked "$_busy" "$(port_holder "$_busy" || echo '?|an unidentified process')" "The recorded app was stopped; nothing was started"
+  blocked "$_busy" "$(mxtk_port_holder "$_busy" || echo '?|an unidentified process')" "The recorded app was stopped; nothing was started"
 elif port_answers "$PORT"; then
   blocked "$PORT" "?|an unidentified process" "The recorded app was stopped; nothing was started"
 fi