Skip to content

fix(exec): pause the project watcher during the gate and retry once on an infra exit - #451

Open
MendixMau wants to merge 1 commit into
masterfrom
fix/gate-watcher-race
Open

MendixMau wants to merge 1 commit into
masterfrom
fix/gate-watcher-race

Conversation

@MendixMau

@MendixMau MendixMau commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Problem

The exec gate's mxbuild and a live mxcli run --local --watch on the same project both write deployment/. In a field migration project (12 modules), 17 of 27 failed gates were "gate could not run" (mxbuild exit 1 or 3, no CE code, no errors file), against 0 of 22 gates run in watcher-free copies of the same project. Each cost 2 to 4 minutes of rerun plus blocked guards.

Refs #441, #444, #171, #285, #424

Change

  • project-bin/_common.sh: mxtk_gate_mxbuild wraps mxtk_mxbuild_error_count. It stops this project's watcher (SIGSTOP, found by pgrep on run --local --watch, matched by project path or cwd) for the length of the gate and continues it (SIGCONT) after. mxbuild has no option to move the deploy target's output out of deployment/, so a separate output dir is not available; pausing was the simplest robust choice. Only the watcher process is stopped, not its runtime.
  • Retry: mxbuild exit 1 or 3 with no CE code in the output and an empty errors file is retried once, printed as "retry after infra exit" and added to the BUILD-LOG detail. An errors file with content (for example a version mismatch) is deterministic and is not retried.
  • project-bin/exec.sh: the three gate call sites (baseline, gate, whose-error rebuild) use the wrapper; the EXIT trap resumes the watcher; the fix(exec): roll back the model write when mxbuild crashes #424 rollback is untouched and applies to the final failure.
  • CHANGELOG entry.

Test

New tests/wave2/test-exec-gate-retry.sh (stub mxcli, mxbuild, java; no real mxbuild): A) infra exit then clean: retried, kept, row and output carry the note; B) infra exit twice: retried once, rolled back, model byte-identical; C) a real CE error list is not retried; D) a live stub watcher is in state T during both mxbuild runs and running afterwards. 14 of 14 pass. Existing test-exec-crash-rollback.sh (13) and test-exec-lock-watch.sh (84) pass. check-pr-discipline.sh and check-portability.sh (changed files) are clean. test-common-windows.sh has 2 ELF failures that are identical on unmodified master.

Risk

  • A watcher rebuild already in flight when the pause starts can still collide; the retry covers it.
  • If exec.sh is SIGKILLed during the gate, the watcher stays stopped until kill -CONT <pid>.
  • A deterministic infra failure with no errors file now costs one extra mxbuild before it is reported.
  • Touches the same BUILD-LOG detail line as the full-log PR (fix/gate-full-log); a one-line rebase conflict is possible.

🤖 Generated with Claude Code

… on an infra exit

The gate's mxbuild and a live `mxcli run --local --watch` both write deployment/.
In a field migration project (12 modules) 17 of 27 failed gates were "gate could
not run" (mxbuild exit 1, no CE code), against 0 of 22 in watcher-free copies.

The watcher of this project is stopped for the gate and resumed after (and in the
EXIT trap). An exit 1 or 3 with no CE code and no errors file is retried once and
logged as "retry after infra exit". A final failure still takes the #424 rollback.

Refs #441, #442, #171, #285, #424

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant