Skip the vlt compat matrix on ci.yml-only changes

[Mikola Lysenko (mikolalysenko)]

No open ci-perf issue was eligible this run. #1170, #1267, #1225, #1176, #1171, #1172, #1173 and #1174 all change ci.yml, which open PR #1247 also changes. #1248 is a settings change. This is measurable waste outside ci.yml and outside the six compat files in #1275.

Problem

vlt-compatibility.yml lists .github/workflows/ci.yml in both its pull_request and push filters. It does that because install-proof leaves out the cells that ci.yml's vlt e2e rows already run (scripts/ci-vlt-proof-suites.py). Most ci.yml edits don't touch those rows, though, and every one of them reran the whole matrix:

run	Linux	Windows	macOS
PR (37939237267)	52 jobs / 41 min	20 / 38	–
push (37942116119, 37939225360)	52 / 39	20 / 36–40	16 / 21–23

I replayed the last 24h through the new gate:

• PRs: 24 of the 83 PRs merged to main triggered the vlt PR filter. In 10 of them, ci.yml was the only matching file and the vlt cells didn't change.
• Push: 45 vlt push runs, each diffed against the previous main push. 5 would skip: the pushes of Fix Maven e2e failing on Central blips after warm-up #1256, Build coverage-docker's binary once per leg #1261, Run e2e suites that share a toolchain in one leg #1234, Fix macOS Composer legs red on main via setup-php 2.40.0 #1215 and Stop Gradle e2e legs failing on services.gradle.org 500s #1165.

Change

• New changes job (ubuntu, ~10 s): runs scripts/vlt-compat-gate.py. It prints matrix=false only when both of these hold:

  • the one changed file that the event's own filter matches is ci.yml (it reads the paths: list from the workflow file, so the list isn't duplicated);
  • ci_cells() from ci-vlt-proof-suites.py returns the same cells on base and head.

  Base is the merge commit's parent 1 on a PR and github.event.before on push. schedule, workflow_dispatch, a missing base and any error all print matrix=true.

• build and plan now need: changes and run only on matrix == 'true'. install-proof, install-proof-macos, native, canary and downgrade follow through needs. lock-diff (!cancelled()) gets the same condition explicitly.

• matrix-coverage (test_ci_vlt_rows.py, which checks ci.yml's vlt rows and hosted-e2e wiring) still runs on every trigger.

• scripts/tests/test_vlt_compat_gate.py: the decision, the glob matching and the job wiring. ci.yml's python3 -m unittest discover -s scripts/tests picks it up. Both filters also list the gate script and its test.

Expected saving

• PR: 10 PRs/day × ~1.7 non-draft pushes per PR × (41 Linux + 38 Windows) ≈ 700 Linux + 650 Windows job-min/day.
• Push: 5 runs/day × (39 Linux + 37 Windows + 22 macOS) ≈ 195 Linux + 185 Windows + 110 macOS job-min/day.
• Total ≈ 0.9k Linux + 0.83k Windows + 0.11k macOS job-min/day, ≈ 2.9k on the dashboard's L×1 / W×2 / M×3 weighting. That is 20 Windows jobs less per skipped run, at a time when Windows queue wait peaked at 6.3 min.
• The workflow doesn't gate merging, so the merge-queue critical path doesn't change.

Measured result

This PR edits `vlt-compatibility.yml` and the gate script, so its own run must run the matrix. The skip itself only shows on later ci.yml-only PRs and pushes, which the profiler can verify.

• vlt run 37947320274 on 2be0fcd: green in 11.3 min.
  • The new `changes` job took 7 s and answered `matrix=true` here, as it should.
  • The matrix ran in full: 53 Linux jobs / 41 min and 20 Windows / 35 min. That matches the 41 / 38 baseline, so the gate adds about 0.1 job-min to a run that does need the matrix.
• `CI` (37947320377): green, with `ci-ok` and `clippy` passing. That run includes `unittest discover -s scripts/tests`, which picks up the new `test_vlt_compat_gate.py`.
• Replay of the last 24h through the gate: 10 of 24 vlt-triggering PRs and 5 of 45 vlt push runs answer `matrix=false`.
• Validation: `actionlint` gives the same 6 pre-existing findings before and after. `zizmor --offline` reports the same 9 unsuppressed low findings before and after. `python3 -m unittest discover -s scripts/tests` passes 301 tests.
• Bugbot:
  • The security review found that `--name-only` + `split()` dropped spaced or non-ASCII paths and the old side of a rename. Fixed in 2be0fcd: the gate now uses `-z --no-renames` and has a regression test against a scratch repo.
  • The re-review found no issues.

Where each test still runs

• Nightly: every vlt-compatibility job still runs in full (schedule always gives matrix=true), and so does workflow_dispatch.
• PR or push changing a vlt file, shared engine code (push) or ci.yml's vlt rows: runs the matrix as before.
• PR or push changing only ci.yml outside its vlt rows: the ci.yml rows the matrix defers to run in ci.yml itself, and matrix-coverage still checks them. The rest of the matrix would run exactly as it did on the base, and it runs again on the next relevant change or nightly.
• ci-ok and clippy are untouched.

Risk

• Hidden ci.yml dependency: a ci.yml change could affect the vlt matrix in a way ci_cells() doesn't capture. Today it is the only part of ci.yml the matrix reads (ci-vlt-proof-suites.py in install-proof). A future reader would need to widen the gate.
• Unparseable filter: if the paths: parser misreads the filter, it returns an empty list, and the gate answers matrix=true.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YVLGaJFexMvqCdiWxbiHXB

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 2be0fcd. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Dequeued on CI_FAILURE, but the failure isn't from this PR. The merge-group CI run 37954543526 failed on hosted-e2e and e2e (ubuntu-latest, e2e_safety_pnpm).

• Both jobs fail the same way on main's own push runs (37954163319, 37954292636, 37954539425). They also fail on the queue entries for Retry CI toolchain installs past rustup download blips #1286, Wire vendored nuget.config through formats::nuget (#594) #1288 and Fix bun.lockb shared bundled pin being unmanageable (#1243) #1247.
• Tracked in Merge queue blocked: production no longer publishes free minimist@1.2.2 patch #1293: production no longer serves the free minimist@1.2.2 patch these suites rely on. No fix PR exists yet.
• This PR only changes vlt-compatibility.yml and two Python scripts, and neither suite reads them. Its PR-head CI is green.

Nothing to port. It needs re-queueing once #1293 is fixed. I'm not re-queueing it myself.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Ready for review (burn-down) at 2be0fcd3b.

• CI: 260/260 checks green on this head (248 success, 11 skipped, 1 neutral). Mergeable, no conflicts. A main-wide failure dequeued it earlier: hosted-e2e and e2e (…, e2e_safety_pnpm) are red on main's own push runs because production no longer serves the free minimist@1.2.2 patch. That isn't this PR's.
• Bugbot: reviewed 2be0fcd3b with no new findings. Its one earlier security finding (the rename case in the changed-file list) was fixed in 2be0fcd and the thread is resolved. Approved by Tanmay182003 on this head.
• Reviewers: scripts/vlt-compat-gate.py decides whether a ci.yml-only change touches the vlt e2e rows. Check its fallback: when the diff can't be read, the full matrix runs.

---

Generated by Claude Code