Skip to content

ci: run mutation tests with Infection - #63

Merged
roxblnfk merged 2 commits into
2.xfrom
ci/mutation-testing
Oct 9, 2026
Merged

roxblnfk merged 2 commits into
2.xfrom
ci/mutation-testing

Conversation

@roxblnfk

@roxblnfk roxblnfk commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

What was changed

  • Mutation testing with Infection and the Testo adapter runs on every PR and on pushes to 2.x, with reports sent to the Stryker dashboard. The README gets a mutation score badge.
  • Thresholds follow the measured score: covered MSI 65% locally (568 mutants) and 63% on CI (569 mutants), so --min-msi=60 --min-covered-msi=60.
  • The get-binary and make-config tests now run in their own temporary working directory. Both commands skip config generation when the working directory already contains a .rr.yaml. A mutant wrote one into the project root, and the config tests of every later run then failed.

Why?

To measure test quality beyond line coverage. Most of the surviving mutants are in console output formatting and the GitHub token header, which the tests do not inspect.

Review notes

  • Infection is installed inside the workflow instead of from the phar. The Testo adapter bundled in the 0.35.6 phar calls the unscoped \Infection\StreamWrapper\IncludeInterceptor, so every mutant errors and is wrongly counted as killed. It stays out of require-dev because it needs PHP 8.3+.

Checklist

  • Tested
    • Infection and Testo run locally (PHP 8.4, Windows)

The commands skip config generation when the working directory already has a .rr.yaml. Under Infection a mutant wrote one into the project root, which then failed the config tests of every later run.

Assisted-By: Claude Opus 5.5
@roxblnfk
roxblnfk requested a review from a team October 9, 2026 20:37
@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 13 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 79a0d677-3708-4be5-853b-f47e4fbb6d44

📥 Commits

Reviewing files that changed from the base of the PR and between 8052c99 and 9f5abf8.


📒 Files selected for processing (6)
  • .gitattributes
  • .github/workflows/mutation.yml
  • README.md
  • infection.json5
  • tests/Unit/GetBinaryCommandTest.php
  • tests/Unit/MakeConfigCommandTest.php


  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Infection is installed in the workflow rather than from the phar: the Testo adapter bundled in the 0.35.6 phar writes a bootstrap that calls the unscoped `\Infection\StreamWrapper\IncludeInterceptor`, while the phar only ships `Infected\...`, so every mutant errors. It stays out of require-dev because it needs PHP 8.3+. Local run (Windows): 568 mutants, covered MSI 65%; CI (Linux): 569 mutants, covered MSI 63%, so both thresholds are 60.

Assisted-By: Claude Opus 5.5
@roxblnfk
roxblnfk force-pushed the ci/mutation-testing branch from 23e9f9f to 9f5abf8 Compare October 9, 2026 20:41
@roxblnfk
roxblnfk merged commit 3818233 into 2.x Oct 9, 2026
10 checks passed
@roxblnfk
roxblnfk deleted the ci/mutation-testing branch October 9, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant