Skip to content

test: remove redundant unit tests and fold lookup tables - #882

Open
Brooooooklyn wants to merge 1 commit into
mainfrom
claude/prune-tests-redundant-unit-tests
Open

Brooooooklyn wants to merge 1 commit into
mainfrom
claude/prune-tests-redundant-unit-tests

Conversation

@Brooooooklyn

Copy link
Copy Markdown
Member

Requested by Brooklyn · project thread

Before: five unit tests could not fail on a regression that no other test catches: a f() == f() tautology, a "hash twice and compare" check, and three tests whose assertions are a subset of a neighbour's. Two fspy_shared_unix tables and the labeled-writer tests were hand-unrolled into one #[test] per row.

After: the five tests are gone, and the three tables are table-driven tests that keep every row, its inputs and its expected value. Only #[cfg(test)] code changes.

How: each removal was defended by mutating product code with the most realistic regression it could catch, running the crate's tests, and reverting.

Totals

  • Files touched: 8, test modules only.
  • Cases deleted: 5. Cases merged: 18 rows folded into 3 table-driven tests (21 tests become 3). Across the six crates run (fspy_shared_unix, vt, vt_glob, vt_powershell, vt_workspace, vt_plan lib tests): 282 → 259 cases. The whole workspace was not run (see below), so there is no workspace-wide total.
  • Net lines: −169 (+107 / −276).
  • Runtime saved: about 34 ms, the summed duration of the deleted cases (npm_workspace_with_exclusions 28 ms, test_file_hashes_are_consistent 6 ms, the rest under 1 ms).

Removals

  1. vt_powershell tests::is_stdin_terminal_is_idempotent — tautology: assert_eq!(is_stdin_terminal(), is_stdin_terminal()) on a LazyLock. Mutation R1 (negate the cached is_terminal()): the test passes, as does everything else in vt_powershell and vt_plan. The behaviour that depends on it is covered by vt_plan ps1_shim::tests::skips_rewrite_when_not_interactive (mutation R1b, skipping the !is_interactive check, fails it).
  2. vt session::execute::glob::tests::test_file_hashes_are_consistent — determinism check. Mutation R2 (seed the file hasher randomly per call) fails it and also session::execute::hash::tests::other_files_hash_raw_content and redacted_file_hashes_without_content, which pin exact hash values.
  3. vt session::reporter::grouped::writer::tests::buffers_output — subset of shared_buffer_interleaves, which appends several chunks from two writers to one shared buffer and checks the joined result. Mutation R5 (replace instead of append) fails both.
  4. vt_glob path::tests::test_match_ignores_directory_only_patterns — build/** with !build/keep/**; test_match_ignores_directory_patterns asserts the same shapes (build/**, dist/** with !dist/public/**). Mutations R4 (ignore negations) and R4b (treat negated globs as positive) fail 8 and 9 other path tests.
  5. vt_workspace tests::test_get_package_graph_npm_workspace_with_exclusions — despite its name it only asserts that the root and packages/normal are found; its closing comment says exclusions "might not work" and asserts nothing about them. Mutation R3 (drop negated workspace globs): it passed (run at the earlier base), while test_get_package_graph_workspace_exclusions, …_workspace_work_with_last_match_wins and …_npm_workspace_with_root_relative_prefixes fail. Mutation R3b (stop adding the root package) fails 13 other graph tests, test_get_package_graph_npm_workspace among them.

Folds (every row kept)

File New test Rows Mutation that fails it
fspy_shared_unix/src/exec/mod.rs appends_value_to_the_env_list 7 RF2: always prepend :
fspy_shared_unix/src/exec/which.rs test_which 8 RF3: ENAMETOOLONG → ENOENT
vt/src/session/reporter/labeled/writer.rs prefixes_each_line 6 RF1: drop the prefix

Each assertion carries the row's name, so a failure still names the case.

Kept after defense

  • vt_subprocess_test tests::test_command_for_fn was nominated and kept: its defense could not be completed safely (see "What ran").

Case counts (lib tests)

crate before after delta
fspy_shared_unix 30 17 −13 (folds)
vt 88 81 −7 (2 deleted, 5 folded)
vt_glob 23 22 −1
vt_powershell 6 5 −1
vt_workspace 85 84 −1
vt_plan 50 50 0

0 failures before and after.

What ran and what did not

  • Ran on Linux at 0346c268 and on this branch: the lib tests of the six crates above with --report-time; mutations R1–R5, R1b, R3b, R4b and RF1–RF3, each reverted afterwards; cargo fmt --check on the touched files; cargo clippy -p fspy_shared_unix -p vt -p vt_glob -p vt_powershell -p vt_workspace --tests -- -D warnings (clean).
  • Not run: the workspace-wide just lint and just test, macOS and Windows, and the e2e/snapshot suites. The vt_subprocess_test and vt_pty_terminal suites were deliberately not run: while defending test_command_for_fn, a mutation that swapped the child's id and payload arguments left children with an empty argument, which libtest took as a filter matching every test, so each child re-ran the suite and spawned without bound.

Open pull requests: of the 61 open, only #601 touches a file here, and it renames crates/vt/src/session/execute/glob.rs without changing its content.

Ages: all five deleted tests are older than a month (newest: is_stdin_terminal_is_idempotent, 2026-06-28). None was added for a bug, and none is in a permission or security suite.

To revert: git revert 2c324106.


Generated by Claude Code

Delete unit tests that cannot fail on a regression another test does not
already catch: a `f() == f()` tautology, a determinism check that the hash
tests already cover, and three strict-subset duplicates. Fold three
hand-unrolled lookup tables into table-driven tests, keeping every row and
comment.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxVsmvB9xfZKfVnUonY8kf
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-24T19:40:54.457436Z 2c32410 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -0.13%  [ -2.79% ..  +2.38%]  overhead   +49.76%
dynamic/access             change  -0.37%  [ -3.21% ..  +3.44%]  overhead   +18.57%
dynamic/access-relative    change  +0.89%  [ -1.74% ..  +3.15%]  overhead   +73.63%
dynamic/access-contended   change  +1.85%  [ -2.76% ..  +5.22%]  overhead   +38.27%
static/launch              change  +0.42%  [ -2.26% ..  +2.82%]  overhead  +183.99%
static/access              change  +0.06%  [ -0.69% ..  +0.73%]  overhead +1437.91%
static/access-relative     change  -0.30%  [ -1.74% ..  +0.54%]  overhead +1937.23%
static/access-contended    change  +0.07%  [ -0.77% ..  +1.07%]  overhead +2074.27%

macos

dynamic/launch             change  -0.30%  [ -4.94% ..  +4.29%]  overhead  +231.78%
dynamic/access             change  +0.00%  [-34.06% ..  +9.44%]  overhead    +5.43%
dynamic/access-relative    change  +0.00%  [ -5.62% ..  +5.96%]  overhead  +260.31%
dynamic/access-contended   change  +1.47%  [-17.70% .. +30.24%]  overhead    +5.54%

windows

dynamic/launch             change  +0.20%  [ -1.86% ..  +2.56%]  overhead   +24.65%
dynamic/access             change  +0.00%  [ -0.79% ..  +0.60%]  overhead    +1.16%
dynamic/access-relative    change  +0.00%  [ -0.58% ..  +0.95%]  overhead    +1.35%
dynamic/access-contended   change  -0.18%  [ -0.91% ..  +0.72%]  overhead    +0.95%

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants