Skip to content

CI: hermetic spawns in apply/rollback tests (#823) - #1405

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-ci-hermetic-apply-rollback
Oct 12, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-ci-hermetic-apply-rollback

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Refs #823 (slice 3; the issue stays open for the remaining files).

What it does

Test-only. The apply/, rollback/, remove/ and get/ suites spawned socket-patch with a bare Command::new. Most swept SOCKET_* by hand, but apply_invariants and in_process_npm_multicopy did not, so a shell exporting SOCKET_DRY_RUN, SOCKET_JSON or SOCKET_ECOSYSTEMS failed 15 of their tests for reasons unrelated to the code under test.

  • 13 files now spawn through common::hermetic_command. Their hand-rolled sweeps, hostile seeds and duplicate SOCKET_TELEMETRY_DISABLED / SOCKET_CLI_API_TOKEN lines are gone, because the builder already forces or removes those.
  • New hermetic::Extra::Bundle replaces the gem suites' private BUNDLE_* sweep (4 files). It seeds BUNDLE_APP_CONFIG, which the gem crawler reads, and then scrubs it. cli_gem_variant_mismatch_policy had no BUNDLE_* scrub before and now gets one. spawn_env_hygiene gains a contract test for it.
  • get/get_edge_cases_e2e.rs was already hermetic (it uses common::run_with_env). It was on the list only because a comment quoted the bare-spawn spelling the detector matches, so that comment is reworded.
  • All 14 files leave PENDING_RAW_SPAWNS (140 → 126), so the ratchet keeps them hermetic.

Left out to avoid conflicts: apply/apply_network.rs and rollback/rollback_multicopy_blob_gate.rs, both changed by open PR #1402.

Evidence (red → green)

Hostile shell: SOCKET_DRY_RUN=true SOCKET_JSON=true SOCKET_ECOSYSTEMS=cargo SOCKET_OFFLINE=true BUNDLE_PATH=/nonexistent/bundle.

target origin/main this branch
apply (migrated files) 15 fail (6 apply_invariants, 9 in_process_npm_multicopy) 0 fail
rollback, remove, get pass pass

apply_network (9 more failures under that shell, on both main and this branch) is the excluded file above.

Normal environment: spawn_env_hygiene 14 passed, apply 132 passed (1 ignored), rollback 57 passed, remove 99 passed, get 86 passed. cargo fmt --all -- --check is clean. Running clippy on these test targets reports only findings that are already on main, in the unchanged common/mod.rs and prebuilt_common/mod.rs. CI's clippy job does not lint test targets.

CI on 49e78d7: all 86 checks either passed or were skipped (including ci-ok and clippy). Bugbot found no issues.

/code-review high

I fixed these findings in 49e78d7: the missing BUNDLE_* scrub in cli_gem_variant_mismatch_policy, a hostile seed the crawler didn't read, dead SOCKET_CLI_API_TOKEN / telemetry lines, and comments that ran past the wrap width. I left these out on purpose:

  • Scrub BUNDLE_* in hermetic::command for every child. Suites that use run_with_env get no bundler scrub. For example, check_verifies_installed_tree::check_fails_on_an_unpatched_copy_in_a_refused_bundle_path fails under an ambient BUNDLE_APP_CONFIG, on main as well as here. Moving the sweep into the shared builder changes the environment of every CLI test child, so it's a follow-up for Spawn every CLI test child through one hermetic Command builder; 10 test files inherit ambient SOCKET_* today #823, not part of this slice.
  • Use hermetic::binary_command() instead of hermetic_command(&binary()). I kept the spelling slice 2 (CI: hermetic spawns in scan/repair tests (#823) #1398) used. A tree-wide switch can be its own cleanup.
  • Share one rollback_cmd between the two rollback invariant files. That's a helper refactor and out of scope here.

CI cost

No workflow change. A lean PR run is unchanged: same jobs, roughly the same job-minutes (about 28 Linux jobs, about 8 min wall time).

🤖 Generated with Claude Code

https://claude.ai/code/session_013sMuYxiSNRUULtv88CtBGM


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
The apply, rollback, remove and get test suites spawned socket-patch
with a bare Command::new. Most swept SOCKET_* by hand, but
apply_invariants and in_process_npm_multicopy did not, so a developer
shell exporting SOCKET_DRY_RUN, SOCKET_JSON or SOCKET_ECOSYSTEMS failed
15 of their tests for reasons unrelated to the code under test.

All 14 files now spawn through the shared hermetic builder in
tests/common/hermetic.rs and leave PENDING_RAW_SPAWNS, so the ratchet
keeps them hermetic. The gem suites' private BUNDLE_* sweep becomes
Extra::Bundle, with its own contract test. get_edge_cases_e2e was
already hermetic; it was only listed because a comment quoted the
bare-spawn spelling the detector matches.

Refs #823

Assisted-by: Claude Code:claude-opus-5-5
Review follow-ups: the gem variant-mismatch suite now drops ambient
BUNDLE_* too, so a developer's BUNDLE_PATH can't aim its apply at a real
gem store. Extra::Bundle seeds BUNDLE_APP_CONFIG, which the gem crawler
actually reads, so a dropped scrub line fails loudly. Remove env lines
the hermetic builder already covers and rewrap the edited comments.

Refs #823

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 12, 2026 00:06
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 49e78d7. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 12, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review (burn-down agent)


Generated by Claude Code

Merged via the queue into main with commit 1779317 Oct 12, 2026
86 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-ci-hermetic-apply-rollback branch October 12, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants