Repository navigation
Print one JSON envelope from every command (v5.0) - #1369
Mikola Lysenko (mikolalysenko) wants to merge 22 commits into
Conversation
* Skip the vlt matrix on ci.yml-only changes vlt-compatibility's PR and push filters list ci.yml because install-proof leaves out the cells ci.yml's vlt e2e rows already run. Most ci.yml edits don't touch those rows, yet each one reran the whole matrix (about 41 Linux + 38 Windows job-min per PR run, plus 22 macOS on push). In the last 24h that was 10 of the 24 merged PRs that triggered the workflow, and 5 of its 45 push runs. A new `changes` job runs scripts/vlt-compat-gate.py. It skips build, plan and everything after them only when ci.yml is the one changed file the event's filter matches and the vlt cells parsed from ci.yml are the same on base and head. matrix-coverage still runs, and schedule, dispatch or any doubt (missing base, parse error) run the full matrix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YVLGaJFexMvqCdiWxbiHXB * Match odd and renamed paths in the vlt gate `git diff --name-only` split on whitespace dropped paths with spaces, quoted non-ASCII names, and reported only the new side of a rename, so a vlt file changed that way next to an inert ci.yml edit could read as "only ci.yml changed". Read NUL-separated paths with --no-renames and test it against a scratch repository. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YVLGaJFexMvqCdiWxbiHXB --------- Co-authored-by: Claude <noreply@anthropic.com>
A DNS lookup failure for static.rust-lang.org on a macOS runner failed `rustup toolchain install 1.93.1` in cargo-vex-matrix and evicted #1180 from the merge queue (run 37942278302). rustup makes one attempt per download, so a single blip on a fresh runner fails the leg. The "Install Rust" steps had the same hole, hidden: since rustup 1.28, `rustup show` reports a failed download of the rust-toolchain.toml channel and still exits 0, leaving the install to the job's first cargo command with no retry at all. Add scripts/rustup-retry.sh (4 attempts, growing pause, like pip-install-retry.sh) and route every ci.yml toolchain and component install through it, using `rustup toolchain install` (no name installs the rust-toolchain.toml channel and components) so a failed download fails or retries the install step itself. Steps on matrix-OS jobs get `shell: bash` so the Windows legs run the script. Claude-Session: https://claude.ai/code/session_01TW6TFHfrADaEw26viJ5bAq Co-authored-by: Claude <noreply@anthropic.com>
* Start refactor for #594 Assisted-by: Claude Code:claude-opus-5-5 * Wire vendored nuget.config via formats::nuget Vendored NuGet now reads the source keys and finds the <packageSources>, <packageSourceMapping> and <configuration> anchors through formats::nuget::parse_config, the reader that hosted, upstream restore and VEX already use. The private substring scanner (blank_comments, parse_config_source_keys, attr_value, self_closing_package_sources, insert_at_line) is deleted. User impact: - A close tag written with whitespace (</packageSources >) is now the section that gets extended; vendor used to append a second section NuGet ignores, so restore failed NU1100/NU1403 (#685). - An empty <packageSourceMapping /> is expanded in place instead of left beside a second mapping section. - A section opened and closed on one line receives the source inside it, not before its open tag. - Catch-all keys are written XML-encoded, so a key with & or a quote keeps its identity. - Malformed XML or a repeated section is refused with "malformed XML or a repeated section; not wired" instead of being spliced at the first substring match, as hosted already does. Output bytes for well-formed configs are unchanged. Fixes #685 Refs #594 Assisted-by: Claude Code:claude-opus-5-5 --------- Co-authored-by: Claude <noreply@anthropic.com>
* Start fix for #1243 Assisted-by: Claude Code:claude-opus-5-5 * Keep bun.lockb shared bundled pins manageable Bun 1.2+ keeps one bun.lockb record for a version that is installed both from the registry and bundled inside a parent's tarball. The hosted scan wires that record for the regular install, but discovery treated it like a bundled-only record and dropped its ref. So `list`, `remove` and `rollback` refused the pin as contested, and the hosted to vendored takeover failed with vendor_lock_entry_not_found. Discovery now classifies a shared record as the regular install and records its version as a bundled copy, so the ref is shadowed: still never attested in VEX (the bundled copy stays unpatched), but visible to every command that manages hosted pins, as the text bun.lock already is. Fixes #1243 Assisted-by: Claude Code:claude-opus-5-5 * Test a hosted pin on a shared bundled bun.lockb record Bun 1.2+ e2e: a root that depends on minimist@1.2.2 and on a local parent that bundles its own minimist@1.2.2. After the hosted scan, `list` must name the pin (it exited 1 with hosted_wiring_contested), the online takeover must vendor over it (it failed with vendor_lock_entry_not_found), `vendor --revert` must restore the original bytes, and `rollback` must refuse it with the checkout remedy like any binary hosted pin. Older Bun keeps no shared record, so the leg skips there. Refs #1243 Assisted-by: Claude Code:claude-opus-5-5 * Run the shared bundled bun.lockb test on Bun 1.4 The Bun compatibility backtest runs every `native_binary_` test and expects exactly three to pass, so the new #1243 test's name broke all three `binary` legs. Rename it out of that prefix, and add it to the Bun 1.4.2 e2e_bun_lockb leg: the 1.0 and 1.1 legs keep no shared record and skip it, so without this no CI leg ran it for real. Refs #1243 Assisted-by: Claude Code:claude-opus-5-5 --------- Co-authored-by: Claude <noreply@anthropic.com>
- Envelope::extra (flattened) carries a command's own payload keys beside the shared ones; set_extra refuses a shared key. - Status gains notInstalled, noMatch, noPackages and selectionRequired (get's outcomes) so no command needs a snake_case status. - PatchAction gains rolledBack; summary.rebuilt and summary.rolledBack are always present. - manifest_load_error is the one mapping from a manifest load failure to manifest_invalid / manifest_unreadable (#931); list and remove use it, and remove reports a manifest that vanished mid-run as manifest_not_found instead of manifest_invalid. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every rollback outcome is now a PatchEvent: agent restores are rolledBack (verified on a dry run, installed copy in details.path), already-original and not-installed entries are skipped with already_original / package_not_installed, failures are failed with the blocking file's code. Vendored and hosted legs carry details.mode; drift-keeps are failed vendor_revert_kept (they still exit 1). Manifest entries the run drops are removed (verified on a dry run) with details.manifest. GC goes through set_gc; a requested GC that could not run adds the gc_skipped warning. Every error path prints a full envelope; manifest load failures use manifest_load_error. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#931) apply, apply --check, repair, vendor, vendor --check and vex now report an unparseable manifest as manifest_invalid and an unreadable one as manifest_unreadable through json_envelope::manifest_load_error, like list and remove. Gone: apply_failed / repair_failed for a manifest load failure, and vendor's undocumented invalid_manifest. vex keeps exit 2. apply's run_locked reports a manifest that vanished mid-run as manifest_not_found. Exit codes are unchanged. remove's GC carrier now matches repair's (details.count + details.checked, bytes); the per-kind totals are only in gc. The in-place rollback leg is reported as rolledBack events (summary.rolledBack) instead of the carrier's details.rolledBack. Sweep failures reach --json as cleanup_failed warnings. Vendored- and hosted-leg events of remove and repair carry details.mode; repair's download carrier key is now details.downloadMode. Sidecar file actions and severities serialize with the envelope's camelCase enum convention (wire-identical for today's one-word values). Adds a regression matrix test over list, remove, apply, apply --check, repair, vendor, vendor --check and vex. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds tests/common/rollback_json.rs, per-leg views projected from events, and moves the rollback-centric suites onto summary/events. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Covers the mixed suites (apply multicopy, dispatch, global scope, json error shape, hosted/vendored takeovers) and the ignored e2e suites, which were fixed by reading. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CLI_CONTRACT.md: manifest_invalid / manifest_unreadable now list every command that loads the manifest (v5.0, MAJOR, #931; vex keeps exit 2); the top-level error table gains the codes apply, remove, vendor, vex and --update already emitted; the remove/repair matrix rows, the rolledBack action row and new notes describe the shared GC carrier (details.count + checked), details.mode on vendored/hosted-leg events, repair's details.downloadMode, and the sidecars[] value-tag casing. apply --check prints a noManifest envelope when the manifest vanishes between the existence probe and the read, instead of printing nothing. Tests pin details.mode on remove's hosted/vendored legs and repair's vendored phase. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Start fix for #1098, #1109 Assisted-by: Claude Code:claude-opus-5-5 * Test gem system-home cases #1098 #1109 hit Add regression tests for the false stale-install warning under Bundler deployment / .bundle-default settings (#1109), and for standalone vex refusing a project whose Bundler path never loads the unpatched system gem-home copy (#1098). Both fail on main. Assisted-by: Claude Code:claude-opus-5-5 * Format a test main left unformatted cargo fmt --check fails on main in the bun lock remedy test; rewrap the assertion so the format gate passes. Assisted-by: Claude Code:claude-opus-5-5 * Skip unused system gem homes in gem checks Bundler stops using the system gem homes under deployment mode as well as under an explicit path, but the stale-install guard only modeled the explicit path. A fresh deployment checkout with an old copy of the patched gem in the machine gem home got a false stale warning, and scan --mode hosted --vex exited 1 with nothing to attest (#1109). Standalone vex and apply --check still judged every gem env copy whenever vendor/bundle was empty, even under an explicit or deployment path, so they refused to attest a project that never loads that copy (#1098). One predicate now answers whether Bundler uses system gems, counting deployment after the path tiers like Bundler::Settings#path. The stale guard and the vex copy lookup both use it; default gems stay judged. The .bundle default from default_install_uses_path or simulate_version depends on the Bundler version, so those keep the system homes judged. Assisted-by: Claude Code:claude-opus-5-5 * Repin live minimist@1.2.2 suites to republished patch 642d7f02 Ports #1301 so this PR's CI is not blocked by #1293: production no longer serves the free minimist@1.2.2 patch 80630680, which breaks hosted-e2e and e2e_safety_pnpm on main as well. Same change as #1301; it becomes a no-op once #1301 lands on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YRcjmQwhWGod7X58Hbe5FW * Read vlt backtest patch file keys with or without the package/ prefix The republished minimist patch (642d7f02) keys its files without the package/ prefix, so key.split('/', 1)[1] raised IndexError in every native vlt leg. Ports the matching line from #1302. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YRcjmQwhWGod7X58Hbe5FW --------- Co-authored-by: Claude <noreply@anthropic.com>
* Start fix for #1271 Assisted-by: Claude Code:claude-opus-5-5 * Patch yarn 1 lock blocks with an empty range A dependency declared as "left-pad": "" is valid npm semver (the same as "*"), and yarn 1 locks it under the key `left-pad@:`, merged with other members' ranges as `left-pad@, left-pad@^1.3.0:`. The shared yarn key parser rejected a pattern with an empty range, so every classic reader dropped that block: a lock-only scan reported nothing to patch, vendored mode failed with vendor_lock_entry_not_found, and hosted mode pinned nothing while exiting 0. Add a classic-grammar splitter that keeps an empty range and route the classic readers and writers through it (lock inventory, vendored and hosted rewriters, upstream restore, lockfile VEX, copy-source classification). Berry keeps the strict parser: it never writes a rangeless descriptor and skips one as malformed. Fixes #1271 Assisted-by: Claude Code:claude-opus-5-5 * Move empty-range tests out of other test bodies Three of the new #1271 tests and the VEX one landed inside a neighbouring test body or between another test's doc comment and its function, so they compiled as nested items and never ran. Move each to its own top-level spot; all of them run and pass now. Assisted-by: Claude Code:claude-opus-5-5 * Repin live minimist suites to the republished patch Port of #1302 (fixes #1293). Production withdrew the free minimist@1.2.2 patch 80630680-… and republished the same fix as 642d7f02-…, with a new patched index.js (afterHash ec956dca…). That turned hosted-e2e and e2e_safety_pnpm red on main and here. No-ops once main carries #1302 or #1301. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mzbk9WDZkhatFrCWUAyaNs * Accept unprefixed patch file keys in the vlt backtest The republished minimist patch (642d7f02-…) lists its files without npm's `package/` prefix. `holds()` split every key on its first `/`, so a bare `index.js` raised IndexError and every native vlt cell errored. Strip the prefix only when present, like the CLI (patch/apply.rs) and backtest-bun.py's oracle already do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mzbk9WDZkhatFrCWUAyaNs --------- Co-authored-by: Claude <noreply@anthropic.com>
The rollback contract's JSON section now maps each outcome to its event, the action matrix and migration status list rollback as an envelope command, and the errorCode / EnvelopeError tables carry its codes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
scan and get now print one serialized Envelope per run (command,
status from the Status enum, dryRun, events, summary, warnings, gc,
vex), built by one emitter each (scan::emit_scan, get::emit):
- The agent download engine records per-patch events into the caller's
envelope (downloaded / updated + oldUuid with the patch metadata in
details, skipped already_in_manifest, failed with download_failed /
patch_no_applicable_files / blob_write_failed or the refusal code),
then applied / failed events for the nested apply; it never prints,
so scan --mode agent --json can no longer put two JSON documents on
stdout when the lock is held or the manifest is unreadable.
- Hosted runs record applied (verified on --dry-run) / skipped events
tagged details.mode hosted; redirect shrinks to {mode, rewrittenFiles}
and its warnings move to the top level.
- Vendored runs merge the vendor engine's events, warnings and sidecars
into the outer envelope (details.mode vendored) instead of nesting a
vendor envelope; the dry-run preview becomes verified / skipped events.
- scan --prune records pruned manifest entries as removed events
(details.manifest), reverted vendored entries as removed, drift-kept
and failed reverts as skipped, the sweep as the envelope's gc, and a
pass that could not take the lock as a gc_skipped warning.
- get's statuses become notFound / noPackages / noMatch / notInstalled /
paidRequired / selectionRequired; selection_required options use
publishedAt; every error prints a full envelope.
- Exit parity (#1062): one fetch_details decision for both outputs, no
human-only fetched == 0 failure, hosted --json uses the policy-gated
prune, and the human vendored dry run previews the --prune GC.
- A manifest that exists but cannot be loaded fails agent-mode scan with
manifest_invalid / manifest_unreadable and is a warning elsewhere.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CLI_CONTRACT.md: scan and get join the envelope (migration status, action matrix, PatchAction rows, error codes, a new "scan and get JSON" section replacing the patches[] entry shape, jq recipes), and the scan mode / get paragraphs describe events, the redirect payload and top-level warnings instead of the nested blocks. Tests: shared envelope invariants in tests/common/envelope.rs, a legacy redirect rebuild for the in-memory engine parity harness, scan envelope tests (agent events, dry run, lock held prints one document, manifest load errors) and the #1062 human/JSON parity tests, get envelope tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every test that read the legacy scan/get JSON (top-level counters, patches[], apply / download / vendor / redirect blocks, nested and string warnings, snake_case statuses and actions, gc sub-keys) now reads the envelope: events (details.mode for the hosted and vendored legs), summary, top-level warnings, redirect.rewrittenFiles and the scan payload keys. The in-memory hosted engine parity tests compare the engine's redirect block against one rebuilt from the disk run's envelope. Env-gated e2e files were updated by reading. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every command now prints the envelope, so the legacy error helpers (legacy_error, set_error, is_legacy_shape, …) are deleted and every usage error is a full envelope. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
| ); | ||
| env.warnings.extend(venv.warnings); | ||
| env.sidecars.extend(venv.sidecars); | ||
| if venv.summary.failed > 0 || matches!(venv.status, Status::PartialFailure | Status::Error) { |
There was a problem hiding this comment.
[agent] merge_vendor_envelope never carries venv.error, and the Ok arm here (and get.rs:2374) doesn't report it either, though the doc says it's the caller's to report. The engine marks an error and still returns Ok for vendor_state_unreadable, the group-commit refusal and vendor_commit_failed. Scenario: corrupt .socket/vendor/state.json, scan --mode vendored --json → status: "partialFailure", exit 1, no error and no failed event; on main the nested vendor.error carried the code, and CLI_CONTRACT now promises top-level error: {code, message}. Fix: if let Some(err) = venv.error { env.mark_error(err) } in the merge, plus a corrupt-ledger JSON test.
Also, most of the CI red is caused by this PR's consumers still reading the old scan shape: the bench gate (socket-patch-bench/src/engine.rs:315-385, packagesWithPatches/totalPatches/redirect.* → "got null, want 60"); the Python backtests (backtest-{pipenv,pdm,poetry,bun,uv}.py read envelope["apply"]/["vendor"]["summary"]/redirect.redirected, so Pipenv/Poetry/PDM and the Bun native cells see 0 applied); and e2e tests e2e_redirect_gem_build.rs:1232 (warnings omitted when empty → expect panics), e2e_vendor_vlt_build.rs:1239 (gc.revertedVendoredEntries removed), e2e_bun_lockb.rs:1102 (redirect.redirected) and :137 (expects error, CLI correctly returns partialFailure with a skipped event).
Every command's
--jsonoutput now uses the one envelope shape for v5.0.scan,getandrollbackwere the last commands printing their own pre-v3 shapes. This finishes the migration thatCLI_CONTRACT.mdlisted as pending. This is a breaking change on purpose (v5.0, MAJOR).Stacked on #1273 (one GC shape and rollback counters); merge that first.
Closes #931. Closes #1062.
What every command now prints
command,status,dryRun,events,summary,warnings, pluserror: {code, message}on failure, andgcwhen a sweep ran.statuscomes from one camelCase set:success,partialFailure,error,noManifest,paidRequired,notFound,notInstalled,noMatch,noPackages,selectionRequired. No snake_case statuses or actions remain.summaryalways equals the event counts. New action:rolledBack.details.mode: "vendored" | "hosted", the conventionlistalready used.{code, detail}. There are no string warnings and no nested*.warningsarrays.Envelope::extra. Examples: scan'spackages,rollout,policyandredirectState; rollback'spathsandhosted.editedFiles.Per command
rollback:
rolledBack,alreadyOriginal,failed,results[],vendored*,manifestandhosted.{reverted,failed,unsupported}became events:rolledBackskipped already_originalskipped package_not_installedfailedwith a specific code (hash_mismatch,vendor_revert_kept,hosted_restore_refused, …)removedwithdetails.manifest: trueverifiedgc: {skipped: true}became an absentgcplus agc_skippedwarning. Exit codes are unchanged.scan / get:
apply.patches,download, the nestedvendorenvelope andredirect.patchesall merged intoevents.found,totalPatches,redirected, …) are replaced bysummary.would_add/would_pin/would_vendorbecameverified.already_vendoredand dry-run refusals becameskippedwith a code.options[].published_atbecamepublishedAt.--prunemanifest entries areremovedevents, and the sweep is the envelope'sgc.Corrupt manifest (An unparseable .socket/manifest.json is reported under five different --json error codes depending on the command #931): one mapping,
manifest_load_error, on every command. An unparseable manifest ismanifest_invalid; an unreadable one ismanifest_unreadable. This replacesapply_failed,repair_failed, the undocumentedinvalid_manifest, and themanifest_unreadablethatapply --check/vendor --check/vex/get/rollbackused for an unparseable file. Agent-modescannow fails on an unloadable manifest. The other scan modes add a warning with the same code.remove:
{count, checked}, matching repair's.rolledBackevent (it wasdetails.rolledBack).cleanup_failedwarnings.Sidecars:
SidecarFileAction/SidecarSeverityserialize camelCase. Every value is one word, so the output bytes don't change.Bugs fixed on the way
scan --mode agent --jsonprinted two JSON documents when the lock was held or the manifest was unreadable. The download engine now only records into the caller's envelope and never prints.scan exits 1 in human output but 0 with --json when every patch query returns nothing #1062: human and
--jsonscan now share each decision, so they agree:--jsonfetches details under the same rule as human.--json--prunerespects the socket.yml write gate.--dry-run --prunepreviews the GC like--jsondoes.The logic from Fix scan human/JSON exit and prune forks (#1062) #1298 and Fix human vendored --prune skipping GC (#1127) #1338 is included here, so whichever lands second needs a mechanical rebase.
Contract
CLI_CONTRACT.mdenvelope, event, action matrix, error-code tables, rollback contract,scan/getJSON section and jq recipes all describe the new shape.legacy_error,set_error,is_legacy_shape, …) are deleted.Tests
cargo test -p socket-patch-cli -p socket-patch-core --tests --lib --no-fail-fast(local, macOS arm64): 13,307 passed, 2 failed. The 2 failures aree2e_vendor_cargo_buildcases that need x86 cargo toolchains, which don't run on this machine ("Bad CPU type"). They are unrelated to this change.commandpresent, status from the enum, summary equal to the event counts, error paths print a full envelope)e2e_scan,e2e_npm,e2e_redirect_*_build,e2e_vendored_production,e2e_vex_*,e2e_bun_lockb,e2e_hosted_production, and others.cargo clippy --locked --workspace --all-features -- -D warningsandcargo fmt --all -- --checkare clean.CHANGELOG.mdis untouched, per AGENTS.md.🤖 Generated with Claude Code
Note
High Risk
Breaking v5.0 JSON contract for automation consuming CLI output, plus broad changes to get/scan/apply error and manifest paths that affect exit codes and failure reporting.
Overview
Completes the v5.0 breaking migration so every command emits a single shared JSON envelope (
command,status,events,summary, structuredwarnings, optionalerror) instead of legacy per-command shapes.get,scan, androllbackpaths now recordPatchEventoutcomes (e.g.downloaded,applied,failed,skipped) and derive counts fromsummary; the agent download engine writes into a caller-ownedEnvelopeand no longer prints JSON itself (fixes double-document output on lock/manifest errors).Manifest handling is unified via
manifest_load_error(manifest_invalidvsmanifest_unreadable) across apply, get, vendor, list, and related flows, replacing ad hoc codes likeapply_failedfor bad manifests. Nested apply failures fold throughrecord_apply_outcomeinstead of mutatingpatches[]JSON.CI switches Rust installs from
rustup showtoscripts/rustup-retry.sh toolchain install(retries download blips; explicit install). vlt-compatibility adds achangesgate (vlt-compat-gate.py) to skip the heavy matrix when only unrelatedci.ymledits triggered the workflow. Bun lockb e2e adds a matrix filter forbinary_shared_bundled_record_hosted_pin_is_managed.Reviewed by Cursor Bugbot for commit 48c63c6. Configure here.