Fix gem unwind dropping source-block ! (#1056)

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -1022,7 +1022,7 @@ v5.0 replaces v4's per-purl reverts and whole-ledger reverse replay (`revert_rem
   * **cargo** — `Cargo.lock` back on crates.io (source + the sparse index's checksum, `SOCKET_CRATES_INDEX`). When the lock already holds a crates.io block for the same `name@version` (a later dependent locked its own copy, #679), the Socket block merges into it instead: it is dropped (with a v1 lock's `[metadata]` line), and dependents' references take the spelling cargo writes, so no duplicate block is left that cargo cannot parse (#863), and no index lookup is needed; every `Cargo.toml` declaration loses its `registry = "socket-patch-<uuid>"` pin (the shorthand the rewriter produced collapses back); every `[registries.socket-patch-<uuid>]` block no manifest or lock still references leaves the project cargo config — including a superseded patch generation's block an earlier re-pin left behind (#864). A declaration it cannot unpin refuses.
   * **golang** — the hosted `replace` and the socket module's go.sum lines go; the upstream module's two go.sum lines come back, hashed from the module proxy (`SOCKET_GOPROXY`, else `GOPROXY` / `GONOPROXY` / `GOPRIVATE` as go reads them) and cross-checked against the checksum database (`SOCKET_GOSUMDB_URL`, else `sum.golang.org` unless `GOSUMDB=off` / `GONOSUMDB` / `GOPRIVATE` say go would not ask it). A `replace` the user had before the hosted run is not recorded anywhere, so the restore lands on the plain upstream module.
   * **pypi** — `Pipfile.lock`, `requirements.txt` (+ in-root `-r` includes), Hatch PEP 508 direct references (`pyproject.toml` / `hatch.toml`), `poetry.lock`, `pdm.lock`, `uv.lock`, PEP 723 script locks and PEP 751 `pylock*.toml` (+ the paired `pyproject.toml` / script metadata): hashes re-derived from PyPI's JSON API (`SOCKET_PYPI_JSON_API`). A restored `requirements.txt` line gets `--hash` options only when the file is in pip's hash-checking mode. The mode is read off the file's other requirement lines (an `-e` / `--editable` line means unhashed). When every requirement is a hosted pin, it is read off the hosted line itself (`--hash` vs a `#sha256=` url fragment) (#410). Refused: a `pdm.lock` without `cross_platform`, or a uv / script / pylock lock, whose release has a wheel that is not pure Python 3 (which files the lock keeps is not re-derivable); a uv lock whose options filter files (`exclude-newer`, `no-binary`, `no-build`), or whose other registry packages name no registry, several, or one other than PyPI's simple index; a pylock whose other registry packages show neither an `index` nor (as `uv pip compile` writes them) only PyPI files with none, which restores the entry without an `index` too; uv 0.2 `[[distribution]]` locks. Restored artifact fields keep the spelling the lock's other entries show, including the `upload_time` that uv 0.6.15–0.6.17 write. A restored pylock entry's `upload-time`s are whole seconds, as uv writes them, unless the lock's other entries show fractions. Its artifacts come back in the TOML spelling the other entries use: uv's inline `wheels = [{ … }]`, or the standard tables `pip lock` writes (`[[packages.wheels]]` with a `[packages.wheels.hashes]` sub-table, `[packages.sdist]`). A `pip lock` file (`created-by = "pip"`) records only the artifact pip selected, so the entry is restored with only the release's wheel (its sdist when it has none), and a release with several wheels is refused. A transitive `override-dependencies` entry hosted mode added is removed (`upstream_uv_override_removed`). Hosted mode keeps no ledger, so it marks the entry it adds with a `# socket-patch hosted: …` comment line above it and removes only a marked entry; a user's own `<name>==<version>` override (which the hosted rewrite reuses rather than adding a second one) is kept, and the lock's `[manifest] overrides` record of it gets its specifier back.
-  * **gem** — `Gemfile.lock` / `gems.locked` + `Gemfile` / `gems.rb`: the spec moves back into the upstream `GEM` section (or the Socket remote leaves a merged section), the `source "<patch registry>" do … end` block is undone, the `CHECKSUMS` entry is re-pinned from the rubygems.org compact index (`SOCKET_RUBYGEMS_URL`) and the `DEPENDENCIES` pin loses its `!`. The declaration's original constraint is not recorded, so it comes back as the exact pin `gem "<name>", "<version>"`. A transitive gem (one the manifest never declared) gets an appended block with a blank line before it; the restore removes that block, its blank line and the `DEPENDENCIES` entry, so the pair comes back byte for byte. An appended block with no blank line before it (written by a release before this one) can't be told apart from an in-place rewrite, so it still comes back as the exact pin. Refused: an ambiguous upstream section, an upstream remote other than rubygems.org. The manifest pair can't make a committed bundler cache upstream: a `<name>-<version>.gem` left in Bundler's cache dir that isn't the upstream archive is named by `upstream_gem_stale_cache`, never deleted.
+  * **gem** — `Gemfile.lock` / `gems.locked` + `Gemfile` / `gems.rb`: the spec moves back into the upstream `GEM` section (or the Socket remote leaves a merged section), the `source "<patch registry>" do … end` block is undone, the `CHECKSUMS` entry is re-pinned from the rubygems.org compact index (`SOCKET_RUBYGEMS_URL`) and the `DEPENDENCIES` pin loses its `!`, unless the declaration comes back inside the user's own `source "…" do` block (Bundler writes the `!` for any dependency declared in one, rubygems.org included), where the `!` stays. The declaration's original constraint is not recorded, so it comes back as the exact pin `gem "<name>", "<version>"`. A transitive gem (one the manifest never declared) gets an appended block with a blank line before it; the restore removes that block, its blank line and the `DEPENDENCIES` entry, so the pair comes back byte for byte. An appended block with no blank line before it (written by a release before this one) can't be told apart from an in-place rewrite, so it still comes back as the exact pin. Refused: an ambiguous upstream section, an upstream remote other than rubygems.org. The manifest pair can't make a committed bundler cache upstream: a `<name>-<version>.gem` left in Bundler's cache dir that isn't the upstream archive is named by `upstream_gem_stale_cache`, never deleted.
   * **composer** — `composer.lock`: `dist` and the deleted `source` block from packagist's composer v2 metadata (`SOCKET_PACKAGIST_URL`). Refused unless the entry is packagist-sourced and packagist still serves the lock's `dist.reference` for the version.
   * **maven** — `pom.xml` (the `-socket.<hex8>` version suffix, the added `<repository>` / `<dependencyManagement>` entry) and the `.mvn/maven.config` / `.mvn/checksums/checksums.sha256` lines hosted mode writes: **no network**, so it restores under `--offline` too. `.mvn` files holding anything else keep the resolver lines (`maven_trusted_checksums_left`).
   * **nuget** — `nuget.config` loses the `socket-patch-<uuid>` source and its exact-id mapping; every `packages.lock.json` entry of the id gets nuget.org's `contentHash` back (`SOCKET_NUGET_URL`). Refused when the restored config would not resolve the id from nuget.org alone. A config hosted mode created from scratch is kept (`nuget_default_config_left`).

crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs

@@ -440,6 +440,10 @@ enum Driver {
     /// A modifier adjacent to a top-level constant is still a modifier,
     /// not a hash label (`if::ENV`, #340).
     ScanVexScopedConstantModifier,
+    /// [`Driver::ScanVex`] on a gem declared inside the user's own
+    /// `source "<upstream>" do` block (#1056): bundler locks it with a `!`
+    /// source pin, which the unwind must keep.
+    ScanVexSourceBlock,
     /// A heredoc option continues beyond the declaration's physical line.
     ScanVexHeredocDeclaration,
     /// A double-quoted interpolation can itself contain a heredoc opener.
@@ -521,6 +525,7 @@ impl Driver {
             Driver::ScanVexCustomLockfile => "scan --mode hosted (lockfile custom.lock)",
             Driver::ScanVexTwin => "scan --mode hosted (Gemfile + gems.rb twin)",
             Driver::ScanVexGroupBlock => "scan --mode hosted (gem in a group block)",
+            Driver::ScanVexSourceBlock => "scan --mode hosted (gem in a source block)",
             Driver::ScanVexSemicolonJoinedDeclaration => {
                 "scan --mode hosted (two `;`-joined gem declarations)"
             }
@@ -814,6 +819,10 @@ async fn redirect_scanned_project(
             "source \"{}/upstream\"\n\ngroup :development do\n  gem \"{DEP}\"\nend\n",
             server.uri()
         ),
+        Driver::ScanVexSourceBlock => format!(
+            "source \"{up}/upstream\"\n\nsource \"{up}/upstream\" do\n  gem \"{DEP}\", \"{DEP_VERSION}\"\nend\n",
+            up = server.uri()
+        ),
         Driver::ScanVexEvalGemfile => {
             std::fs::write(proj.join("Gemfile.common"), format!("gem \"{DEP}\"\n")).unwrap();
             format!(
@@ -1061,6 +1070,7 @@ async fn redirect_scanned_project(
         | Driver::ScanVexDuplicateDeclaration
         | Driver::ScanVexEvalGemfile
         | Driver::ScanVexGroupBlock
+        | Driver::ScanVexSourceBlock
         | Driver::ScanVexCustomGitSource
         | Driver::ScanVexMultiLineDeclaration
         | Driver::ScanVexConditionalDeclaration
@@ -1263,6 +1273,7 @@ async fn redirect_scanned_project(
     match driver {
         Driver::ScanVex
         | Driver::ScanVexGroupBlock
+        | Driver::ScanVexSourceBlock
         | Driver::ScanVexTrailingSemicolonDeclaration => {
             assert_eq!(env["vex"]["statements"], 1, "vex block: {env}");
             assert_eq!(
@@ -2329,6 +2340,106 @@ async fn gem_hosted_group_block_pin_survives_a_refused_vendored_takeover() {
     }
 }
 
+/// #1056: a gem the user declared inside their own `source "…" do` block
+/// is locked as `name (= v)!`. After the hosted scan converges (an
+/// unfrozen install), `rollback` / `remove` must hand back the exact
+/// pre-scan pair, `!` included, so a FROZEN install of the restored pair
+/// still succeeds (it exited 16 when the unwind dropped the `!`).
+/// Converged without CHECKSUMS so the restore needs no rubygems.org sha.
+#[tokio::test(flavor = "multi_thread")]
+#[ignore = "host capstone: shells out to a real ruby/gem/bundler (>= 2.2); \
+            the unpinned `test` job skips it, an e2e job with a pinned toolchain runs it via --ignored"]
+async fn gem_hosted_unwind_keeps_a_source_block_bang() {
+    for command in ["rollback", "remove"] {
+        let Some(fx) = redirect_scanned_project(
+            &format!("source-block {command}"),
+            Spelling::Gemfile,
+            false,
+            true,
+            None,
+            Driver::ScanVexSourceBlock,
+        )
+        .await
+        else {
+            return;
+        };
+        if !fx.bundler.at_least(2, 2) {
+            println!(
+                "SKIP e2e_redirect_gem_build (source-block unwind): bundler {} merges every \
+                 rubygems source into one GEM section",
+                fx.bundler.version
+            );
+            return;
+        }
+        let pristine_lock = String::from_utf8(fx.pristine_lock.clone()).unwrap();
+        assert!(
+            pristine_lock.contains(&format!("  {DEP} (= {DEP_VERSION})!\n")),
+            "bundler pins a source-block gem with `!` (test premise):\n{pristine_lock}"
+        );
+        // Converge the mixed pair: the lock now resolves from the registry.
+        let install = bundle(&fx.proj, &["install"]);
+        assert!(
+            install.status.success(),
+            "converging install:\n{}",
+            String::from_utf8_lossy(&install.stderr)
+        );
+        let lock = std::fs::read_to_string(fx.proj.join(fx.lock_name)).unwrap();
+        assert!(
+            lock.contains(&format!("remote: {}", fx.index_url)),
+            "converged:\n{lock}"
+        );
+
+        let api = fx._server.uri();
+        let cwd = fx.proj.to_str().expect("utf8 tmp path");
+        let mut argv = vec![command];
+        if command == "remove" {
+            argv.extend([PURL, "--yes"]);
+        }
+        argv.extend([
+            "--json",
+            "--cwd",
+            cwd,
+            "--api-url",
+            &api,
+            "--org",
+            ORG,
+            "--api-token",
+            "fake",
+            "--patch-server-url",
+            &api,
+        ]);
+        let (code, stdout, stderr) = run_socket(&fx.proj, &argv);
+        assert_eq!(code, 0, "{command}:\nstdout:\n{stdout}\nstderr:\n{stderr}");
+        assert_eq!(
+            std::fs::read_to_string(fx.proj.join(fx.lock_name)).unwrap(),
+            pristine_lock,
+            "{command}: the lock comes back byte for byte, `!` included"
+        );
+        assert_eq!(
+            std::fs::read(fx.proj.join(fx.gemfile_name)).unwrap(),
+            fx.pristine_gemfile,
+            "{command}: the Gemfile comes back byte for byte"
+        );
+        let fresh = stage_fresh_checkout(&fx, &format!("source-block-{command}"));
+        let install = bundle_env(&fresh, &["install"], &[("BUNDLE_FROZEN", "true")]);
+        assert!(
+            install.status.success(),
+            "{command}: frozen install of the restored pair:\n{}",
+            String::from_utf8_lossy(&install.stderr)
+        );
+        assert_eq!(
+            std::fs::read(fresh_installed_lib(
+                &fresh,
+                &format!("{DEP}-{DEP_VERSION}"),
+                "vuln_gem.rb"
+            ))
+            .unwrap(),
+            orig_lib().into_bytes(),
+            "{command}: the upstream bytes are installed"
+        );
+    }
+}
+
 /// One refused takeover of the group-block fixture: the
 /// `gemfile_declaration_not_editable` refusal (exit 1 wet; a `would_refuse`
 /// preview row dry), no revert (nor a preview of one), and the hosted