diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 863dbcbca..346596a9e 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -959,7 +959,7 @@ v5.0 replaces v4's per-purl reverts and whole-ledger reverse replay (`revert_rem * **Scope.** The hosted pins are what lockfile discovery finds — `(purl, patch uuid, files wiring it)`, recognized only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin. A scoped rollback (paths / identifiers / `--ecosystems`) restores exactly the pins in scope; each pin restores or refuses on its own (there is no whole-ledger replay, and a pre-v5 ledger's edits are never replayed). A pin discovery cannot see is out of reach: a lockless cargo `registry = "socket-patch-"` pin, a nuget exact-id mapping with no `packages.lock.json`, a gem wired only in the `Gemfile` (pre-bundler-2.6 mixed state) — restore those files from version control. * **What a restore does.** Every file wiring the pin is rewritten back to the DEFAULT UPSTREAM registry entry for `name@version`, re-resolving whatever the entry pins (tarball URL, integrity, checksum, hashes) from the public registry; only the hosted entries change and every other byte stays the file's own. A pin is **all-or-nothing**: refused in one of its files, it is restored in none of them, so no pin is left half hosted. Nothing reaches disk until every pin has resolved, and `--dry-run` resolves exactly like a wet run — registry lookups included — and skips only the write. Per format: - * **npm family** — `package-lock.json` / `npm-shrinkwrap.json`, `yarn.lock` (classic and berry), `pnpm-lock.yaml` / `shrinkwrap.yaml`, `bun.lock`: resolution + integrity (+ shasum where recorded) from the npm registry's version document (`SOCKET_NPM_REGISTRY`); a yarn berry lock whose `.yarnrc.yml` names another `npmRegistryServer` reads that registry's document instead, so a mirror's off-path `dist.tarball` keeps its `::__archiveUrl=` binding, and a pnpm lock whose sibling settings name a registry reads that registry's document — `.npmrc` `registry` (or, for a scoped name, `@scope:registry`); on pnpm 10 a pnpm-workspace.yaml `registries` map instead when present; on pnpm 11+ (or an unknown major) pnpm-workspace.yaml `registries."@scope"` / `registry` / `registries.default` too, ahead of the matching `.npmrc` key — so a mirror's `tarball:` comes back as pnpm recorded it and a URL conventional under that registry stays derived (falling back to the default registry, with `upstream_registry_fallback`, when the mirror can't be read; a value holding an unexpanded `${VAR}` is read as unset). Whether a restored pnpm entry gets its `tarball:` back follows `lockfileIncludeTarballUrl` as the pnpm that wrote the lock read it (#902), from the strongest evidence available: (1) the lock's own unpinned registry resolutions (a bare one proves it off; a URL pnpm could have derived proves it on; pnpm 11+'s env lockfile document does not count); else (2) the settings file the installed pnpm major reads (`node_modules/.modules.yaml` `packageManager`, else package.json `packageManager`; a pre-9 lock or shrinkwrap means pnpm <= 8): `.npmrc` `lockfile-include-tarball-url` on pnpm <= 9, pnpm-workspace.yaml `lockfileIncludeTarballUrl` on pnpm >= 11 (also assumed for a lock carrying an env lockfile document), the workspace file then `.npmrc` on pnpm 10; else (3) pnpm 10's reading. A Rush lock (`common/config/rush/pnpm-lock.yaml` or a subspace lock, with `rush.json` at the Rush root) takes its pnpm major from rush.json `pnpmVersion` instead of tier 2's install record and package.json pin. A 9.0 lock may come from pnpm 9, 10 or 11+, so when tier 3's reading differs from pnpm 9's (`.npmrc` only) or pnpm >= 11's (pnpm-workspace.yaml only) — e.g. `.npmrc` on with the workspace file silent, or the workspace file setting it with `.npmrc` silent or disagreeing — the restore follows pnpm 10 but warns `upstream_pnpm_tarball_setting_guessed` (once per lock, naming the entries); a URL pnpm records anyway (not derivable from the registry) never warns. A `bun.lock` 4-tuple's registry slot is rebuilt the way Bun writes it (#992): `""` for a package from registry.npmjs.org, otherwise the full tarball URL — Bun 1.1.39–1.3.6 read `""` as npmjs whatever the project configures. The registry is the one Bun resolves the package against: a scope's `.npmrc` `@scope:registry` or `bunfig.toml` `[install.scopes]` entry, else `BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY`, the `.npmrc` `registry`, then `bunfig.toml` `[install] registry`; its version document's `dist.tarball` fills the slot, and when it can't be read (`upstream_registry_fallback`) the default registry's conventional URL is re-based on it. The `bun.lockb` takeover restore records the same URL. Side settings: a project `.npmrc` that is exactly `allow-remote=all\n` is deleted once no root npm lock entry is hosted, otherwise a remaining top-level `allow-remote=all` warns `npm_allow_remote_left`; a `pnpm-workspace.yaml` that is exactly the scaffold hosted mode creates is deleted once `pnpm-lock.yaml` is no longer hosted, otherwise a remaining `trustLockfile: true` warns `pnpm_trust_lockfile_left`. **`bun.lockb` (binary)**: `rollback` and `remove` refuse it (the checkout remedy). The hosted → vendored takeover and the eject DO restore it, since the vendor ledger then records the rebuilt record as its pre-vendor original: the native codec turns each hosted remote-tarball record back into Bun's npm registry record for `name@version` (the registry's `dist.tarball` + `dist.integrity`, the package metadata hash re-derived, the hosted URL string dropped from the string pool). The hosted rewrite keeps the registry record's inactive bytes (padding, semver) in the tarball record, so a lock it wrote comes back byte for byte — early writers' uninitialized padding included; a record without them (an older socket-patch or a Bun re-save) is rebuilt the way Bun writes one, and refused for a prerelease/build version. A lock the hosted rewrite had to normalize is marked in the root package's resolution value bytes (which no Bun reader reads): a binary format 1 lock it promoted to format 2 is demoted back to its exact format-1 bytes (verified by promoting it again, otherwise refused), and a lock whose workspace dependency behaviors it normalized is refused with the `git checkout -- bun.lockb` remedy. + * **npm family** — `package-lock.json` / `npm-shrinkwrap.json`, `yarn.lock` (classic and berry), `pnpm-lock.yaml` / `shrinkwrap.yaml`, `bun.lock`: resolution + integrity (+ shasum where recorded) from the npm registry's version document (`SOCKET_NPM_REGISTRY`); a yarn berry lock whose `.yarnrc.yml` names another `npmRegistryServer` reads that registry's document instead, so a mirror's off-path `dist.tarball` keeps its `::__archiveUrl=` binding, and a pnpm lock whose sibling settings name a registry reads that registry's document — `.npmrc` `registry` (or, for a scoped name, `@scope:registry`); on pnpm 10 a pnpm-workspace.yaml `registries` map instead when present; on pnpm 11+ (or an unknown major) pnpm-workspace.yaml `registries."@scope"` / `registry` / `registries.default` too, ahead of the matching `.npmrc` key — so a mirror's `tarball:` comes back as pnpm recorded it and a URL conventional under that registry stays derived (falling back to the default registry, with `upstream_registry_fallback`, when the mirror can't be read; a value holding an unexpanded `${VAR}` is read as unset). Whether a restored pnpm entry gets its `tarball:` back follows `lockfileIncludeTarballUrl` as the pnpm that wrote the lock read it (#902), from the strongest evidence available: (1) the lock's own unpinned registry resolutions (a bare one proves it off; a URL pnpm could have derived proves it on; pnpm 11+'s env lockfile document does not count); else (2) the settings file the installed pnpm major reads (`node_modules/.modules.yaml` `packageManager`, else package.json `packageManager`; a pre-9 lock or shrinkwrap means pnpm <= 8): `.npmrc` `lockfile-include-tarball-url` on pnpm <= 9, pnpm-workspace.yaml `lockfileIncludeTarballUrl` on pnpm >= 11 (also assumed for a lock carrying an env lockfile document), the workspace file then `.npmrc` on pnpm 10; else (3) pnpm 10's reading. A Rush lock (`common/config/rush/pnpm-lock.yaml` or a subspace lock, with `rush.json` at the Rush root) takes its pnpm major from rush.json `pnpmVersion` instead of tier 2's install record and package.json pin. A 9.0 lock may come from pnpm 9, 10 or 11+, so when tier 3's reading differs from pnpm 9's (`.npmrc` only) or pnpm >= 11's (pnpm-workspace.yaml only) — e.g. `.npmrc` on with the workspace file silent, or the workspace file setting it with `.npmrc` silent or disagreeing — the restore follows pnpm 10 but warns `upstream_pnpm_tarball_setting_guessed` (once per lock, naming the entries); a URL pnpm records anyway (not derivable from the registry) never warns. A `bun.lock` 4-tuple's registry slot is rebuilt the way Bun writes it (#992): `""` for a package from registry.npmjs.org, otherwise the full tarball URL — Bun 1.1.39–1.3.6 read `""` as npmjs whatever the project configures. The registry is the one Bun resolves the package against: a scope's `.npmrc` `@scope:registry` or `bunfig.toml` `[install.scopes]` entry, else `BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY`, then the `.npmrc` `registry` or `bunfig.toml` `[install] registry`. Bun reads these from the files beside the lock and from the user's own (#1276): `$XDG_CONFIG_HOME/.npmrc` when it exists, else `~/.npmrc` (never `NPM_CONFIG_USERCONFIG`), and the global bunfig `$XDG_CONFIG_HOME/.bunfig.toml` when `XDG_CONFIG_HOME` is set, else `~/.bunfig.toml`. A key set beside the lock wins over the user's own file of the same kind. Between the two kinds, Bun ≤ 1.3 takes any `.npmrc` over any bunfig and Bun ≥ 1.4 any bunfig over any `.npmrc`. A lockfileVersion-2 `bun.lock` is Bun ≥ 1.4's (Bun 1.3 ignores it); when the kinds disagree on a package of a lockfileVersion-0/1 `bun.lock` (which Bun 1.4 keeps as is) or a `bun.lockb`, the restore refuses that pin rather than guess. Only the conventional token variables (`NPM_TOKEN`, `NODE_AUTH_TOKEN`, `BUN_AUTH_TOKEN`) expand in the project's files; the user's own files expand any variable, as Bun does. Its version document's `dist.tarball` fills the slot, and when it can't be read (`upstream_registry_fallback`) the default registry's conventional URL is re-based on it. The `bun.lockb` takeover restore records the same URL. Side settings: a project `.npmrc` that is exactly `allow-remote=all\n` is deleted once no root npm lock entry is hosted, otherwise a remaining top-level `allow-remote=all` warns `npm_allow_remote_left`; a `pnpm-workspace.yaml` that is exactly the scaffold hosted mode creates is deleted once `pnpm-lock.yaml` is no longer hosted, otherwise a remaining `trustLockfile: true` warns `pnpm_trust_lockfile_left`. **`bun.lockb` (binary)**: `rollback` and `remove` refuse it (the checkout remedy). The hosted → vendored takeover and the eject DO restore it, since the vendor ledger then records the rebuilt record as its pre-vendor original: the native codec turns each hosted remote-tarball record back into Bun's npm registry record for `name@version` (the registry's `dist.tarball` + `dist.integrity`, the package metadata hash re-derived, the hosted URL string dropped from the string pool). The hosted rewrite keeps the registry record's inactive bytes (padding, semver) in the tarball record, so a lock it wrote comes back byte for byte — early writers' uninitialized padding included; a record without them (an older socket-patch or a Bun re-save) is rebuilt the way Bun writes one, and refused for a prerelease/build version. A lock the hosted rewrite had to normalize is marked in the root package's resolution value bytes (which no Bun reader reads): a binary format 1 lock it promoted to format 2 is demoted back to its exact format-1 bytes (verified by promoting it again, otherwise refused), and a lock whose workspace dependency behaviors it normalized is refused with the `git checkout -- bun.lockb` remedy. * **vlt** — `vlt-lock.json`: slot [2] from the registry's `dist.integrity`, slot [3] per the lock's own convention (see the vlt hosted-mode contract); every hosted instance of the pin together. * **cargo** — `Cargo.lock` back on crates.io (source + the sparse index's checksum, `SOCKET_CRATES_INDEX`); every `Cargo.toml` declaration loses its `registry = "socket-patch-"` pin (the shorthand the rewriter produced collapses back); every `[registries.socket-patch-]` block no manifest or lock still references leaves the project cargo config — including a superseded patch generation's block an earlier re-pin left behind (#864). A declaration it cannot unpin refuses. * **golang** — the hosted `replace` and the socket module's go.sum lines go; the upstream module's two go.sum lines come back, hashed from the module proxy (`SOCKET_GOPROXY`, else `GOPROXY` / `GONOPROXY` / `GOPRIVATE` as go reads them) and cross-checked against the checksum database (`SOCKET_GOSUMDB_URL`, else `sum.golang.org` unless `GOSUMDB=off` / `GONOSUMDB` / `GOPRIVATE` say go would not ask it). A `replace` the user had before the hosted run is not recorded anywhere, so the restore lands on the plain upstream module. diff --git a/crates/socket-patch-cli/tests/e2e_redirect_bun_build.rs b/crates/socket-patch-cli/tests/e2e_redirect_bun_build.rs index 683a4f2cd..4f05eb596 100644 --- a/crates/socket-patch-cli/tests/e2e_redirect_bun_build.rs +++ b/crates/socket-patch-cli/tests/e2e_redirect_bun_build.rs @@ -284,11 +284,19 @@ fn bun_toolchain(tag: &str) -> Option<(String, BunVersion)> { /// pre-rewrite assertions). Scrub BEFORE seeding: `Command`'s last env call /// for a name wins, and the scrub removes `BUN_INSTALL_CACHE_DIR`. fn bun(cwd: &Path, args: &[&str], cache_dir: &Path) -> Output { + bun_env(cwd, args, cache_dir, &[]) +} + +/// [`bun`] with extra env applied last (a fixture's user-level config). +fn bun_env(cwd: &Path, args: &[&str], cache_dir: &Path, env: &[(String, String)]) -> Output { let mut cmd = Command::new("bun"); cmd.args(args).current_dir(cwd); cache_env::scrub_ambient_bun_env(&mut cmd); cache_env::isolate(&mut cmd); cmd.env("BUN_INSTALL_CACHE_DIR", cache_dir); + for (k, v) in env { + cmd.env(k, v); + } cmd.output().expect("failed to run bun") } @@ -460,6 +468,16 @@ async fn mount_scoped_registry(server: &MockServer, tgz: Vec) { }))) .mount(server) .await; + // The version document a hosted unwind's restore reads. + Mock::given(method("GET")) + .and(path_regex(r"^/@scope(%2[fF]|/)pkg/1\.0\.0$")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "name": SCOPED_NAME, + "version": SCOPED_VERSION, + "dist": { "tarball": tarball_url, "integrity": sri(&tgz) } + }))) + .mount(server) + .await; Mock::given(method("GET")) .and(path(format!("/@scope/pkg/-/pkg-{SCOPED_VERSION}.tgz"))) .respond_with(ResponseTemplate::new(200).set_body_raw(tgz, "application/octet-stream")) @@ -546,9 +564,23 @@ struct BunRedirectFixture { /// `bun --version`, verbatim, for messages. bun_raw: String, bun_version: BunVersion, + /// Env every bun and socket-patch run of this fixture gets: the + /// `XDG_CONFIG_HOME` holding a [`ScopeConfig::UserNpmrc`] scope. + user_env: Vec<(String, String)>, _server: MockServer, } +/// Where the scoped target's registry is configured. +#[derive(Clone, Copy, PartialEq)] +enum ScopeConfig { + /// The project's committed `bunfig.toml` `[install.scopes]`. + ProjectBunfig, + /// Only the user's `$XDG_CONFIG_HOME/.npmrc` `@scope:registry`, where + /// private scopes and their tokens usually live (#1276): no + /// committable file names the registry. + UserNpmrc, +} + /// Which socket-patch invocation drives the hosted rewrite (step 3). Both /// route through the same hosted engine (`get --mode hosted` hands its /// selected (purl, uuid) pair to scan's `run_redirect_selected`), so every @@ -598,6 +630,27 @@ async fn bun_hosted_project( driver: HostedDriver, shape: LockShape, target: Target, +) -> Option { + bun_hosted_project_with( + tag, + tamper_served_tarball, + driver, + shape, + target, + ScopeConfig::ProjectBunfig, + ) + .await +} + +/// [`bun_hosted_project`] with the scoped target's registry configured +/// where `scope_config` says. +async fn bun_hosted_project_with( + tag: &str, + tamper_served_tarball: bool, + driver: HostedDriver, + shape: LockShape, + target: Target, + scope_config: ScopeConfig, ) -> Option { let (bun_raw, bun_version) = bun_toolchain(tag)?; if shape == LockShape::V1OnNewerBun && bun_version < LOCK_V2_FROM { @@ -619,19 +672,35 @@ async fn bun_hosted_project( // before the fixture install), the patch API, and the hosted tarball. let server = MockServer::start().await; let mut registry_field = String::new(); + let mut user_env = Vec::new(); if target == Target::ScopedWithDeps { let registry_tgz = scoped_registry_tgz(); mount_scoped_registry(&server, registry_tgz).await; - // bun's scoped-registry config — a committable file, so it travels - // with every fresh checkout below. - std::fs::write( - proj.join("bunfig.toml"), - format!( - "[install.scopes]\n\"@scope\" = {{ url = \"{}/\" }}\n", - server.uri() - ), - ) - .unwrap(); + match scope_config { + // bun's scoped-registry config — a committable file, so it + // travels with every fresh checkout below. + ScopeConfig::ProjectBunfig => std::fs::write( + proj.join("bunfig.toml"), + format!( + "[install.scopes]\n\"@scope\" = {{ url = \"{}/\" }}\n", + server.uri() + ), + ) + .unwrap(), + ScopeConfig::UserNpmrc => { + let xdg = tmp.path().join("xdg-config"); + std::fs::create_dir_all(&xdg).unwrap(); + std::fs::write( + xdg.join(".npmrc"), + format!("@scope:registry={}/\n", server.uri()), + ) + .unwrap(); + user_env.push(( + "XDG_CONFIG_HOME".to_string(), + xdg.to_str().unwrap().to_string(), + )); + } + } // For a non-default registry bun records the TARBALL URL as the // 4-tuple's registry field. registry_field = format!("{}/@scope/pkg/-/pkg-{SCOPED_VERSION}.tgz", server.uri()); @@ -639,7 +708,7 @@ async fn bun_hosted_project( // 1. REAL fixture: bun install (network here, private cache). Text lockfile. let cache = tmp.path().join("bun-cache"); - let install = bun(&proj, &fixture_install_args(bun_version), &cache); + let install = bun_env(&proj, &fixture_install_args(bun_version), &cache, &user_env); if !install.status.success() { assert!( !bun_required(), @@ -881,7 +950,7 @@ async fn bun_hosted_project( "fake", ], }; - let (code, stdout, stderr) = run_socket(&proj, &argv); + let (code, stdout, stderr) = run_socket_env(&proj, &argv, &user_env); assert_eq!( code, 0, "{} --mode hosted failed.\nstdout:\n{stdout}\nstderr:\n{stderr}", @@ -1016,6 +1085,7 @@ async fn bun_hosted_project( lock_version, bun_raw, bun_version, + user_env, _server: server, }) } @@ -1084,10 +1154,11 @@ fn fresh_checkout(fx: &BunRedirectFixture, name: &str) -> PathBuf { fn fresh_frozen_install(fx: &BunRedirectFixture, name: &str) -> (PathBuf, Output) { let fresh = fresh_checkout(fx, name); let fresh_cache = fx.tmp.path().join(format!("{name}-bun-cache")); - let ci = bun( + let ci = bun_env( &fresh, &["install", "--frozen-lockfile", "--ignore-scripts"], &fresh_cache, + &fx.user_env, ); (fresh, ci) } @@ -1157,7 +1228,12 @@ fn assert_patched_fresh_install(fx: &BunRedirectFixture) { let wired_lock = std::fs::read(fx.proj.join("bun.lock")).unwrap(); std::fs::remove_dir_all(fresh.join("node_modules")).unwrap(); let plain_cache = fx.tmp.path().join("fresh-plain-bun-cache"); - let plain = bun(&fresh, &["install", "--ignore-scripts"], &plain_cache); + let plain = bun_env( + &fresh, + &["install", "--ignore-scripts"], + &plain_cache, + &fx.user_env, + ); assert!( plain.status.success(), "plain `bun install` on the redirected lock must succeed.\nstdout:\n{}\nstderr:\n{}", @@ -1501,6 +1577,83 @@ async fn bun_redirect_rollback_restores_lock_and_original_install() { ); } +/// #1276: the scoped target's registry is set only in the user's +/// `$XDG_CONFIG_HOME/.npmrc`, as a private scope usually is. `rollback` +/// must read that registry (not the default one, which does not know the +/// package) and give back the tarball URL bun recorded, byte for byte, so +/// a fresh frozen install with the same user config lands the original +/// bytes. +#[tokio::test(flavor = "multi_thread")] +#[serial_test::serial] +async fn bun_redirect_rollback_reads_a_scope_set_only_in_the_user_npmrc() { + let Some(fx) = bun_hosted_project_with( + "user-npmrc-scope", + false, + HostedDriver::ScanVex, + LockShape::Native, + Target::ScopedWithDeps, + ScopeConfig::UserNpmrc, + ) + .await + else { + return; + }; + assert!( + !fx.proj.join("bunfig.toml").exists() && !fx.proj.join(".npmrc").exists(), + "no project file names the scope's registry" + ); + assert_patched_fresh_install(&fx); + + let proj = &fx.proj; + let mut env = unwind_env(&fx).await; + env.extend(fx.user_env.iter().cloned()); + let (code, stdout, stderr) = run_socket_env( + proj, + &[ + "rollback", + "--yes", + "--json", + "--cwd", + proj.to_str().unwrap(), + ], + &env, + ); + assert_eq!( + code, 0, + "rollback failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let envelope: serde_json::Value = serde_json::from_str(&stdout) + .unwrap_or_else(|e| panic!("rollback --json output is not JSON: {e}\nstdout:\n{stdout}")); + assert_eq!( + envelope["status"], "success", + "rollback envelope: {envelope}" + ); + assert!( + !stdout.contains("upstream_registry_fallback"), + "the user's scope registry was read: {envelope}" + ); + assert_eq!( + String::from_utf8(std::fs::read(proj.join("bun.lock")).unwrap()).unwrap(), + String::from_utf8(fx.lock_before.clone()).unwrap(), + "rollback must restore bun.lock byte-identical to the pre-redirect snapshot, \ + the scope registry's tarball URL in the slot" + ); + + let (fresh, ci) = fresh_frozen_install(&fx, "fresh-rolled-back"); + assert!( + ci.status.success(), + "fresh-checkout `bun install --frozen-lockfile` of the restored lock must \ + succeed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&ci.stdout), + String::from_utf8_lossy(&ci.stderr), + ); + let installed = std::fs::read(fx.target.installed_dir(&fresh).join("index.js")).unwrap(); + assert_eq!( + installed, fx.orig, + "after rollback the fresh install must be byte-identical to the pristine package" + ); +} + // ── digest-dropping lock re-saves (Bun 1.1.39–1.3.9) ───────────────── /// A local `file:` tarball dep added to `proj`'s package.json plus this diff --git a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs index 05851c204..b02169380 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs @@ -38,7 +38,9 @@ //! 6. A project with its own registries (bunfig `[install] registry` and //! `[install.scopes]`, #992): `remove ` and the takeover + //! `vendor --revert` chain keep each registry's tarball URL in the -//! slot (`in_process_vendor_bun_takeover/registry.rs`). +//! slot (`in_process_vendor_bun_takeover/registry.rs`), including +//! registries set only in the user's `.npmrc` or global bunfig +//! (#1276). //! //! Every child process gets the ambient `SOCKET_*` vars scrubbed and //! telemetry hard-disabled; each test runs in its own tempdir. @@ -353,6 +355,12 @@ fn registry_uri() -> &'static str { /// (`SOCKET_PATCH_SERVER_URL`) and the registry is the shared mirror /// (`SOCKET_NPM_REGISTRY`). Returns `(exit_code, stdout, stderr)`. fn run_cli(cwd: &Path, args: &[&str]) -> (i32, String, String) { + run_cli_env(cwd, args, &[]) +} + +/// [`run_cli`] with `env` set last, over the stand-in `HOME` (a test's own +/// user-level Bun config, #1276). +fn run_cli_env(cwd: &Path, args: &[&str], env: &[(&str, String)]) -> (i32, String, String) { let mut cmd = Command::new(env!("CARGO_BIN_EXE_socket-patch")); cmd.current_dir(cwd); for (key, _) in std::env::vars() { @@ -362,16 +370,22 @@ fn run_cli(cwd: &Path, args: &[&str]) -> (i32, String, String) { } // A registry exported by npm (`npm_config_registry`) or Bun would // steer the Bun restores off the fixtures' registries (#992). + // So would a user-level `.npmrc` / bunfig under `XDG_CONFIG_HOME` + // (#1276); `HOME` is the stand-in `prepare_command` pins. for key in [ "BUN_CONFIG_REGISTRY", "NPM_CONFIG_REGISTRY", "npm_config_registry", + "XDG_CONFIG_HOME", ] { cmd.env_remove(key); } cmd.env("SOCKET_TELEMETRY_DISABLED", "1") .env("SOCKET_NPM_REGISTRY", registry_uri()); let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); + for (key, value) in env { + cmd.env(key, value); + } let out = cmd.output().expect("spawn socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -382,7 +396,12 @@ fn run_cli(cwd: &Path, args: &[&str]) -> (i32, String, String) { /// `--json` invocation returning the parsed envelope. fn run_json(cwd: &Path, args: &[&str]) -> (i32, Value) { - let (code, stdout, stderr) = run_cli(cwd, args); + run_json_env(cwd, args, &[]) +} + +/// [`run_json`] with [`run_cli_env`]'s extra `env`. +fn run_json_env(cwd: &Path, args: &[&str], env: &[(&str, String)]) -> (i32, Value) { + let (code, stdout, stderr) = run_cli_env(cwd, args, env); // The child's stderr rides the harness's captured output so a failing // assertion downstream shows the CLI's own diagnostics. if !stderr.trim().is_empty() { diff --git a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs index 3aaf3a7c2..f2281a6f0 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs @@ -24,8 +24,8 @@ use wiremock::{Mock, MockServer, ResponseTemplate}; use super::{ assert_no_event_code, find_event, hosted_line, line_integrity, lock_line, patch_record, read, - run_json, vendor_cli, write_bun_project, HOSTED_URL, LEFT_PAD_REGISTRY_LINE, NAME, - PATCHED_INDEX, PATCHED_SHA512, PURL, UUID, VERSION, + run_json, run_json_env, vendor_cli, write_bun_project, HOSTED_URL, LEFT_PAD_REGISTRY_LINE, + NAME, PATCHED_INDEX, PATCHED_SHA512, PURL, UUID, VERSION, }; const SCOPED_NAME: &str = "@corp/widget"; @@ -135,13 +135,29 @@ impl Registries { /// leaves it (the lock's URL 3-tuples, no ledger) beside its bunfig.toml; /// returns the pristine registry lock. fn write_hosted_project(root: &Path, registries: &Registries) -> String { + let pristine = write_hosted_lock(root, registries, 2); + std::fs::write(root.join("bunfig.toml"), registries.bunfig()).unwrap(); + pristine +} + +/// [`write_hosted_project`] with no project registry settings, its lock +/// written as `lock_version` (2: Bun ≥ 1.4; 1: Bun 1.2 – 1.3, which Bun +/// 1.4 keeps as is); returns the pristine registry lock. +fn write_hosted_lock(root: &Path, registries: &Registries, lock_version: u8) -> String { let pristine = registries.pristine_lock(); + let pristine = if lock_version == 2 { + pristine + } else { + pristine.replace( + "\"lockfileVersion\": 2,\n \"configVersion\": 1,", + &format!("\"lockfileVersion\": {lock_version},"), + ) + }; write_bun_project( root, &pristine, &[(NAME, VERSION), (SCOPED_NAME, SCOPED_VERSION)], ); - std::fs::write(root.join("bunfig.toml"), registries.bunfig()).unwrap(); let hosted = pristine .replace( ®istries.left_pad_line(), @@ -282,3 +298,195 @@ async fn bun_takeover_then_vendor_revert_keeps_the_bunfig_and_scope_registry_url "the revert lands on the Bun-written registry lock" ); } + +// ── #1276: Bun's user and global config layers ────────────────────────────── + +/// The variable a user's own `.npmrc` takes the scope's token from. Not one +/// of the token variables a project's files may expand: the user wrote +/// this file, and Bun expands any variable in it. +const USER_TOKEN_VAR: &str = "CORP_REGISTRY_TOKEN"; + +impl Registries { + /// The registries as a user `.npmrc` spells them: the default registry, + /// the `@corp` scope and its token keyed by the scope registry's path. + fn user_npmrc(&self) -> String { + let uri = self.server.uri(); + let host_path = uri.trim_start_matches("http:"); + format!( + "registry={uri}/mirror/\n@corp:registry={uri}/corp/\n\ + {host_path}/corp/:_authToken=${{{USER_TOKEN_VAR}}}\n" + ) + } + + /// A registry with nothing on it: a restore that reads it lands on the + /// default registry, which does not know `@corp/widget`. + fn decoy(&self) -> String { + format!("{}/decoy/", self.server.uri()) + } +} + +/// `remove` of both pins in a project whose registries are set only at +/// user level, with `home` as `HOME` and `env` on top: lands on the +/// pristine lock. +fn assert_removes_restore_pristine( + root: &Path, + home: &Path, + env: &[(&str, String)], + pristine: &str, +) { + let cwd = root.to_str().unwrap(); + let mut env = env.to_vec(); + env.push(("HOME", home.to_str().unwrap().to_string())); + env.push(("USERPROFILE", home.to_str().unwrap().to_string())); + env.push((USER_TOKEN_VAR, SCOPE_TOKEN.to_string())); + for purl in [PURL, SCOPED_PURL] { + let (code, env) = run_json_env( + root, + &["remove", purl, "--yes", "--json", "--cwd", cwd], + &env, + ); + assert_eq!(code, 0, "remove {purl}: {env:#}"); + assert!(env["error"].is_null(), "{env:#}"); + assert_no_registry_fallback(&env); + } + assert_eq!( + read(root, "bun.lock"), + pristine, + "each slot holds the tarball URL of the registry Bun resolved it against" + ); +} + +/// #1276: a private scope (and the default registry) set only in the +/// user's `~/.npmrc`, the usual home of a scope's token, is the registry +/// the restore reads, with the token that file gives it. +#[tokio::test(flavor = "multi_thread")] +async fn bun_remove_reads_registries_set_only_in_the_user_npmrc() { + let registries = Registries::start().await; + let tmp = tempfile::tempdir().unwrap(); + let (root, home) = (tmp.path().join("proj"), tmp.path().join("home")); + std::fs::create_dir_all(&root).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + let pristine = write_hosted_lock(&root, ®istries, 2); + std::fs::write(home.join(".npmrc"), registries.user_npmrc()).unwrap(); + assert_removes_restore_pristine(&root, &home, &[], &pristine); +} + +/// #1276: with `XDG_CONFIG_HOME` set, Bun reads `$XDG_CONFIG_HOME/.npmrc` +/// in place of `~/.npmrc` and only `$XDG_CONFIG_HOME/.bunfig.toml` as the +/// global bunfig (measured on Bun 1.1.39 – 1.4.2); the `~` copies here +/// would send both packages to a registry that has neither. +#[tokio::test(flavor = "multi_thread")] +async fn bun_remove_reads_the_xdg_config_home_npmrc_and_bunfig() { + let registries = Registries::start().await; + for global in [".npmrc", ".bunfig.toml"] { + let tmp = tempfile::tempdir().unwrap(); + let (root, home) = (tmp.path().join("proj"), tmp.path().join("home")); + let xdg = tmp.path().join("xdg"); + for dir in [&root, &home, &xdg] { + std::fs::create_dir_all(dir).unwrap(); + } + let pristine = write_hosted_lock(&root, ®istries, 2); + let decoy = registries.decoy(); + std::fs::write( + home.join(".npmrc"), + format!("registry={decoy}\n@corp:registry={decoy}\n"), + ) + .unwrap(); + std::fs::write( + home.join(".bunfig.toml"), + format!("[install]\nregistry = \"{decoy}\"\n"), + ) + .unwrap(); + let config = if global == ".npmrc" { + registries.user_npmrc() + } else { + registries.bunfig() + }; + std::fs::write(xdg.join(global), config).unwrap(); + // The XDG `.npmrc` replaces `~/.npmrc` only when it exists; the + // bunfig leg has none, so its `~/.npmrc` decoy must lose to the + // global bunfig on this lockfileVersion-2 (Bun ≥ 1.4) lock. + assert_removes_restore_pristine( + &root, + &home, + &[("XDG_CONFIG_HOME", xdg.to_str().unwrap().to_string())], + &pristine, + ); + } +} + +/// #1276: a global `~/.bunfig.toml` (no `XDG_CONFIG_HOME`) carries the +/// registries, the scope's token included. +#[tokio::test(flavor = "multi_thread")] +async fn bun_remove_reads_registries_set_only_in_the_global_bunfig() { + let registries = Registries::start().await; + let tmp = tempfile::tempdir().unwrap(); + let (root, home) = (tmp.path().join("proj"), tmp.path().join("home")); + std::fs::create_dir_all(&root).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + let pristine = write_hosted_lock(&root, ®istries, 2); + std::fs::write(home.join(".bunfig.toml"), registries.bunfig()).unwrap(); + assert_removes_restore_pristine(&root, &home, &[], &pristine); +} + +/// #1276: Bun ≥ 1.4 (the first to write lockfileVersion 2) takes a key +/// any bunfig sets over any `.npmrc`; Bun ≤ 1.3 the other way round. On a +/// lockfileVersion-2 lock the global bunfig's registries win over the +/// project `.npmrc`'s. +#[tokio::test(flavor = "multi_thread")] +async fn bun_remove_on_a_v2_lock_takes_bunfig_over_npmrc() { + let registries = Registries::start().await; + let tmp = tempfile::tempdir().unwrap(); + let (root, home) = (tmp.path().join("proj"), tmp.path().join("home")); + std::fs::create_dir_all(&root).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + let pristine = write_hosted_lock(&root, ®istries, 2); + let decoy = registries.decoy(); + std::fs::write( + root.join(".npmrc"), + format!("registry={decoy}\n@corp:registry={decoy}\n"), + ) + .unwrap(); + std::fs::write(home.join(".bunfig.toml"), registries.bunfig()).unwrap(); + assert_removes_restore_pristine(&root, &home, &[], &pristine); +} + +/// #1276: on a lockfileVersion-1 lock, which Bun 1.2 – 1.3 write and Bun +/// 1.4 keeps, a `.npmrc` and a bunfig that name different registries for +/// the package leave the registry Bun resolves it against unknown. The +/// restore refuses with the checkout remedy instead of guessing, and +/// leaves the pin in place. +#[tokio::test(flavor = "multi_thread")] +async fn bun_remove_refuses_when_npmrc_and_bunfig_disagree_on_a_v1_lock() { + let registries = Registries::start().await; + let tmp = tempfile::tempdir().unwrap(); + let (root, home) = (tmp.path().join("proj"), tmp.path().join("home")); + std::fs::create_dir_all(&root).unwrap(); + std::fs::create_dir_all(&home).unwrap(); + write_hosted_lock(&root, ®istries, 1); + let hosted = read(&root, "bun.lock"); + let decoy = registries.decoy(); + std::fs::write(home.join(".npmrc"), format!("@corp:registry={decoy}\n")).unwrap(); + std::fs::write(root.join("bunfig.toml"), registries.bunfig()).unwrap(); + let cwd = root.to_str().unwrap(); + let env = [ + ("HOME", home.to_str().unwrap().to_string()), + ("USERPROFILE", home.to_str().unwrap().to_string()), + ]; + let (code, env) = run_json_env( + &root, + &["remove", SCOPED_PURL, "--yes", "--json", "--cwd", cwd], + &env, + ); + assert_ne!(code, 0, "the restore can't tell the registry: {env:#}"); + let text = env.to_string(); + assert!( + text.contains("Bun 1.4") && text.contains(&decoy) && text.contains("/corp/"), + "the refusal names both registries and the Bun versions behind them: {env:#}" + ); + assert_eq!( + lock_line(&read(&root, "bun.lock"), SCOPED_NAME), + lock_line(&hosted, SCOPED_NAME), + "the pin stays in place" + ); +} diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/bun_lockb.rs b/crates/socket-patch-core/src/patch/redirect/upstream/bun_lockb.rs index 2e90bfb7c..3d12f9384 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/bun_lockb.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/bun_lockb.rs @@ -17,7 +17,9 @@ use std::collections::BTreeSet; -use super::npm::{bun_tarball_url, by_uuid, fetch_dists_on, refuse_all_in, BunRegistrySettings}; +use super::npm::{ + bun_tarball_url, by_uuid, fetch_dists_on, refuse_all_in, BunConfigOrder, BunRegistrySettings, +}; use super::{Ctx, FormatResult, HostedPin, View}; use crate::vendor::bun_lockb::{BunLockb, NORMALIZED_FORMAT_1, NORMALIZED_WORKSPACE}; @@ -104,7 +106,9 @@ pub(super) async fn restore( .collect(); // The record keeps the tarball URL Bun fetches from, which is the // project registry's for a mirror (#992). - let settings = BunRegistrySettings::read(view, rel).await; + // A binary lock does not say which Bun wrote it (#1276). + let settings = BunRegistrySettings::read(view, rel, BunConfigOrder::Unknown).await; + let wanted = settings.refuse_ambiguous(rel, wanted, &mut result); let dists = fetch_dists_on( &wanted, |n| settings.registry_with_credentials(n), diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index c3d0d26e7..4fda8b0f6 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -1525,43 +1525,133 @@ pub(crate) async fn restore_pnpm_locks( /// lock (#992): a scoped package's `.npmrc` `@scope:registry`, else its /// `bunfig.toml` `[install.scopes]` entry; otherwise `env_registry` /// (`BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY`), the `.npmrc` -/// `registry`, then `bunfig.toml` `[install] registry` — Bun's own order. -/// `None` means Bun's default registry, npmjs. +/// `registry`, then `bunfig.toml` `[install] registry` — Bun ≤ 1.3's +/// order. [`bun_lookup_layered`] is the general form, with the user's own +/// files and Bun ≥ 1.4's order. +#[cfg(test)] +fn bun_lookup_registry( + npmrc: Option<&str>, + bunfig: Option<&str>, + env_registry: Option<&str>, + var: &dyn Fn(&str) -> Option, + name: &str, +) -> Option { + let project = |text: Option<&str>| { + text.map(|text| BunConfigFile { + text: text.to_string(), + users: false, + }) + .into_iter() + .collect() + }; + let files = BunConfigFiles { + npmrc: project(npmrc), + bunfig: project(bunfig), + }; + bun_lookup_layered(&files, env_registry, var, name, false) +} + +/// One Bun config file: its text, and whether it is the user's own (the +/// user `.npmrc` or the global bunfig) rather than the project's. +pub(super) struct BunConfigFile { + text: String, + users: bool, +} + +/// The config files Bun reads registries from, each kind highest +/// precedence first: the project `.npmrc` then the user's, the project +/// `bunfig.toml` then the global one. A key the project file sets wins +/// over the user's or global file; any other key still comes from those +/// (#1276, measured on Bun 1.1.39 – 1.4.2). +#[derive(Default)] +pub(super) struct BunConfigFiles { + npmrc: Vec, + bunfig: Vec, +} + +/// Where Bun reads the user's `.npmrc` and the global bunfig, measured on +/// Bun 1.1.39 – 1.4.2 (#1276): `$XDG_CONFIG_HOME/.npmrc` when that file +/// exists, else `~/.npmrc` (Bun ignores `NPM_CONFIG_USERCONFIG`); and +/// `$XDG_CONFIG_HOME/.bunfig.toml` when `XDG_CONFIG_HOME` is set — Bun +/// then never looks in `~` — else `~/.bunfig.toml`. The home dir is +/// `HOME`, or `USERPROFILE` on Windows. +fn bun_user_config_paths( + var: &dyn Fn(&str) -> Option, + exists: &dyn Fn(&std::path::Path) -> bool, +) -> (Option, Option) { + use std::path::PathBuf; + let dir = |key: &str| var(key).filter(|v| !v.is_empty()).map(PathBuf::from); + let xdg = dir("XDG_CONFIG_HOME"); + let home = dir(if cfg!(windows) { "USERPROFILE" } else { "HOME" }); + let npmrc = xdg + .as_ref() + .map(|xdg| xdg.join(".npmrc")) + .filter(|path| exists(path)) + .or_else(|| home.as_ref().map(|home| home.join(".npmrc"))); + let bunfig = match xdg { + Some(xdg) => Some(xdg.join(".bunfig.toml")), + None => home.map(|home| home.join(".bunfig.toml")), + }; + (npmrc, bunfig) +} + +/// The variables a config file may expand: any for the user's own files, +/// as Bun does, but only the [`BUN_EXPANDED_VARS`] token variables for the +/// project's — those files come with the project, and any other reference +/// (say `$GITHUB_TOKEN`) would hand that secret to a host the project +/// names. It expands to nothing instead. +fn bun_file_var<'a>( + var: &'a dyn Fn(&str) -> Option, + users: bool, +) -> impl Fn(&str) -> Option + 'a { + move |key: &str| { + (users || BUN_EXPANDED_VARS.contains(&key)) + .then(|| var(key)) + .flatten() + } +} + +/// The registry Bun resolves `name` against (#992, #1276), from `files`: +/// a scoped package's scope registry, else `env_registry` +/// (`BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY`), else the configured +/// default registry. `None` means Bun's default registry, npmjs. +/// +/// Bun keys every file's settings into one config, so a key comes from the +/// highest-precedence file that sets it: a project file over the user's or +/// global one of its kind, and between kinds, any `.npmrc` over any +/// bunfig on Bun ≤ 1.3 but any bunfig over any `.npmrc` on Bun ≥ 1.4 +/// (`bunfig_first`). A scope's entry with no URL (a bunfig token only) +/// takes the configured default registry with its own token. /// /// The registry carries the credentials Bun sends it: the bunfig entry's /// own `token` (Bearer) or `username` / `password` (Basic), else the /// `.npmrc` `//host/path/:_authToken` / `:_auth` / `:username` + /// `:_password` whose path covers the registry URL. `$VAR` / `${VAR}` in a /// bunfig value and `${VAR}` in an `.npmrc` value read `var`, as Bun -/// expands them, but only for the [`BUN_EXPANDED_VARS`] token variables: -/// these files come with the project, and any other reference (say -/// `$GITHUB_TOKEN`) would hand that secret to a host the project names. -/// It expands to nothing instead. A private scope registry answers 401 -/// without them. -fn bun_lookup_registry( - npmrc: Option<&str>, - bunfig: Option<&str>, +/// expands them, limited by [`bun_file_var`]. A private scope registry +/// answers 401 without them. +fn bun_lookup_layered( + files: &BunConfigFiles, env_registry: Option<&str>, var: &dyn Fn(&str) -> Option, name: &str, + bunfig_first: bool, ) -> Option { use super::super::npmrc::npmrc_top_level_value; - let var = &|key: &str| BUN_EXPANDED_VARS.contains(&key).then(|| var(key)).flatten(); fn url(value: &str) -> Option { let value = value.trim().trim_matches(['"', '\'']); (value.starts_with("https://") || value.starts_with("http://")).then(|| value.to_string()) } // A bunfig registry is a URL string or a table carrying `url` and // maybe its credentials. - let toml_url = |item: Option<&toml_edit::Item>| -> Option { - let item = item?; + let toml_url = |item: &toml_edit::Item, var: &dyn Fn(&str) -> Option| { let value = item .as_str() .or_else(|| item.get("url").and_then(toml_edit::Item::as_str))?; url(&expand_url_env(value, var, true)) }; - let toml_auth = |item: &toml_edit::Item| -> Option { + let toml_auth = |item: &toml_edit::Item, var: &dyn Fn(&str) -> Option| { let field = |key: &str| { item.get(key) .and_then(toml_edit::Item::as_str) @@ -1578,15 +1668,12 @@ fn bun_lookup_registry( base64::engine::general_purpose::STANDARD.encode(format!("{user}:{password}")) )) }; + // An `.npmrc` key from the highest-precedence `.npmrc` that sets it. let npmrc_value = |key: &str| { - npmrc - .and_then(|text| npmrc_top_level_value(text, key)) - .map(|v| expand_env(&v, var, false)) - }; - let npmrc_url = |key: &str| { - npmrc - .and_then(|text| npmrc_top_level_value(text, key)) - .and_then(|v| url(&expand_url_env(&v, var, false))) + files.npmrc.iter().find_map(|file| { + let value = npmrc_top_level_value(&file.text, key)?; + Some(expand_env(&value, &bun_file_var(var, file.users), false)) + }) }; // Credentials in the URL itself (`https://user:${TOKEN}@host/`) go on // the request only: the base is written into bun.lock and warnings. @@ -1600,53 +1687,97 @@ fn bun_lookup_registry( authorization, } }; - let bunfig = bunfig.and_then(|text| text.parse::().ok()); - let install = bunfig.as_ref().and_then(|doc| doc.get("install")); - // The configured default registry before the environment applies. - let configured = || -> Option { - if let Some(base) = npmrc_url("registry") { - return Some(with_npmrc_auth(base, None)); - } - let item = install?.get("registry")?; - let base = toml_url(Some(item))?; - Some(with_npmrc_auth(base, toml_auth(item))) + enum Source<'a> { + Npmrc(&'a BunConfigFile), + Bunfig(toml_edit::DocumentMut, bool), + } + let npmrcs = files.npmrc.iter().map(Source::Npmrc); + let bunfigs = files.bunfig.iter().filter_map(|file| { + let doc = file.text.parse::().ok()?; + Some(Source::Bunfig(doc, file.users)) + }); + let sources: Vec = if bunfig_first { + bunfigs.chain(npmrcs).collect() + } else { + npmrcs.chain(bunfigs).collect() + }; + // The configured default registry before the environment applies, + // and whether the user's own file (not the project's) set it. + let configured = || { + sources.iter().find_map(|source| match source { + Source::Npmrc(file) => { + let value = npmrc_top_level_value(&file.text, "registry")?; + let base = url(&expand_url_env( + &value, + &bun_file_var(var, file.users), + false, + ))?; + Some((with_npmrc_auth(base, None), file.users)) + } + Source::Bunfig(doc, users) => { + let item = doc.get("install")?.get("registry")?; + let var = bun_file_var(var, *users); + let base = toml_url(item, &var)?; + Some((with_npmrc_auth(base, toml_auth(item, &var)), *users)) + } + }) }; if let Some((scope, _)) = name.strip_prefix('@').and_then(|rest| rest.split_once('/')) { - if let Some(scoped) = npmrc_url(&format!("@{scope}:registry")) { - return Some(with_npmrc_auth(scoped, None)); - } - let entry = install.and_then(|i| i.get("scopes")).and_then(|scopes| { - scopes - .get(scope) - .or_else(|| scopes.get(format!("@{scope}"))) - }); - if let Some(entry) = entry { - if let Some(scoped) = toml_url(Some(entry)) { - return Some(with_npmrc_auth(scoped, toml_auth(entry))); + let key = format!("@{scope}:registry"); + let scoped = sources.iter().find_map(|source| match source { + Source::Npmrc(file) => { + let value = npmrc_top_level_value(&file.text, &key)?; + let base = url(&expand_url_env( + &value, + &bun_file_var(var, file.users), + false, + ))?; + Some(Some(with_npmrc_auth(base, None))) } - // A scope entry with no URL (a token only) takes the configured - // default registry, never the environment's, with its own - // credentials. With no default configured that is npmjs, which - // still gets the scope's token: a private npmjs scope 401s - // without it. - if entry.is_table_like() && entry.get("url").is_none() { - let own = toml_auth(entry); - return match configured() { - Some(r) => Some(ProjectRegistry { - authorization: own.or(r.authorization), + Source::Bunfig(doc, users) => { + let scopes = doc.get("install")?.get("scopes")?; + let entry = scopes + .get(scope) + .or_else(|| scopes.get(format!("@{scope}")))?; + let var = bun_file_var(var, *users); + if let Some(base) = toml_url(entry, &var) { + return Some(Some(with_npmrc_auth(base, toml_auth(entry, &var)))); + } + // A scope entry with no URL (a token only) takes the + // configured default registry, never the environment's, + // with its own credentials. With no default configured + // that is npmjs, which still gets the scope's token: a + // private npmjs scope 401s without it. A token from the + // user's own file never goes to a default registry the + // project's file names: the repository would pick the host + // that receives the user's credential. + if !entry.is_table_like() || entry.get("url").is_some() { + return None; + } + let own = toml_auth(entry, &var); + Some(match configured() { + Some((r, from_users)) => Some(ProjectRegistry { + authorization: if *users && !from_users { + r.authorization + } else { + own.or(r.authorization) + }, ..r }), None => own.map(|own| ProjectRegistry { base: format!("{}/", crate::vendor::registry_fetch::DEFAULT_NPM_REGISTRY), authorization: Some(own), }), - }; + }) } + }); + if let Some(scoped) = scoped { + return scoped; } } match env_registry.and_then(url) { Some(base) => Some(with_npmrc_auth(base, None)), - None => configured(), + None => configured().map(|(r, _)| r), } } @@ -1795,52 +1926,93 @@ fn bun_env_registry(var: impl Fn(&str) -> Option) -> Option { .find_map(|key| var(key).filter(|v| v.starts_with("https://") || v.starts_with("http://"))) } -/// The settings beside a Bun lock that decide which registry Bun resolves -/// each package against, and so which tarball URL it recorded (#992). +/// Which kind of Bun config file wins a key both kinds set, as far as the +/// lock tells (#1276, measured on Bun 1.1.39 – 1.4.2). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(super) enum BunConfigOrder { + /// A lockfileVersion-2 `bun.lock`, which only Bun ≥ 1.4 writes (Bun + /// 1.3 ignores it): any bunfig over any `.npmrc`. + BunfigFirst, + /// A lockfileVersion-0/1 `bun.lock`, which Bun 1.4 keeps as is, or a + /// `bun.lockb`: Bun ≤ 1.3 takes any `.npmrc` over any bunfig and Bun + /// ≥ 1.4 the other way round, so the lock does not say which. + Unknown, +} + +impl BunConfigOrder { + /// The order for the `bun.lock` `text`. + pub(super) fn of_text_lock(text: &str) -> Self { + match crate::vendor::bun_lock_text::lock_version(text) { + Some(v) if v >= 2 => Self::BunfigFirst, + _ => Self::Unknown, + } + } +} + +/// The settings that decide which registry Bun resolves each package of a +/// lock against, and so which tarball URL it recorded (#992): the +/// `.npmrc` and `bunfig.toml` beside the lock, the user's `.npmrc` and the +/// global bunfig (#1276), and the registry environment variables. pub(super) struct BunRegistrySettings { - npmrc: Option, - bunfig: Option, + files: BunConfigFiles, env_registry: Option, + order: BunConfigOrder, } impl BunRegistrySettings { - pub(super) async fn read(view: &mut View<'_>, rel: &str) -> Self { + pub(super) async fn read(view: &mut View<'_>, rel: &str, order: BunConfigOrder) -> Self { let dir_prefix = match rel.rsplit_once('/') { Some((dir, _)) => format!("{dir}/"), None => String::new(), }; - let npmrc = view - .read(&format!("{dir_prefix}.npmrc")) - .await - .ok() - .flatten(); - let bunfig = view - .read(&format!("{dir_prefix}bunfig.toml")) - .await - .ok() - .flatten(); - // Unit tests read no ambient registry: npm exports + let mut files = BunConfigFiles::default(); + for (name, kind) in [ + (".npmrc", &mut files.npmrc), + ("bunfig.toml", &mut files.bunfig), + ] { + if let Some(text) = view + .read(&format!("{dir_prefix}{name}")) + .await + .ok() + .flatten() + { + kind.push(BunConfigFile { text, users: false }); + } + } + // Unit tests read no ambient registry or user config: npm exports // `npm_config_registry` to child processes whenever one is - // configured, which would otherwise steer the fixtures' restores. + // configured, and a developer's `~/.npmrc` would steer the + // fixtures' restores the same way. let env_registry = if cfg!(test) { None } else { + let (npmrc, bunfig) = + bun_user_config_paths(&|key| std::env::var_os(key), &|path| path.is_file()); + for (path, kind) in [(npmrc, &mut files.npmrc), (bunfig, &mut files.bunfig)] { + let Some(path) = path else { continue }; + if let Ok(text) = crate::utils::fs::read_regular_to_string(&path).await { + kind.push(BunConfigFile { text, users: true }); + } + } bun_env_registry(|key| std::env::var(key).ok()) }; Self { - npmrc, - bunfig, + files, env_registry, + order, } } - /// The registry Bun resolves `name` against; `None` means npmjs. - pub(super) fn registry(&self, name: &str) -> Option { - self.registry_with_credentials(name).map(|r| r.base) + #[cfg(test)] + fn from_files(files: BunConfigFiles, order: BunConfigOrder) -> Self { + Self { + files, + env_registry: None, + order, + } } - /// [`Self::registry`] with the credentials Bun sends it. - pub(super) fn registry_with_credentials(&self, name: &str) -> Option { + fn lookup(&self, name: &str, bunfig_first: bool) -> Option { // Unit tests read no ambient variables, as for `env_registry`. let var = |key: &str| { if cfg!(test) { @@ -1849,14 +2021,80 @@ impl BunRegistrySettings { std::env::var(key).ok() } }; - bun_lookup_registry( - self.npmrc.as_deref(), - self.bunfig.as_deref(), + bun_lookup_layered( + &self.files, self.env_registry.as_deref(), &var, name, + bunfig_first, ) } + + /// The registry Bun resolves `name` against; `None` means npmjs. + pub(super) fn registry(&self, name: &str) -> Option { + self.registry_with_credentials(name).map(|r| r.base) + } + + /// [`Self::registry`] with the credentials Bun sends it. Under + /// [`BunConfigOrder::Unknown`] it is Bun ≤ 1.3's answer: call it only + /// for a package [`Self::refuse_ambiguous`] kept, where both agree. + pub(super) fn registry_with_credentials(&self, name: &str) -> Option { + self.lookup(name, self.order == BunConfigOrder::BunfigFirst) + } + + /// Why the registry Bun resolves `name` against can't be told: under + /// [`BunConfigOrder::Unknown`], Bun ≤ 1.3 and Bun ≥ 1.4 resolve it + /// against different registries (or credentials) because an `.npmrc` + /// and a bunfig both set the deciding key. + fn ambiguity(&self, rel: &str, name: &str) -> Option { + if self.order != BunConfigOrder::Unknown { + return None; + } + let (old, new) = (self.lookup(name, false), self.lookup(name, true)); + let same = |a: &Option, b: &Option| match (a, b) { + (Some(a), Some(b)) => a.base == b.base && a.authorization == b.authorization, + (None, None) => true, + _ => false, + }; + if same(&old, &new) { + return None; + } + // Never a credential: only the bases, which carry no userinfo. + let shown = |r: &Option| match r { + Some(r) => r.base.clone(), + None => "npmjs".to_string(), + }; + let (old_shown, new_shown) = (shown(&old), shown(&new)); + let against = if old_shown == new_shown { + format!("{old_shown} with different credentials") + } else { + format!("{old_shown} (an .npmrc setting wins) but Bun 1.4+ against {new_shown} (a bunfig setting wins)") + }; + Some(format!( + "Bun ≤ 1.3 resolves {name} against {against}, and {rel} does not say which Bun \ + installs it" + )) + } + + /// `wanted` (uuid, name, version) without the packages whose registry + /// can't be told ([`Self::ambiguity`]), each refused in `result`. + pub(super) fn refuse_ambiguous( + &self, + rel: &str, + wanted: BTreeSet<(String, String, String)>, + result: &mut FormatResult, + ) -> BTreeSet<(String, String, String)> { + wanted + .into_iter() + .filter(|(uuid, name, version)| match self.ambiguity(rel, name) { + Some(why) => { + result.refuse(uuid, format!("{name}@{version}: {why}")); + false + } + None => true, + }) + .collect() + } } /// The tarball URL Bun recorded for `name@version` resolved against @@ -1975,8 +2213,10 @@ pub(crate) async fn restore_bun_locks( .map(|(_, u, n, v, _)| (u.clone(), n.clone(), v.clone())) .collect(); // Bun records the tarball URL of a package from any registry but - // npmjs, so the restore reads the project's registry settings. - let settings = BunRegistrySettings::read(view, rel).await; + // npmjs, so the restore reads the registry settings Bun does. + let settings = + BunRegistrySettings::read(view, rel, BunConfigOrder::of_text_lock(&text)).await; + let wanted = settings.refuse_ambiguous(rel, wanted, &mut result); let dists = fetch_dists_on( &wanted, |n| settings.registry_with_credentials(n), @@ -2112,6 +2352,263 @@ mod tests { }; use crate::patch::redirect::upstream::client::NpmDist; + fn files(npmrc: &[(&str, bool)], bunfig: &[(&str, bool)]) -> super::BunConfigFiles { + let layer = |list: &[(&str, bool)]| { + list.iter() + .map(|(text, users)| super::BunConfigFile { + text: text.to_string(), + users: *users, + }) + .collect() + }; + super::BunConfigFiles { + npmrc: layer(npmrc), + bunfig: layer(bunfig), + } + } + + #[test] + fn bun_user_config_paths_follow_bun_s_lookup() { + use std::ffi::OsString; + use std::path::{Path, PathBuf}; + let home_var = if cfg!(windows) { "USERPROFILE" } else { "HOME" }; + let env = |pairs: &'static [(&'static str, &'static str)]| { + move |key: &str| { + let key = if key == home_var { "HOME" } else { key }; + pairs + .iter() + .find(|(k, _)| *k == key) + .map(|(_, v)| OsString::from(v)) + } + }; + let paths = |pairs: &'static [(&'static str, &'static str)], existing: &[&str]| { + let existing: Vec = existing.iter().map(PathBuf::from).collect(); + super::bun_user_config_paths(&env(pairs), &|p: &Path| existing.iter().any(|e| e == p)) + }; + let some = |p: &str| Some(PathBuf::from(p)); + // No XDG: both under the home dir. + assert_eq!( + paths(&[("HOME", "/h")], &[]), + (some("/h/.npmrc"), some("/h/.bunfig.toml")) + ); + // XDG set: the XDG `.npmrc` replaces `~/.npmrc` only when it exists, + // while the global bunfig is the XDG one whether or not it exists. + assert_eq!( + paths(&[("HOME", "/h"), ("XDG_CONFIG_HOME", "/x")], &[]), + (some("/h/.npmrc"), some("/x/.bunfig.toml")) + ); + assert_eq!( + paths( + &[("HOME", "/h"), ("XDG_CONFIG_HOME", "/x")], + &["/x/.npmrc", "/h/.npmrc"] + ), + (some("/x/.npmrc"), some("/x/.bunfig.toml")) + ); + // An empty variable counts as unset; no home and no XDG: nothing. + assert_eq!( + paths(&[("HOME", "/h"), ("XDG_CONFIG_HOME", "")], &[]), + (some("/h/.npmrc"), some("/h/.bunfig.toml")) + ); + assert_eq!(paths(&[], &[]), (None, None)); + // Bun never reads `NPM_CONFIG_USERCONFIG`. + assert_eq!( + paths(&[("NPM_CONFIG_USERCONFIG", "/u/npmrc")], &[]), + (None, None) + ); + } + + #[test] + fn bun_layers_take_each_key_from_the_highest_file_that_sets_it() { + let lookup = |files: &super::BunConfigFiles, name: &str, bunfig_first: bool| { + super::bun_lookup_layered(files, None, &|_| None, name, bunfig_first).map(|r| r.base) + }; + let some = |s: &str| Some(s.to_string()); + // The project `.npmrc` wins its own keys; the user's file still + // supplies the rest (a scope, the default registry). + let npmrc = files( + &[ + ("@p:registry=https://proj.example/\n", false), + ( + "@p:registry=https://user-p.example/\n@u:registry=https://user.example/\nregistry=https://ureg.example/\n", + true, + ), + ], + &[], + ); + assert_eq!(lookup(&npmrc, "@p/a", false), some("https://proj.example/")); + assert_eq!(lookup(&npmrc, "@u/a", false), some("https://user.example/")); + assert_eq!(lookup(&npmrc, "a", false), some("https://ureg.example/")); + // The project bunfig wins over the global one, key by key. + let bunfig = files( + &[], + &[ + ("[install.scopes]\np = \"https://proj.example/\"\n", false), + ( + "[install]\nregistry = \"https://greg.example/\"\n\n[install.scopes]\np = \"https://glob-p.example/\"\ng = \"https://glob.example/\"\n", + true, + ), + ], + ); + assert_eq!( + lookup(&bunfig, "@p/a", false), + some("https://proj.example/") + ); + assert_eq!( + lookup(&bunfig, "@g/a", false), + some("https://glob.example/") + ); + assert_eq!(lookup(&bunfig, "a", false), some("https://greg.example/")); + // Between kinds: Bun ≤ 1.3 takes any `.npmrc` first, Bun ≥ 1.4 any + // bunfig — the user `.npmrc` against the project bunfig here. + let both = files( + &[("@s:registry=https://user-npmrc.example/\n", true)], + &[( + "[install.scopes]\ns = \"https://proj-bunfig.example/\"\n", + false, + )], + ); + assert_eq!( + lookup(&both, "@s/a", false), + some("https://user-npmrc.example/") + ); + assert_eq!( + lookup(&both, "@s/a", true), + some("https://proj-bunfig.example/") + ); + // A scope beats the default registry wherever each is set. + let mixed = files( + &[("registry=https://proj-reg.example/\n", false)], + &[( + "[install.scopes]\ns = \"https://glob-scope.example/\"\n", + true, + )], + ); + for bunfig_first in [false, true] { + assert_eq!( + lookup(&mixed, "@s/a", bunfig_first), + some("https://glob-scope.example/") + ); + } + // A global token-only scope entry takes the user's default registry. + let token_only = files( + &[("registry=https://ureg.example/\n", true)], + &[("[install.scopes]\ns = { token = \"t\" }\n", true)], + ); + let r = super::bun_lookup_layered(&token_only, None, &|_| None, "@s/a", false).unwrap(); + assert_eq!( + (r.base.as_str(), r.authorization.as_deref()), + ("https://ureg.example/", Some("Bearer t")) + ); + // It never goes to a default registry the project's file names, + // in either order: the repository would pick the host that gets + // the user's token. + for (npmrc, bunfig) in [ + (vec![("registry=https://evil.example/\n", false)], vec![]), + ( + vec![], + vec![("[install]\nregistry = \"https://evil.example/\"\n", false)], + ), + ] { + let mut bunfig = bunfig; + bunfig.push(("[install.scopes]\ns = { token = \"t\" }\n", true)); + let leaked = files(&npmrc, &bunfig); + for bunfig_first in [false, true] { + let r = super::bun_lookup_layered(&leaked, None, &|_| None, "@s/a", bunfig_first) + .unwrap(); + assert_eq!( + (r.base.as_str(), r.authorization.as_deref()), + ("https://evil.example/", None) + ); + } + } + } + + #[test] + fn bun_user_files_expand_any_variable_project_files_only_token_ones() { + let vars = |key: &str| (key == "GITHUB_TOKEN").then(|| "gh".to_string()); + let auth = |users: bool| { + let files = files( + &[( + "@s:registry=https://npm.pkg.example/\n//npm.pkg.example/:_authToken=${GITHUB_TOKEN}\n", + users, + )], + &[], + ); + super::bun_lookup_layered(&files, None, &vars, "@s/a", false) + .and_then(|r| r.authorization) + }; + // The user wrote `~/.npmrc`: Bun expands any variable in it. + assert_eq!(auth(true), Some("Bearer gh".to_string())); + // The project's file can't send `$GITHUB_TOKEN` anywhere. + assert_eq!(auth(false), None); + // The token key from the user's file covers a project-set scope + // registry on the same host, as in Bun. + let split = files( + &[ + ("@s:registry=https://npm.pkg.example/\n", false), + ("//npm.pkg.example/:_authToken=${GITHUB_TOKEN}\n", true), + ], + &[], + ); + assert_eq!( + super::bun_lookup_layered(&split, None, &vars, "@s/a", false) + .and_then(|r| r.authorization), + Some("Bearer gh".to_string()) + ); + } + + #[test] + fn bun_refuses_a_registry_the_lock_s_bun_version_would_decide() { + use super::{BunConfigOrder, BunRegistrySettings}; + assert_eq!( + BunConfigOrder::of_text_lock("{\n \"lockfileVersion\": 2,\n}"), + BunConfigOrder::BunfigFirst + ); + for v in ["0", "1"] { + assert_eq!( + BunConfigOrder::of_text_lock(&format!("{{\n \"lockfileVersion\": {v},\n}}")), + BunConfigOrder::Unknown + ); + } + let conflict = || { + files( + &[("@s:registry=https://n.example/\n", true)], + &[("[install.scopes]\ns = \"https://b.example/\"\n", false)], + ) + }; + let unknown = BunRegistrySettings::from_files(conflict(), BunConfigOrder::Unknown); + let why = unknown.ambiguity("bun.lock", "@s/a").expect("ambiguous"); + assert!( + why.contains("https://n.example/") + && why.contains("https://b.example/") + && why.contains("bun.lock does not say which Bun"), + "{why}" + ); + // A package no conflicting key decides is not ambiguous. + assert_eq!(unknown.ambiguity("bun.lock", "a"), None); + assert_eq!(unknown.ambiguity("bun.lock", "@t/a"), None); + // A v2 lock is Bun ≥ 1.4's: the bunfig wins, nothing is refused. + let v2 = BunRegistrySettings::from_files(conflict(), BunConfigOrder::BunfigFirst); + assert_eq!(v2.ambiguity("bun.lock", "@s/a"), None); + assert_eq!(v2.registry("@s/a").as_deref(), Some("https://b.example/")); + // Same registry, different credentials: still refused. + let creds = files( + &[("//r.example/:_authToken=n\n", true)], + &[( + "[install]\nregistry = { url = \"https://r.example/\", token = \"b\" }\n", + false, + )], + ); + let mut creds = creds; + creds.npmrc[0] + .text + .insert_str(0, "registry=https://r.example/\n"); + let creds = BunRegistrySettings::from_files(creds, BunConfigOrder::Unknown); + assert!(creds + .ambiguity("bun.lock", "a") + .is_some_and(|why| why.contains("different credentials") && !why.contains("Bearer"))); + } + #[test] fn bun_reads_the_registry_in_bun_s_own_order() { let bunfig = "[install]\nregistry = \"https://b.example/\"\n\n\ diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 759d7d182..345b14187 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -242,9 +242,15 @@ The backticked slug in each row is the value `-e`/`--ecosystems` accepts (e.g. remedy (then `bun install --force`: a plain install keeps the patched copy), while the hosted → vendored takeover rebuilds its npm registry record natively and vendors over it. Each restore reads the package's version document from the registry Bun resolves it against (`.npmrc` / `bunfig.toml` scope and default - registries, `BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY`), sending the credentials + registries, `BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY`), read from the project's + files and the user's own: `$XDG_CONFIG_HOME/.npmrc` or `~/.npmrc`, and the global + `$XDG_CONFIG_HOME/.bunfig.toml` or `~/.bunfig.toml` (#1276). Bun ≥ 1.4 takes a bunfig + key over an `.npmrc` one and Bun ≤ 1.3 the reverse, so when they disagree on a package + of a lockfileVersion-0/1 `bun.lock` or a `bun.lockb` (either Bun may install it) the + restore refuses that pin with the checkout remedy. It sends the credentials those settings give it — a bunfig `token` or `username` / `password` (`$VAR` - expanded only for `NPM_TOKEN`, `NODE_AUTH_TOKEN` and `BUN_AUTH_TOKEN`; any other + expanded only for `NPM_TOKEN`, `NODE_AUTH_TOKEN` and `BUN_AUTH_TOKEN` in the project's + files, any variable in the user's own; any other variable expands to nothing, so a project's config cannot send other secrets; in a registry URL only its `user:password@` part expands, which is sent as the `Authorization` header and never printed or written into the lock), else the `.npmrc` `//host/path/:_authToken` / `_auth` / `username` +