socket-patch supports hosted and vendored Python patches in native uv locks,
PEP 723 script locks, PEP 751 locks, and requirements.txt. The tests use real
uv binaries, real PyPI artifacts, and the public Socket patch service. Successful
rewriting alone is not an installation result: the backtest reinstalls from the
rewritten files and compares the installed bytes with the published patch.
This supplements the existing hosted and vendored production suites. See the ecosystem matrix for other package managers.
| Input | Hosted | Vendored |
|---|---|---|
requirements.txt, including uv-generated hash continuations |
Exact version pins become direct artifact URLs with the patched SHA-256. Extras and markers are retained; hashes for the replaced artifact are removed. | Requirements refer to a committed wheel under .socket/vendor/pypi/ with its hash. |
uv.lock, native version = 1, [[package]] |
The package source and artifact entry agree on the hosted URL and hash. A paired pyproject.toml receives the corresponding uv source configuration. |
The package source refers to the committed wheel. The paired pyproject.toml records that source. |
uv.lock, experimental [[distribution]] (uv 0.1.45–0.2.34) |
Follows the entry's own shape: direct+ string or { url = … } table source, [[distribution.wheel]] sub-table or inline wheels entry, and source-qualified dependency references where the lock carries them. |
Refused (pypi_uv_legacy_lock_unsupported): the experimental grammar cannot carry a portable local wheel — through 0.2.6 a relative path source does not parse, and on 0.2.17–0.2.34 --locked rejects it while uv lock and plain uv sync absolutize it. Use uv 0.2.35 or newer. |
*.py.lock with its PEP 723 *.py script |
Rewrites the lock and the script's uv source metadata together. | Rewrites the lock and script metadata together and commits the patched wheel. |
pylock.toml and pylock.<name>.toml, PEP 751 lock-version = "1.0" |
Uses one archive URL with the patched SHA-256. |
Uses one archive path with the committed wheel's SHA-256. |
Replacing a wheel removes stale sdist entries, sizes, and upload times. A source archive occupies an sdist/archive entry rather than a wheel entry. Native uv dependency references are updated when they identify the replaced source.
The native project and script metadata edits matter for ordinary resolution:
changing only the lock's source can make uv sync --locked reject the lock, or
let an ordinary uv sync restore the registry source. The backtest records
frozen, locked, and ordinary installation outcomes separately where supported.
- The installed env is the one uv syncs into. With
UV_PROJECT_ENVIRONMENTset (absolute, or relative to the project), the crawler uses that env in place of./.venvor an activatedVIRTUAL_ENV, asuv sync/uv rundo. This holds for a project withuv.lock, or apyproject.tomlthat no other manager's lock claims. Agent mode patches that env, and the hosted stale-install warning andvexcheck it. Run the scan with the sameUV_PROJECT_ENVIRONMENTthatuv syncused. - uv 0.0 has no native
uv.lock; its compatibility lane is compiled requirements.uv pip syncrejects the bare local wheel paths emitted by vendored requirements through uv 0.1.23 (Unexpected '.', expected '-c', '-e', '-r' or the start of a requirement) and accepts them from 0.1.24; use hosted mode or upgrade uv for older binaries. uv lockitself has a boundary: the subcommand appears in 0.1.42 but panics (not yet implemented) through 0.1.44 and first writes a lock at 0.1.45, the last 0.1 release. Below that, hosted mode covers uv through compiled requirements (from 0.0.5).- uv 0.1.45–0.2.34 write the experimental
[[distribution]]lock grammar;[[package]]starts at 0.2.35. The grammar went through three shapes, and the hosted rewriter follows the entry's own shape on each axis: string sources with sub-table artifacts (source = "registry+…",[distribution.sdist],[[distribution.wheel]], source-qualified[[distribution.dependencies]]) through 0.2.5; string sources with inline artifacts (sdist = { … },wheels = [ … ]) from 0.2.6 through 0.2.17; and inline-table sources (source = { registry = … }) from 0.2.18. Emitting the wrong shape is not a parse error the user sees: 0.2.18–0.2.34 reject the string source and silently ignore the lock (--frozen/--lockedfail, an ordinaryuv syncstill installs the patch via the pyproject source), and 0.2.6–0.2.17 ignore an unexpected[[distribution.wheel]]and try to build the direct wheel URL as a source archive. The[[distribution]]grammar is hosted-only: vendored native wiring is refused withpypi_uv_legacy_lock_unsupported. The refusal is kept because no spelling of a committed wheel is stable across that era: through 0.2.6 a relative path source cannot be parsed at all (path+<rel>is an invalid URL,path+file:forms resolve against the filesystem root or panic); 0.2.17–0.2.34 install a relative{ path = … }source under--frozenand plainuv sync, but--lockedrejects the non-canonical spelling,uv lockand plainuv syncrewrite the path to an absolute one, resolution is relative to the current directory rather than the project, and hashes are not verified before 0.2.34. Use uv 0.2.35 or newer for native vendoring; vendored requirements work on the same binaries. - Vendored native wiring covers every
[[package]]-grammar release, uv 0.2.35 onward. uv 0.2.35 and 0.2.36 wrote no root[package.metadata]yet (it arrived in 0.2.37); on those locks the requires-dist repoint is skipped rather than refused, and the package source plus the pyproject[tool.uv.sources]entry carry the redirect (verified with--frozen,--locked, and ordinary installs). A lock that has metadata but no entry for the package is stale and is still refused withpypi_uv_lock_package_missing. - Native lock versions other than
version = 1, and PEP 751 versions other thanlock-version = "1.0", are refused. Lockrevisionvalues 1 (0.6.0–0.6.14), 2 (0.6.15–0.8.3) and 3 (0.8.4 onward) are covered by the release-family runner. - Command availability boundaries observed with real binaries:
uv lockwrites a lock from 0.1.45 (see above);uv exportfrom 0.4.1, its--output-fileflag from 0.4.7 (the harness reads the export from stdout below that);uv lock --scriptfrom 0.5.17; PEP 751uv pip compile --output-file pylock.tomlfrom 0.6.15. Earlier binaries record those lanes as unavailable, not as failures. - After an unwind (
rollback,remove,vendor --revert)uv synckeeps the patched build, because the restored lock pins the same version. The unwind warnsvendor_pypi_reinstall_required/redirect_pypi_reinstall_requiredand namesuv sync --reinstall-package <name>(#477). [tool.uv] dev-dependencies(the pre-PEP 735 dev group) is classified as a direct dependency, and every duplicaterequires-dist/requires-deventry for the package (extras, markers) is repointed, souv sync --lockedaccepts the lock. The hosted unwind (rollback,remove, the hosted → vendored takeover) puts each entry's specifier back from the declaration uv lowered it from, so one package declared with different specifiers independencies, extras or marker-split lines, or reached through a PEP 735include-group, rolls back byte for byte (theextrasandinclude-grouplanes ofe2e_redirect_uv_build, uv ≥ 0.4.27). An entry whose marker matches no declaration is still refused. Declaration-owned simple equality markers (extra == 'name') are matched explicitly. More complexextrapredicates with differing version clauses remain refused, as do declarations whose lowered markers are indistinguishable: hosted URLs erase the specifiers needed to recover their provenance. Refusals leave the lock and paired metadata unchanged.[tool.uv] constraint-dependencies/build-constraint-dependenciesnaming the package are repointed in the lock's[manifest]constraints/build-constraintsentries, which uv ≥ 0.5.6 serializes with the package's source. uv 0.2.37–0.5.3 serialize constraints as{ name, specifier }regardless of sources, so on those releases the repointed entry makesuv sync --lockedfail (--frozenand a plainuv syncstill install the patch; the plain sync rewrites the entry back). The CLI cannot tell those binaries apart from the lock, so the repoint emits the advisorypypi_uv_constraints_require_uv_0_5_6. The project-variant lane below exercises both shapes. Measured with the real 0.5.4 and 0.5.5 binaries (scripts/uv-vex-matrix.sh): 0.5.4 still rejects the repointed entry under--locked, 0.5.5 accepts it — the effective boundary is 0.5.5; the advisory keeps its0_5_6name.- Transitive targets are wired through
[tool.uv] override-dependenciesplus a[tool.uv.sources]entry. Hosted mode puts a# socket-patch hosted: …comment line above the override entry it adds; rollback / remove remove only a marked entry, so a user's own<name>==<version>override survives the round trip (#411). uv applies sources to overrides only from 0.5.6: on 0.2.35–0.5.3--frozeninstalls the patched wheel from the lock, but a plainuv syncre-resolves the override against the registry and reinstalls the pristine wheel (and rewrites the lock). The CLI cannot tell those binaries apart from the lock, so the override branch emits the advisory warningpypi_uv_override_requires_uv_0_5_6instead of refusing. Measured with the real binaries: 0.5.4 still re-resolves the override, 0.5.5 already keeps it (effective boundary 0.5.5). When the environment already holds the patched install (a--frozensync ran first), 0.2.37 – 0.5.4 leave that same-version install in place while rewriting the lock to the registry, so only the NEXT install from the lock is pristine; manifest-less VEX follows the lock and stops attesting either way. - Symlinked
uv.lock,pyproject.toml,pylock*.toml,*.py.lockand script files are discovered for inventory andrepair, but every writer refuses before touching anything withredirect_symlinked_file_unsupported(the one symlink code, hosted and vendored alike) — because uv writes through the link while socket-patch's atomic stage-and-rename would replace the link with a regular file, leaving the target unpatched and the checkout with a type change. A symlink that is not one of the files to be written does not block vendoring its regular siblings. - Vendored requirements install on uv ≥ 0.1.24 (the bare
./wheelpath grammar). The--hashon that line is enforced only byuv pip sync --require-hashes, which exists from 0.1.32, and by default from 0.5.x; through 0.1.29 uv silently ignores hashes. uv ≤ 0.1.23 has no local path grammar at all — hosted requirements work there. - Both uv backends preserve CRLF line endings: the hosted rewriter and the
vendored
uv.lock/pyproject.tomlwriter (including the appended[manifest]and[package.metadata]fragments and their revert) keep the file's convention. - uv workspaces are refused from the root (
pypi_uv_workspace_unsupported/redirect_uv_project_unsupported) and from a member directory: a member listed by the nearest ancestor[tool.uv.workspace] members(no standalone[project]in between) installs from the root'suv.lock, whatever locks sit in the member, so both modes refuse it before writing (redirect_workspace_lockfile_elsewherehosted,pypi_uv_workspace_unsupportedvendored) instead of rewriting a Hatch-configured member as a lockless Hatch project (#1138). - A script lock requires its paired script and a valid PEP 723 metadata block. Missing metadata or an incompatible existing source is reported before either file is rewritten.
- A PEP 508 direct reference to the patched package (
six @ https://…whl,six @ git+…) in[project]dependencies, an extra, a[dependency-groups]group, the legacy[tool.uv] dev-dependenciesor a PEP 723 script, or a lock entry resolved from a direct URL (uv.locksource = { url }, pylockarchive = { url }), is the user's own source. Both modes refuse it before any file is written: vendored withpypi_uv_source_already_exists, hosted withredirect_uv_project_unsupported/redirect_uv_script_unsupported/redirect_uv_lock_unsupported(#767). Before, vendored left a lock thatuv sync --lockedrejects, and hosted replaced the user's URL with one its own rollback then refused to restore. - Native projects and scripts resolving multiple versions of the same package are refused when a global uv source would replace another version. Supporting those cases requires marker-specific source mappings. Standalone PEP 751 rewriting selects the exact package version; duplicate entries for the same name and version are refused when source selection is ambiguous.
- Lock-only discovery (a fresh checkout, no venv) queries the patch API with
every PEP 440 spelling of an exact pure-release pin as well as the one
written (
six==1.16asks for@1.16and@1.16.0;==1.16.0also asks for@1.16), so it finds the patch the registry keys under its own spelling, as a venv-backed run does, and reports the package as not installed (#604). - Hosted requirements select exact
==/===pins or socket-patch's own hosted archive URLs. A user-authored direct reference to the patched release (name @ <url>on any other origin,files.pythonhosted.organdfile://included) is refused withredirect_requirements_direct_referenceand left unchanged. Other versions remain unchanged. A bare requirement is rewritten only when one row and one override version identify the selection. Ranges, wildcard pins, opaque URLs, and ambiguous unpinned rows are reported asredirect_requirements_version_ambiguousand preserved. - A script lock does not replace the main project's lockfile selection merely
by sharing its directory: its packages supplement the
poetry.lockorrequirements.txtinventory rather than hiding it, and an unrelated script lock does not block vendoring a package from the project's requirements or Poetry lock.uv.lockkeeps its exclusive precedence. When multiple applicable package-manager locks coexist, the CLI reports its precedence choice and the locks it leaves unchanged. - Vendored installation needs the committed artifact tree. uv 0.2.35–0.3.5
cannot build the ROOT fixture from an empty cache under
--offline(setuptools>=40.8.0is a build dependency of the fixture, not of the patched wheel).uv sync --no-install-projectexists from 0.3.3 and the harness passes it wheneversync --helplists it, which is why the 0.3.5 row passes cold; from 0.4.0 uv no longer builds a root without a[build-system], so ≥ 0.4.0 passes regardless; ≤ 0.2.34 vendoring is refused, so the case is never exercised there. On 0.2.35–0.3.0 the harness retries the install with network access and records it asproject-vendored-frozen-sync-root-build-networked, distinct from any failure to install the patched wheel. vendor --revertrefuses to delete a vendored Python wheel whileuv.lock, a PEP 751 lock, a script, orrequirements.txtstill references it and the ledger entry has no wiring to replay (the shapesocket-patch repairrebuilds whenstate.jsonis lost);vendor_wiring_unknown_revert_blockednames the file. Restore the pre-vendor files (or re-lock) first.
Revert state retains the original wiring. Script and lock edits are treated as a
pair: conflicting changes preserve both files and their recovery state rather
than restoring only one side. When any uv.lock or pyproject.toml record has
drifted, neither file is written. A relock that only re-serializes an array
around socket-patch's unchanged element is not drift: uv add --dev x rewrites
the dev group's requires-dev line, and uv add y sorts [manifest] overrides
into its multi-line form. Revert restores or removes just that element.
A path source records no specifier, so changing the vendored package's own
declaration (uv add "six>=1.16") leaves uv.lock unchanged. Revert then
restores the requires-dist, requires-dev and [manifest] constraints
entries with the specifier pyproject.toml declares at revert time, not the one
recorded when vendoring. When uv's spelling can't be derived, as with a
multi-clause range (uv orders clauses differently across releases), the edit is
treated as drift and both files are kept. Tests also cover restoring one package while
preserving another package's vendored entries.
The matrix pins 42 binaries: the first and the latest release of every uv 0.x
family (0.0 through 0.12), plus the releases on either side of every behaviour
boundary the probes found — 0.1.23/0.1.24 (local wheel paths in
requirements), 0.1.44/0.1.45 (uv lock writes a lock; the subcommand
exists from 0.1.42 and panics not yet implemented through 0.1.44),
0.2.5/0.2.6 (sub-table → inline lock artifacts),
0.2.17/0.2.18 (string → inline-table lock sources),
0.2.34/0.2.35 ([[distribution]] → [[package]]), 0.2.36/0.2.37
(root [package.metadata] appears),
0.4.0/0.4.1 (uv export), 0.5.16/0.5.17 (uv lock --script),
0.6.14/0.6.15 (PEP 751 export; lock revision 2) and 0.8.3/0.8.4 (lock
revision 3). The full list is VERSIONS in scripts/backtest-uv.py; the
boundaries were bisected with scripts/probe-uv-boundaries.py, which records
the lock grammar, lock revision, command availability (including whether the
uv lock subcommand exists and actually writes a lock, and whether uv export
accepts --output-file), and local-wheel requirement support of any set of uv
releases. The one probe-pinned boundary the matrix does not bracket is the
uv export --output-file flag (0.4.6/0.4.7): the harness reads the export
from stdout, so it is a harness detail rather than a compatibility boundary.
This is release-family plus boundary coverage, not a claim that every patch
release was tested.
From the repository root on macOS or Linux:
cargo build -p socket-patch-cli
cp target/debug/socket-patch /tmp/socket-patch-backtest-bin
python3 scripts/backtest-uv.py \
--socket-patch /tmp/socket-patch-backtest-bin \
--socket-patch-revision "$(git rev-parse HEAD)" \
--python /path/to/python3 \
--output /tmp/socket-patch-uv-backtestUse Python 3.9 to match the recorded probes; the fixtures declare it as their
minimum. Copy the CLI out of target/ first so a rebuild cannot swap the binary
under a running matrix. The default list takes roughly half an hour with the
harness's four workers; --versions selects a smaller diagnostic run. The
script downloads pinned uv binaries and the pristine urllib3 wheel from PyPI and
verifies their registry hashes. It runs against the public patch proxy without
an API token.
For each binary, the run records command lines, exit codes, output, artifact
hashes, and installed urllib3/response.py hashes. It exercises native locks,
plain and hashed requirements, requirements/PEP 751 exports, standalone PEP 751
compilation, and script locks where the uv binary provides those commands.
Unsupported commands remain visible in the results; they are not counted as
successful installation tests. A successful CLI exit with a refusal warning is
also not counted as a successful rewrite.
On every [[package]]-grammar binary (uv ≥ 0.2.35) the run adds a
project-variant lane (variant_matrix in the script, variant-<name>-<mode>-*
cases in results.json). Five pyproject shapes are locked fresh, scanned in
hosted and vendored mode, and installed from the patched lock into a fresh
environment with uv sync --frozen, uv sync --locked (where the binary has
it) and a plain uv sync, recording the exit code, whether the lock survived
untouched, and the installed bytes:
tool-uv-dev—dependencies = []plus[tool.uv] dev-dependencies;dependency-groups—dependencies = []plus PEP 735[dependency-groups]dev(honoured from uv 0.4.27; older binaries lock an empty project and the row recordsformatSupported: false);extras-duplicate— the package both independenciesand in[project.optional-dependencies], so the lock carries tworequires-distentries for it;constraints— the package independenciesplus[tool.uv] constraint-dependencies, giving the lock a[manifest]constraints entry (skipped as unsupported when the binary records none);transitive—requests==2.28.2with[tool.uv] exclude-newer = "2024-01-01T00:00:00Z"so urllib3 resolves to 1.26.18 as a transitive dependency; the CLI takes the override-dependencies branch, and on uv < 0.5.6 the plainuv syncrow is expected to reinstall the pristine wheel (recorded asinstalledPatch: false, see Limits).
A plain uv sync that rewrites the lock is recorded (lockUnchanged: false),
not raised: it is a real observation, not a harness error. No export or PEP 751
lanes run for the variants. Each version row in results.json carries a
variants summary. Render a summary alongside the run's output with
--render-doc-table:
python3 scripts/backtest-uv.py --render-doc-table /tmp/socket-patch-uv-backtest/results.jsonKeep live download grants out of committed evidence. Published patch UUIDs, archive filenames, hashes, uv versions, and redacted command results are enough to identify a run. Compare the fresh installed bytes with the patched artifact, not just with a URL or a success message.
Hosted and vendored checkouts carry no .socket/manifest.json; socket-patch vex discovers the patch from the wiring files (uv.lock + pyproject.toml,
<script>.py.lock + the script's PEP 723 block, pylock*.toml), takes the
record from the ledgers or the patch API, and verifies the installed tree
(hosted) or the committed wheel (vendored). Three layers cover it:
crates/socket-patch-cli/tests/e2e_vex_lockfile/uv.rs— hermetic, every OS: every lock shape × hosted / vendored, online / offline / 404, ledger without manifest, reverted and half-reverted pairs (including script pairs), tampered installed trees and wheel members, spoofed hosts and vendor paths, record mismatches, and the embeddedapply --vex/vendor --vex/scan --mode hosted|vendored --vexpaths.e2e_redirect_uv_build(hosted, wiremock patch API serving the patched wheel) ande2e_vendor_pypi_build(vendored) — the REAL uv under test (SOCKET_PATCH_UV_E2E_BIN/_VERSION/_PYTHON/_REQUIRED) builds each lane (project, constraints, transitive override, script lock,uv export/uv pip compile/pip lockpylock), our CLI wires it, a fresh checkout installs from an empty cache and imports the patched bytes, then VEX runs with the manifest deleted, with the ledgers deleted, offline (record_unavailable, zero requests), embedded, and after the wiring is reverted with the ledgers left behind (not attested,--no-verifytoo).scripts/uv-vex-matrix.shruns every uv 0.N line (0.1.45, 0.2.37, 0.3.5, 0.4.30, 0.5.3–0.5.6, 0.6.17, 0.7.22, 0.8.24, 0.9.30, 0.10.12, 0.11.33, 0.12.17) through both suites, plus the live-production uv legs withUV_VEX_MATRIX_PRODUCTION=1.- This backtest's
vex_matrixphase (vex-backtest.json): the same steps over the installed production cases (project,export-pylock,pylock-direct), recordingvexAttested,vexMarkersandvexSkipper row.
Lanes a release lacks are reported n/a: uv 0.1.45 writes the
[[distribution]] grammar (hosted only; vendored is refused as above), script
locks need uv lock --script (0.5.17), pylock lanes need uv pip sync pylock.toml (0.7).
Full run results belong with the source revision and toolchain versions in CI artifacts or a local output directory. See the testing guide.