Skip to content

feat(watcher): add the Subversion working-copy observation core - #2715

Open
DeusData wants to merge 2 commits into
mainfrom
feat/svn-state-reader
Open

DeusData wants to merge 2 commits into
mainfrom
feat/svn-state-reader

Conversation

@DeusData

Copy link
Copy Markdown
Owner

What this adds

The first, inert part of the Subversion support agreed in #1127 ("PR-A" in the plan posted there on 21 September): src/watcher/svn_state.c / .h, the observation core, plus its tests. Nothing calls it yet, so there is no behaviour change for any user.

Two pieces, and neither runs a process:

  • cbm_svn_client_init resolves and pins a system svn client from the absolute PATH entries. A candidate that resolves under the watched root or under the current directory is refused, relative PATH entries are ignored, and on POSIX the file must be executable. It only inspects files; it never executes the client.
  • cbm_svn_parse_status_stream reads one svn status --xml --verbose document from a FILE * (the capture file a bounded subprocess will write in the follow-up) and folds it into a semantic signature over every reported entry, plus a content signature over the indexable files the status reports as modified, added, replaced, conflicted, obstructed, unversioned or ignored.

What changed relative to the code in #1127, and why

The parser, the PATH pinning, the path-safety rules and the signature design are carried over from #1127 largely unchanged. The changes are the review items from 21 and 27 September:

  1. No recursion. Unversioned trees are walked from an explicit heap work stack, so tree depth never reaches the native stack.
  2. Open first, then decide. Every file and directory is opened one path component at a time without following it, and its type and size come from the open handle: openat(O_NOFOLLOW | O_NONBLOCK) + fstat on POSIX. On Windows the full path is opened with FILE_FLAG_OPEN_REPARSE_POINT, and the handle's own final path must equal root/relative, so a junction on any ancestor is caught on the handle itself. There is no stat-then-open window, and a FIFO never blocks the walk.
  3. Working-copy content is never fatal. Symlinks and reparse points, FIFOs, sockets, devices, unreadable files and oversized files are skipped and counted (new skipped_* fields) instead of turning the whole observation UNCERTAIN. UNCERTAIN is kept for output that cannot be trusted (malformed or truncated XML, a path escaping the root) and for I/O errors.
  4. The size cap applies to the bytes actually read, not to a size measured before the read: at most cbm_max_file_bytes() + 1 bytes are read, and a file that grew past the cap is counted as oversized.
  5. Memory core. Every heap allocation goes through cbm_alloc / cbm_realloc / cbm_mem_strdup / cbm_free. The parser state and the 16 KB tag buffer are on the heap, not on the caller's stack. scripts/lint-memory-core.py reports no new raw site.
  6. POSIX names. : and \ are ordinary filename bytes on POSIX. a:b.py and back\slash.c are read like any other file (\ is still a separator on Windows, and : is still refused there).
  7. Windows client detection. MSYS2 and Cygwin builds of svn are recognised by the runtime DLL beside the executable (msys-2.0.dll, cygwin1.dll), not by the install prefix. Cygwin paths get the /cygdrive/<drive>/... spelling (Cygwin's default prefix).
  8. No process spawning in this file at all, as the plan asked. cbm_svn_probe and its popen call are not part of this change.

Deliberately left for later

  • PR-B (watcher integration): the opt-in config key (default off, composed with watcher_enabled), running svn status through the bounded watcher_git_run route (deadline, output cap, cancellation, expiry maps to UNCERTAIN), the strategy dispatch in the watcher, and the per-poll cost and --ignore-externals questions from 21 September. This change does not touch src/watcher/watcher.c, the watch_non_git tree poll from feat(watcher): opt-in polling of non-git project roots (#1948) #2333, or any config key.
  • PR-C (maintainer side): Subversion in the test images, the Windows VM and CI, and the real-svn fixture tests. This change edits no workflow, no scripts/security-allowlist.txt and no test image.
  • Not carried over: the MCP store-update gate, the Windows cbm_popen_argv API, the dump_and_persist hunk (already on main via 0e00ef5), and the docs/plans/ document.

Proof

Run before any suite, with the production compile flags, through a small driver that hands the parser a status document in the svn 1.14 --xml --verbose layout. The trees are real source (this repository's src/ and tests/, 841 files) laid out as a working copy: 25 files with a real content edit reported as modified, an unversioned new_module/ with a 100-level directory chain, and an ignored node_modules/. Subversion is not installed on the proof machine, so the status documents are generated in that layout rather than captured from a real working copy.

input result entries files hashed bytes hashed skipped time
working copy, no quirks OK 890 66 2,048,015 0 0.06-0.09 s
same + the 27 September quirks (a bazel-out-style symlink to outside the tree, a symlink inside the unversioned tree, a FIFO, an unreadable file, a:b.py) OK 894 67 2,048,021 (+6 = a:b.py) 2 symlinks, 1 special, 1 unreadable 0.07-0.08 s
same input, built with those four quirks made fatal again (the behaviour described on 27 September) UNCERTAIN - - - - 0.004 s
src/ tests/ scripts/ docs/ as unversioned trees OK 7 972 32,111,399 0 0.04-0.18 s (4 runs; warm 0.043 s)
internal/ src/ tests/ as unversioned trees (1.2 GB; vendored/ excluded by discovery's rules) OK 6 1,140 38,129,506 0 0.11-0.25 s
  • An edit to an unversioned file with no status change moves only the content signature (3d481fb205f610ef -> 34c8858dae4de356). Restoring the file restores the exact signature, and every input above gives identical signatures run after run.
  • The quirk-free control gives bit-identical signatures with and without the quirk handling, so the skip rules change nothing else.
  • Known cost: each directory taken off the work stack is re-opened from the root one component at a time, so a pathological 400-level chain costs about 1.9 s. Caching ancestor descriptors would avoid that, but a cached directory renamed out of the root stays readable, which is exactly the window item 2 closes. It is an input to the PR-B per-poll cost question.

Tests

Suite svn_state (tests/test_svn_state.c, wired in Makefile.cbm and tests/test_main.c): 22 cases on POSIX, 16 on Windows, where the six symlink/FIFO/permission/POSIX-name cases are compiled out. None needs Subversion: status documents are written in the svn layout, and the client tests resolve fake executables that are never run.

  • parser: content edit vs semantic signature, declarations and truncated documents refused, clean properties and externals, the svn 1.14 layout with nested commit and lock, an empty target, entries escaping the root;
  • tree walk: discovery-excluded directories, symlinks never followed and counted, FIFOs never blocking, unreadable files, : and \ in POSIX names, the measured size cap, the read cap on a file that grows after it was measured (through a test-only seam, no timing), and a 64-level unversioned tree walked on a 128 KB thread;
  • client: refusal of an svn in the current directory, under the watched root, from a relative PATH entry, and without the execute bit, the all-refused case, runtime detection by DLL, and path spelling per runtime.

Each of the 22 cases was shown to fail against a deliberately broken implementation, rebuilt in the same runner, and to pass on this tree: 21 by assertion (for example, symlinks followed reads 6 files instead of 1, and dropping the read cap hashes a file that grew past it) and the deep-tree case by a stack overflow on its 128 KB thread when the walk recurses. Also run: the watcher suite (90 passed), scripts/lint-memory-core.py --strict (no new raw site), cppcheck 2.20.0, clang-format, scripts/security-audit.sh (no allowlist change needed), and a Windows compile of both files for x86_64 and aarch64 (llvm-mingw, -Werror, with and without the test API).

On Linux (arm64 container, GCC 13.3, ASan + UBSan) svn_state and watcher pass, 112 cases. On the trapping UBSan build (-fno-sanitize-recover=undefined) svn_state passes too, after a fix that build found: a status with no content candidates handed qsort a NULL array, which glibc declares nonnull even for a zero count. macOS and recovering UBSan do not catch that, so it is its own commit, RED on the trap leg before the guard and GREEN after.

Credit

This is @lzl51230's design and much of their code: the XML parser (bounds-checked throughout, <! refused outright, entities and code points validated), the executable pinning that is stricter than our own git lookup, the path containment, the semantic/content signature split, and the test shapes. The first commit carries a Co-authored-by trailer for them. Thank you, @lzl51230, for the care that went into it.

Refs #1127

DeusData and others added 2 commits October 20, 2026 00:39
Add src/watcher/svn_state.c, the inert first part of the SVN support
agreed in #1127: a parser for `svn status --xml --verbose` output read
from a FILE *, and resolution of a system `svn` client from PATH. It runs
no process and nothing calls it yet; running the client through the
bounded subprocess route and the opt-in watcher strategy are the
follow-up.

The parser, the PATH pinning, the path-safety rules and the signature
design are carried over from the #1127 contribution. Changed for this
landing, per the review there:

- unversioned trees are walked from an explicit heap work stack, so tree
  depth never reaches the native stack;
- every file and directory is opened one component at a time without
  following it and measured on the open handle (openat + fstat on POSIX;
  on Windows the handle's final path must equal root/relative);
- symlinks, FIFOs, sockets, devices, unreadable and oversized files are
  skipped and counted instead of turning the observation UNCERTAIN;
- the size cap applies to the bytes actually read;
- every heap allocation goes through the memory core;
- ':' and '\' are ordinary bytes in POSIX names;
- MSYS2 and Cygwin clients are recognised by the runtime DLL beside the
  executable instead of the install prefix, and Cygwin paths get the
  /cygdrive spelling.

The svn_state suite needs no Subversion install: status documents in
the svn 1.14 layout and fake client executables that are never run.

Refs #1127

Co-authored-by: lzl51230 <14934913+lzl51230@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A status document with no content candidates (only clean entries,
externals or property changes) left the candidate array NULL, and
fingerprint_candidates sorted it anyway. glibc declares qsort's base
nonnull even for a zero count, so the trapping UBSan leg aborts there;
macOS and recovering UBSan only print or ignore it. Sort only when there
are at least two candidates.

parse_clean_props_and_externals_are_not_local_changes reaches this path:
RED on the Linux arm64 GCC leg built with -fno-sanitize-recover=undefined
before the guard, GREEN after.

Refs #1127

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant