Repository navigation
Conversation
Add src/watcher/svn_state.c, the inert first part of the SVN support agreed in #1127: a parser for `svn status --xml --verbose` output read from a FILE *, and resolution of a system `svn` client from PATH. It runs no process and nothing calls it yet; running the client through the bounded subprocess route and the opt-in watcher strategy are the follow-up. The parser, the PATH pinning, the path-safety rules and the signature design are carried over from the #1127 contribution. Changed for this landing, per the review there: - unversioned trees are walked from an explicit heap work stack, so tree depth never reaches the native stack; - every file and directory is opened one component at a time without following it and measured on the open handle (openat + fstat on POSIX; on Windows the handle's final path must equal root/relative); - symlinks, FIFOs, sockets, devices, unreadable and oversized files are skipped and counted instead of turning the observation UNCERTAIN; - the size cap applies to the bytes actually read; - every heap allocation goes through the memory core; - ':' and '\' are ordinary bytes in POSIX names; - MSYS2 and Cygwin clients are recognised by the runtime DLL beside the executable instead of the install prefix, and Cygwin paths get the /cygdrive spelling. The svn_state suite needs no Subversion install: status documents in the svn 1.14 layout and fake client executables that are never run. Refs #1127 Co-authored-by: lzl51230 <14934913+lzl51230@users.noreply.github.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A status document with no content candidates (only clean entries, externals or property changes) left the candidate array NULL, and fingerprint_candidates sorted it anyway. glibc declares qsort's base nonnull even for a zero count, so the trapping UBSan leg aborts there; macOS and recovering UBSan only print or ignore it. Sort only when there are at least two candidates. parse_clean_props_and_externals_are_not_local_changes reaches this path: RED on the Linux arm64 GCC leg built with -fno-sanitize-recover=undefined before the guard, GREEN after. Refs #1127 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
The first, inert part of the Subversion support agreed in #1127 ("PR-A" in the plan posted there on 21 September):
src/watcher/svn_state.c/.h, the observation core, plus its tests. Nothing calls it yet, so there is no behaviour change for any user.Two pieces, and neither runs a process:
cbm_svn_client_initresolves and pins a systemsvnclient from the absolutePATHentries. A candidate that resolves under the watched root or under the current directory is refused, relativePATHentries are ignored, and on POSIX the file must be executable. It only inspects files; it never executes the client.cbm_svn_parse_status_streamreads onesvn status --xml --verbosedocument from aFILE *(the capture file a bounded subprocess will write in the follow-up) and folds it into a semantic signature over every reported entry, plus a content signature over the indexable files the status reports as modified, added, replaced, conflicted, obstructed, unversioned or ignored.What changed relative to the code in #1127, and why
The parser, the PATH pinning, the path-safety rules and the signature design are carried over from #1127 largely unchanged. The changes are the review items from 21 and 27 September:
openat(O_NOFOLLOW | O_NONBLOCK)+fstaton POSIX. On Windows the full path is opened withFILE_FLAG_OPEN_REPARSE_POINT, and the handle's own final path must equal root/relative, so a junction on any ancestor is caught on the handle itself. There is no stat-then-open window, and a FIFO never blocks the walk.skipped_*fields) instead of turning the whole observationUNCERTAIN.UNCERTAINis kept for output that cannot be trusted (malformed or truncated XML, a path escaping the root) and for I/O errors.cbm_max_file_bytes() + 1bytes are read, and a file that grew past the cap is counted as oversized.cbm_alloc/cbm_realloc/cbm_mem_strdup/cbm_free. The parser state and the 16 KB tag buffer are on the heap, not on the caller's stack.scripts/lint-memory-core.pyreports no new raw site.:and\are ordinary filename bytes on POSIX.a:b.pyandback\slash.care read like any other file (\is still a separator on Windows, and:is still refused there).svnare recognised by the runtime DLL beside the executable (msys-2.0.dll,cygwin1.dll), not by the install prefix. Cygwin paths get the/cygdrive/<drive>/...spelling (Cygwin's default prefix).cbm_svn_probeand itspopencall are not part of this change.Deliberately left for later
watcher_enabled), runningsvn statusthrough the boundedwatcher_git_runroute (deadline, output cap, cancellation, expiry maps toUNCERTAIN), the strategy dispatch in the watcher, and the per-poll cost and--ignore-externalsquestions from 21 September. This change does not touchsrc/watcher/watcher.c, thewatch_non_gittree poll from feat(watcher): opt-in polling of non-git project roots (#1948) #2333, or any config key.svnfixture tests. This change edits no workflow, noscripts/security-allowlist.txtand no test image.cbm_popen_argvAPI, thedump_and_persisthunk (already onmainvia 0e00ef5), and thedocs/plans/document.Proof
Run before any suite, with the production compile flags, through a small driver that hands the parser a status document in the svn 1.14
--xml --verboselayout. The trees are real source (this repository'ssrc/andtests/, 841 files) laid out as a working copy: 25 files with a real content edit reported as modified, an unversionednew_module/with a 100-level directory chain, and an ignorednode_modules/. Subversion is not installed on the proof machine, so the status documents are generated in that layout rather than captured from a real working copy.bazel-out-style symlink to outside the tree, a symlink inside the unversioned tree, a FIFO, an unreadable file,a:b.py)a:b.py)src/ tests/ scripts/ docs/as unversioned treesinternal/ src/ tests/as unversioned trees (1.2 GB;vendored/excluded by discovery's rules)3d481fb205f610ef->34c8858dae4de356). Restoring the file restores the exact signature, and every input above gives identical signatures run after run.Tests
Suite
svn_state(tests/test_svn_state.c, wired inMakefile.cbmandtests/test_main.c): 22 cases on POSIX, 16 on Windows, where the six symlink/FIFO/permission/POSIX-name cases are compiled out. None needs Subversion: status documents are written in the svn layout, and the client tests resolve fake executables that are never run.commitandlock, an empty target, entries escaping the root;:and\in POSIX names, the measured size cap, the read cap on a file that grows after it was measured (through a test-only seam, no timing), and a 64-level unversioned tree walked on a 128 KB thread;svnin the current directory, under the watched root, from a relativePATHentry, and without the execute bit, the all-refused case, runtime detection by DLL, and path spelling per runtime.Each of the 22 cases was shown to fail against a deliberately broken implementation, rebuilt in the same runner, and to pass on this tree: 21 by assertion (for example, symlinks followed reads 6 files instead of 1, and dropping the read cap hashes a file that grew past it) and the deep-tree case by a stack overflow on its 128 KB thread when the walk recurses. Also run: the
watchersuite (90 passed),scripts/lint-memory-core.py --strict(no new raw site), cppcheck 2.20.0, clang-format,scripts/security-audit.sh(no allowlist change needed), and a Windows compile of both files for x86_64 and aarch64 (llvm-mingw,-Werror, with and without the test API).On Linux (arm64 container, GCC 13.3, ASan + UBSan)
svn_stateandwatcherpass, 112 cases. On the trapping UBSan build (-fno-sanitize-recover=undefined)svn_statepasses too, after a fix that build found: a status with no content candidates handedqsorta NULL array, which glibc declares nonnull even for a zero count. macOS and recovering UBSan do not catch that, so it is its own commit, RED on the trap leg before the guard and GREEN after.Credit
This is @lzl51230's design and much of their code: the XML parser (bounds-checked throughout,
<!refused outright, entities and code points validated), the executable pinning that is stricter than our own git lookup, the path containment, the semantic/content signature split, and the test shapes. The first commit carries aCo-authored-bytrailer for them. Thank you, @lzl51230, for the care that went into it.Refs #1127