Repository navigation
fix(core): warn when the watcher cannot read a new directory - #1686
Closed
sammywachtel wants to merge 1 commit into
Closed
sammywachtel wants to merge 1 commit into
sammywachtel wants to merge 1 commit into
Conversation
On Linux the watcher adds a watch on a new directory when it appears. If the directory cannot be read at that moment, adding the watch fails and notify discards the error. Files written into the directory then produce no events and are never indexed, and nothing is logged. The directory's own creation event still arrives, because the parent's watch is fine. When a batch reports a new directory that cannot be listed, log a warning that names it and says to run `bm project index <name>` (or `bm reindex`) once its permissions are fixed. Signed-off-by: sammywachtel <subp@wachtel.us>
Member
|
Thank you, @sammywachtel. This is the right-sized fix. Your commit is cherry-picked unchanged into #1692, with your authorship kept, so the full CI suite runs; fork PRs skip it. #1692 also adds a changelog entry. It lands in v0.24.0. |
phernandez
added a commit
that referenced
this pull request
Oct 9, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
phernandez
added a commit
that referenced
this pull request
Oct 9, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
This is the smaller change suggested in the review of #1664.
On Linux the watcher adds a watch on each new directory when it appears. If the directory cannot be read at that moment, adding the watch fails and notify discards the error. Files written into that directory then produce no events and are never indexed, and nothing is logged. The fix is to correct the permissions and run
bm project index <name>, but nothing told anyone that was needed.The directory's own creation event still arrives, because its parent's watch is fine. That is the one place the watch service can see the problem.
What changed
src/basic_memory/index/watch_service.py: newwarn_unreadable_new_directories(), called at the start ofhandle_changes. For eachaddedpath in the batch that is a directory (not a symlink), it tries to list it. If that fails, it logs one warning:Nothing else changes. The check is one
scandirfor each new directory in a batch, and none for files.What it detects, and what it does not
index_delay, 1 s by default) after it appeared. For example, a directory created by another user, or by root, with permissions the server's user lacks.install -d -o <user>run as root creates the directory owned by root and closed, then hands it over a moment later. The watch can fail inside that moment, but by the time the batch is handled the directory is readable, so there is nothing left to see. The watch service has no other signal for it, because notify reports no error.Testing
tests/index/test_watch_service.py:test_handle_changes_warns_when_a_new_directory_cannot_be_read(skipped as root): a mode-000 directory passed throughhandle_changesproduces exactly one warning, naming the directory andbm project index <project>. Fails without the call inhandle_changes.test_warn_unreadable_new_directories_ignores_readable_directories_and_files: a readable directory, a file, a path that no longer exists, and amodifiedevent produce no warning.Runs:
tests/indexon macOS, SQLite, Python 3.13: 188 passed, 1 failed (test_local_watcher_embeds_indexed_file, which fails the same way onmainhere).tests/index/test_watch_service.pyon Linux (Debian, Python 3.12, non-root user): passed.WatchService.run) on macOS and on Linux, with a mode-000 directory created in a watched project: the warning was logged within the debounce interval on both. This was a throwaway test, not committed.ruff check,ruff format --check: clean.ty check src: clean apart from thepymilvusimports, which are not installed here.