Repository navigation
fix(sync): make the index delete check case-exact - #1699
Merged
Merged
Conversation
The project-index delete verifier confirmed absence with stat(). On a case-insensitive filesystem (APFS, NTFS) stat finds `case/config.md` when only `case/Config.md` exists, so after a case-only rename the old spelling was always "present in storage again" and its ghost entity was never deleted. Check each path component against its parent's directory listing, one listing per directory per batch. The old spelling now counts as absent, and the next index pass deletes the ghost row. Part of #1627. The watcher race that creates the ghost is not fixed here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 72a06a4a89
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The verifier cached each directory listing for the batch. A file created after its directory was listed passed stat() but was missing from the stale listing, so it was confirmed deleted. Deleting its entity and NoteContent loses its external_id, which a later index does not restore. A cached listing now only proves presence; a miss is re-listed before it counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
This was referenced Oct 9, 2026
phernandez
added a commit
that referenced
this pull request
Oct 9, 2026
Entries for #1697, #1698, #1699, #1700, #1701, #1702, #1704 and #1705, which landed without changelog edits so the parallel PRs would not conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
phernandez
added a commit
that referenced
this pull request
Oct 9, 2026
Entries for #1697, #1698, #1699, #1700, #1701, #1702, #1704 and #1705, which landed without changelog edits so the parallel PRs would not conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1627.
What was wrong
After a case-only rename with the watcher on, a ghost entity for the old lowercase path can appear (
case/config.mdnext to the realcase/Config.md). The ghost never went away.bm project indexplanned its delete, thenLocalProjectIndexDeletePathVerifier.confirm_deleted_pathschecked existence with(base_path / path).stat(). On a case-insensitive filesystem (APFS, NTFS) that stat succeeds for any case, so the verifier reported the path as present and the maintenance runner logged "Skipping planned index deletes for paths present in storage again" on every pass.Fix
path_spelled_exactly_on_diskinsrc/basic_memory/index/local_project.pyruns after a successful stat. It checks every path component against its parent directory's listing, which returns the stored spelling. Listings are cached per directory for the batch, so each parent is listed once. Names are compared in NFC because APFS is also normalization-insensitive and only case is in question here. Listing errors go through the verifier's existingOSErrorhandling, so a failed probe still skips the delete instead of confirming it.The old spelling now counts as absent, so the next index pass deletes the ghost. This is derived state converging through the existing index pass. No locks or extra machinery.
Not fixed here (follow-up)
The watcher race that creates the ghost in the first place (the stale lowercase path indexed as a new file after the move, and the brief
-1permalink rewrite of the file bytes) is not addressed. That needs separate work in the watcher's new-file handling. In the index-pass test below, the renamed file is created before the old row is deleted in the same pass, so it can still be assigned a-1permalink; that is also part of the follow-up.Tests
tests/index/test_local_project_index.py:path_spelled_exactly_on_diskwithCase/Config.mdon disk: the exact spelling is present;Case/config.mdandcase/Config.mdare absent. On a case-sensitive filesystem those paths simply do not exist, which also reports absent. One listing per directory.case/config.mdabsent and keepscase/Config.md.case/config.mdwithcase/Config.md(different content, so no move detection) leaves exactly one entity,case/Config.md.On macOS (APFS) the two verifier/index tests fail without the fix and pass with it. On a case-sensitive filesystem they pass either way.
Verification
just fast-check: passes.BASIC_MEMORY_TESTMON_SELECT_FLAGS="--import-mode=importlib" just fast-test tests/index tests/indexing test-int/test_project_index_delete_materialization_race.py: 793 passed, 3 skipped.🤖 Generated with Claude Code
https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea