Repository navigation
[core] Detect fixed-bucket deletion vector commit conflicts - #10700
Open
discivigour wants to merge 8 commits into
Open
discivigour wants to merge 8 commits into
discivigour wants to merge 8 commits into
Conversation
added 7 commits
October 20, 2026 15:47
Carry the previous bucket DV through deletedIndexFiles for both compaction and direct DV deletes. Validate index replacements against the latest snapshot, reject rewrites that missed an input DV, and reject new DV references to files absent after applying the commit. Keep unchanged bucket indexes and allow concurrent DV changes on unrelated rewrite inputs. Use the existing index increment serialization and the common Optional conflict/rollback path without passing compact messages. Cover stale same-file, cross-file and cross-level PK updates, direct DELETE recomputation, physical rewrites, consecutive replacements, checkpointed writer state, commit retries, and serialization compatibility.
Represent DV updates as unchanged, replacement, or removal, and clear old indexes without writing empty files. Reuse file-level DV dependency checks for fixed buckets and retain bucket checks for initial additions and legacy empty indexes. Cover DV clearing, stale concurrent commits, and ordered prepared commits; remove unsupported cross-prepare aggregation handling and the restored-index count assertion. Validation: 288 focused tests passed with Maven package, including formatting and static checks.
discivigour
marked this pull request as ready for review
October 21, 2026 03:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Concurrent writers or compactors can restore the same fixed-bucket deletion vector (DV) index and prepare separate replacements. Because each replacement covers the entire bucket, a stale commit can discard another writer's deletion marks and expose duplicate primary keys. A concurrent data-file rewrite can also invalidate a DV's references without changing the bucket's DV version.
This change emits explicit DELETE entries for the previous DV index when replacing or clearing it, for both compaction and direct DV DML. Unchanged, replacement, and removal results are distinguished so clearing an index does not write an empty DV file, and intermediate compaction outputs do not become published dependencies.
For
HASH_FIXED,ConflictDetection.checkConflictsvalidates:Tests
Regression coverage includes: