Skip to content

fix(bigtable): preserve empty row keys in RowRange and fix UTF-8 prefix bounds - #18882

Open
mutianf wants to merge 3 commits into
mainfrom
fix-row-range-and-prefix-bounds
Open

mutianf wants to merge 3 commits into
mainfrom
fix-row-range-and-prefix-bounds

Conversation

@mutianf

@mutianf mutianf commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@mutianf
mutianf requested a review from a team as a code owner October 28, 2026 15:23

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the handling of row ranges and keys in the Bigtable client, specifically improving how empty keys and inclusiveness flags are parsed and represented. It replaces truthiness checks on protobuf fields with explicit membership tests to prevent incorrect evaluation of empty byte keys. Additionally, add_row_range_with_prefix has been updated to support both string and byte prefixes, correctly handling UTF-8 boundaries and empty inputs. The review feedback correctly identifies an issue in read_rows_query.py where comparing end_key to an empty string fails to match empty byte keys, and suggests using 'not end_key' for a more robust and idiomatic check.

Comment thread packages/google-cloud-bigtable/google/cloud/bigtable/data/read_rows_query.py Outdated
# but preserve explicit empty-key bounds when inclusiveness is specified.
if not start_key and start_is_inclusive is not False:
start_key = None
if not end_key and end_is_inclusive is None:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this not end_is_inclusive is not False?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because b"" is the smallest possible key in lexicographical byte order, [b"", ...) (start_is_inclusive=True) is equivalent to an unbounded start (None), whereas (..., b""] (end_is_inclusive=True) matches only the empty key b"" (k <= b"") rather than all rows. Using end_is_inclusive is not False would turn RowRange(end_key=b"", end_is_inclusive=True) into end_key=None (unbounded), matching the entire table instead of just b"". Checking end_is_inclusive is None only normalizes "" / b"" when the caller omitted end_is_inclusive (such as RowRange("", "")), while preserving both explicit end_is_inclusive=True (end_key_closed=b"") and end_is_inclusive=False (end_key_open=b"").

prefix_bytes = _to_bytes(row_key_prefix, encoding="utf-8")
if not prefix_bytes:
raise ValueError("row_key_prefix cannot be empty.")
stripped = prefix_bytes.rstrip(b"\xff")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is \xff a special case?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0xFF (255) is the maximum value of a single byte, so incrementing the last byte of a prefix ending in \xff would overflow range(0, 256) (ValueError: byte must be in range(0, 256)). To find the smallest byte string greater than all keys with that prefix, the increment must carry into the rightmost non-0xFF byte and drop the trailing 0xFF bytes (e.g., prefix b"abc\xff" has exclusive upper bound b"abd"). If every byte in the prefix is 0xFF (e.g., b"\xff\xff"), no larger byte prefix exists so end_key must be None (unbounded).

@mutianf
mutianf force-pushed the fix-row-range-and-prefix-bounds branch from 274d3dc to 8d23a80 Compare October 28, 2026 18:46
@mutianf
mutianf force-pushed the fix-row-range-and-prefix-bounds branch from 8d23a80 to 6735ff7 Compare October 28, 2026 19:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant