Skip to content

feat(replication): map HBase table names to Cloud Bigtable table IDs - #4639

Open
vermas2012 wants to merge 1 commit into
googleapis:mainfrom
vermas2012:feat/replication-table-name-mapping
Open

vermas2012 wants to merge 1 commit into
googleapis:mainfrom
vermas2012:feat/replication-table-name-mapping

Conversation

@vermas2012

Copy link
Copy Markdown
Member

Support mapping HBase table names (such as namespaced tables like my_namespace:my_table) to valid Cloud Bigtable table IDs during live WAL replication via three mutually exclusive configuration mechanisms:

  • google.bigtable.replication.table_name_mapping: inline JSON object (identical format to HBaseSchemaTranslator's google.bigtable.schema.mapping.filepath)
  • google.bigtable.replication.table_name_mapping_file: JSON file path resolved via Hadoop FileSystem (with local filesystem fallback when URI scheme is omitted)
  • google.bigtable.replication.table_name_mapping.<hbase_table>: direct per-table properties in hbase-site.xml or peer configuration

Key Details

  • Enforces 1:1 injectivity across canonicalized HBase source tables (e.g. default:t and t) and target Cloud Bigtable table IDs (including unmapped identity fallback collision detection).
  • Coalesces WAL entries by mapped target table prior to row-key grouping to preserve per-row mutation ordering.
  • Hardens JSON token validation, bounded file streaming, bidirectional marker detection, and batch threshold checks.

@vermas2012
vermas2012 requested a review from a team as a code owner October 13, 2026 14:12
@product-auto-label product-auto-label Bot added size: xl Pull request size is extra large. api: bigtable Issues related to the googleapis/java-bigtable-hbase API. labels Oct 13, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to map HBase table names to Cloud Bigtable table IDs, which is particularly useful for handling HBase tables in non-default namespaces. The changes include a new TableNameMapper class to handle these mappings via inline JSON, external files, or prefixed configuration properties, and updates to CloudBigtableReplicator to apply these mappings during replication. Additionally, the PR includes robust error handling for mapping collisions, improved batching logic for replication tasks, and comprehensive unit tests to verify the new functionality.

@vermas2012
vermas2012 force-pushed the feat/replication-table-name-mapping branch 2 times, most recently from 3880a61 to 37af92b Compare October 13, 2026 15:23
Support mapping HBase table names (such as namespaced tables like
'my_namespace:my_table') to valid Cloud Bigtable table IDs during live
WAL replication via three mutually exclusive configuration mechanisms:

- google.bigtable.replication.table_name_mapping: inline JSON object
- google.bigtable.replication.table_name_mapping_file: JSON file path
  resolved via Hadoop FileSystem (with local filesystem fallback when
  URI scheme is omitted)
- google.bigtable.replication.table_name_mapping.<hbase_table>: direct
  per-table properties in hbase-site.xml or peer configuration

Enforce 1:1 injectivity across canonicalized HBase source tables and
target Cloud Bigtable table IDs (including unmapped identity fallback
collision detection), coalesce WAL entries by mapped target table prior
to row-key grouping, harden JSON and file streaming validation, and
strengthen bidirectional marker detection and batch threshold checks.
@vermas2012
vermas2012 force-pushed the feat/replication-table-name-mapping branch from 37af92b to e66517c Compare October 13, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the googleapis/java-bigtable-hbase API. size: xl Pull request size is extra large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant