Skip to content

[python] Support getting a table by id in the REST catalog - #10697

Open
plusplusjiajia wants to merge 1 commit into
apache:masterfrom
plusplusjiajia:feat/python-get-table-by-id
Open

plusplusjiajia wants to merge 1 commit into
apache:masterfrom
plusplusjiajia:feat/python-get-table-by-id

Conversation

@plusplusjiajia

@plusplusjiajia plusplusjiajia commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Purpose

Java has had Catalog#getTableById since #7085, but pypaimon can only load a table by name. This adds Catalog.get_table_by_id(table_id).

  • RESTCatalog implements it on GET /v1/{prefix}/tables/id/{tableId}. As in Java, the table is built from the database and name in the response, a 404 raises TableIdNotExistException, and a 403 raises TableIdNoPermissionException. Other catalogs raise NotImplementedError.
  • GetTableResponse now keeps the database field that the server sends, which Java already reads. It is left out of the JSON when it is None, because native readers reuse a REST response only when its database matches the table's.
  • As in table_details, ~ stays raw in the path: requests sends %7E as ~ after the DLF signer has signed the path.

Tests

  • RESTCatalogTest#test_get_table_by_id: loads a table by id from a database with a dot in its name, checks its identifier and path, checks that native readers can reuse the response, and reads back the written row. It also checks the exceptions for an unknown id and for a table without permission.
  • ApiTest: RESTApi.get_table_by_id against the mock server and the empty-id check. A fixed JSON fixture, as the Java server sends it, checks that database is parsed and written under that key, and that it is left out when None.
  • native_plan_rest_test.py#test_rest_dotted_database_and_table_keep_identity gains a case that loads the table by id and reads it through the native planner and reader, with no fallback to Python and no catalog reload.
  • ResourcePathsTest#test_signature_matches_the_path_sent now also covers the table-id path, for both DLF signing algorithms.
  • RESTApiCoverageTest checks the new request against the OpenAPI spec and the openapi-v4 actions.
  • The mock REST server serves the new endpoint and returns database, as the Java server does.
  • pypaimon/tests/rest and the native-plan tests: 631 pass. The one failure, the Lance-format commit test, needs lance, which is not installed locally. The new tests also pass on Python 3.6, and flake8 is clean.
  • With pypaimon-rust built from paimon-rust main and the native flags on, the REST and native tests fail the same way as on master: tests that need ray or lance, Avro reads of long-dt, and the flaky shard tests in py36/ao_simple_test.py.

@plusplusjiajia
plusplusjiajia force-pushed the feat/python-get-table-by-id branch 2 times, most recently from 2b177b3 to d14e4c2 Compare October 20, 2026 11:14
@plusplusjiajia
plusplusjiajia marked this pull request as ready for review October 20, 2026 11:23
Java has Catalog#getTableById since apache#7085, but pypaimon could only load a
table by name. Add Catalog.get_table_by_id, implemented by RESTCatalog on
GET /v1/{prefix}/tables/id/{tableId}. As in Java, the table is built from
the database and name in the response, a 404 raises
TableIdNotExistException and a 403 raises TableIdNoPermissionException.
Other catalogs raise NotImplementedError. As in table_details, '~' stays
raw in the path, so the DLF signature covers the path requests sends.

GetTableResponse now keeps the database field the server sends. It is left
out of the JSON when unknown, since native readers only reuse a REST
response whose database matches the table.
@plusplusjiajia
plusplusjiajia force-pushed the feat/python-get-table-by-id branch from d14e4c2 to 54f0b3d Compare October 21, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant