Support atomic multi-table commits through the REST catalog

[kotwal-itpro]

Closes #4095

Rationale for this change

The REST spec's POST /v1/{prefix}/transactions/commit applies changes to several tables in one atomic operation, and the Java client exposes it as RESTCatalog.commitTransaction(...). PyIceberg had no way to call it, so write-audit-publish across tables or a backfill of related tables couldn't be published atomically from Python.

This adds Catalog.commit_transactions(*transactions):

orders_tx = orders.transaction()
orders_tx.append(new_orders)
items_tx = items.transaction()
items_tx.append(new_items)

catalog.commit_transactions(orders_tx, items_tx)  # both tables, or neither

What it does

• REST catalog: sends each transaction's requirements and updates as one CommitTransactionRequest, adding the same AssertTableUUID that commit_transaction adds. The endpoint returns 204, so each table is refreshed afterwards and the transactions are cleared.
• Other catalogs: the base Catalog raises NotImplementedError, so they can add support later.
• Errors map like commit_table: 409 → CommitFailedException, 404 → NoSuchTableError, 500/502/504 → CommitStateUnknownException.
• Files: on a definite failure (409, 404) the files written for the transactions are deleted. On an unknown outcome they're kept, because the catalog may already reference them. That's the same rule commit_transaction follows. A failed transaction can't be reused.
• Rejected before anything is sent: no transactions, the same transaction twice, two transactions on one table, a transaction from another catalog, and CreateTableTransaction. Transactions without staged changes are skipped, and a call with only empty ones sends nothing.
• Endpoint check: uses the server's advertised endpoints. For servers that don't list endpoints, transactions/commit is assumed supported, as in the Java client's DEFAULT_ENDPOINTS.

Not in this PR (open questions)

• No retry on conflict. A 409 doesn't say which table conflicted, and replaying snapshot producers across tables is a bigger design. The caller re-stages and commits again, as with the Java client. Happy to follow up if you'd like retries.
• Empty transactions with snapshot producers (for example a delete whose filter matched nothing) are skipped, so their concurrency validation doesn't run here. Should they be validated, or rejected?
• Creating tables in a multi-table commit (with assert-create) could come later.
• Entry point: I went with Catalog.commit_transactions(*transactions) as proposed in Support atomic multi-table commits through the REST catalog (transactions/commit) #4095. Glad to change it, for example to a context manager.

Are these changes tested?

• tests/catalog/test_rest.py: 12 tests with a mocked server. They cover the request body, the prefix, each error status, file cleanup only when nothing was applied, skipping empty transactions, the input checks and an unsupported endpoint.
• tests/catalog/test_base.py: the base catalog raises NotImplementedError.
• tests/integration/test_rest_catalog.py, against the apache/iceberg-rest-fixture with S3-compatible storage:
  • two tables each get a real append (and a property change) in one call, and both have the rows afterwards;
  • another writer appends to one table after the transactions were staged: the commit fails with CommitFailedException, and neither table gets the staged rows.

make test: 4273 passed. make lint passes on the changed files.

Are there any user-facing changes?

Yes: the new Catalog.commit_transactions method, and a "Multi-table commits" section in mkdocs/docs/api.md.

I used an AI coding assistant while writing this change; I reviewed and tested all of it.

🤖 Generated with Claude Code

https://claude.ai/code/session_011ZBQVsXRmKT8WXDr2qSfbd