Skip to content

fix(connection): make close() and UDF callbacks safe while a query runs - #65

Open
zahariash wants to merge 1 commit into
LadybugDB:mainfrom
zahariash:fix/close-with-query-in-flight
Open

zahariash wants to merge 1 commit into
LadybugDB:mainfrom
zahariash:fix/close-with-query-in-flight

Conversation

@zahariash

Copy link
Copy Markdown
Contributor

This work was produced with the help of language models.

Closing a Connection while another thread ran a query on it destroyed the native connection under the query (hang, heap corruption, segfault, or close() waiting for a parameterized query). AsyncConnection.close() hit the same path (ladybug 0.21.2, CPython 3.13/3.14). A Python UDF calling back into the connection running its query waited for itself forever, also when folded while binding.

Fix. Connection counts calls in flight (both backends). close() refuses new calls, interrupts a running query (repeatedly), waits for in-flight calls, then releases the native connection, and raises if called from inside one of its own calls. Any call from a UDF into a connection with a call in flight raises RuntimeError. UDFs are database-wide and the driver cannot tell which query runs one, so a UDF may not call any busy connection; create_function() documents this. A UDF using an idle other connection works.

Tests. Subprocess tests cover unparameterized, parameterized, write and AsyncConnection queries in flight, re-entrant close, close from a UDF, execute after close, and a UDF calling the same connection (engine thread or binding) or an idle other one. The close scenarios except execute-after-close, and both same-connection UDF ones, fail on main; all pass here on pybind and C-API (UDF ones xfailed there). Full suites: no new failures.

Fixes #60, fixes #61

Closing a connection while another thread was executing a query on it
destroyed the native connection under the running query. Depending on the
query this hung the process, aborted it with heap corruption, or segfaulted
(a write). AsyncConnection.close() hit the same path, since it closes its
connections before shutting down its executor.

Connection now counts calls in flight in every method that reaches the
native connection. close() refuses new calls, interrupts a running query
(repeating the interrupt until the call returns, because an interrupt that
arrives while the query is still compiling is cleared when execution
starts), and waits for in-flight calls to leave the engine before releasing
the native connection. Calls made after close() raise "Connection is
closed." The guard lives in the Python layer, so it covers both backends.

close() called from inside one of the connection's own calls on the same
thread raises instead of waiting for itself. A Python UDF that calls into a
connection while it runs a query (close(), execute() or any other call)
raises too: UDFs run inside the query in flight, so such calls used to wait
for themselves forever, on main as well. A per-thread stack of active calls
and UDFs tells these apart from the connection's own nested calls, and
covers UDFs folded on the calling thread while binding. UDFs are shared by
all connections of a database and the driver cannot tell which query runs
one, so a UDF may not call any connection that has a call in flight;
create_function() documents this. interrupt() also reaches the pybind
connection used as a fallback by the C-API backend, and is a no-op on a
closed connection. The scan-object lookup for LOAD FROM no longer assumes a
fixed call depth and walks up to the first frame outside this module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant