Skip to content

Free-threaded CPython with PYTHON_GIL=0: first calls from several threads at once crash the process #68

Closed

Description

@zahariash

This work was produced with the help of language models.

On free-threaded CPython with the GIL disabled (PYTHON_GIL=0 or -Xgil=0), several threads making their first driver calls at the same time crash the process intermittently: SIGSEGV, free(): invalid pointer or malloc(): corrupted top size. The driver fills its import cache (the Python objects it loads on first use: datetime, uuid, pandas, ...) lazily with no synchronization and relies on the GIL, so concurrent first loads race on a shared vector. Making one driver call from a single thread before starting the others avoids it.

The module does not declare free-threading support, so by default importing it re-enables the GIL (with a RuntimeWarning) and nothing crashes; the crash needs the GIL disabled explicitly.

import threading, ladybug as lb

db = lb.Database(":memory:")
conns = [lb.Connection(db) for _ in range(8)]
start = threading.Barrier(len(conns))

def run(conn, t):
    start.wait()
    for i in range(300):
        conn.execute(f"RETURN $x + {t * 100000 + i}", {"x": 1}).get_next()

threads = [threading.Thread(target=run, args=(c, t)) for t, c in enumerate(conns)]
for th in threads: th.start()
for th in threads: th.join()

Run it as PYTHON_GIL=0 python repro.py, in a fresh interpreter each time, a few times over.

Expected: the script finishes.
Actual: it crashes in about 1 run of 7 (2 of 15).

Reproduced on main (548c183), built from source, with free-threaded CPython 3.15.0rc3.

Fix: #69

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions