Repository navigation
Update PyO3 to 0.29 and explore abi3t support #1603
Description
I'm working on free-threaded support in polars with @kumaraditya303: pola-rs/polars#27955. Datafusion shows up in the polars dependency tree.
Updating to PyO3 0.29 enables builds with the new Python 3.15 free-threaded stable ABI, which should allow datafusion to enable free-threaded support without an ongoing need to upload version-specific wheels every time Python does a release.
Activity
[-]Update PyO3 to 0.29[/-][+]Update PyO3 to 0.29 and explore abi3t support[/+]timsaucer commentedon Jun 16, 2026
Thanks for the issue! We have to update pyo3 in lock step with arrow-rs. I just merged a PR to updated it to 0.29, so it will be the next major arrow release (60). Then we need datafusion upstream to update, which it generally does very quickly. After that we'll follow suit.
Can you point out where datafusion shows up in the dependency tree for polars? That's new to me.
FWIW our next release of datafusion-python does have free threaded support with 3.13 (experimental) and 3.14 (stable) python.
kumaraditya303 commentedon Jun 16, 2026
polarshas dependency on deltalake which in turn depends ondatafusion. It is heavily used bypolarsfor io.ngoldbaum commentedon Jun 16, 2026
OK great. Just an FYI though that PyO3 0.29 dropped 3.13t support. See the release notes and the free-threading guide for more detail about that. I'd suggest simply never shipping 3.13t wheels. 3.13t never gained support for a few key packages and that makes it difficult to use 3.13t with a nontrivial set of dependencies. 3.14t is supported much more widely so we're trying to get people to start support there.
timsaucer commentedon Jun 16, 2026
Sounds good! I will exclude them when I do the upload! Thanks for the feedback!
timsaucer commentedon Jun 29, 2026
DataFusion-54.0.0 is released with free threaded Python wheels available. Do you want to keep this issue open? We will auto-upgrade pyo3 as our dependencies update.
ngoldbaum commentedon Jun 29, 2026
Let's close it if it's not useful for you.