Tootfinder

Opt-in global Mastodon full text search. Join the index!

@mgorny@social.treehouse.systems
2025-06-16 10:22:27

Some fun facts about #Python limited API / stable ABI.
1. #CPython supports "limited API". When you use it, you get extensions that are compatible with the specified CPython version and versions newer than that. To indicate this compatibility, such extensions use `.abi3.so` suffix (or equivalent) rather than the usual `.cpython-313-x86_64-linux-gnu.so` or alike.
2. The actual support is split between CPython itself and #PEP517 build systems. For example, if you use #setuptools and specify `py_limited_api=` argument to the extension, setuptools will pass appropriate C compiler flags and swap extension suffix. There's a similar support in #meson, and probably other build systems.
3. Except that CPython freethreading builds don't support stable ABI right now, so building with "limited API" triggers an explicit error from the headers. Setuptools have opted for building explicit about this: it emits an error if you try to use `py_limited_api` on a freethreading interpreter. Meson currently just gives the compile error. This implies that package authors need to actively special-case freethreading builds and enable "limited API" conditionally.
4. A some future versions of CPython will support "limited API" in freethreading builds. I haven't been following the discussions closely, but I suspect that it will only be possible when you target that version or newer. So I guess people will need to be building two stable ABI wheels for a time — one targeting older Python versions, and one targeting newer versions plus freethreading. On top of that, all these projects will need to update their "no 'limited API' on freethreading" conditions.
5. And then there's #PyPy. PyPy does not feature a stable ABI, but it allows you to build extensions using "limited API". So setuptools and meson just detect that there is no `.abi3.so` on PyPy, and use regular suffix for the extensions built with "limited API".

@mgorny@pol.social
2025-07-20 16:01:29

W przypadku #PyPy, przyczyną jest zawsze jakiś wyciek zasobów…
github.com/vutran1710/PyrateLi

@mgorny@social.treehouse.systems
2025-07-05 15:24:22

A while ago, I've followed the example given by #Fedora and unbundled ensurepip wheels from #Python in #Gentoo (just checked — "a while ago" was 3 years ago). This had the important advantage that it enabled us to update these wheels along with the actual pip and setuptools packages, meaning new virtual environments would get fresh versions rather than whatever CPython happened to bundle at the time of release.
I had considered using our system packages to prepare these wheels, but since we were already unbundling dependencies back then, that couldn't work. So I just went with fetching upstream wheels from PyPI. Why not build them from source instead? Well, besides feeling unnecessary (it's not like the PyPI wheels are actually binary packages), we probably didn't have the right kind of eclass support for that at the time.
Inspired by @…, today I've tried preparing new revisions of ensurepip packages that actually do build everything from source. So what changed, and why should building from source matter now? Firstly, as part of the wheel reuse patches, we do have a reasonably clean architecture to grab the wheels created as part of the PEP517 build. Secondly, since we're unbundling dependencies from pip and setuptools, we're effectively testing different packages than these installed as ensurepip wheels — and so it would be meaningful to test both variants. Thirdly, building from source is going to make patching easier, and at the very least enable user patching.
While at it, I've refreshed the test suite runs in all three regular packages (pip, setuptools and wheel — we need an "ensurepip" wheel for the last because of test suites). And of course, I hit some test failures in testing the versions with bundled dependencies, and I've discovered a random bug in #PyPy.
github.com/gentoo/gentoo/pull/ (yes, we haven't moved yet)
github.com/pypy/pypy/issues/53

@mgorny@social.treehouse.systems
2025-07-20 16:01:04

For #PyPy, it's always a resource leak…
github.com/vutran1710/PyrateLi