Tootfinder

New on #Quansight PBC blog: Python Wheels: from Tags to Variants
#Python distributions are uniform across different Python versions and platforms. For these distributions, it is sufficient to publish a single wheel that can be installed everywhere. However, some packages are more complex than that; they include compiled Python extensions or binaries. In order to robustly deploy these software on different platforms, you need to publish multiple binary packages, and the installers need to select the one that fits the platform used best.
For a long time, Python wheels made do with a relatively simple mechanism to describe the needed variance: Platform compatibility tags. These tags identified different Python implementations and versions, operating systems, and CPU architectures. Over time, they were extended to facilitate new use cases. To list a couple: PEP 513 added manylinux tags to standardize the core library dependencies on GNU/Linux systems, and PEP 656 added musllinux tags to facilitate Linux systems with musl libc.
However, not all new use cases can be handled effectively within the framework of tags. To list a few:
• The advent of GPU-backed computing made distinguishing different acceleration frameworks such as NVIDIA CUDA or AMD ROCm important.
• As the compatibility with older CPUs became less desirable, many distributions have set baselines for their binary packages to x86-64-v2 microarchitecture level, and Python packages need to be able to express the same requirement.
• Numerical libraries support different BLAS/LAPACK, MPI, OpenMP providers, and wish to enable the users to choose the build matching their desired provider.
While tags could technically be bent to facilitate all these use cases, they would grow quite baroque, and, critically, every change to tags needs to be implemented in all installers and package-related tooling separately, making the adoption difficult.
Facing these limitations, software vendors have employed different solutions to work around the lack of an appropriate mechanism. Eventually, the #WheelNext initiative took up the challenge to design a more robust solution.
"""
#packaging