Pixi adoption overcomes Conda workflow liabilities

Conda environments create management friction

I see why teams are abandoning the Conda and Mamba workflows. Having environments detached from project directories encourages bad practices like reusing environments for unrelated tasks or installing everything into the same environment to get broken dependency resolutions. Pixi manages project-specific dependencies through project files and handles global utilities through pixi global. This allows you to install tools like git or git-annex without cluttering a project. While Conda allows for non-Python dependencies like Firefox for automation, Pixi also records these versions in its lock files. Poetry limits lock files to Python dependencies, but Pixi includes the browser version in the same lock file to ensure reproducibility. You should probably stop using Conda for project-specific dependencies if you want reliable lock files. Conda environments are a bad fit for global tools or for packages that need frequent updates because they are not easily updatable and do not tie themselves to a specific project directory. Pixi addresses these needs through pixi global and project-specific environments. Pixi uses the same repositories as Conda, so you do not lose access to conda-forge packages.

New environment sets solve dependency scaling

The new environment set mechanism in Pixi 0.40 allows for tailored dependency management across different use cases. This feature remains invisible to users who do not need multiple environments, which keeps the tool simple. To prevent a combinatorial explosion of dependency sets, the tool requires users to define environments manually rather than inferring them from a feature matrix. The design only allows one environment to be active at a time, which simplifies the resolution process and prevents conflicts.

Developers use these sets for several specific reasons. In the polarify project, developers test multiple versions of polars combined with multiple package versions. They previously used a matrix in GitHub actions for this, but they can now replace that with multiple environments. The solve-group field in the environment definition helps group environments together during the solve stage. This grouping is useful when environments need the same dependencies but require additional ones, such as testing a production environment with additional test dependencies. Other use cases include testing multiple package versions like python 3.9 and 3.10, creating small, single-tool environments for tasks like linting or documentation, or building large developer environments that combine multiple smaller environments. The target field allows all the above features to be separated by targets.

The pixi.toml file contains these fields. Within the pixi.lock file, a package may include an additional environments field to specify which environment it belongs to. This avoids duplication and keeps the lock file of minimal size.

Solving conflicts and hardware compatibility

I find the current conflict with user site-packages frustrating. When a user installs numpy v2 in their local user site via PEP 370, it can override a pixi environment requesting numpy v1, which creates a situation where the version in the shell does not match the version defined in the project. A potential solution involves adding an option to ignore packages not explicitly in the environment, similar to the PYTHONNOUSERSITE environment variable.

The implementation of pixi add also contains a pinning error. When users add packages that belong to NON_SEMVER_PACKAGES, the remaining packages use a minor pinning strategy instead of the expected behavior. For instance, adding pyodbc, python, and sqlalchemy can lead to unexpected version ranges.

Pixi’s adoption continues in scientific communities. A core maintainer of both SciPy and Pixi has already introduced Pixi into SciPy to improve the user experience. The tool also now supports RISC-V (riscv64) platforms, bringing conda ecosystem power to open hardware. Can the developers maintain this growth while fixing the remaining dependency resolution bugs?