Show HN: Wetlands – a lightweight Python library for managing Conda environments

“Pixi instead of uv” would be a more fair comparison, as Pixi is a more modern tool which still uses the conda package format and ecosystem, much like uv is a modernised pip which still uses the PyPI package format.

One thing an conda package can do which an PyPI package cannot is have binary dependencies: a conda package is linked upon installation, and packages can declare dependencies on shared libraries. As common example is numeric libraries depending on a BLAS implementation: in a conda/pixi environment you will get exactly one BLAS shared library linked into your process, used by numpy, scipy, optimisers, etc. For some foundational libraries like BLAS which have multiple implementations, the user even has the power to consistently switch the implementation within the environment, eg from OpenBLAS to Intel’s MKL.

The PyPI package format does not allow binary dependencies: wheels must be self-contained when it comes to binary code (not when it comes to Python code - which hopefully makes it clear that something here is inconsistent). Take any numerical python environment and enumerate the copies of BLAS you have, it is probably 3-5. All running their own threadpools.

Another very simple example is with inbuilt modules depending on native code, like the sqlite3 module. In a conda/pixi installation you are guaranteed that the python binary links against the same sqlite3 code as the command-line sqlite3 cli tool in the same environment. Stuff like this removes many cross-language or cross-tool hassles.

I prefer uv or poetry if I’m doing anything simple or pure python (or perhaps with a small binary dependency like an event loop). But pixi is the way to go for large environments with lots of extra tools and numerical libraries.

Thanks for your comments! I fully agree that the Python ecosystem is (overly) complex, and Conda is generally not necessary for Python tools.

I made this library for a workflow management system, which can use any tool packaged with Conda, not just Python tools. The tools can be binaries made in C++, Java programs, or anything Conda can containerize. Note that Docker is not an option because it cannot be installed automatically on all platforms (and because of performances on non-Linux OS).

My users do not have to worry about command lines to install tools since Wetlands is installed in the workflow management system. Each tool is installed when the user executes a workflow using it.

In the bio-image analysis and medical imaging communities —as well as many others— scientists are often unfamiliar with the Python ecosystem and the concept of virtual environments. However, they rely heavily on a wide range of tools, each with numerous dependencies written in various languages. Applications with a built-in package management system like Wetlands greatly simplify their workflow by handling the complex task of setting up environments for these tools behind the scenes.

For example, Napari is an excellent viewer for multi-dimensional images written in Python which can be easily extended via plugins. There are hundreds of plugins, to do things like image denoising, registration, segmentation, particle tracking, etc. Plugins depend on tools (like Segment-Anything-Model, Cellpose, Stardist, etc.) which cannot be installed in the same environment. Wetlands can come to the rescue and isolate each plugin in its own environment.

I hope the purpose of Wetlands is clearer now :)

When I started the project, Spack was not available on Windows so it was not an option. Now I should reconsider this. Thanks!