Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Wetlands – a lightweight Python library for managing Conda environments

arthursw.github.io

Post reply on HN
Print view View original

1–10 of 45 posts

Show HN: Wetlands – a lightweight Python library for managing Conda environments

#1

Post by arthursw »

When building a plugin system for an application, avoiding dependency conflicts is critical. To address this, I created Wetlands – a lightweight Conda environment manager.

Wetlands not only simplifies the creation of isolated Conda environments with specific dependencies, but also allows you to run arbitrary Python code within those environments and retrieve the results. It uses the multiprocessing.connection and pickle modules for inter-process communication. Additionally, one can easily use shared memory between the environments, making data exchange more efficient.

Docs: https://arthursw.github.io/wetlands/latest/ Source: https://github.com/arthursw/wetlands/

I’d really appreciate any feedback. Thanks!

Show HN: Wetlands – a lightweight Python library for managing Conda environments
arthursw.github.io

Re: Show HN: Wetlands – a lightweight Python library for managing Conda environments

#2

Post by mushufasa »

> Wetlands not only simplifies the creation of isolated Conda environments with specific dependencies, but also allows you to run

I've been using Conda for 10 years as my default package manager on my devices (not pipenv or poetry etc). I started because it was "the way" for data science but I kept with it because the syntax is really intuitive to me (conda create, conda activate).

I'm not sure what problem you are solving here -- the issues with conda IMO are that it is overkill for the rest of the python community, so conda-forge has gradually declined and I typically create a conda environment then use pip for the latest libraries. Managing the conda environments though is not my issue -- that part works so well that I keep with it.

If you could explain why you created this and what problems you are solving with an example, that would be helpful. All package managers are aimed at "avoiding dependency conflicts" so that doesn't really communicate to me what this is and what real problem it solves.

Re: Show HN: Wetlands – a lightweight Python library for managing Conda environments

#3

Post by mrweasel »

> If the user doesn't have pixi or micromamba installed, Wetlands will download and set it up automatically.

Please don't. Never have a tool that automatically reaches out onto the internet to get a binary and then run it. Just let the user know that they need to install either pixi or micromamba. It's inherently unsafe and you don't know what will be put into those binaries in the future.

Maybe it's because I don't have a use case for this, but I don't really get what this is for. It's interesting, but I'm not really sure where I'd use it.

Re: Show HN: Wetlands – a lightweight Python library for managing Conda environments

#6

Post by ElectricalUnion »

barapa wrote: post #4

Why do people use Conda instead of uv?

The conda ecosystem was a early adopter of standardized binary packages.

Now it's mostly behind us, but there used to be a time where pypi didn't have wheels (a 2012 thing), or manylinux wheels (a 2016 thing) for most libraries. pip install was a world of pain if you didn't have the "correct source packages" in your system.

And now several of those projects built back then, they're no longer projects but deployed systems, might as well stick to what is working.

Re: Show HN: Wetlands – a lightweight Python library for managing Conda environments

#7

Post by krapht »

ElectricalUnion wrote: post #6
barapa wrote: post #4

Why do people use Conda instead of uv?

The conda ecosystem was a early adopter of standardized binary packages. Now it's mostly behind us, but there used to be a time where pypi didn't have wheels (a 2012 thing), or manylinux wheels (a 2016 thing) for most libraries. pip install was a world of pain if you didn't have the "correct source packages" in your system. And now several of those projects built back then, they're no longer projects but deployed sys…

Conda is still the recommended way to work with Intel's Python distribution, so there's a reason for it to live on my work computer.

The fastest graph library I know, graph-tool, needs to be installed with Conda too.

Re: Show HN: Wetlands – a lightweight Python library for managing Conda environments

#8

Post by jpecar »

Why not just call it swamp? It would better describe the python ecosystem mess ;)

Jokes aside, this feels very meta: package manager for a package manager for a package manager. Reminds me of the old RFC1925: "you can always add another layer of abstraction". That RFC also says "perfection has been reached not when there is nothing left to add, but when there is nothing left to take away".

And as a hpc admin, I'm not offering my users any help with conda and let them suffer on their own. Instead I'm showing them the greener pastures with spack, easybuild and eessi whenever I can. And they're slowly migrating over.

Re: Show HN: Wetlands – a lightweight Python library for managing Conda environments

#9

Post by N1H1L »

mushufasa wrote: post #2

> Wetlands not only simplifies the creation of isolated Conda environments with specific dependencies, but also allows you to run I've been using Conda for 10 years as my default package manager on my devices (not pipenv or poetry etc). I started because it was "the way" for data science but I kept with it because the syntax is really intuitive to me (conda create, conda activate). I'm not sure what problem you are s…

I used to use conda, but have switched entirely to uv now

Re: Show HN: Wetlands – a lightweight Python library for managing Conda environments

#10

Post by jessekv »

barapa wrote: post #4

Why do people use Conda instead of uv?

These days I see Conda (and micromamba) used as a reliable cross-platform winget/apt/yum/brew. For example, to install GDAL.

uv replaces pip, conda and pip have been complementary for a long time. But I would be surprised if uv does not take on conda at some point, e.g. with a micromamba subcommand.

Post reply on HN