Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Conda: A package management disaster?

pyherald.com

Post reply on HN
Print view View original

201–210 of 234 posts

Re: Conda: A package management disaster?

#201

Post by Kwpolska »

marcosdumay wrote: post #160
These335 wrote: post #67

Can somebody please eli5 why it is so unanimously accepted that Python's package management is terrible? For personal projects venv + requirements.txt has never caused problems for me. For work projects we use poetry because of an assumption that we would need something better but I remain unconvinced (nothing was causing a problem for that decision to be made).

> For personal projects venv + requirements.txt has never caused problems for me. That means you don't use Windows. What is great. Keep not using it. But most people will have a different experience.

There are no Windows-specific issues in venv + pip. Windows can be more painful if you need to compile C extensions, but you usually don't, since most commonly used packages have had binary wheels for Windows on PyPI for many years.

Re: Conda: A package management disaster?

#202

Post by LarsDu88 »

traversaro wrote: post #177
LarsDu88 wrote: post #49

Earlier quoted context omitted.

`uv` is not a drop-in replacement for `conda` in the sense that `conda` also handles non-python dependencies, has its own distinct api server for packages, and has its own packaging yaml standard. `pixi` basically covers `conda` while using the same solver as `uv` and is written in Rust like `uv`. Now is it a good idea to have python's package management tool handle non-python packages? I think that's debateable. I p…

I am not sure pixi uses the same solver of uv, at least in general. pixi uses resolvo ( https://github.com/mamba-org/resolvo ) for conda packages, while uv (that in turns uses pubgrub https://github.com/pubgrub-rs/pubgrub ) for pip packages.

Hmmm I'll update that point

Re: Conda: A package management disaster?

#203

Post by LarsDu88 »

traversaro wrote: post #177
LarsDu88 wrote: post #49

Earlier quoted context omitted.

`uv` is not a drop-in replacement for `conda` in the sense that `conda` also handles non-python dependencies, has its own distinct api server for packages, and has its own packaging yaml standard. `pixi` basically covers `conda` while using the same solver as `uv` and is written in Rust like `uv`. Now is it a good idea to have python's package management tool handle non-python packages? I think that's debateable. I p…

I am not sure pixi uses the same solver of uv, at least in general. pixi uses resolvo ( https://github.com/mamba-org/resolvo ) for conda packages, while uv (that in turns uses pubgrub https://github.com/pubgrub-rs/pubgrub ) for pip packages.

Pixi uses uv for resolving pypi deps: https://prefix.dev/blog/uv_in_pixi If you look closely, pixi used `resolvo to power `rip` then switched from a `rip` solver to a `uv` solver

Re: Conda: A package management disaster?

#204

Post by mdaniel »

I hope you read it while it was available, because the domain has expired

    Domain Name: pyherald.com
    Registry Domain ID: 2663190918_DOMAIN_COM-VRSN
    Registrar WHOIS Server: whois.namesilo.com
    Registrar URL: https://www.namesilo.com/
    Updated Date: 2024-12-21T07:00:00Z
    Creation Date: 2021-12-21T07:00:00Z
    Registrar Registration Expiration Date: 2024-12-21T07:00:00Z
https://web.archive.org/web/20241220211119/https://pyherald.... is the most recent snap

Re: Conda: A package management disaster?

#205

Post by btrettel »

mdaniel wrote: post #204

I hope you read it while it was available, because the domain has expired Domain Name: pyherald.com Registry Domain ID: 2663190918_DOMAIN_COM-VRSN Registrar WHOIS Server: whois.namesilo.com Registrar URL: https://www.namesilo.com/ Updated Date: 2024-12-21T07:00:00Z Creation Date: 2021-12-21T07:00:00Z Registrar Registration Expiration Date: 2024-12-21T07:00:00Z https://web.archive.org/web/20241220211119/https://pyhera…

Looks like the webpage is on GitHub too: https://github.com/pyherald/pyherald/blob/main/docs/articles...

Re: Conda: A package management disaster?

#206

Post by coredog64 »

disgruntledphd2 wrote: post #194
wruza wrote: post #189

Earlier quoted context omitted.

Oh right, recently I started learning classic ML and “just” tried to install tensorflow, which, itself or through one of dependencies, stopped providing windows binaries since x.y.z and so my python has to be downgraded to 3.a and then other dependencies stop installing. Eventually I managed to find a proper version intersection for everything together with some shady repo, but it felt like one more requirement and I…

Tensorflow is the worst. Basically every time my python env was borked (with multiple incompatible versions of Numpy) it was down to tensorflow.

You said it. I was working with an official Google library that used TF and it didn’t work at all with 3.12. I spent a day building the wheels for 3.12 only to find there was a bug with dataclasses. :|

I can’t recall the library, but there was another major project that just deprecated TF because it was the cause of so many build problems.

Re: Conda: A package management disaster?

#207

Post by wakawaka28 »

michaelmrose wrote: post #178
wakawaka28 wrote: post #134

Earlier quoted context omitted.

If you install a package in a fresh environment then it does actually get installed. It can be inherited from the global environment but I don't think disparate venvs that separately install a package actually share the package files. If they did, then a command executed in one tree could destroy the files in another tree. I have not done an investigation to look into this today but I think I'm right about this.

In better designed systems than python they do. To share them with python you need something with dedup. Eg BTRFS ZFS

Python's venv design is not obviously unintelligent. It must work on all sorts of filesystems, which limits how many copies can be stored and how they can be associated. More advanced filesystems can support saving space explicitly for software that exploits them, and implicitly for everyone, but there is a cost to everything.

Re: Conda: A package management disaster?

#208

Post by retrochameleon »

KolenCh wrote: post #168
SamPatt wrote: post #28

Conda is the only package manager I've used on Ubuntu that intermittently and inexplicably gets stuck when installing or uninstalling. It will sometimes resolve itself if left alone for hours, but often won't. I avoid it as much as possible.

The situation has since changed as the solver is rewritten. (Upstreamed from the work done in mamba.) I encourage you to try again.

Does that mean I should consider using conda again over mamba? (It's nice to not be a black sheep, but conda's performance was embarrasingly abysmal)

Re: Conda: A package management disaster?

#209

Post by KolenCh »

retrochameleon wrote: post #208
KolenCh wrote: post #168

Earlier quoted context omitted.

The situation has since changed as the solver is rewritten. (Upstreamed from the work done in mamba.) I encourage you to try again.

Does that mean I should consider using conda again over mamba? (It's nice to not be a black sheep, but conda's performance was embarrasingly abysmal)

As far as the solver is concerned, there should be no difference as it has been streamed. But I personally can’t see a reason to go back, as mamba is supposed to be a drop in replacement of conda. I default to use mamba and switch to conda only when necessary. There are some cases mamba can’t handle correctly, such as the case where you want to roll back to an earlier revision: https://github.com/mamba-org/mamba/issues/803

Re: Conda: A package management disaster?

#210

Post by mardifoufs »

vel0city wrote: post #138
mardifoufs wrote: post #101

Earlier quoted context omitted.

Try building a package and you will get hundreds of little paper cuts. Need a different index for some packages? It will work with a cli "pip install -from-index", but pip will not let you add an index in a requirement.txt for... security reasons. That means, good luck trying to "enforce" the CUDA version of pytorch without using third party tooling. So you either hard code a direct link (breaks platform portability)…

You can include command line parameters like index-url and include-extra-index-url and find-links and what not in a requirements file.

And have them build with "Pip install ." or python build? With the default setuptools config (or even any tweaks to it)? It works until you actually try to package the app, that's where the edge cases start piling up, a lot of them due to very weird decisions made on a whim on some random discourse thread.

Adding index URLs is explicitly not supported in the requirements.txt in setuptools or the default python build tool.

Post reply on HN