Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python Has Too Many Package Managers

dublog.net

Post reply on HN
Print view View original

31–40 of 170 posts

Re: Python Has Too Many Package Managers

#31

Post by sealedservant »

doctorpangloss wrote: post #29
sealedservant wrote: post #25

Earlier quoted context omitted.

Virtual environments are cool, and necessary, but at the same time, they are incredibly limited and I always get frustrated at the lack of features they should have. They are too fragile. Nuking a whole venv when you mess up isn't really efficient. They aren't portable. You need to package up your editable project for an offline system? Too bad, virtual environments use hardcoded paths and symlinks that will be broke…

> There's no concept of caching venv packages elsewhere A venv's pip shares a systemwide cache for downloads.

Ah, that's news to me! Updated previous comment for accuracy.

I mostly use pipx these days for stuff I need on the PATH.

Re: Python Has Too Many Package Managers

#33

Post by imglorp »

slt2021 wrote: post #23

I see a lot of package managers I never head of, am a happy venv & pip user. One of the key faults of pip is what happens when you decide to remove a dependency. Removing a dependency does not actually remove the sub-dependencies that were brought in by the original dependency, leaving a lot of potential cruft. This is not really an issue if your virtual environments are disposable. Just nuke and recreate venv from s…

> Just nuke and recreate venv from scratch using only what you need.

That's great until you have dynamic dependencies which means you can run apparently happy yet fail later on an import. This means you have to take special care generating your dep list.

Re: Python Has Too Many Package Managers

#34

Post by theideaofcoffee »

marcosdumay wrote: post #28
theideaofcoffee wrote: post #13

Earlier quoted context omitted.

Maybe it's just a joke, but for those that don't know that is Perl's motto and even better because there's the venerable, wonderful CPAN [0] which is the de facto package repository for the language. [0] https://www.cpan.org/

It's certainly a joke, because the Python original motto was "there's only one way to do it", and the current one is "there's only one (obvious) way to do it". The Python's motto was created as an obvious reference to the Perl's one, purposefully negating it.

Thank you for that context!

Re: Python Has Too Many Package Managers

#35

Post by slt2021 »

sealedservant wrote: post #25
slt2021 wrote: post #23

I see a lot of package managers I never head of, am a happy venv & pip user. One of the key faults of pip is what happens when you decide to remove a dependency. Removing a dependency does not actually remove the sub-dependencies that were brought in by the original dependency, leaving a lot of potential cruft. This is not really an issue if your virtual environments are disposable. Just nuke and recreate venv from s…

Virtual environments are cool, and necessary, but at the same time, they are incredibly limited and I always get frustrated at the lack of features they should have. They are too fragile. Nuking a whole venv when you mess up isn't really efficient. They aren't portable. You need to package up your editable project for an offline system? Too bad, virtual environments use hardcoded paths and symlinks that will be broke…

Dockerfile and requirements.txt is everything you need to know to recreate environment, and this is how apps are usually ran in production. This is how you can create consistent recreatable and testable env and carry it from dev machine to test, staging, CI/CD, and prod environments

Never really had an issue with python apps not being portable.

Building regular python app? Just use official python docker image.

Using advanced CUDA stuff with nvidia cards? Just use nvidia’s docker image and forget about fiddling with configuring and compiling dependencies. It is all as easy as carrying dockerfile and requirements.txt

Re: Python Has Too Many Package Managers

#37

Post by lexicality »

slt2021 wrote: post #23

I see a lot of package managers I never head of, am a happy venv & pip user. One of the key faults of pip is what happens when you decide to remove a dependency. Removing a dependency does not actually remove the sub-dependencies that were brought in by the original dependency, leaving a lot of potential cruft. This is not really an issue if your virtual environments are disposable. Just nuke and recreate venv from s…

I do a lot of maintenance work and every time I've encountered a (complex) project set up in this way that's older than ~6 months, it's bitrotted from breaking changes in the dependencies.

The python ecosystem does not stand still and seems quite happy to introduce breaking changes in non-major versions.

To my mind, there are 3 ways to make sure your python project of today will work in 2 years time:

  1. Have no dependencies 
  2. Destroy your virtual environment and reinstall it every day for the next 2 years and fix anything that breaks
  3. Freeze your entire dependency tree where it stands and occasionally do large breaking change updates
All 3 options suck but #3 is the least worst

Re: Python Has Too Many Package Managers

#39

Post by Scaevolus »

sealedservant wrote: post #25
slt2021 wrote: post #23

I see a lot of package managers I never head of, am a happy venv & pip user. One of the key faults of pip is what happens when you decide to remove a dependency. Removing a dependency does not actually remove the sub-dependencies that were brought in by the original dependency, leaving a lot of potential cruft. This is not really an issue if your virtual environments are disposable. Just nuke and recreate venv from s…

Virtual environments are cool, and necessary, but at the same time, they are incredibly limited and I always get frustrated at the lack of features they should have. They are too fragile. Nuking a whole venv when you mess up isn't really efficient. They aren't portable. You need to package up your editable project for an offline system? Too bad, virtual environments use hardcoded paths and symlinks that will be broke…

Pip will cache the downloaded package tarballs to make repeated installs faster at least, but that's little help when you have a dozen venvs with multiple gigabytes of pytorch.

Re: Python Has Too Many Package Managers

#40

Post by slt2021 »

imglorp wrote: post #33
slt2021 wrote: post #23

I see a lot of package managers I never head of, am a happy venv & pip user. One of the key faults of pip is what happens when you decide to remove a dependency. Removing a dependency does not actually remove the sub-dependencies that were brought in by the original dependency, leaving a lot of potential cruft. This is not really an issue if your virtual environments are disposable. Just nuke and recreate venv from s…

> Just nuke and recreate venv from scratch using only what you need. That's great until you have dynamic dependencies which means you can run apparently happy yet fail later on an import. This means you have to take special care generating your dep list.

Thats what pytest is for, it is important to have tests that cover your most important code paths so these these dynamic imports are ran.

Due to dynamic natire of language, coding in python without strong test culture is masochism and full of footguns anyways

Post reply on HN