Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Conda: A package management disaster?

pyherald.com

Post reply on HN
Print view View original

91–100 of 234 posts

Re: Conda: A package management disaster?

#91

Post by thangngoc89 »

BiteCode_dev wrote: post #85
The_Colonel wrote: post #32

It's rare to see something as systematically broken as Python package/dependencies ecosystem. What I don't understand - what makes this so difficult to solve in Python? It seems that many other platforms solved this a long time ago - maven 2.0 was released almost 20 years ago. While it wasn't / isn't by no means perfect, its fundamentals were decent already back then. One thing which I think messed this up from the b…

Python packaging is broken mostly because bootstrapping is broken, and it cascades to packaging but people don't know the bootstrapping is responsible and blame packaging. Not saying packaging doesn't have faults, but on it's own, on a good Python setup, it's actually better than average. But few people have a good setup. In fact most people don't know what a good setup looks like. And here is why bootstrapping is br…

uv solves this issue nicely. Uv manages Python version and being a single binary, installing uv involved downloading a file and add it to PATH

Re: Conda: A package management disaster?

#92

Post by dccsillag »

gdiamos wrote: post #75
These335 wrote: post #67

Can somebody please eli5 why it is so unanimously accepted that Python's package management is terrible? For personal projects venv + requirements.txt has never caused problems for me. For work projects we use poetry because of an assumption that we would need something better but I remain unconvinced (nothing was causing a problem for that decision to be made).

It's popular enough that it causes pain for a lot of people. Coming from C++, IMO, it is vastly better.

Well, yes, but that's an extremely low bar!!

Re: Conda: A package management disaster?

#93

Post by 9dev »

cdavid wrote: post #55
The_Colonel wrote: post #32

It's rare to see something as systematically broken as Python package/dependencies ecosystem. What I don't understand - what makes this so difficult to solve in Python? It seems that many other platforms solved this a long time ago - maven 2.0 was released almost 20 years ago. While it wasn't / isn't by no means perfect, its fundamentals were decent already back then. One thing which I think messed this up from the b…

A lot of path dependency, but essentially 1. A good python solution needs to support native extensions. Few other languages solve this well, especially across unix + windows. 2. Python itself does not have package manager included. I am not sure solving 2 alone is enough, because it will be hard to fix 1 then. And ofc 2 would needs to have solution for older python versions. My guess is that we're stuck in a local ma…

PHP and composer do. You can specify native extensions in the composer.json file, along with an optional version requirement, and install them using composer just fine. Dependencies can in turn depend on specific extensions, or just recommend them without mandating an installation. This works across UNIX and Windows, as far as I’m aware.

Re: Conda: A package management disaster?

#94

Post by 9dev »

turboponyy wrote: post #57
reactordev wrote: post #40

Earlier quoted context omitted.

The answer is not Yet Another Tool In The Chain. Python community itself needs to address this. Because if they don’t then you’ll have requirements.txt, setuptools, pyproject, pip, pipx, pipenv, pyenv, venv, nix.

The thing is, Nix is not Yet Another Tool, it is the tool.

The inevitable reality: https://xkcd.com/927/

Re: Conda: A package management disaster?

#95

Post by dagw »

9dev wrote: post #93
cdavid wrote: post #55

Earlier quoted context omitted.

A lot of path dependency, but essentially 1. A good python solution needs to support native extensions. Few other languages solve this well, especially across unix + windows. 2. Python itself does not have package manager included. I am not sure solving 2 alone is enough, because it will be hard to fix 1 then. And ofc 2 would needs to have solution for older python versions. My guess is that we're stuck in a local ma…

PHP and composer do. You can specify native extensions in the composer.json file, along with an optional version requirement, and install them using composer just fine. Dependencies can in turn depend on specific extensions, or just recommend them without mandating an installation. This works across UNIX and Windows, as far as I’m aware.

PHP and composer do.

Is that a new feature? Pretty sure it didn't a few years ago. If the thing I need needed the libfoo C library then I first had to install libfoo on my computer using apt/brew/etc. If a new version of the PHP extension comes out that uses libfoo 2.0, then it was up to me to update libfoo first. There was no way for composer to install and manage libfoo.

Re: Conda: A package management disaster?

#96

Post by Tainnor »

throwawaymaths wrote: post #12
duped wrote: post #10

Earlier quoted context omitted.

My experience with conda is that its fine if you're the original author of whatever you're using it for and never share it with anyone else. But as a professional I usually have to pull in someone else's work and make it function on a completely different machine/environment. I've only had negative experiences with conda for that reason. IME the hard job of package management is not getting software to work in one lo…

imagine being a beginner to programming and being told "use venvs" or worse, imagine being a longtime user of shells but not python and then being presented a venv as a solution to the problem that for some reason python doesn't stash deps in a subdirectory of your project

You don't need to stash deps in a subdirectory, IMHO that's a node.js design flaw that leads to tons of duplication. I don't think there's any other package manager for a popular language that works like this by default (Bundlers does allow you to version dependencies which can be useful for deployment, but you still only ever get one version of any dependency unlike node).

You just need to have some sort of wrapper/program that knows how to figure out which dependencies to use for a project. With bundler, you just wrap everything in "bundle exec" (or use binstubs).

Re: Conda: A package management disaster?

#97

Post by neonsunset »

ElectricalUnion wrote: post #90
neonsunset wrote: post #64

Earlier quoted context omitted.

FWIW Nuget to .NET is what Cargo crates are to Rust instead of what Maven and Gradle are to Java. The package manager is just a part of the SDK. Even the CLI workflow is identical: dotnet add package / cargo add (.NET had it earlier too, it's nice that Cargo now also has it).

Wait, newer versions of thr JDK Java SDK now bundle maven and gradle? Why does everyone use mvnw/gradlew for?

This was referring to package manager being just a part of .NET's SDK. Gradle and Maven continue to ship separately.

Re: Conda: A package management disaster?

#98

Post by BiteCode_dev »

thangngoc89 wrote: post #91
BiteCode_dev wrote: post #85

Earlier quoted context omitted.

Python packaging is broken mostly because bootstrapping is broken, and it cascades to packaging but people don't know the bootstrapping is responsible and blame packaging. Not saying packaging doesn't have faults, but on it's own, on a good Python setup, it's actually better than average. But few people have a good setup. In fact most people don't know what a good setup looks like. And here is why bootstrapping is br…

uv solves this issue nicely. Uv manages Python version and being a single binary, installing uv involved downloading a file and add it to PATH

Yes, that's one of the most important success of the tool. Being in rust, it is completely independent from the Python setup, and therefore it doesn't care if you botched it. And with the indy greg build, it can even avoid the pyenv pitfall of compiling on your machine on linux.

Re: Conda: A package management disaster?

#99

Post by rcxdude »

alsodumb wrote: post #4

As someone with admittedly no formal CS education, I've been using conda for all of my grad school and never managed to break it. I create a virtual environment for every project. I install almost all packages with pip, except for any binaries or CUDA related things from conda. I always exported the conda yaml file and managed to reproduce the code/environment including the Python version. I've seen a lot of posts ov…

This is exactly the kind of thing that causes python package nightmares. Pip is barely aware of packages it's installed itself, let alone packages from other package managers and especially other package repositories. Mixing conda and pip is 100% doing it wrong (not that there's an easy way to do it right, but stick to one or the other, I would generally recommend just using pip, the reasons for conda's existance are mostly irrelevant now)

Re: Conda: A package management disaster?

#100

Post by The_Colonel »

neonsunset wrote: post #64
The_Colonel wrote: post #43

Earlier quoted context omitted.

Agreed. Often there's a quite tight coupling between the core platform devs and package management - node.js has its npm, rust cargo, go has one as well and for the most part it seems to have worked out fine for them. Java and .NET (and I think PHP) are different in the sense that the package management systems have no relation to the platform developers, but industry standards (maven, gradle, NuGET, Composer) still…

FWIW Nuget to .NET is what Cargo crates are to Rust instead of what Maven and Gradle are to Java. The package manager is just a part of the SDK. Even the CLI workflow is identical: dotnet add package / cargo add (.NET had it earlier too, it's nice that Cargo now also has it).

Right, I forgot NuGet got adopted by Microsoft. But it started and gained prominence independently.
Post reply on HN