Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python Has Too Many Package Managers

dublog.net

Post reply on HN
Print view View original

71–80 of 170 posts

Re: Python Has Too Many Package Managers

#71

Post by woodruffw »

> Naturally this led to a proliferation of new Python package managers which leverage the new standard. Enter poetry, PDM, Flit, and Hatch.

An important qualification: Poetry uses pyproject.toml, but it doesn't use the standard (i.e. PEP 518, and 621) metadata layout. This in practice means that it doesn't follow the standard; it just happens to (confusingly) use a file with the same name.

To the best of my knowledge, the others fully comply with the listed PEPs. In practice this means that the difference between them is abstracted away by tools like `build`, thanks to PEP 517.

Re: Python Has Too Many Package Managers

#72

Post by LarsDu88 »

kelseyfrog wrote: post #61
turndown wrote: post #44

Earlier quoted context omitted.

Is option 3 exactly what you’re supposed to do? Freezing your dependency graph and/or explicitly denoting what version of the dependency you want are your best bets for avoiding problems like this

3 is exactly what you're supposed to do. A lot of people will assume that specifying major version upper bounds on dependencies is what you're supposed to do, but I've seen this fail more often than freezing dependencies. The problem with major version upper bounds is that if it's possible to write a test case for a bug, it's possible to depend on broken behavior. Changing behavior in a way that breaks users should b…

I mention this ceiling pinning footgun in the article. It's an enormous pain in the ass to explain to folks, and some software engineers I've met are totally incredulous that that's "not the right thing to do"

Poetry makes it 10x worse with its `^` operator

Re: Python Has Too Many Package Managers

#73

Post by slt2021 »

rangerelf wrote: post #66
slt2021 wrote: post #51

Earlier quoted context omitted.

Can I ask you a question, what do you think is better approach: 1) publish packages as wheels or 2) publish apps as docker images (or docker-compose files or helm charts)? Why some People prefer 1 to 2? I think 2 is more “production friendly” and universal across other languages and stacks (same approach used for java js ruby etc)

Vastly different use cases. Publish a module as a wheel if you want to be able to use pip to install it in your servers, or anywhere really. Once you've built out your virtual environment you can build your docker image and push it to a registry. Your helm chart should include references to docker images that are getting deployed to a kubernetes cluster, among other entities that need to be built out for your app to…

By publishing module you refer to situation when a single module is reused across different applications?

So if you dont have modules that used across apps, then you dont need to package and publish wheel?

or if your modules are already vendored in your main application module (monorepo) then no need to package app as a wheel?

Re: Python Has Too Many Package Managers

#74

Post by simonw »

On Rye: "This project was ultimately abandoned by its author in 2023 and given to Astral.sh in favor of supporting uv instead"

I don't think that's quite the right way to frame this. Handing Rye over to a company that could maintain it full time isn't the same thing as "abandoning" it - and the new maintainers are active on that project: https://github.com/astral-sh/rye/commits/main/

Re: Python Has Too Many Package Managers

#76

Post by LarsDu88 »

VagabundoP wrote: post #30

Use Rye. It wasn't abandoned it ownership was transferred. Rye uses other pretty standard stuff under the hood, tools that follow PEPs, its just a front end that is sane. uv is fast as well. It downloads the pinned version of standalone Python, it keeps everything in its own venv and theres very little messing/tweaking of the environment. It is messy, although its getting better. I doubt everything will ever standard…

I made a mistake about Rye when I posted the first draft of this article. That has since been corrected.

Rye is incredibly young, and when I first encountered it, I looked at its source code and found it was simply calling piptools under the hood.

I did not realize that it had changed and now uses uv (or perhaps just the same solver as uv) under the hood.

Re: Python Has Too Many Package Managers

#77

Post by LarsDu88 »

stairlane wrote: post #20
AtlasBarfed wrote: post #17

"Finally, because dependency resolution is a directed acylic graph (DAG)" ... Is it? :-) Mwahahaha

We analyze firmware and binaries, amongst other things at my day job. Reconstructing dependency graphs is a necessary part of our day to day. Assuming dependency graphs cant have cycles has shot us in the foot. There are absolutely real circular dependencies in assets in the wild.

Original author here... wait, what?

I believe pip (or at least older versions of pip) makes it possible to have cycles. Are you analyzing dep graphs made with older versions of pip?

Re: Python Has Too Many Package Managers

#78

Post by LarsDu88 »

dmd wrote: post #27
p1esk wrote: post #21

Looks like Conda is still the best package/env manager for ML engineers.

Unfortunately I'm still waiting for "Solving environment". It's been 8 years. Please send help. I'm so hungry. (Yes, I know about mamba.)

I lol'd at this.

At my last job, we lost days if not weeks to "solving environment" due to poetry and many folks not understanding that ceiling pinning is BAD when you use a tool like poetry to manage your deps.

Re: Python Has Too Many Package Managers

#80

Post by beaugunderson »

I share the author's excitement about `uv`. I was a big fan of `pip-compile` because it was the simplest possible way to have clear top-level dependencies that also froze sub-dependencies (you note your top-level dependencies in `requirements.in`, then use `pip-compile` to freeze those plus the sub-dependencies in `requirements.txt`, and it adds comments noting what top-level dependency brought it in).

`uv` is basically that but faster.

Post reply on HN