Live data from Hacker News

Overview of Python dependency management tools

modelpredict.com

21–30 of 184 posts

Re: Overview of Python dependency management tools

#21
This is a good overview of something that took me an annoyingly long time to learn. My personal preference is to keep things simple with pyenv, venv, and pip.

Tangentially related is the tool tox [1], which is often used to run a test suite inside of virtual environments created by venv, on multiple versions of Python managed by pyenv.

Now if only setuptools could work well without hackery...

[1]: https://tox.readthedocs.io/en/latest/

Re: Overview of Python dependency management tools

#26
post #17
post #5

Earlier quoted context omitted.

I'm only familiar with Python, Javascript and Rust. It seems to me that Rust is the only one that has "solved" this problem. I dont think there are any real Python devs who thinks dependency management is solved. However, why would you claim Javascript has a good solution? The inconsistencies between node and web dev is odd at best. Babel compilation is annoying and slow. Are we even standardized on webpack yet? Can…

> Can anyone say with a straight face that getting a new JS dev caught up on what all these different parts to compile a JS program is a solved problem? You are intermixing dependency management with build tools. Webpack and Babel have very little to do with dependency management.

I loosely included Babel and Webpack as part of dependency management since Python does not have a similar compile step. I don't think it is unfair since they fall into the same category of wtf when trying to get your dev environment working.

With that said, I should have included Yarn and Npm with their own problems. I can't remember how often I solved my dependency problems with rm -rf node_modules.

Re: Overview of Python dependency management tools

#27

This is a good overview of something that took me an annoyingly long time to learn. My personal preference is to keep things simple with pyenv, venv, and pip. Tangentially related is the tool tox [1], which is often used to run a test suite inside of virtual environments created by venv, on multiple versions of Python managed by pyenv. Now if only setuptools could work well without hackery... [1]: https://tox.readthe…

Wouldn’t you still need something like pip-tools to lock down subdependencies and handle conflicts?

Re: Overview of Python dependency management tools

#28
If you're interested in the technical issues behind Python packaging, a recent Podcast.__init__ episode features three people working on improving Pip's dependency resolution algorithm. My use cases are simple enough that I've gotten by for years just using pip and venv with requirements.txt files, but it was still fascinating to listen to how package management is approached in more complex situations.

Dependency Management in Pip's Resolver: https://www.pythonpodcast.com/pip-resolver-dependency-manage...

Re: Overview of Python dependency management tools

#29
post #18

It's 2020, but the python community still has not converged to a small set of sane solutions. It seems to me that Ruby, PHP, JS, and Rust communities have solved the problem.

Hasn't it? pypi is really the de-facto package index, Pipenv/Poetry/Conda are all venv handlers (using the standard venv tools) + dependency graph, and using pip which is standard as well. I would call this a small set of solutions (3), and they are all sane (any will do, just pick one).

Inviting the question: why don't the core devs have a serious bakeoff and bring that functionality into the core distro?

Re: Overview of Python dependency management tools

#30
post #15

Every attempt to solve this problem in Python seems to eventually end up in a pretty terrible place. Pipenv got off to a great start but got slower and slower to the point that it was more painful to use than not. Poetry (which is still my preferred option) started off with something seemingly beautifully thought through, and very fast too. But after only a few version updates, it seems to be hitting the same problem…

I get that resolving dependencies is a SAT problem and inherently intensive; however, I don't understand why it's so much slower in Python. Is it just that all of these resolvers are implemented in Python (and Python is really that much slower than other languages?), or does Python require you to download an entire package just to determine its dependencies? In the latter case, that seems pretty dumb, right? Like as bad as exposing the entire interpreter as the extension interface, rendering optimizations and competing interpreters virtually impossible.
Post reply on HN