Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overview of Python dependency management tools

modelpredict.com

Post reply on HN
Print view View original

91–100 of 184 posts

Re: Overview of Python dependency management tools

#91

Post by cbarrick »

brummm wrote: post #77

Using conda for environment and dependency management works really well.

Conda is a great tool.

But it forks the ecosystem, twice:

First, Conda packages have to be maintained separately from PyPI packages.

Second, the "default" repo is maintained by Anaconda, but the community maintained Conda Forge repo is also separate, and officially the packages in one are not compatible with the packages in the other. (In practice they usually play nice).

Having three incompatible package repos is not ideal.

Re: Overview of Python dependency management tools

#92

Post by ptx »

a_cool_username wrote: post #72
madelyn wrote: post #50

People always get up in arms about this, but as someone who has used Python as her daily driver for years it's really... never been this serious of an issue for me? I have used virtualenv/venv and pip to install dependencies for years and years, since I was a teen hacking around with Python. Packaging files with setup.py doesn't really seem that hard. I've published a few packages on pypi for my own personal use and…

Let me start by saying: I love python, and I love developing in it. It's the "a pleasure to have in class" of languages: phenomenal library support, not too painful to develop in, nice and lightweight so it's easy to throw together test scripts in the shell (contrast that with Java!), easy to specify simple dependencies + install them. (contrast that with C!). That said... if you work on software that is distributed…

> struggling to get Python installed and into their `PATH` ... it's frankly still amateur hour over there

But that has been solved on Windows for quite a while hasn't it?

Python installs the "py" launcher on the path, which allows you to run whichever version you want of those you have installed. Just type "py" instead of "python". Or "py -3.5-32" to specifically run 32-bit Python 3.5, or "py -0" to list the available versions.

Re: Overview of Python dependency management tools

#93

Post by sosodev »

Personally I just try to avoid Python development because I hate feeling like I'm dealing with what should be a solved problem. Recently I had to work with an outdated Python Tensorflow framework and the only way we could get it to work correctly across different dev and deployment machines was with a fat Docker image that took hours of head scratching to build. It was miserable.

Re: Overview of Python dependency management tools

#94

Post by mariokostelac »

tarruda wrote: post #6
nouveaux wrote: post #5

Earlier quoted context omitted.

I'm only familiar with Python, Javascript and Rust. It seems to me that Rust is the only one that has "solved" this problem. I dont think there are any real Python devs who thinks dependency management is solved. However, why would you claim Javascript has a good solution? The inconsistencies between node and web dev is odd at best. Babel compilation is annoying and slow. Are we even standardized on webpack yet? Can…

One thing node.js got right is the module resolution logic. Node doesn't even need something like venv since module lookup is always local. Also no problem with dependency hell. Each dependency can have its own private dependencies, even different versions of dependencies shared by sibling modules. Tools like yarn/npm can remove duplicates across a project.

How come other tools don't just replicate successful models?

Re: Overview of Python dependency management tools

#95

Post by mariokostelac »

cycomanic wrote: post #44

Everytime I read an article about all these tools I really can't help but think what would happened if Linus would have taken over the desktop. All the tools really largely seem to try to poorly replicate Linux package management and the fact that because of this devs now don't care anymore about api stability and not always building against the latest and greatest. I admit a pyenv is nice for testing against differe…

I think that works when you use Python cli tools, but not when you're working on 5 different projects, each running different python version.

Re: Overview of Python dependency management tools

#96

Post by robertlagrant »

slavoingilizov wrote: post #46

The missing ingredient to really, REALLY solve these problems once and for all is an authoritative decision to switch package formats and run the whole dependency resolution stack by the core python language contributor team. I get backwards compatibility and open-source governance and bla-bla, but the reality is that this cannot be done by a third-party library author and needs to become part of the core stack, incl…

The default is pip + python3 -m venv.

Re: Overview of Python dependency management tools

#97

Post by 6gvONxR4sf7o »

Oof, that footnote:

> It’s 2020, but reliably compiling software from source, on different computer setups, is still an unsolved problem. There are no good ways to manage different versions of compilers, different versions of libraries needed to compile the main program etc.

I wonder how much stuff like this has to do with python's popularity. When I have opaque issues like "libaslkdjfasf.so is angry with you and/or out to lunch and/or not doing expected things," it's the most frustrating part of programming. I'd pay devops people infinite money to not have to deal with installation/setup issues anymore.

Re: Overview of Python dependency management tools

#98

Post by mariokostelac »

xapata wrote: post #38

> [Pipenv] loads packages from PyPI so it does not suffer from the same problem as Conda does. False. Conda manages packages installed from PyPI. This is discussed under the Conda section, so I'm surprised the quoted line wound up in the article.

Hey xapata, thanks for pointing this out.

Any chance you could give me some reference so I can fix it in the original article?

Re: Overview of Python dependency management tools

#100

Post by throwaway894345 »

philwelch wrote: post #55
madelyn wrote: post #50

People always get up in arms about this, but as someone who has used Python as her daily driver for years it's really... never been this serious of an issue for me? I have used virtualenv/venv and pip to install dependencies for years and years, since I was a teen hacking around with Python. Packaging files with setup.py doesn't really seem that hard. I've published a few packages on pypi for my own personal use and…

As an individual that probably works just fine. In a team setup, it takes a lot of training and effort for everyone to consistently follow a manual pip/venv workflow, so it becomes valuable to minimize and standardize it.

Especially if you have to deploy to production and you want fast, reproducible builds, or you don't want to run a bunch of tests for things that haven't changed.
Post reply on HN