Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overview of Python dependency management tools

modelpredict.com

Post reply on HN
Print view View original

141–150 of 184 posts

Re: Overview of Python dependency management tools

#141

Post by petters »

kissgyorgy wrote: post #119

> Pipenv or poetry? If you used pipenv for a complex project with huge dependency tree, or used it for a long time, you definitely run into a blocker issue with it. That is the worst package manager of all, and probably the reason why Python has such a bad reputation in this area. It's because it's fundamentals are terrible. Just go with Poetry. It's very stable, easy to use, has a superior dependency resolver and wa…

Agreed. A large part of Pipenv's raison d'être is resolving dependencies. Yet it seems to install packages in random order. This sometimes fails, so it retries failures at the end.

We actually used this in production before switching to poetry.

Re: Overview of Python dependency management tools

#142

Post by franey »

I think this is a good basic overview of the dependency management landscape. I have a few things to add.

One is that because Python has been around for so long, it's easy to find outdated or conflicting advice about how to manage Python packages.

I think it's important to stress that pyenv isn't strictly a dependency manager, too, and depending on your OS, isn't necessary. (Supported Python versions are in the AUR[0].)

A lot of pain from Python 2 -> 3 is that many operating systems were so slow to switch their default Python version to 3. Unless something has changed in the last month or so, Mac OS _still_ uses Python 2 as the default.

It's a shame to see Python take a beating for OS-level decisions.

[0] https://aur.archlinux.org/

Re: Overview of Python dependency management tools

#143

Post by franey »

sixhobbits wrote: post #135

I've spent far too many hours fighting with these tools in two completely different scenarios * Developing and deploying production Python solutions * Helping beginners run their first script While it's great for beginners to use the same tools that are used in industry, I strongly believe that the problem nearly all of these tools face is that they can't decide whether they want to _manage_ complexity or _hide_ comp…

This is a good point about complexity. I started with pip + virtualenv, and I'd recommend pip + venv to anyone learning Python. venv is in the standard library, so there's official documentation for it.

I picked up Pipenv when a point-point release of a dependency broke a production deployment. Pipenv's dependency locking meant that I wouldn't get surprised like that again.

Part of why this topic comes up so much is the desire to run with a language before learning to walk with it, perhaps. I'm a big fan of Poetry, but I like it because I know what it gives me compared to vanilla pip and a setup.py file.

Installing dependencies at the OS level will get you far as a beginner. And when the time comes that you need a virtual environment, you'll probably know.

Re: Overview of Python dependency management tools

#144

Post by infraredcabbage »

madelyn wrote: post #50

People always get up in arms about this, but as someone who has used Python as her daily driver for years it's really... never been this serious of an issue for me? I have used virtualenv/venv and pip to install dependencies for years and years, since I was a teen hacking around with Python. Packaging files with setup.py doesn't really seem that hard. I've published a few packages on pypi for my own personal use and…

This sort of attitude is the reason why the world doesn't move away from awful solutions. It is a testament to the lack of ability to see beyond your own nose.

A lot of people who use Python, don't have the luxury of it being their "daily driver for years", so the conflicting documentation, decision paralysis and other problems that come with it end up being a huge time sink.

A lot of non-programmers are being forced to use Python for various automation tasks. A lot of the CAD-software that construction engineers use, support Python-plugins. Network admins that have been configuring switches and routers on CLI for decades now have to configure them using Python.

Look at "cargo" to see what the world could be like.

Re: Overview of Python dependency management tools

#145

Post by sivakon »

kissgyorgy wrote: post #119

> Pipenv or poetry? If you used pipenv for a complex project with huge dependency tree, or used it for a long time, you definitely run into a blocker issue with it. That is the worst package manager of all, and probably the reason why Python has such a bad reputation in this area. It's because it's fundamentals are terrible. Just go with Poetry. It's very stable, easy to use, has a superior dependency resolver and wa…

The last time I tried poetry, it had dependency problems. The maintainers acknowledged it and resolved it with a patch, but somehow it was not working. This was at poetry 1.0.0b3

Re: Overview of Python dependency management tools

#146

Post by Izkata »

Sohcahtoa82 wrote: post #114
mariokostelac wrote: post #95

Earlier quoted context omitted.

I think that works when you use Python cli tools, but not when you're working on 5 different projects, each running different python version.

Outside of Python 2/3 differences, are Python interpreters not backwards compatible? In other words, while obviously a program written for 3.3 won't work in 2.7, but will a program written for 3.3 fail to run in 3.8? If it runs fine, why the need for multiple interpreters? I'd think you'd get by just fine by having the latest 2.x and 3.x installed.

Because underscore-functions aren't truly private, I have once seen an upgrade from 2.7.8 to 2.7.13 fail. A commonly-used package was importing one from a core python module.

Re: Overview of Python dependency management tools

#147

Post by anxq11 »

emmelaich wrote: post #99
pjs_ wrote: post #42

Anyone installs Conda on my shit I hit the roof... I sympathize with the individual but cannot tolerate the act.

Can you elaborate?

Perhaps I can guess some causes of the irritation. Let me start by saying that on Windows conda is probably an improvement.

On Linux, however, I do not see much benefit, unless you frequently install large binary C library based packages.

To me it feels cleaner to compile these packages from source. You are sure to have no glibc mismatches etc.

Conda, despite its advertising, does have library issues. C Libraries are shared between environments, compiling inside an environment can lead to surprising results when stale libraries are in the miniconda path.

All in all, it feels like a second OS shoehorned into the user's home directory. Compared to apt-get it is really slow and bloated.

It feels too intrusive on a Unix system.

Also, I'm not sure if the repositories are secured in any meaningful way.

Re: Overview of Python dependency management tools

#148

Post by nouveaux »

diarrhea wrote: post #52
nouveaux wrote: post #5

Earlier quoted context omitted.

I'm only familiar with Python, Javascript and Rust. It seems to me that Rust is the only one that has "solved" this problem. I dont think there are any real Python devs who thinks dependency management is solved. However, why would you claim Javascript has a good solution? The inconsistencies between node and web dev is odd at best. Babel compilation is annoying and slow. Are we even standardized on webpack yet? Can…

> As pioneers in dependency management Maybe a noob question, but how come they are pioneers? Weren't there languages with package systems before Python and JS?

Actually no. As diegof79 pointed out, Perl was the first one with a centralized package management system with CPAN. Perl was first released in 1987. Python was first released in 1990. Javascript was first released in 1995. The internet was the launch pad for these open source languages, which allowed the sharing of code written in these languages.

Another thing to note is that dependencies were not that complex in the 90s. The idea of dependency management started when there was too many dependencies to manage and the existing tools could not reliably reproduce builds. My guess is that all these package use took off after the dotcom boom in 2000 and everyone started building websites with Ruby, Python and eventually Nodejs.

Re: Overview of Python dependency management tools

#149

Post by gen220 »

infraredcabbage wrote: post #144
madelyn wrote: post #50

People always get up in arms about this, but as someone who has used Python as her daily driver for years it's really... never been this serious of an issue for me? I have used virtualenv/venv and pip to install dependencies for years and years, since I was a teen hacking around with Python. Packaging files with setup.py doesn't really seem that hard. I've published a few packages on pypi for my own personal use and…

This sort of attitude is the reason why the world doesn't move away from awful solutions. It is a testament to the lack of ability to see beyond your own nose. A lot of people who use Python, don't have the luxury of it being their "daily driver for years", so the conflicting documentation, decision paralysis and other problems that come with it end up being a huge time sink. A lot of non-programmers are being forced…

You're right of course.

Still, it's worth keeping in mind that Rust was born 20 years after Python was. Python was being written before Mosaic, Netscape, and Yahoo! were around. I think it can be forgiven for failing to conceive of a perfect package management system in 1990s. There were bigger fish to fry back then, so to speak.

Over the decades (!) there have been many, well-documented attempts at coming up with a package management story. pip and virtualenv have been the obvious winners here for years.

So, in conclusion, again you're right. But 30 years of history produces a lot of "conflicting documentation". It's only the last 10 years or so, that people have fought over the superiority of one language's package management ecosystem or another.

Re: Overview of Python dependency management tools

#150

Post by [deleted] »

madelyn wrote: post #50

People always get up in arms about this, but as someone who has used Python as her daily driver for years it's really... never been this serious of an issue for me? I have used virtualenv/venv and pip to install dependencies for years and years, since I was a teen hacking around with Python. Packaging files with setup.py doesn't really seem that hard. I've published a few packages on pypi for my own personal use and…

[deleted]
Post reply on HN