Python: Please stop screwing over Linux distros

The scale and quantity of software being developed is just too much for traditional Linux distributions approaches. I personally rely on Debian to know that most of the packages I use are reasonably secure. But often I need to sacrifice flexibility and bleeding-edgedness.

And it couldn't be otherwise. There are 339,267 projects on PyPi.org. I bet a good percentage of these are riddled with security flaws. But people use them anyway. How many distro maintainers would you need to handle this workload? Is it worth it? Does anyone care?

It looks like people care much more about experimentation and speed of development than stability, security or coherence and cleanliness of the solution. The author seems to feel this is wrong from an engineering (or even moral?) perspective.

I am also uncomfortable when I have to deal with Python and I have my own favourite few tools... but perhaps if Python users really wanted a single solution then it would already exist.

If you are instead convinced that there is this need and no adequate solution, then congratulations, you just found a gap in the market. Go on and do better than everyone else before you. Relevant XKCD comic is already in the article...

I think this requirement made sense when disk space was scarce.

I think this requirement makes sense if you trust that your distro is always better at choosing the 'best' version of a dependency that some software should use than the software author.

Nowadays, I think neither is generally true. Disk is plentiful, distro packages are almost always far more out of date than the software's original source, and allowing authors to ship software with specific pinned dependency versions reduces bugs caused by dependency changes and makes providing support for software (and especially reproducing end-user's issues) significantly easier.

Isolating dependencies by application, with linking to avoid outright duplication of identical versions (a la pnpm's approach for JS: https://pnpm.io/) is the way to go I think. Honestly, it feels like the way it's already gone, and it's just that the distros are fighting it tooth & nail whilst that approach takes over regardless.

There are distributions that keep up-to-date, though, e.g., archlinux.

Serving the needs of python developers vs. sysadmins is a false dichotomy. Python developers develop on a system that they need to admin. One great thing about linux is that everything on a system can be kept up-to-date using just one software tool (a package manager). You are not going to convince me in a million years that this is a bad idea.

Now, for things that are needed in development you sometimes need different versions, in particular if you happen to have an OS that ships very old versions. For that purpose there should be some sort of tool and not a gazillion tools that are all incompatible and behave in slightly (or not so slightly) different ways. 'python -m venv' is different from 'virtualenv'? WTF?

Also, if your distribution is up-to-date (e.g., archlinux) and you are pinning to older version there is also a problem. As the article puts it 'pin their dependencies to 10 versions and 6 vulnerabilities ago'. And if you actually want to maintain your software in the future you, at some point, have to go to the new version anyway.

Yes, that is in the context of a giant and very successful team using this approach, believe or not. :)