Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python: Please stop screwing over Linux distros

drewdevault.com

Post reply on HN
Print view View original

201–210 of 384 posts

Re: Python: Please stop screwing over Linux distros

#201

Post by rolandog »

simonh wrote: post #129
cjfd wrote: post #97

Earlier quoted context omitted.

Perhaps the distributions would be more inclined to include up-to-date version if the standard in the python community was not to break everything all the time. There are distributions that keep up-to-date, though, e.g., archlinux. Serving the needs of python developers vs. sysadmins is a false dichotomy. Python developers develop on a system that they need to admin. One great thing about linux is that everything on…

>Serving the needs of python developers vs. sysadmins is a false dichotomy. Python developers develop on a system that they need to admin. I couldn't disagree more. Even if the same person is admining a system they develop on, they almost certainly aren't going to admin the systems their users deploy on. The admin role and developer role should be completely separate with different goals and requirements. The system…

Interesting; how would you isolate the system python from the user in Debian/Ubuntu. Do you have a link where I could learn more about this?

Re: Python: Please stop screwing over Linux distros

#202

Post by toyg »

ajsnigrutin wrote: post #183
toyg wrote: post #134

Earlier quoted context omitted.

> Perl is a godsend... take a code from 20 years ago, run it on a modern system, and everything works. That's more or less like "take a VB6 binary from 20 years ago, run it on some modern Windows, and everything works" - that's just because the ecosystem is effectively dead, so supporting it on new releases just means carrying over some stuff that worked 20 years ago.

But it works... it's universally supported on pretty much every os, even by default on most *nix based ones, and new versions and new CPAN modules are still written. In 10 years, with python 4, or even maybe 5, everything will still be broken, and you'll still be claiming perl is dead, and i'll still be using the same stuff I use now, that worked years ago, works now, and will work then.

And people will still be shipping code faster and more efficiently than they would do in Perl. Because in the end, the advantages of using Python, in terms of readability and productivity, easily offset a bit of packaging pain - whereas the disadvantages of using Perl don't offset whatever marginal gain you get by using old infrastructure.

Re: Python: Please stop screwing over Linux distros

#205

Post by toyg »

thinker5555 wrote: post #165

As a "self taught and still learning developer-lite", I love the python language, but the ecosystem drives me nuts. I feel a lot of the pain expressed in the article, and it pretty much speaks to my current conclusion of "I'm trying to do things the 'right way' but there doesn't seem to be a 'right way'". I've seen a few comments here about how Nix/NixOS fixes the whole python binary/library mess, but I'm having trou…

> I've seen a few comments here about how Nix/NixOS fixes the whole python binary/library mess

Nix is a bit of a cult. Its theoretical aims are laudable, but in order to get there it forces you to do a lot of work and reason strictly in its own way. Whether all this work is worth the rewards, I think is open for debate.

Re: Python: Please stop screwing over Linux distros

#206

Post by nerdponx »

AnIdiotOnTheNet wrote: post #199
nerdponx wrote: post #92

Earlier quoted context omitted.

Homebrew does this very well with its "cellar" system. Every version of every package gets installed to its own root tree, eg `/usr/local/Cellar/python/3.9.7/`. The currently-active version is then symlinked into `/usr/local/opt/python` and from there into `/usr/local`. I believe Nix and Guix also work this way.

IIRC GoboLinux was the first distro to do things this way. Sadly, it didn't catch on and the Linux world doubled down on labor intensive volunteer package maintenance.

Great shout out. I still ought to try using it one of these days! It seems like a good option for people who want a better file system hierarchy without the extra complexity of Nix/Guix.

Re: Python: Please stop screwing over Linux distros

#207

Post by pwang »

pfranz wrote: post #107
WhyNotHugo wrote: post #47

Earlier quoted context omitted.

> Use pip install in user mode This is a great recipe for disaster. Whatever you install in user mode will shadow anything installed system-wide, so when you try to run some system-wide project, it may now fail. I'm also not a fan of how it drops scripts into `./.local/bin`, since that's where I keep my own script, and is version controlled. The installation will also be frozen and never get updated -- unless you rem…

So you're suggesting always using virtualenv? I used to just use pip to install to the system. Months/years later I would try to untangle the mess of packages I was just playing with, what the OS wanted/needed, I got those conflicting dependencies you mention, etc. I usually ended up reinstalling the OS. At the time I may not have been as knowledgeable about where the OS package manager keeps packages vs pip--but the…

This is one of the reasons people use Anaconda/miniconda for non-data science work: conda environments are self-contained Python installs, so if you conda/pip install packages into those environments, they will not break each other. This design requirement arose from the specific needs of numerical computing (which always drags in a ton of system-level C/C++/FORTRAN dependencies), but is a generically useful design construct.

Anaconda is a distro, and conda is a package manager, that works across OS platforms and hardware architectures, and installs cleanly into userland without requiring admin privileges. The only way we achieve this difficult goal is by creating a distro and build system that creates "portable" packages that can be relocated/relinked at install-time.

Ultimately, Python's challenges in this department come from the fact that it has such great integration with low-level C/C++ libraries. This gives it super powers as duct tape/glue language, but it also drags it down into the packaging tech debt of C/C++. Hmm... maybe I should write that blog post: "Python Packaging Isn't The Problem; C/C++ Is." :-)

Re: Python: Please stop screwing over Linux distros

#208

Post by michaelt »

ivirshup wrote: post #19

Can someone make the case for distributing python packages at the distro level to me? Especially scientific software for data analysis? Our project gets a few issues opened by distro maintainers who have trouble with some part of their build process. Is it really worth our project maintainers time to help troubleshoot esoteric build processes when we already provide source, wheel, and conda distributions?

The main reason to distribute python packages at the distro level is: Because something else distributed at the distro level depends on them.

For example, if gnuradio requires numpy then a distro that won't distribute numpy can't distribute gnuradio

Re: Python: Please stop screwing over Linux distros

#209

Post by pwang »

nerdponx wrote: post #27
withinboredom wrote: post #13

Earlier quoted context omitted.

I’m not a Python dev. I do need to occasionally run things written in Python. I made the mistake of trying to get pip + Conda + pyenv (or whatever) to install a fairly simple tool. I have no idea how the dev got their setup working, but it was totally and utterly unreproducable, even after they sat down on my computer for several hours. In that amount of time, I could have probably rewritten it in PHP (that’s actuall…

This is especially frustrating to read because one of the main selling points of Conda is reproducibility. In data science teams, I've found it indispensable for making sure people can all run each others' project code. So for anyone reading this in the future: don't try to use Pyenv to install Conda. Pyenv tries to set up shims for every binary in the Conda env, which will likely break your PATH. Pyenv supports inst…

Thanks for the feedback... I had no idea that pyenv was hosing conda envs. Do you have more details on this brokenness?

Re: Python: Please stop screwing over Linux distros

#210

Post by dagw »

pwang wrote: post #198
dagw wrote: post #170

Earlier quoted context omitted.

hands down the best way to manage python and its packages. Agreed, especially on Windows. It just works. This is pushing it. It's not hard to break conda or put yourself in situation where the updater/dependency checker gets stuck and doesn't know what to do, especially once you start adding conda-forge packages. But it does do a better job than anything else I've tried (although poetry + pyenv on Linux is getting mu…

Thanks! FWIW, we are soon going to be releasing a much faster depedency resolver. We are also thinking hard about how best to address the "growing ecosystem" problem, in a future-proof way.

FWIW, we are soon going to be releasing a much faster depedency resolver

Fantastic. Easily my biggest complaint about conda.

Post reply on HN