Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python: Please stop screwing over Linux distros

drewdevault.com

Post reply on HN
Print view View original

261–270 of 384 posts

Re: Python: Please stop screwing over Linux distros

#261

Post by BiteCode_dev »

nerdponx wrote: post #12

> Every one of these package managers is designed for a reckless world in which programmers chuck packages wholesale into ~/.pip, set up virtualenvs and pin their dependencies to 10 versions and 6 vulnerabilities ago, and ship their computers directly into production in Docker containers which aim to do the minimum amount necessary to make their user’s private data as insecure as possible. Is this any different from…

There are some differences indeed:

- python is more active than most alternatives, you have new packages created every day.

- python is massively used outside of the web, unlike JS, ruby or PHP, that are 99% web. You get Python in SIG softwares, data analytics, automation, pen testing, sysadmin, biology, etc. It's a huge graph.

- python is used by the distro themselves to code features of the OS. E.G: you remove Python, there is no yum.

- python has a rich compiled extensions ecosystem, produced from c, c++, fortran, and assembly. It's very complicated to ship them.

- it's much more common to have several Python installed than for other dynamic languages. So isolation matters even more.

So the difference is the sheer size of the problem.

Re: Python: Please stop screwing over Linux distros

#262

Post by southerntofu »

justin66 wrote: post #245
toyg wrote: post #130

Earlier quoted context omitted.

> That's because there is one obvious distribution of Python to use Lol, no. There is one in the Windows appstore too, which has issues with privileges. And I'm almost sure now MS ships it in some other tool too.

I wonder what percentage of the six people who use Windows app store have found that situation confusing.

You may be surprised outside of the HN crowd how many people use that crappy app store.

Re: Python: Please stop screwing over Linux distros

#263

Post by zaptheimpaler »

If i understand correctly, this is the kind of problem Flatpak and Snap can solve. As a developer, relying on a distro's provided packages just seems too cumbersome - there are 10s of distros all with different versions of your dependency and different packaging systems. I guess the only way to distribute a Python application today that works across distro would be to basically build a single application that works simultaneously on many different python versions, not to mention the shared C libraries and other system software used by important Python packages. What do you do if you use psycopg2 X that needs libpq-dev version between 1 and 2 but the distro bundles in an incompatible version of libpq-dev Y? You have to now support 2 versions of libpq-dev across the project, and all dependents of libpq-dev such as psycopg2?

The shared dependency model is just too complex to work with, and we have enough disk space that its not really necessary anymore. Sandboxes seem like the way to go.

Re: Python: Please stop screwing over Linux distros

#264

Post by nerdponx »

drunkpotato wrote: post #193
toyg wrote: post #127

Earlier quoted context omitted.

I'll remember that "Homebrew does this very well" the next time I have to fix a bunch of shit because it has updated the currently-active version, or removed this or that bugfix release, as part of a general upgrade. After the third time this happened, I started using pyenv - which is another mountain of brokenness, I grant you, but at least I have some degree of control on what happens when.

pyenv is pretty good for working around this problem. I've recently switched to asdf-vm which I like even more, since it handles versions for multiple languages and tools.

I have been meaning to try out ASDF-VM. Currently my shell initialization script has at least 4 of these "version managers". While I don't really mind them (and they are mostly well-behaved), it might be nice to have something a bit more centralized.

Re: Python: Please stop screwing over Linux distros

#266

Post by nerdponx »

BiteCode_dev wrote: post #261
nerdponx wrote: post #12

> Every one of these package managers is designed for a reckless world in which programmers chuck packages wholesale into ~/.pip, set up virtualenvs and pin their dependencies to 10 versions and 6 vulnerabilities ago, and ship their computers directly into production in Docker containers which aim to do the minimum amount necessary to make their user’s private data as insecure as possible. Is this any different from…

There are some differences indeed: - python is more active than most alternatives, you have new packages created every day. - python is massively used outside of the web, unlike JS, ruby or PHP, that are 99% web. You get Python in SIG softwares, data analytics, automation, pen testing, sysadmin, biology, etc. It's a huge graph. - python is used by the distro themselves to code features of the OS. E.G: you remove Pyth…

That's a good analysis, thanks.

But I don't think any of it justifies the ire towards Python, its community, or its devs that was expressed in the blog post.

I also do want to point out that there are quite a lot of general-purpose CLI tools written in Perl, Ruby, and Node.

Re: Python: Please stop screwing over Linux distros

#267

Post by thereddaikon »

josefx wrote: post #237
silvestrov wrote: post #195

Earlier quoted context omitted.

> cost-per-TB There you have it: You measure in TB, not gigabytes, not megabytes. Python packages are megabytes.

Not if they pull in all of their dependencies, PyQt would have a complete copy of all Qt binaries and a complete chrome install because of course Qt includes a browser based html viewer. Python packages are gigabytes.

Fortunately there are a thousand gigabytes in a terabyte and nobody is using a system with single or double digit gigs of storage anymore.

Unless you are trying to save a buck it seems 1TB is the standard today. My primary desktop has 4.

Re: Python: Please stop screwing over Linux distros

#268

Post by 0xbadcafebee »

Python's trend of "reinvent it yourself" engineering leads inexorably to a sprawling mess of duplicated and incompatible projects. The source of this problem is Python's lack of "lead by example" on the topic of DRY code / implementations.

With Perl modules (like Python packages), the intent is to never have to rewrite them, but to make them so generic that they are just extended by new modules (that inherit the old modules) to gain more features. Their naming convention reinforces this intention, with "Net.pm", "Net/IMAP.pm", "Net/IMAP/SSL.pm", etc. Each one of those files can be uploaded to CPAN by a completely different developer, without having to completely re-write all the base stuff from scratch. You can already do that with Python, but because everyone names their packages "fizzbaggy", "unclib2", "OtherThingHere", etc, there's no sane convention that clearly tells you what this thing is, what it does, or what it depends on.

The end result of all that is a nightmare in terms of regular users figuring out how to install and use most Python scripts, packages, and tools. This is part of why Go is so popular now: no need for an engineering degree just to run a Python program!

I do not believe there is any way for Python to re-invent itself and suddenly become less sprawling or confusing. The shift would be too huge and take too long.

Re: Python: Please stop screwing over Linux distros

#269

Post by dagw »

rolandog wrote: post #201
simonh wrote: post #129

Earlier quoted context omitted.

>Serving the needs of python developers vs. sysadmins is a false dichotomy. Python developers develop on a system that they need to admin. I couldn't disagree more. Even if the same person is admining a system they develop on, they almost certainly aren't going to admin the systems their users deploy on. The admin role and developer role should be completely separate with different goals and requirements. The system…

Interesting; how would you isolate the system python from the user in Debian/Ubuntu. Do you have a link where I could learn more about this?

I would start by putting it in a path that won't be included in the users default $PATH, perhaps /sbin or /usr/sbin

Re: Python: Please stop screwing over Linux distros

#270

Post by thetwentyone »

microtonal wrote: post #48
mangecoeur wrote: post #9

Everyone complained, no one addressed the fact that for years the whole of python packaging was handled by like 2.5 people. But as a rant this, like most of the ‘but just fix it’ rants, fails to acknowledge the hugely diverging needs of different users. I could not live without conda, since it’s the only sane way to get a working recent geospatial stack. Others need to run embedded environments, or portable ones, som…

And yet, other languages ship package managers [1] that are widely loved within their ecosystems and cover everything from microcontrollers to server applications. I think what it makes it more difficult in the case of Python is that it has decades of legacy to deal with. No consistent semantic versioning, packages that expect that they can modify their package path in-place (this is a nightmare for immutable systems…

And Julia's Pkg! One of my primary reasons for switching away from Python to Julia was how ergonomic modern dependency management was.
Post reply on HN