[deleted]
Python: Please stop screwing over Linux distros
271–280 of 384 posts
Re: Python: Please stop screwing over Linux distros
#272Earlier quoted context omitted.
Not if they pull in all of their dependencies, PyQt would have a complete copy of all Qt binaries and a complete chrome install because of course Qt includes a browser based html viewer. Python packages are gigabytes.
PyQt only contains the bindings. You share the same Qt environment across your system (hence qmake needs to be in your path). The python package itself is not that big (~10 MB).
The comment on top of this chain is about letting every package specify its own versioned dependencies. So how would that global version work out when python needs 5.1 and some other software specifies 5.2?
Re: Python: Please stop screwing over Linux distros
#273Python libraries shipped by distributions are so old that this mechanism is mostly useless for python development. This also applies to many others programming languages which have their own packaging systems. While python packaging is indeed messy the needs of traditional linux distributions are by far the least important. Python packaging needs to better serve the needs of python developers, not those of sysadmins.…
Yes, exactly this. Only I view it from the other way round: to try to do development with distro-supplied language packages is a category error.
Distro-supplied interpreters and their associated libraries are there for the applications supplied and supported by the distribution. Unless you are developing something to be part of the distro, they are not for you.
Do not let the distro get between you and your libraries. Supply your own.
Re: Python: Please stop screwing over Linux distros
#274Earlier quoted context omitted.
Interesting; how would you isolate the system python from the user in Debian/Ubuntu. Do you have a link where I could learn more about this?
You may not be able to completely, but basically if users can write python scripts and run them using system python that's a side effect. It's purpose is to run system scripts, and any other use must not interfere or compete with that in any way. To be hones that wasn't always the case. In the early days Python was provided on these systems for users just as much as administrators and in the early days there weren't…
RHEL / CentOS 8 did this a few years ago. System scripts use a separate Python binary.
Re: Python: Please stop screwing over Linux distros
#275Earlier quoted context omitted.
And yet, other languages ship package managers [1] that are widely loved within their ecosystems and cover everything from microcontrollers to server applications. I think what it makes it more difficult in the case of Python is that it has decades of legacy to deal with. No consistent semantic versioning, packages that expect that they can modify their package path in-place (this is a nightmare for immutable systems…
Is it fair to compare an interpreted language and its package manager to Rust and Cargo? Python packages ship their source (in most cases), depend on a locally installed interpreter (with semantics possibly changing by version). Yes, Python packages often make poor assumptions about what setup.py can do (i.e., _anything_), and so you end up choosing between "tested, supported by the author, and old" or "untested, uns…
Cargo started as a port of Ruby's Bundler. If Ruby can do it, Python certainly can.
Re: Python: Please stop screwing over Linux distros
#276> What is it about Linux distros that makes our use-case unimportant? Have we offered no value to Python over the past 30 years? Indeed you haven't. Worse, you've actively damaged Python's efforts to improve. I mostly work on the JVM these days, and I think one of the main reasons dependency management there is so gloriously simple and effective is that the Debian packagers weren't around to fuck it up.
There is plenty of Java packages in APT.
Re: Python: Please stop screwing over Linux distros
#277Earlier quoted context omitted.
And yet, other languages ship package managers [1] that are widely loved within their ecosystems and cover everything from microcontrollers to server applications. I think what it makes it more difficult in the case of Python is that it has decades of legacy to deal with. No consistent semantic versioning, packages that expect that they can modify their package path in-place (this is a nightmare for immutable systems…
Worth noting that the PSF has no authority to tell people how to do their packaging. Rust's Cargo is 20 years newer than Python and benefits from those decades of experience. It's a very different proposition to start a new system from fresh than to try to migrate a huge and diverse community towards it.
This is actually where a BDFL should step in. "We're doing X for Python 5, we're using it to manage the stdlib as well as community packages, everyone get ready because I have decided."
Re: Python: Please stop screwing over Linux distros
#278Distros, please stop screwing over Python packaging. It is incredible that Debian/Ubuntu pick Python apart and put modules into different packages. They even create a fake venv command that tells you to please install python-venv. What they should just do is offer a bunch of packages like python3.7, python3.8 that install the official python package wholesale into /usr/python or someplace and then symlink one of them…
Sorry, but to be frank, I think you're a bit ignorant here. Let me explain, starting from the bottom:
> I would have one package manager that installs everything into a global package cache, […]
There is exactly one package manager. If you're on Debian or Ubuntu, it's dpkg. If you're on RedHat, it's rpm. If you're on Arch, it's pacman. Yes, some of the BSDs have two (base packages + ports tree), but they're the odd ones out.
pip, cargo, go*, etc. are not the same thing. I know they're called that but they don't perform the same function: none of them can create a working system installation. Let's call them module managers to have a distinct label.
> and then pick the correct version of libraries at run time (for Python: at import time).
That's easy for Python, and incredibly hard for a whole lot of other things. A module manager can do that. A package manager needs to work for a variety of code and ecosystems. Could it try to do it where possible? Maybe. But then the behavior is not uniform and made harder for users to understand. Could it still be worth it? Sure. But not obviously so.
I would also say that this is just giving up on trying to keep a reasonable ecosystem. It's not impossible to reduce the dependency hell that some things have devolved into. It just needs interest in doing so, and discipline while effecting it. I'd really prefer not giving up on this.
> Get rid of the requirement that there is only one stable (minor) version of a package in the distribution at one time. This has become unworkable.
This is to some degree why distros are breaking apart Python. Some bits are easy to install in parallel, some aren't. There can only be one "python". Worse, there can only be one "libpython3.9.so.1.0".
> Distros, please stop screwing over Python packaging. It is incredible that Debian/Ubuntu pick Python apart and put modules into different packages. They even create a fake venv command that tells you to please install python-venv.
They're trying to achieve the very goals you're describing. Trying to give you a working python without having to download and "install" some weird thing somewhere else. And at the same time trying to keep the module managers working when they're replacing some module but not all of them.
On a subjective level, it's obvious you have a strong distaste for this ("they even create a fake") — but could you please make objective arguments how and why this breaks things? If you're getting an incomplete Python installation, that seems like a packaging bug the distro needs to fix. Is that it? Or are there other issues?
> If I would get to redesign package management (both for Linux distros and for languages),
Well, and now we're here: https://xkcd.com/927/
And, I'm sorry to say this, but your post does not convey to me the existence of any essential C codebase packaging knowledge on your end. I don't know about other ecosystems, but I have done packaging work on C codebases (with Python bindings no less), and you don't seem to be aware of very basic issues like header/library mismatches and runtime portability.
If you are interested in this topic, please familiarize yourself with the world of existing package managers, the problems they run into, and how they solve them. There's a lot to learn there, and they're quite distinct from each other on some fronts too. Some problems are still unsolved even.
Re: Python: Please stop screwing over Linux distros
#279As a "self taught and still learning developer-lite", I love the python language, but the ecosystem drives me nuts. I feel a lot of the pain expressed in the article, and it pretty much speaks to my current conclusion of "I'm trying to do things the 'right way' but there doesn't seem to be a 'right way'". I've seen a few comments here about how Nix/NixOS fixes the whole python binary/library mess, but I'm having trou…
> I've seen a few comments here about how Nix/NixOS fixes the whole python binary/library mess Nix is a bit of a cult. Its theoretical aims are laudable, but in order to get there it forces you to do a lot of work and reason strictly in its own way. Whether all this work is worth the rewards, I think is open for debate.
Chemistry is a bit of a cult. Its theoretical aims are laudable, but in order to get there it forces you to do a lot of work and reason strictly in its own way. Whether all this work is worth the rewards, I think is open for debate.
Re: Python: Please stop screwing over Linux distros
#280Yes, python packaging is a mess. And agreed, there are two separate use cases: development and using the software. But, are distros creating too much work for themselves by trying to package every itty-bitty python library (and for that matter, every npm library)? Are distros doing anything more than scanning CVE databases with the library versions, or are they _actually_ auditing the versions they choose? (Not that…
> Are distros doing anything more than scanning CVE databases with the library versions, or are they _actually_ auditing the versions they choose? Debian Developer here. Part of packaging work, for Python libraries or anything else, is to verify the reliability of the upstream developers, audit the code, set hardening flags, add sandboxing and so on. I spotted and reported vulnerability myself and it's not uncommon.
Thanks for your work. And I have to say that for internal development, when there is no need for the latest features, it's much easier to develop based on a Debian release as much as possible. A stable distribution provides an easy to track baseline not only for Python libraries, but any other tools that may be needed.