161–170 of 384 posts
we need one, only one build system, not many. Feels like atm python is a bit shattered around like c++ Using JS or rust seems easier with the unified build system.
Is your unified JavaScript build system using NPM or Yarn? ;)
Earlier quoted context omitted.
And yet, other languages ship package managers [1] that are widely loved within their ecosystems and cover everything from microcontrollers to server applications. I think what it makes it more difficult in the case of Python is that it has decades of legacy to deal with. No consistent semantic versioning, packages that expect that they can modify their package path in-place (this is a nightmare for immutable systems…
Rust has and “advantage” here that it’s not generally shipped with your distro’s package manager. I think my biggest problem with this article is that the distros put Python there in the first place, and all of them apply patches to make Python work how they want it. And when it doesn’t work, it’s Python’s fault…? I mean yeah Python can definitely do something to make distributing Python easier, but it can only do so…
Available Packages
Name : rust
Version : 1.56.1
Release : 1.fc34Can someone make the case for distributing python packages at the distro level to me? Especially scientific software for data analysis? Our project gets a few issues opened by distro maintainers who have trouble with some part of their build process. Is it really worth our project maintainers time to help troubleshoot esoteric build processes when we already provide source, wheel, and conda distributions?
As a matter of principle, I prefer to avoid software from non-curated repositories like pip and the like. Installing the debian-provided scipy and numpy is more than enough for me.
I don't understand what's so special about python that needs its own "package manager" when the distro-provided one is already good. If I need something, I install it using "apt install", regardless of what language it is written in.
Earlier quoted context omitted.
Your last sentence doesn't parse for me at all. How exactly do you get a pip installed on Python 3.6 to modify anything about another install of Python, let alone a different version?
Apt install python-pip3 You'll get pip3 from Python 3.6. apt install python3.8 You'll get system-level python3. Use pip3 in 3.6 to install the latest pip3 that requires Python 3.8, and drops that package in the dist-packages folder, which is shared between system Python runtime versions. Behold as further attempts to do something sane with APT blow up, because now you have 3.8 packages on 3.6's runtime path. I've los…
I mean... I understand how hypothetically this could in theory make some sense because a distro might naively expect to ship the same package version for different Python versions and so all files could be the same except for native extensions, which are disambiguated by implementation name, so theoretically one could imagine a position where this could be conceived of as a thing one could do to save some space in case someone installs the same package for multiple Python versions.
But in reality? Jesus fucking christ. However, I don't see how this can be blamed on Python. This isn't something that "just happens because Python" and is certainly far, far away from any kind of default Python configuration, where all default paths are below the prefix, which contains the Python version. This is something that can only happen because someone went extraordinarily far out of their way to shoot someones else's foot off.
I've seen a few comments here about how Nix/NixOS fixes the whole python binary/library mess, but I'm having trouble understanding how. Does anyone have any insight to share about that?
Additionally, the whole thing kind of makes me want to move away from python wholesale. I was wondering if there are other languages that are great general languages like python that don't suffer from this whole packaging and versioning mess. Ruby? Go? Something else? I'm looking for something high level, somewhat easy to learn, and with good library support for things like working with databases and tabular data. Though I don't know much about them, I just feel like I don't want something like Java or C++ or anything like that. I want to "get things done" and not have to worry about tons of boilerplate or working at really low nitty gritty levels.
The day packaging for distros will be easy, we will use it. Right now, making a deb is hard. Isolating 2 projects with different deb verions is hard. Distributing debs is hard. Upgrading your OS but not your python debs is hards. Then rinse and repeat for red hat, arch, nix, mac and windows ? Yeah right.
Since Nix packages are distribution-independent, once you have it packaged with Nix, you could theoretically skip packaging for Ubuntu etc., but of course that may raise the bar of entry for your users.
Also, unless you repackage thousands of compiled c extensions and you play well with anaconda and can plug into the entire python ecosystem of platforms such as heroku, python anywhere, databrick and so on, you then lose 90%
Distros, please stop screwing over Python packaging. It is incredible that Debian/Ubuntu pick Python apart and put modules into different packages. They even create a fake venv command that tells you to please install python-venv. What they should just do is offer a bunch of packages like python3.7, python3.8 that install the official python package wholesale into /usr/python or someplace and then symlink one of them…
However, I thought the point was helping distro package management, which, to my knowledge, is not really built to support multiple installed versions of a package at the time: `dnf upgrade` for example, will upgrade all single instances of each packages to their newer release.
Earlier quoted context omitted.
> Get rid of the requirement that there is only one stable (minor) version of a package in the distribution at one time. I think this requirement made sense when disk space was scarce. I think this requirement makes sense if you trust that your distro is always better at choosing the 'best' version of a dependency that some software should use than the software author. Nowadays, I think neither is generally true. Dis…
It's not just about disk space, though. Distros assume responsibility for fixing major bugs and security vulnerabilities in the packages they ship. Old versions often contain bugs and vulnerabilities that new versions don't. Distros have two choices here: either ship the new version and remove the old version, or backport the fix to the old version. Continuing to ship the old version without the fix is not an option…
I think they should just ship Python programs, not libraries. They could check what libraries given Python program uses are safe in the version that it uses them.
And just don't care if each of Python programs has a separate copy of the libraries or if particular version of particular library is shared between Python programs by Python environment.
Distributions might just give up responsibility for sharing Python packages between Python programs without giving up the responsibility for security of those programs.
Earlier quoted context omitted.
> I haven't found an instance where python3-X and python3-Y can't be installed because one requires python3-Z v1.2 and the other python3-Z v2.1. With Python packages, that happens way more often. Exactly, because the packages were created/updated at a point in time were they were compatible: a distro ships python 3.x with packages of python modules that are compatible with python 3.x. Now if you as a user want to use…
> Again, if all python package were up to date as in "working together at this point in time" then `pip upgrade` would work. That is impossible, it will never happen. Packages will have different maintenance cycles, some will get deprecated, others abandoned... You can't base your upgrade policy on an impossible situation. > Are you sure you checked in the `*.dist-info` directories? There should be one per package, c…
You realize that if that was "impossible" and "never happening", then absolutely no Python environment would be working ever?
> A version conflict should be reported as an installation failure, not allow you to continue.
Please don't make this mandatory.
> Pip will install Y >= 2.0 and won't care that package X is now broken.
Fine, I'll just quickly fix X and open a pull request, like I probably did a hundred times, then eventually if necessary deploy my fork with the fix meanwhile they do their maintenance release.
Should we not take the responsibility of the dependencies we use and contribute back??
> The "live at master" philosophy only works for small groups of similar output capacity. It won't work for an ecosystem as wide as Python.
I don't understand why, but I'm talking about "live at latest release", not "at master".
> Sometimes it will be necessary, such as for example a package dropping support for a feature that another one needs.
Then the package can just paste the code of the feature in their own, until a new lib does it, I remember having to do that twice in 20 years (except I didn't just "paste" it, but implemented a much smaller version).
Overall, it seems my approach produces versions of my software and software that I depend on that are compatible with all versions because you can always use an earlier version if you really want to deploy on an old python or whatnot, whereas you approach leads to broken packages, tech debt, and blaming the package manager.
Earlier quoted context omitted.
I'll remember that "Homebrew does this very well" the next time I have to fix a bunch of shit because it has updated the currently-active version, or removed this or that bugfix release, as part of a general upgrade. After the third time this happened, I started using pyenv - which is another mountain of brokenness, I grant you, but at least I have some degree of control on what happens when.
I'm using Anaconda, it's so far hands down the best way to manage python and its packages. It just works.
Agreed, especially on Windows.
It just works.
This is pushing it. It's not hard to break conda or put yourself in situation where the updater/dependency checker gets stuck and doesn't know what to do, especially once you start adding conda-forge packages. But it does do a better job than anything else I've tried (although poetry + pyenv on Linux is getting much better)