When this releases it will be crazy, Ive always wondered why something like this didn't already exist.
Really useful concept especially for school.
PYX: The next step in Python packaging
211–220 of 479 posts
Re: PYX: The next step in Python packaging
#212Earlier quoted context omitted.
Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…
I felt like python packaging was more or less fine, right up until pip started to warn me that I couldn't globally install packages anymore. So I need to make a billion venvs to install the same ml, plotting libraries and dependencies, that I don't want in a requirements.txt for the project. I just want packaging to fuck off and leave me alone. Changes here are always bad, because they're changes.
You can turn that off and allow global packages again if you want.
Or install it with the os package manager or something simmilar
Re: PYX: The next step in Python packaging
#213I'm brushing up with Python for a new job, and boy what a ride. Not because of the language itself but the tooling around packages. I'm coming from Go and TS/JS and while these two ecosystems have their own pros and cons, at least they are more or less straightforward to get onboarded (there are 1 or 2 tools you need to know about). In Python there are dozens of tools/concepts related to packaging: pip, easy_install,…
> In Python there are dozens of tools/concepts related to packaging: pip, easy_install, setuptools, setup.py, pypy, poetry, uv, venv, virtualenv, pipenv, wheels, Some of those are package tools, some are dependency managers, some are runtime environments, some are package formats... Some are obsolete at this point, and others by necessity cover different portions of programming language technologies. I guess what I'm…
You're right, it's not like there are actually 14 competing standards, but there are still too many—and that goes for Javascript as well.
Re: PYX: The next step in Python packaging
#214Earlier quoted context omitted.
Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…
I felt like python packaging was more or less fine, right up until pip started to warn me that I couldn't globally install packages anymore. So I need to make a billion venvs to install the same ml, plotting libraries and dependencies, that I don't want in a requirements.txt for the project. I just want packaging to fuck off and leave me alone. Changes here are always bad, because they're changes.
I'd otherwise agree but this problem seems unique to Python. I don't have problems like this with npm or composer or rubygems. Or at least very infrequently. It's almost every time I need to update dependencies or install on a new machine that the Python ecosystem decides I'm not worthy.
Re: PYX: The next step in Python packaging
#215Earlier quoted context omitted.
Thanks that’s bit less cryptic than the linked page. Still don’t get how they are solving what they claim to solve.
I suspect that, in order to succeed, they will need to build something that is isomorphic to Nix.
Yeah, and uv2nix is already pretty good! I wonder if pyx will be competing with uv2nix.
It's easy to compete with Nix tooling, but pretty hard to compete with the breadth of nixpkgs.
Re: PYX: The next step in Python packaging
#216All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.
Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…
Even the way you import packages is kinda wack
Re: PYX: The next step in Python packaging
#217I feel like I must be the crazy one for never having a problem with just vanilla pip, PyPi, and venv (or virtualenv for old Python 2 stuff). Maybe it's just my use case?
Re: PYX: The next step in Python packaging
#218Earlier quoted context omitted.
I share your concern but I have saved so much time with uv already that I figure ill ride it till the VC enshitification kills the host. Hopefully at the point the community is centralized enough to move in one direction.
I agree, now I just use uv and forget about it. It does use up a fair bit of disk, but disk is cheap and the bootstrapping time reduction makes working with python a pleasure again
I recently did the same at work, just converted all our pip stuff to use uv pip but otherwise no changes to the venv/requirements.txt workflow and everything just got much faster - it's a no-brainer.
But the increased resource usage is real. Now around 10% of our builds get OOM killed because the build container isn't provisioned big enough to handle uv's excessive memory usage. I've considered reducing the number of available threads to try throttle the non-deterministic allocation behavior, but that would presumably make it slower too, so instead we just click the re-run job button. Even with that manual intervention 10% of the time, it is so much faster than pip it's worth it.
Re: PYX: The next step in Python packaging
#219Earlier quoted context omitted.
Another piece is that PyPI has no index— it's just a giant list of URLs [1] where any required metadata (eg, the OS, python version, etc) is encoded in the filename. That makes it trivial to throw behind a CDN since it's all super static, but it has some important limitations: - there's no way to do an installation dry run without pre-downloading all the packages (to get their dep info) - there's no way to get hashes…
Your information is out of date. > there's no way to do an installation dry run without pre-downloading all the packages (to get their dep info) Not true for wheels; PyPI implements https://peps.python.org/pep-0658/ here. You can pre-download just the dependency info instead. For sdists, this is impossible until we can drop support for a bunch of older packages that don't follow modern standards (which is to say, inc…
Ah interesting, thanks for that! I was frustrated once again recently to note that `pip install --dry-run` required me to pre-download all packages, so I assumed nothing had changed.
Re: PYX: The next step in Python packaging
#220> There should be one-- and preferably only one --obvious way to do it.
It’s been pointed out to me before that:
the two ways of spacing the em dash on that quote is a joke about how it's not actually possible to do that? (And there's a third way in another line of the zen)
Do I buy it? Not sure. But apparently there’s more to this line than what it suggests.