Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PYX: The next step in Python packaging

astral.sh

Post reply on HN
Print view View original

221–230 of 479 posts

Re: PYX: The next step in Python packaging

#221

Post by [deleted] »

bastawhiz wrote: post #214
anothernewdude wrote: post #209

Earlier quoted context omitted.

I felt like python packaging was more or less fine, right up until pip started to warn me that I couldn't globally install packages anymore. So I need to make a billion venvs to install the same ml, plotting libraries and dependencies, that I don't want in a requirements.txt for the project. I just want packaging to fuck off and leave me alone. Changes here are always bad, because they're changes.

I'd otherwise agree but this problem seems unique to Python. I don't have problems like this with npm or composer or rubygems. Or at least very infrequently. It's almost every time I need to update dependencies or install on a new machine that the Python ecosystem decides I'm not worthy.

[deleted]

Re: PYX: The next step in Python packaging

#222

Post by frollogaston »

nemomarx wrote: post #187
JonChesterfield wrote: post #178

Earlier quoted context omitted.

I've been dealing with python vs debian for the last three hours and am deeply angry with the ecosystem. Solved it is not. Debian decided you should use venv for everything. But when packages are installed in a venv, random cmake nonsense does not find them. There are apt-get level packages, some things find those, others do not. Names are not consistent. There's a thing called pipx which my console recommended for m…

from what I hear uv is the "solved" and venv by hand is the old way

pip is the default still

Re: PYX: The next step in Python packaging

#223

Post by thayne »

o11c wrote: post #28
woodruffw wrote: post #19

Earlier quoted context omitted.

For what it's worth, I understand this concern. However, I want to emphasize that pyx is intentionally distinct from Astral's tools. From the announcement post: > Beyond the product itself, pyx is also an instantiation of our strategy: our tools remain free, open source, and permissively licensed — forever. Nothing changes there. Instead, we'll offer paid, hosted services that represent the "natural next thing you ne…

The entire reason people choose "permissive licenses" is so that it won't last forever. At best, the community can fork the old version without any future features. Only viral licenses are forever.

Or they want to get more people to use it.

Re: PYX: The next step in Python packaging

#224

Post by Eduard »

bastawhiz wrote: post #204
runningmike wrote: post #103

All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.

Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…

I started with "sudo apt install python" a long time ago and this installed python2. This was during the decades-long transition from python2 to python3, so half the programs didn't work so I installed python3 via "sudo apt install python3". Of course now I had to switch between python2 and python3 depending on the program I wanted to run, that's why Debian/Ubuntu had "sudo update-alternatives --config python" for managing the symlink for "python" to either python2 or python3. But shortly after that, python3-based applications also didn't want to start with python3, because apt installed python3.4, but Python developers want to use the latest new features offered by python3.5 . Luckily, Debian/Ubuntu provided python3.5 in their backports/updates repositories. So for a couple of weeks things sort of worked, but then python3.7 was released, which definitely was too fresh for being offered in the OS distribution repositories, but thanks to the deadsnakes PPA, I could obtain a fourth-party build by fiddling with some PPA commands or adding some entries of debatable provenance to /etc/apt/lists.conf. So now I could get python3.7 via "sudo apt install python3.7". All went well again. Until some time later when I updated Home Assistant to its latest monthly release, which broke my installation, because the Home Assistant devs love the latest python3.8 features. And because python3.8 wasn't provided anymore in the deadsnakes PPA for my Ubuntu version, I had to look for a new alternative. Building python from source never worked, but thank heavens there is this new thing called pyenv (cf. pyenv), and with some luck as well as spending a weekend for understanding the differences between pyenv, pyvenv, venv, virtualenv (a.k.a. python-virtualenv), and pyenv-virtualenv, Home Assistant started up again.

This wall of text is an abridged excursion of my installing-python-on-Linux experience.

There is also my installing-python-on-Windows experience, which includes: official installer (exe or msi?) from python.org; some Windows-provided system application python, installable by setting a checkbox in Windows's system properties; NuGet, winget, Microsoft Store Python; WSL, WSL2; anaconda, conda, miniconda; WinPython...

Re: PYX: The next step in Python packaging

#225

Post by voicedYoda »

bastawhiz wrote: post #204
runningmike wrote: post #103

All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.

Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…

You forgot the wheels and eggs

Re: PYX: The next step in Python packaging

#226

Post by normerdfe »

6thbit wrote: post #127
notatallshaw wrote: post #2

Probably the more useful blog post: https://astral.sh/blog/introducing-pyx

Thanks that’s bit less cryptic than the linked page. Still don’t get how they are solving what they claim to solve.

I spend little time with Python, but I didn’t have any problems using uv. Given how great uv is, I’d like to use pyx, but first it would be good if they could provide a solid argument for using it.

Re: PYX: The next step in Python packaging

#227

Post by busterarm »

int_19h wrote: post #180
nickserv wrote: post #60

Earlier quoted context omitted.

The speed of Ruby with the memory management of Java, what's not to love? Also, now you have two problems.

Java has quite possibly the best production GC of all the VMs out there.

And yet operating distributed systems built on it is a world of pain. Elasticsearch, I am looking at you. Modern hardware resources leave the limitations of running/scaling on top JVM to be an expensive, frustrating endeavor.

In addition to elasticsearch's metrics, there's like 4 JVM metrics I have to watch constantly on all my clusters to make sure the JVM and its GC is happy.

Re: PYX: The next step in Python packaging

#228

Post by typpilol »

ghshephard wrote: post #206
nemomarx wrote: post #187

Earlier quoted context omitted.

from what I hear uv is the "solved" and venv by hand is the old way

uv is venv + insanely fast pip. I’ve used it every day for 5+ months and I still stare in amazement every time I use it. It’s probably the most joy I’ve ever gotten out of technology.

Installing packages it the most joy you've ever gotten outta tech?

Not a project you built, or something you're proud of? Installing packages?

Re: PYX: The next step in Python packaging

#229

Post by 6thbit »

woodruffw wrote: post #13
TheChaplain wrote: post #6

Pyx is just a registry, just like Pypi, or did I misunderstood it?

Not exactly -- part of pyx is a registry (and that part speaks the same standards as PyPI), but the bigger picture is that pyx part of a larger effort to make Python packaging faster and more cohesive for developers. To be precise: pyx isn't intended to be a public registry or a free service; it's something Astral will be selling. It'll support private packages and corporate use cases that are (reasonably IMO) beyond…

  > but the bigger picture is that pyx part of a larger effort to make Python packaging faster and more cohesive for developers
Can you elaborate on what will make it faster and cohesive?

Re: PYX: The next step in Python packaging

#230

Post by zahlman »

notatallshaw wrote: post #92
zahlman wrote: post #74

Earlier quoted context omitted.

Pyx represents the server side, not the client side. The analogue in the pre-existing Python world is PyPI. Many ideas are being added to recent versions of pip that are at least inspired by what uv has done — and many things are possible in uv specifically because of community-wide standards development that also benefits pip. However, pip has some really gnarly internal infrastructure that prevents it from taking a…

> However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original). FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. I'm not saying here that pip doesn't have…

Good to see you again.

> FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.

Fair enough. I'm sure if someone were paying me a competitive salary to develop my projects, they'd be getting done much faster, too.

> I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy.

I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files.

> uv's ability to prefetch package metadata,

You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays.

> I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver.

Ugh, yeah. I know the resolution logic has been extracted as a specific package, but it's been challenging trying to figure out how to actually use that in a project that isn't pip.

Post reply on HN