Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How uv got so fast

nesbitt.io

Post reply on HN
Print view View original

281–290 of 468 posts

Re: How uv got so fast

#281

Post by pnt12 »

zahlman wrote: post #75
blintz wrote: post #4

> PEP 658 went live on PyPI in May 2023. uv launched in February 2024. The timing isn’t coincidental. uv could be fast because the ecosystem finally had the infrastructure to support it. A tool like uv couldn’t have shipped in 2020. The standards weren’t there yet. How/why did the package maintainers start using all these improvements? Some of them sound like a bunch of work, and getting a package ecosystem to move i…

> If setup.py was working okay for folks, what incentivized them to start using pyproject.toml? It wasn't working okay for many people, and many others haven't started using pyproject.toml. For what I consider the most egregious example: Requests is one of the most popular libraries, under the PSF's official umbrella, which uses only Python code and thus doesn't even need to be "built" in a meaningful sense. It has a…

That's really unfortunate, and it sounds like a quick thing to fix. Is there a pull request with that?

Re: How uv got so fast

#282

Post by pnt12 »

stavros wrote: post #119
quectophoton wrote: post #104

Earlier quoted context omitted.

I wouldn't say without fear, since you're one typo away from executing a typo-squatted malicious package. I do use it on CI/CD pipelines, but I wouldn't dare type uvx commands myself on a daily basis.

uvx isn't more risky than `pip install`, which is what I used before.

But with pip you only need to be careful on install - with uvx you need to be careful forever.

I'm a big fan of uv, but don't like that part of uvx.

(makes me wonder if a small wrapper can do this - safe uvx, or suvx for short)

Re: How uv got so fast

#283

Post by BiteCode_dev »

zahlman wrote: post #263
BiteCode_dev wrote: post #134

Other design decisions that made uv fast: - uncompressing packages while they are still being downloaded, in memory, so that you only have to write to disk once - design of its own locking format for speed But yes, rust is actually making it faster because: - real threads, no need for multi-processing - no python VM startup overhead - the dep resolution algo is exactly the type of workload that is faster in a compile…

> uncompressing packages while they are still being downloaded ... but the archive directory is at the end of the file? > no python VM startup overhead This is about 20 milliseconds on my 11-year-old hardware.

HTTP range strikes again.

As for 20 ms, if you deal with 20 dependencies in parallel, that's 400ms just to start working.

Shaving half a second on many things make things fast.

Althought as we saw with zeeek in the other comment, you likely don't need multiprocessing since the network stack and unzip in the stdlib release the gil.

Threads are cheaper.

Maybe if you'd bundle pubgrub as a compiled extension, you coukd get pretty close to uv's perf.

Re: How uv got so fast

#284

Post by raincole »

Nextgrid wrote: post #158
orliesaurus wrote: post #148

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

It's not just greenfield-ness but the fact it's a commercial endeavor (even if the code is open-source). Building a commercial product means you pay money (or something they equally value) to people to do your bidding. You don't have to worry about politics, licensing, and all the usual FOSS-related drama. You pay them to set their opinions aside and build what you want, not what they want (and if that doesn't work,…

Is there any sign telling Astral is actually making money via uv? How sustainable is it?

I suggest everyone save this comment and review it five years later.

Re: How uv got so fast

#285

Post by IshKebab »

zahlman wrote: post #126
IshKebab wrote: post #89

Mmm I don't buy it. Not many projects use setup.py now anyway and pip is still super slow. > Plenty of tools are written in Rust without being notably fast. This also hasn't been my experience. Most tools written in Rust are notably fast.

> Not many projects use setup.py now anyway and pip is still super slow. Yes, but that's still largely not because of being written in Python. The architecture is really just that bad. Any run of pip that touches the network will end up importing more than 500 modules and a lot of that code will simply not be used. For example, one of the major dependencies is Rich, which includes things like a 3600-entry mapping of…

> Any run of pip that touches the network will end up importing more than 500 modules and a lot of that code will simply not be used.

That's a property of Python though. The fact that it isn't compiled (and that importing is very slow).

> a 3600-entry mapping of string names to emoji

Which can easily be zero-cost in Rust.

> It literally doesn't know how to access its own package download cache without hitting the network

This is the only example you've given that actually fits with your thesis.

Re: How uv got so fast

#286

Post by eru »

dpark wrote: post #261
goalieca wrote: post #235

Earlier quoted context omitted.

> In large companies people engage in politics because it becomes necessary to accomplish large things. At a large company, your job after a certain level depends on your “impact” and “value delivered”. The challenge is getting 20 other teams to work on your priorities and not their priorities. They too need to play to win to keep their job or get that promotion.

For software engineering, “impact” or “value delivered” are pretty much always your job unless you work somewhere really dysfunctional that’s measuring lines of code or some other nonsense. But that does become a lot about politics after some level. I would not say it’s about getting other people aligned with your priorities instead of theirs but rather finding ways such that your priorities are aligned. There’s alwa…

If you are a fresh grad, you can mostly just chug along with your tickets and churn out code. Your boss (if you have a good boss) will help you make sure the other people work with you.

When you are higher up, that is when you become said good boss, or that boss's boss, the dynamics of the grandfather comment kick in fully.

Re: How uv got so fast

#287

Post by firesteelrain »

oefrha wrote: post #242
zahlman wrote: post #237

Earlier quoted context omitted.

> the selling point with the most traction is that you don't already need a working python install to get UV. And once you have UV, you can just go! I still genuinely do not understand why this is a serious selling point. Linux systems commonly already provide (and heavily depend upon) a Python distribution which is perfectly suitable for creating virtual environments, and Python on Windows is provided by a tradition…

Linux systems commonly already provide an outdated system Python you don’t want to use, and it can’t be used to create a venv of a version you want to use. A single Python version for the entire system fundamentally doesn’t work for many people thanks to shitty compat story in the vast ecosystem. Even languages with great compat story are moving to support multi-toolchains natively. For instance, go 1.22 on Ubuntu 24…

Why not just use a Python container rather than rely on having the latest binary installed on the system? Then venv inside the container. That would get you the “venv of a version” that you are referring to

Re: How uv got so fast

#288

Post by IshKebab »

scottlamb wrote: post #127
IshKebab wrote: post #89

Mmm I don't buy it. Not many projects use setup.py now anyway and pip is still super slow. > Plenty of tools are written in Rust without being notably fast. This also hasn't been my experience. Most tools written in Rust are notably fast.

Mine either. Choosing Rust by no means guarantees your tool will be fast—you can of course still screw it up with poor algorithms. But I think most people who choose Rust do so in part because they aspire for their tool to be "blazing fast". Memory safety is a big factor of course, but if you didn't care about performance, you might have gotten that via a GCed (and likely also interpreted or JITed or at least non-LLV…

Yeah sometimes you get surprisingly fast Python programs or surprisingly slow Rust programs, but if you put in a normal amount of effort then in the vast majority of cases Rust is going to be 10-200x faster.

I actually rewrote a non-trivial Python program in Rust once because it was so slow (among other reasons), and got a 50x speedup. It was mostly just running regexes over logs too, which is the sort of thing Python people say is an ideal case (because it's mostly IO or implemented in C).

Re: How uv got so fast

#289

Post by firesteelrain »

zahlman wrote: post #239
mkoubaa wrote: post #159

Earlier quoted context omitted.

1000% this. uv is trivially installable and is completely unrelated to installations of python.

If I want to install Python on Windows and start using pip, I grab an installer from python.org and follow a wizard. On Linux, I almost certainly already have it anyway. If I want to bootstrap from uv on Windows, the simplest option offered involves Powershell. Either way, I can write quite a bit with just the standard library before I have to understand what uv really is (or what pip is). At that point, yes, the pip…

Traditional Windows install didn’t include things Microsoft doesn’t make. But, any PC distributor could always include Python as part of their base Windows install with all the other stuff that bloats the typical third party Windows installs. They don’t which indicates the market doesn’t want it. Your indictment of the lack of Python out of the box is less on Windows than on the “distro” served by PC manufacturers

Re: How uv got so fast

#290

Post by raincole »

zahlman wrote: post #270
fragmede wrote: post #251

Earlier quoted context omitted.

> has simply not read and understood Contemporary Python Development 101. They haven't. At the end of the day, they just want their program to work. You and I can design a utopian packaging system, but the physics PhD with a hand-me-down windows laptop and access to her university's Linux research cluster don't care about python other than it has a PITA library situation that UV addresses.

If they are not developers, it's the developer's responsibility to fix that. The developers have many options available for this.

They're not applications developers, but they need to write code. That's the whole point. Python is popular within academia because it replaces R/Excel/VB.Net, not Java/C++.
Post reply on HN