Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How uv got so fast

nesbitt.io

Post reply on HN
Print view View original

251–260 of 468 posts

Re: How uv got so fast

#251

Post by fragmede »

zahlman wrote: post #236
crote wrote: post #216

Earlier quoted context omitted.

So basically, it avoids the whole chicken-and-egg problem. With UV you've simply always got "UV -> project Python 1.23 -> project". UV is your dependency manager, and your Python is just another dependency . With other dependency managers you end up with "system Python 3.45 -> dep manager -> project Python 1.23 -> project". Or worse, "system Python 1.23 -> dep manager -> project Python 1.23 -> project". And of course…

Sorry, but that is simply incorrect, on many levels. Virtual environments are the fundamental way of setting up a Python project, whether or not you use uv, which creates and manages them for you . And these virtual environments can freely either use or not use the system environment, whether or not you use uv to create them. It's literally a single-line difference in the `pyvenv.cfg` file, which is a standard requir…

> has simply not read and understood Contemporary Python Development 101.

They haven't. At the end of the day, they just want their program to work. You and I can design a utopian packaging system, but the physics PhD with a hand-me-down windows laptop and access to her university's Linux research cluster don't care about python other than it has a PITA library situation that UV addresses.

Re: How uv got so fast

#252

Post by zahlman »

annexrichmond wrote: post #233

> This reduces resolver backtracking dramatically since upper bounds are almost always wrong. I am surprised by this because Python minor versions break backwards compatibility all the time. Our company for example is doing a painful upgrade from py39 to py311

Could you explain what major pain points you've encountered? I can't think of any common breakages cited in 3.10 or 3.11 offhand. 3.12 had a lot more standard library removals, and the `match` statement introduced in 3.10 uses a soft keyword and won't break code that uses `match` as an identifier.

Re: How uv got so fast

#254

Post by zahlman »

VorpalWay wrote: post #102
breischl wrote: post #98

Earlier quoted context omitted.

I read the article as saying it ignores all upper-bounds, and 4.0 is just an example. I could be wrong though - it seems ambiguous to me. But if we accept that it currently ignores any upper-bounds checks greater than v3, that's interesting. Does that imply that once Python 4 is available, uv will slow down due to needing to actually run those checks?

Are there any plans to actually make a 4.0 ever? I remember hearing a few years ago that after the transition to 3.0, the core devs kind of didn't want to repeat that mess ever again. That said, even if it does happen, I highly doubt that is the main part of the speed up compared to pip.

There are indeed not any such plans.

Re: How uv got so fast

#255

Post by zahlman »

beacon294 wrote: post #13
yjftsjthsd-h wrote: post #6

Earlier quoted context omitted.

That sounds like yes? Instead of doing it once at install time, it's done once at first use. It's only once so it's not persistently slower, but that is a perf hit. My first cynical instinct is to say that this is uv making itself look better by deferring the costs to the application, but it's probably a good trade-off if any significant percentage of the files being compiled might not be used ever so the overall cos…

Probably for any case where an actual human is doing it. On an image you obviously want to do it at bake time, so I feel default off with a flag would have been a better design decision for pip. I just read the thread and use Python, I can't comment on the % speedup attributed to uv that comes from this optimization.

> On an image you obviously want to do it at bake time

It seems like tons of people are creating container images with an installer tool and having it do a bunch of installations, rather than creating the image with the relevant Python packages already in place. Hard to understand why.

For that matter, a pre-baked Python install could do much more interesting things to improve import times than just leaving a forest of `.pyc` files in `__pycache__` folders all over the place.

Re: How uv got so fast

#256

Post by BiteCode_dev »

zzzeek wrote: post #142
BiteCode_dev wrote: post #134

Other design decisions that made uv fast: - uncompressing packages while they are still being downloaded, in memory, so that you only have to write to disk once - design of its own locking format for speed But yes, rust is actually making it faster because: - real threads, no need for multi-processing - no python VM startup overhead - the dep resolution algo is exactly the type of workload that is faster in a compile…

> real threads, no need for multi-processing parallel downloads don't need multi-processing since this is an IO bound usecase. asyncio or GIL-threads (which unblock on IO) would be perfectly fine. native threads will eventually be the default also.

Indeed, but unzipping while downloading do. Analysing multiple metadata files and exporting lock data as well.

Now I believe unzip releases the GIL already so we could already benefit from that and the rest likely don't dominate perfs.

But still, rust software is faster on average than python software.

After all, all those things are possible in python, and yet we haven't seen them all in one package manager before uv.

Maybe the strongest advantage of rust, on top of very clean and fast default behaviors, is that it attracts people that care about speed, safety and correctness. And those devs are more likely to spend time implementing fast software.

Thought the main benefit of uv is not that it's fast. It's very nice, and opens more use cases, but it's not the killer feature.

The killer feature is, being a stand alone executable, it bypasses all python bootstrapping problems.

Again, that could technically be achieved in python, but friction is a strong force.

Re: How uv got so fast

#257

Post by 112233 »

zahlman wrote: post #239
mkoubaa wrote: post #159

Earlier quoted context omitted.

1000% this. uv is trivially installable and is completely unrelated to installations of python.

If I want to install Python on Windows and start using pip, I grab an installer from python.org and follow a wizard. On Linux, I almost certainly already have it anyway. If I want to bootstrap from uv on Windows, the simplest option offered involves Powershell. Either way, I can write quite a bit with just the standard library before I have to understand what uv really is (or what pip is). At that point, yes, the pip…

Not many normal people want to install python. Instead, author of the software they are trying to use wants them to install python. So they follow readme, download windows installer as you say, pip this pipx, pipx that conda, conda this requirements.txt, and five minutes later they have magic error telling that tensorflow version they are installing is not compatible with pytorch version they are installing or some such.

The aftertaste python leaves is lasting-disgusting.

Re: How uv got so fast

#258

Post by 4k93n2 »

pecheny wrote: post #25

The content is nice and insightful! But God I wish people stopped using LLMs to 'improve' their prose... Ironically, some day we might employ LLMs to re-humanize texts that had been already massacred.

there is going to be a point where people have read so much slop that they will start regurgitating the same style without even realising it. or we could already be at that point

Re: How uv got so fast

#259

Post by nxpnsv »

blintz wrote: post #4

> PEP 658 went live on PyPI in May 2023. uv launched in February 2024. The timing isn’t coincidental. uv could be fast because the ecosystem finally had the infrastructure to support it. A tool like uv couldn’t have shipped in 2020. The standards weren’t there yet. How/why did the package maintainers start using all these improvements? Some of them sound like a bunch of work, and getting a package ecosystem to move i…

Hmm... poetry got me into using pyproject.toml, and with that migrating to uv was surprisingly easy.

Re: How uv got so fast

#260

Post by zahlman »

plorkyeran wrote: post #50
yjftsjthsd-h wrote: post #2

> No bytecode compilation by default. pip compiles .py files to .pyc during installation. uv skips this step, shaving time off every install. You can opt in if you want it. Are we losing out on performance of the actual installed thing, then? (I'm not 100% clear on .pyc files TBH; I'm guessing they speed up start time?)

If you have a dependency graph large enough for this to be relevant, it almost certainly includes a large number of files which are never actually imported. At worst the hit to startup time will be equal to the install time saved, and in most cases it'll be a lot smaller.

> a large number of files which are never actually imported

Unfortunately, it typically doesn't work out as well as you might expect, especially given the expectation of putting `import` statements at the top of the file.

Post reply on HN