Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How uv got so fast

nesbitt.io

Post reply on HN
Print view View original

431–440 of 468 posts

Re: How uv got so fast

#431

Post by LtWorf »

IgorPartola wrote: post #429
LtWorf wrote: post #382

Earlier quoted context omitted.

You can validate user input with types using stuff like typedload (which i wrote) or similar runtime type checkers.

“The user can choose between starting their new policy on the first day of employment, the first day of the fiscal year, on a specific date, or some number of days after their prior policy expires. If they choose the first day of the fiscal year, the user must specify when their company’s fiscal year starts. If they choose a specific date they must choose a date that is after the first business day of the next month…

attrs and dataclasses let you define custom validators that can be used together with typedload…

Re: How uv got so fast

#432

Post by BiteCode_dev »

zahlman wrote: post #391
BiteCode_dev wrote: post #283

Earlier quoted context omitted.

HTTP range strikes again. As for 20 ms, if you deal with 20 dependencies in parallel, that's 400ms just to start working. Shaving half a second on many things make things fast. Althought as we saw with zeeek in the other comment, you likely don't need multiprocessing since the network stack and unzip in the stdlib release the gil. Threads are cheaper. Maybe if you'd bundle pubgrub as a compiled extension, you coukd g…

Why are you starting a separate Python process for each dependency?

Real thread are very recent and didn't exist when uv was created. So you needed multiprocesses.

Re: How uv got so fast

#433

Post by BiteCode_dev »

zzzeek wrote: post #406
BiteCode_dev wrote: post #256

Earlier quoted context omitted.

Indeed, but unzipping while downloading do. Analysing multiple metadata files and exporting lock data as well. Now I believe unzip releases the GIL already so we could already benefit from that and the rest likely don't dominate perfs. But still, rust software is faster on average than python software. After all, all those things are possible in python, and yet we haven't seen them all in one package manager before u…

> Maybe the strongest advantage of rust, on top of very clean and fast default behaviors, is that it attracts people that care about speed, safety and correctness. And those devs are more likely to spend time implementing fast software. people who have this opinion should use Rust, not Python, at all. if Python code does not have sufficient speed, safety, and correctness for someone, it should not be used. Python's t…

For the latter point, you are blinded by your own competence.

Bootstrapping a clean python env is the single biggest problem for people that are not daily coding in python.

That's half of the community in the python world.

When you write sqla that's not obvious, because you know a lot. But for the average user, uv was a savior.

I wrote a pretty long article on that here:

https://www.bitecode.dev/p/why-not-tell-people-to-simply-use

We also discuss it with brett cannon there:

https://www.bitecode.dev/p/brett-cannon-on-python-humans-and

But the most convincing argument is to teach python to kids, accountants, mathematicians, java coders and sysadmin.

After 20 years of doing that, I saw the same problems again and again.

And then uv arrived. And they disapeared for those people.

Re: How uv got so fast

#434

Post by IshKebab »

zahlman wrote: post #399
IshKebab wrote: post #285

Earlier quoted context omitted.

> Any run of pip that touches the network will end up importing more than 500 modules and a lot of that code will simply not be used. That's a property of Python though. The fact that it isn't compiled (and that importing is very slow). > a 3600-entry mapping of string names to emoji Which can easily be zero-cost in Rust. > It literally doesn't know how to access its own package download cache without hitting the net…

> That's a property of Python though. The fact that it isn't compiled (and that importing is very slow). Bytecode compilation is compilation. There are many things that could be used to improve import speed that I never even see discussed, let alone implemented. But most importantly, pip doesn't need to have all these modules imported . They already proved they could defer the Requests imports; but the actual network…

> Bytecode compilation is compilation.

It was pretty clear I meant AoT compilation.

> Which is irrelevant to the point.

Sorry, the fact that typical Rust programming style is much faster than typical Python style is irrelevant to the fact that a Rust tool is faster than its Python competitor? Riiight.

> My thesis is that pip doesn't have to be the way it is in order to actually solve the problem of installing Python packages.

Ok but the actual debate was about whether `uv` is faster because it is written in Rust. The answer is yes because when you put a normal amount of effort into writing Python or Rust you run into all these problems in Python that you don't run into in Rust.

Sure you can defer imports, but that's a pretty uncommon thing to do in Python.

It's like, you wouldn't say "bicycles are just as fast as cars; as long as you attach a jet engine to the bicycle!". Yes, technically possible, but nobody does that.

Re: How uv got so fast

#435

Post by akoboldfrying »

zahlman wrote: post #407
akoboldfrying wrote: post #328

Earlier quoted context omitted.

> you have to add them yourself to be compatible with your dependencies This is no more true for version upper bounds than it is for version lower bounds, assuming that package installers ensure all package version constraints are satisfied. I presume you think version lower bounds should still be honoured?

The point is that you can know that a lower bound is necessary at the time of publication; an upper bound is either speculative or purely defensive, and has possibly unnecessary consequences for your dependents.

You can also know that an upper bound is necessary at the time of publication -- for example, if your foo project uses bar 2.0, and bar 3.0 has already come out, and you have tried it and found it incompatible.

In the reverse direction, many version lower bounds are also "purely defensive" -- arising from nothing more than the version of the dep that you happened to get when you started the project. (Just because you installed "the latest baz" and got version 2.3.4, without testing there is nothing to say that version 2.3.3 would also work fine, so adding the version lower bound >=2.3.4 is purely defensive).

Basically, the two bound types are isomorphic.

Re: How uv got so fast

#436

Post by Woodi »

LOL

All that years (decades?) Python had build-in code executed on install ??? That's functionally adequate to MS autoexec.bat viruses spreading mechanism!

Just LOL...

Re: How uv got so fast

#437

Post by BiteCode_dev »

andy99 wrote: post #42

I remain baffled about these posts getting excited about uv’s speed. I’d like to see a real poll but I personally can’t imagine people listing speed as one of the their top ten concerns about python package managers. What are the common use cases where the delay due to package installation is at all material? Edit to add: I use python daily

The golden rule to understand the success of uv is to remind yourself how big and diverse the python community is. Your context is only a tiny fraction of the contexts in which python is going to be executed.

E.G: if you compare it to your machine, it's a different thing that if you compare it to a locked down corporate machines.

I have clients that have Python setup so bad installing all deps for a project takes... 18 minutes. Those are not crazy projects either. It's just the context that is bad. And you won't be able to change the context. But we are in talk to change the package manager to uv.

There are so many different setups that are different than yours. If you are a professional trainer, and you get a new group every week, having 12 people installing their env in a blink is a win. If you are a researcher and you want to download the top 100 pypi packages and attempt to install them, speed is a bliss. If you are a blogger and try a lot of new stuff for an article, it's great. If you are working at repl.io and you get millions of venv created every day, boy does that matter. If you are sysadmin in charge of deploying kub pods, you might be looking at serious savings. Etc.

Speed affects many things:

- CI runs

- AI iterations

- docker builds

- isolated builds over multiple versions of python

But it also unlocks some use cases.

E.G:

- uvx is great only because uv is fast. Because uv calls are virtually instants, using uvx feels like magic.

- "uv run --with" exist only because overlaying a new venv on top of the other is basically free. And it's a killer feature.

- You never create lock files in uv. Because the operation is transparently done in the background since it's so fast. I can't recall the last time I ran uv sync. Because uv run automatically call it, since it's so fast you don't notice. So you just skip the middle man and go straight to coding.

I was a big proponent of "speed is not that important". Until I got speed.

And then I realized I missed a lot, because they are things you just can't do if you are slow.

Re: How uv got so fast

#438

Post by aragilar »

jacquesm wrote: post #331
aragilar wrote: post #306

Earlier quoted context omitted.

Honestly, they should be using conda (if they're working on their laptops) and the cluster package manager otherwise.

Conda has slowly but surely gone down the drain as well. It used to be bullet proof but there too you now get absolutely unsolvable circular dependencies.

I'd be curious as to seeing what these circular dependencies you're seeing are (not saying I don't believe you, and I do recall in the early days of conda it doing some dumb stuff, but that particular issue seems odd)?

As for why conda: wheels do not have post-installation hooks (which given the issues with npm, I'm certainly a fan of), and while for most packages this isn't an issue, I've encountered enough packages where sadly they are required (for integration purposes), and the PyPI packages are subtlety broken on install without them. Additionally, conda (especially Anaconda Inc's commercial repositories) have significantly more optimised builds (not as good as the custom build well-run clusters provide, but better than PyPI-provided ones). I personally do not use conda (because I tend to want to test/modify/patch/upstream packages lower down the chain and test with higher up packages), but for novices (especially novices on Windows), conda for all its faults is the best option for those in the "data science" ecosystem.

Re: How uv got so fast

#439

Post by jaapz »

codeflo wrote: post #269
WD-42 wrote: post #214

Earlier quoted context omitted.

I don't know what you think "typical Foss projects" are but in my experience they are exactly like your systemd example: one person that does what they want and share it with the world. The rest of your argument doesn't really make any sense with that in mind.

That's no longer as true as it once was. I get the feeling that quite a few people would consider "benevolent dictator for life" an outdated model for open source communities. For better or worse, there's a lot of push to transition popular projects towards being led by committee. Results are mixed (literally: I see both successes and failures), but that doesn't seem to have any effect on the trend.

In reality, most people don't really care about a project's "governance model". When it solves a problem well, they will use it. There are loads of weird tiny libraries you don't even know about that rely one the one guy living in some small town in lithuania to spend his saturdays on maintaining.

Re: How uv got so fast

#440

Post by WiSaGaN »

Nextgrid wrote: post #158
orliesaurus wrote: post #148

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

It's not just greenfield-ness but the fact it's a commercial endeavor (even if the code is open-source). Building a commercial product means you pay money (or something they equally value) to people to do your bidding. You don't have to worry about politics, licensing, and all the usual FOSS-related drama. You pay them to set their opinions aside and build what you want, not what they want (and if that doesn't work,…

This argument falls apart when you look at Rust and Cargo. uv is literally trying to be "Python's Cargo." The entire blueprint came from a flagship FOSS project.

Rust's development used a structured, community RFC process—endless planning by your definition. The result was a famously well-designed toolchain that the entire community praises. FOSS didn't hold it back; it made it good.

So no, commercial backing isn't the only way to ship something good. FOSS is more than capable to ship great software when done right.

Post reply on HN