Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How uv got so fast

nesbitt.io

Post reply on HN
Print view View original

361–370 of 468 posts

Re: How uv got so fast

#361

Post by CamouflagedKiwi »

zahlman wrote: post #241
collinmanderson wrote: post #165

Earlier quoted context omitted.

> the conversation here keeps collapsing back to "Rust rewrite good/bad." That feels like cargo-culting the toolchain instead of asking the uncomfortable question: why did it take a greenfield project to give Python the package manager behavior people clearly wanted for the last decade? I think there's a few things going on here: - If you're going have a project that's obsessed with speed, you might as well use rust/…

> the entire python ecosystem generally does not put much emphasis on startup time. You'd think PyPy would be more popular, then. > even modules in the standard library will pre-compile regular expressions at import time, even if they're never used, like the "email" module. Hmm, that is slower than I realized (although still just a fraction of typical module import time): $ python -m timeit --setup 'import re' 're.co…

> You'd think PyPy would be more popular, then.

PyPy doesn't do anything to help startup time. In fact, it's typically a bit slower to start up than CPython.

You reap the speed benefits from PyPy once it's been running for a little while and it can JIT compile the hot bits of code.

Re: How uv got so fast

#362

Post by jbs789 »

orliesaurus wrote: post #148

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

“Why did it take a greenfield project…?”

By definition greenfield projects literally means free from constraints.

So the answer is in your question: Why did it take a team unbound by constraints to try something new, as compared to a project with millions of existing stakeholders?

Single vision. Smaller team. What they landed on is a hit (no guarantee of that in advance!)

Conversely, with so many stakeholders, getting everyone to rally around a change (in advance) is hard.

In my experience this is about human nature/organisation and spans all types of organisations, not just python or open source etc.

It also looks like python would have got there, given the foundations put in place as noted in the article.

Re: How uv got so fast

#363

Post by PartiallyTyped »

Nextgrid wrote: post #158
orliesaurus wrote: post #148

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

It's not just greenfield-ness but the fact it's a commercial endeavor (even if the code is open-source). Building a commercial product means you pay money (or something they equally value) to people to do your bidding. You don't have to worry about politics, licensing, and all the usual FOSS-related drama. You pay them to set their opinions aside and build what you want, not what they want (and if that doesn't work,…

You often pay them for their opinions too!

Re: How uv got so fast

#364

Post by [deleted] »

CamouflagedKiwi wrote: post #361
zahlman wrote: post #241

Earlier quoted context omitted.

> the entire python ecosystem generally does not put much emphasis on startup time. You'd think PyPy would be more popular, then. > even modules in the standard library will pre-compile regular expressions at import time, even if they're never used, like the "email" module. Hmm, that is slower than I realized (although still just a fraction of typical module import time): $ python -m timeit --setup 'import re' 're.co…

> You'd think PyPy would be more popular, then. PyPy doesn't do anything to help startup time. In fact, it's typically a bit slower to start up than CPython. You reap the speed benefits from PyPy once it's been running for a little while and it can JIT compile the hot bits of code.

[deleted]

Re: How uv got so fast

#365

Post by rmah »

firesteelrain wrote: post #287
oefrha wrote: post #242

Earlier quoted context omitted.

Linux systems commonly already provide an outdated system Python you don’t want to use, and it can’t be used to create a venv of a version you want to use. A single Python version for the entire system fundamentally doesn’t work for many people thanks to shitty compat story in the vast ecosystem. Even languages with great compat story are moving to support multi-toolchains natively. For instance, go 1.22 on Ubuntu 24…

Why not just use a Python container rather than rely on having the latest binary installed on the system? Then venv inside the container. That would get you the “venv of a version” that you are referring to

Our firm uses python extensively and the virtual environment for every script or script is ... difficult. We have dozens of python scripts running for team research and in production, from small maintenance tools to rather complex daemons. Add to that the hundreds of Jupyter notebooks used by various people. Some have a handful of dependencies, some dozens of dependencies. While most of those scripts/notebooks are only used by a handful of people, many are used company-wide.

Further, we have a rather largish set of internal libraries most of our python programs rely on. And some of those rely on external 3rd party API's (often REST). When we find a bug or something changes, more often than not, we want to roll out the changed internal lib so that all programs that use it get the fix. Having to get everyone to rebuild and/or redeploy everything is a non-starter as many of the people involved are not primarily software developers.

We usually install into the system dirs and have a dependency problem maybe once a year. And it's usually trivially resolved (the biggest problem was with some google libs which had internally inconsistent dependencies at one point).

I can understand encouraging the use of virtual environments, but this movement towards requiring them ignores what, I think, is a very common use case. In short, no one way is suitable for everyone.

Re: How uv got so fast

#366

Post by dpark »

eru wrote: post #286
dpark wrote: post #261

Earlier quoted context omitted.

For software engineering, “impact” or “value delivered” are pretty much always your job unless you work somewhere really dysfunctional that’s measuring lines of code or some other nonsense. But that does become a lot about politics after some level. I would not say it’s about getting other people aligned with your priorities instead of theirs but rather finding ways such that your priorities are aligned. There’s alwa…

If you are a fresh grad, you can mostly just chug along with your tickets and churn out code. Your boss (if you have a good boss) will help you make sure the other people work with you. When you are higher up, that is when you become said good boss, or that boss's boss, the dynamics of the grandfather comment kick in fully.

Agree. A fresh grad is still measured on “impact” but that impact is generally localized. e.g. Quality of individual code and design vs ability to wrangle others to work with you.

Impact is a handwavy way of saying “is your work good for the company”.

Re: How uv got so fast

#367

Post by SAI_Peregrinus »

webstrand wrote: post #154
IgorPartola wrote: post #138

Earlier quoted context omitted.

Rust is also a systems language. I am still wrapping my mind around why it is so popular for so many end projects when its main use case and goals were basically writing a browser a maybe OS drivers. But that’s precisely why it is good for developer tools. And it turns out people who write systems code are really damn good at writing tools code. As someone who cut my teeth on C and low level systems stuff I really ou…

I write scripts in rust as a replacement for bash. Its really quite good at it. Aside from perl, its the only scripting language that can directly make syscalls. Its got great libraries for: parsing, configuration management, and declarative CLIs built right into it. Sure its a little more verbose than bash one-liners, but if you need any kind of error handling and recovery, its way more effective than bash and doesn…

Paths are hard because they usually look like printable text, but don't have to be text. POSIX filenames are octet strings not containing 0x2F or 0x00. They aren't required to contain any "printable" characters, or even be valid text in any particular encoding. Most of the Rust stdlib you're thinking of is for handling text strings, but paths aren't text strings. Python also has the same split between Pathlib paths & all other strings.

Re: How uv got so fast

#368

Post by whakim »

zahlman wrote: post #236
crote wrote: post #216

Earlier quoted context omitted.

So basically, it avoids the whole chicken-and-egg problem. With UV you've simply always got "UV -> project Python 1.23 -> project". UV is your dependency manager, and your Python is just another dependency . With other dependency managers you end up with "system Python 3.45 -> dep manager -> project Python 1.23 -> project". Or worse, "system Python 1.23 -> dep manager -> project Python 1.23 -> project". And of course…

Sorry, but that is simply incorrect, on many levels. Virtual environments are the fundamental way of setting up a Python project, whether or not you use uv, which creates and manages them for you . And these virtual environments can freely either use or not use the system environment, whether or not you use uv to create them. It's literally a single-line difference in the `pyvenv.cfg` file, which is a standard requir…

> Most of the time you don't need a different Python version from the system one.

Except for literally anytime you’re collaborating with anyone, ever? I can’t even begin to imagine working on a project where folks just use whatever python version their OS happens to ship with. Do you also just ship the latest version of whatever container because most of the time nothing has changed?

Re: How uv got so fast

#369

Post by beeboop0 »

orliesaurus wrote: post #148

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

[dead]

Re: How uv got so fast

#370

Post by metaltyphoon »

mlrtime wrote: post #317
formerly_proven wrote: post #311

Earlier quoted context omitted.

Cargo is not really good. The very much non-zero frequency of something with cargo not working for opaque reasons and then suddenly working again after "cargo clean", the "no, I invoke your binaries"-mentality (try running a benchmark without either ^C'ing out of bench to copy the binary name or parsing some internal JSON metadata) because "cargo build" is the only build system in the world which will never tell you…

I have empathy for anyone who was required to use cargo on a nfs mounted fs. The number of files and random IO cargo uses makes any large project unusable. I had to stop telling people to stop syncing their cargo env around nfs so many times, but sometimes they have no choice.

> nfs mounted fs

Anything doing locks on nfs, including trying to use sqlite, is a mistake. This is not a cargo problem this is a nsf problem.

Post reply on HN