Live data from Hacker News

How uv got so fast

nesbitt.io

151–160 of 468 posts

Re: How uv got so fast

#151
post #44

Earlier quoted context omitted.

“Why this matters” being the final section is a guaranteed give away, among innumerable others.

I realized once I was in the "optimizations that dont need rust" section. Specifically "This is concurrency, not language magic."

Yup. The author has now swapped that part out for “Any language can do this.”

Just commenting to preempt any comments telling you that the article doesn’t say this.

Re: How uv got so fast

#152
post #25

The content is nice and insightful! But God I wish people stopped using LLMs to 'improve' their prose... Ironically, some day we might employ LLMs to re-humanize texts that had been already massacred.

To me, unless it is egregious, I would be very sensitive to avoid false positives before saying something is LLM aided. If it is clearly just slop, then okay, but I definitely think there is going to be a point where people claim well-written, straightforward posts as LLM aided. (Or even the opposite, which already happens, where people purposely put errors in prose to seem genuine).

Re: How uv got so fast

#153

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

Consensus building and figuring out what was actually needed?

Someone on this site said most tech problems are people problems - this feels like one.

Greenfield mostly solves the problem because it's all new people.

Re: How uv got so fast

#154

Earlier quoted context omitted.

I'm my experience this is definitely where rust shined. The language wasn't really what made the project succeed so much as having relatively curious, meticulous, detail-oriented people on hand who were interested in solving hard problems. Sometimes I thought our teams would be a terrible fit for more cookie-cutter applications where rapid development and deployment was the primary objective. We got into the weeds al…

Rust is also a systems language. I am still wrapping my mind around why it is so popular for so many end projects when its main use case and goals were basically writing a browser a maybe OS drivers. But that’s precisely why it is good for developer tools. And it turns out people who write systems code are really damn good at writing tools code. As someone who cut my teeth on C and low level systems stuff I really ou…

I write scripts in rust as a replacement for bash. Its really quite good at it. Aside from perl, its the only scripting language that can directly make syscalls. Its got great libraries for: parsing, configuration management, and declarative CLIs built right into it.

Sure its a little more verbose than bash one-liners, but if you need any kind of error handling and recovery, its way more effective than bash and doesn't break when you switch platforms (i.e. mac/bsd utility incompatibilities with gnu utilities).

My only complaint would be that dealing with OsString is more difficult than necessary. Way to much of the stdlib encourages programmers to just do "non-utf8 paths don't exist" and panic/ignore when encountering one. (Not a malady exclusive to rust, but I wish they'd gotten it right)

Example I had handy: https://gist.github.com/webstrand/945c738c5d60ffd7657845a654...>

Re: How uv got so fast

#155

Earlier quoted context omitted.

Sure, but that kind of incompetence is already filtered out (in the https://www.lesswrong.com/w/screening-off-evidence sense) by the task of creating a package installer.

You would think so, yet here I am sitting with a node_modules full of crud placed there by npm, waiting for the next supply chain attack.

npm isn't the issue there it's the ts\js community and their desire to use a library for everything. in communities that do not consider dependencies to be a risk you will find this showing up in time.

The node supply chain attacks are also not unique to node community. you see them happening on crates.io and many other places. In fact the build time scripts that cause issues on node modules are probably worse off with the flexibility of crate build scripts and that they're going to be harder to work around than in npm.

Re: How uv got so fast

#156
post #22

uv seems to be a pet peeve of HN. I always thought pipenv was good but yeah, seems like I was being ignorant

My biggest complaint with pipenv is/was(?) that it's lockfile format only kept the platform identifiers of the platform you locked it on - so if you created it on Mac, then tried to install from the lockfile on a Linux box, you're building from source because it's only locked in wheels for MacOS.

Poetry and uv avoid this issue.

Re: How uv got so fast

#157

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

I largely agree but don't want to entirely discount the effect that using a compiled language had.

At least in my limited experience, the selling point with the most traction is that you don't already need a working python install to get UV. And once you have UV, you can just go!

If I had a dollar for every time I've helped somebody untangle the mess of python environment libraries created by an undocumented mix of python delivered through the distributions package management versus native pip versus manually installed...

At least on paper, both poetry and UV have a pretty similar feature set. You do however need a working python environment to install and use poetry though.

Re: How uv got so fast

#158

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

It's not just greenfield-ness but the fact it's a commercial endeavor (even if the code is open-source).

Building a commercial product means you pay money (or something they equally value) to people to do your bidding. You don't have to worry about politics, licensing, and all the usual FOSS-related drama. You pay them to set their opinions aside and build what you want, not what they want (and if that doesn't work, it just means you need to offer more money).

In this case it's a company that believes they can make a "good" package manager they can sell/monetize somehow and so built that "good" package manager. Turns out it's at least good enough that other people now like it too.

This would never work in a FOSS world because the project will be stuck in endless planning as everyone will have an opinion on how it should be done and nothing will actually get done.

Similar story with systemd - all the bitching you hear about it (to this day!) is the stuff that would've happened during its development phase had it been developed as a typical FOSS project and ultimately made it go nowhere - but instead it's one guy that just did what he wanted and shared it with the world, and enough other people liked it and started building upon it.

Re: How uv got so fast

#159

The most surprising part of uv's success to me isn't Rust at all, it's how much speed we "unlocked" just by finally treating Python packaging as a well-specified systems problem instead of a pile of historical accidents. If uv had been written in Go or even highly optimized CPython, but with the same design decisions (PEP 517/518/621/658 focus, HTTP range tricks, aggressive wheel-first strategy, ignoring obviously de…

I largely agree but don't want to entirely discount the effect that using a compiled language had. At least in my limited experience, the selling point with the most traction is that you don't already need a working python install to get UV. And once you have UV, you can just go! If I had a dollar for every time I've helped somebody untangle the mess of python environment libraries created by an undocumented mix of p…

1000% this. uv is trivially installable and is completely unrelated to installations of python.

Re: How uv got so fast

#160

Earlier quoted context omitted.

I think the framing in the post is that it's specific to Rust, relative to what Python packaging tools are otherwise written in (Python). It's not very easy to do zero-copy deserialization in pure Python, from experience. (But also, I think Rust can fairly claim that it's made zero-copy deserialization a lot easier and safer.)

I can't even imagine what "safety" issue you have in mind. Given that "zero-copy" apparently means "in-memory" (a deserialized version of the data necessarily cannot be the same object as the original data), that's not even difficult to do with the Python standard library. For example, `zipfile.ZipFile` has a convenience method to write to file, but writing to in-memory data is as easy as with zipfile.ZipFile(archive…

As a quick and kind of oversimplified example of what zero copy means, imagine you read the following json string from a file/the network/whatever:

    json = '{"user":"nugget"}' // from somewhere
A simple way to extract json["user"] to a new variable would be to copy the bytes. In pythony/c pseudo code

    let user = allocate_string(6 characters)
    for i in range(0, 6)
      user[i] = json["user"][i]
    // user is now the string "nugget"
instead, a zero copy strategy would be to create a string pointer to the address of json offset by 9, and with a length of 6.

    {"user":"nugget"}
             ^     ]end
The reason this can be tricky in C is that when you call free(json), since user is a pointer to the same string that was json, you have effectively done free(user) as well.

So if you use user after calling free(json), You have written a classic _memory safety_ bug called a "use after free" or UAF. Search around a bit for the insane number of use after free bugs there have been in popular software and the havoc they have wreaked.

In rust, when you create a variable referencing the memory of another (user pointing into json) it keeps track of that (as a "borrow", so that's what the borrow checker does if you have read about that) and won't compile if json is freed while you still have access to user. That's the main memory safety issue involved with zero-copy deserialization techniques.

Post reply on HN