141–150 of 468 posts
I remain baffled about these posts getting excited about uv’s speed. I’d like to see a real poll but I personally can’t imagine people listing speed as one of the their top ten concerns about python package managers. What are the common use cases where the delay due to package installation is at all material? Edit to add: I use python daily
Other design decisions that made uv fast: - uncompressing packages while they are still being downloaded, in memory, so that you only have to write to disk once - design of its own locking format for speed But yes, rust is actually making it faster because: - real threads, no need for multi-processing - no python VM startup overhead - the dep resolution algo is exactly the type of workload that is faster in a compile…
parallel downloads don't need multi-processing since this is an IO bound usecase. asyncio or GIL-threads (which unblock on IO) would be perfectly fine. native threads will eventually be the default also.
Earlier quoted context omitted.
Sure, but that kind of incompetence is already filtered out (in the https://www.lesswrong.com/w/screening-off-evidence sense) by the task of creating a package installer.
You would think so, yet here I am sitting with a node_modules full of crud placed there by npm, waiting for the next supply chain attack.
Earlier quoted context omitted.
> Given that "zero-copy" apparently means "in-memory" (a deserialized version of the data necessarily cannot be the same object as the original data), that's not even difficult to do with the Python standard library This is not what zero-copy means. Here's a working definition[1]. Specifically, it's not just about keeping things in memory; copying in memory is normal. The goal is to not make copies (or more precisely…
> Yes, that's what makes it not zero-copy. Yeah, so you'd have to pass around the `BytesIO` instead. I know that zero-copy doesn't ordinarily mean what I described, but that seemed to be how TFA was using it , based on the logic in the rest of the sentence.
That wouldn’t be zero-copy either: BytesIO is an I/O abstraction over a buffer, so it intentionally masks the “lifetime” of the original buffer. In effect, reading from the BytesIO creates new copies of the underlying data by design, in new `bytes` objects.
(This is actually a great capsule example of why zero-copy design is difficult in Python: the Pythonic thing to do is to make lots of bytes/string/rich objects as you parse, each of which owns its data, which in turn means copies everywhere.)
I think this post does a really good job of covering how multi-pronged performance is: it certainly doesn't hurt uv to be written in Rust, but it benefits immensely from a decade of thoughtful standardization efforts in Python that lifted the ecosystem away from needing `setup.py` on the hot path for most packages.
Earlier quoted context omitted.
Sure, but that kind of incompetence is already filtered out (in the https://www.lesswrong.com/w/screening-off-evidence sense) by the task of creating a package installer.
You would think so, yet here I am sitting with a node_modules full of crud placed there by npm, waiting for the next supply chain attack.
uv doesn't exactly stop python package supply chain attacks...
Earlier quoted context omitted.
Someone once told me a benefit of staffing a project for Haskell was it made it easy to select for the types of programmers that went out of their way to become experts in Haskell. Tapping the Rust community is a decent reason to do a project in Rust.
Paul Graham said the same thing about Python 20 years ago [1], and back then it was true. But once a programming langauge hits mainstream, this ceases to be a good filter. [1] https://paulgraham.com/pypar.html
Earlier quoted context omitted.
It's an interesting debate. The flip side of this coin is getting hires who are more interested in the language or approach than the problem space and tend to either burn out, actively dislike the work at hand, or create problems that don't exist in order to use the language to solve them. With that said, Rust was a good language for this in my experience. Like any "interesting" thing, there was a moderate bit of lan…
Clojure engineers not interested in doing work? That's surprising
> there was way too much academic interest in language concepts and way too little practical interest in doing work.
They are communicating something real, but perhaps misattributing the root cause.
The purely abstract ‘ideal’ form of software development is unconstrained by business requirements. In this abstraction, perfect software would be created to purely express an idea. Academia allows for this, and to a lesser extent some open source projects.
In the real world, the creation of software must always be subordinate to the goals of the business. The goals are the purpose, and the software is the means.
Languages that are academically interesting, unsurprisingly, attract a greater preponderance of academically minded individuals. Of these, only a percentage have the desire or ability to let go of the pure abstract, and instead focus on the business domain. So it inevitably creates a management challenge; not an insurmountable one, but a challenge.
Hence the simplified ‘these people won’t do the work!’.
Earlier quoted context omitted.
> Yes, that's what makes it not zero-copy. Yeah, so you'd have to pass around the `BytesIO` instead. I know that zero-copy doesn't ordinarily mean what I described, but that seemed to be how TFA was using it , based on the logic in the rest of the sentence.
> Yeah, so you'd have to pass around the `BytesIO` instead. That wouldn’t be zero-copy either: BytesIO is an I/O abstraction over a buffer, so it intentionally masks the “lifetime” of the original buffer. In effect, reading from the BytesIO creates new copies of the underlying data by design, in new `bytes` objects. (This is actually a great capsule example of why zero-copy design is difficult in Python: the Pythonic…
I'm not convinced this is going to bottleneck things, though.
(On the flip side, I guess the OS is likely to cache any disk write in memory anyway.)