Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How uv got so fast

nesbitt.io

Post reply on HN
Print view View original

451–460 of 468 posts

Re: How uv got so fast

#451

Post by zahlman »

ldjnsozhzkdh wrote: post #345
pnt12 wrote: post #280

Earlier quoted context omitted.

I didn't notice that - can you give some examples?

how they claim fetching from a single index magically solves dependency confusion attacks, when in reality it makes the attack much more trivial and able to succeeded. typical llm syncopation.

It says:

> uv picks from the first index that has the package, stopping there. This prevents dependency confusion attacks and avoids extra network requests.

As long as the "first" index is e.g. your organization's internal one, that does ensure that some random thing on PyPI won't override that. A tool that checks every index first still has to have the right rule to choose one.

It is, however, indeed a terrible point. I don't think I've even seen evidence that pip does anything different here. But it's the sort of problem best addressed in other ways

By "syncopation" perhaps you mean "sycophancy"? I don't see how musical rhythms are relevant here.

Re: How uv got so fast

#452

Post by zahlman »

firesteelrain wrote: post #400
baby_souffle wrote: post #377

Earlier quoted context omitted.

> Why not just use a Python container rather than rely on having the latest binary installed on the system? Sometimes this is the right answer. Sometimes docker/podman/runc are not an option nor would the headache of volumes/mounts/permissions/hw-pass-through be worth the additional mess. It is hard to over-state how delightful putting `uv` in the shebang is: in `demo.py`: #!/usr/bin/env -S uv run # /// script # requ…

Yes, PATH-driven interpreter selection is the source of the detours. uv eliminates interpreter ambiguity but requires uv as a prerequisite. This improves portability inside environments that standardize uv; it’s not “portable to machines with nothing installed.” Though, this isn’t about avoiding installs; it’s about making the one install (uv) the only thing you have to get right, instead of debugging whatever python…

> PATH-driven interpreter selection is the source of the detours. uv eliminates interpreter ambiguity but requires uv as a prerequisite.

Also, to use uv like this you either need to specify its path, or as shown in the example invoke /usr/bin/env. The Linux shebang requires a path rather than an executable name, and a relative path only works if you're in the exact right directory.

So in practical terms we have gained nothing, since if we want to avoid "PATH-driven interpreter selection" we could specify an absolute path like /usr/bin/python in the shebang, and uv doesn't let us avoid that.

That said, the PEP 723 interface is really nice (there's a lot more going on in the example than just figuring out which Python to use), and the experience of using uv as the interpreter is nicer in the sense that you only need uv to exist in one place. (This, too, is a problem that can be solved just fine in Python, and there are many approaches to it out there already.)

Re: How uv got so fast

#453

Post by MrDrMcCoy »

whakim wrote: post #368
zahlman wrote: post #236

Earlier quoted context omitted.

Sorry, but that is simply incorrect, on many levels. Virtual environments are the fundamental way of setting up a Python project, whether or not you use uv, which creates and manages them for you . And these virtual environments can freely either use or not use the system environment, whether or not you use uv to create them. It's literally a single-line difference in the `pyvenv.cfg` file, which is a standard requir…

> Most of the time you don't need a different Python version from the system one. Except for literally anytime you’re collaborating with anyone, ever? I can’t even begin to imagine working on a project where folks just use whatever python version their OS happens to ship with. Do you also just ship the latest version of whatever container because most of the time nothing has changed?

If you're writing Python tools to support OS operations in prod, you need to target the system Python. It's wildly impractical to deploy venvs for more than one or two apps, especially if they're relatively small. Developing in a local venv can help with that targeting, but there's no substitute for doing that directly on the OS you're deploying to.

Re: How uv got so fast

#455

Post by BiteCode_dev »

zahlman wrote: post #450
BiteCode_dev wrote: post #432

Earlier quoted context omitted.

Real thread are very recent and didn't exist when uv was created. So you needed multiprocesses.

No, I mean why are you starting them for each dependency , rather than having a few workers pulling build requests from a queue?

At least one worker for each virtual cpu core you get for CPU. I got 16 on my laptop. My servers have much more.

If I have 64 cores, and 20 dependencies, I do want the 20 of them to be uncompressed in parallel. That's faster and if I'm installing something, I wanna prioritize that workload.

But it doesn't have to be 20. Even say 5 with queues, that's 100ms. It adds up.

Re: How uv got so fast

#456

Post by SequoiaHope »

Defletter wrote: post #219
WD-42 wrote: post #214

Earlier quoted context omitted.

I don't know what you think "typical Foss projects" are but in my experience they are exactly like your systemd example: one person that does what they want and share it with the world. The rest of your argument doesn't really make any sense with that in mind.

It depends on governance, for want of a better word: if a project has a benevolent dictator then that project will likely be more productive than one that requires consensus building.

Maybe for a project of a given size and popularity? But BDFL projects might be more likely to be smaller. Projects with a lot of contributors might be more likely to need consensus building, but if they are productive at doing so they can be very productive due to their larger size. This is to say, project structure is not the only indicator of productivity.

Re: How uv got so fast

#457

Post by MikePlacid »

optionalsquid wrote: post #185
xvector wrote: post #181

Earlier quoted context omitted.

Hard disagree, most of my coworkers make well north of $1M and office politics is at an all time high. I believe office politics happens when there are simply too many people at a company or org.

Office politics happen when the number of people at an office exceeds 2

If left alone I can argue with myself indefinitely.

Re: How uv got so fast

#458

Post by burntsushi »

formerly_proven wrote: post #441
burntsushi wrote: post #343

Earlier quoted context omitted.

> or you yolo it with reserve+set_len, which is invalid Rust because you didn't properly use MaybeUninit for locations which are only ever written `Vec::spare_capacity_mut`[1] gives you a view into the unused capacity. There's nothing "invalid" about it. [1]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.spa...

Yes, but now you have a slice of MaybeUninit instead of T. This is totally fine for code you control, but out-parameters of the shape &mut [T] are very common for data en/decoding crates, and those require an initialized slice per Rust initialization rules, even if/though most of these will only write to out or reference elements previously written. In practice you can still reserve+set_len, but it is undefined behav…

If something is asking for `&mut [T]`, then yes, it's required that it be initialized. This is a good thing, because a `&mut [T]` permits reading that `T` in safe code, which would be UB if the `T` were uninitialized.

It seems like your complaint is more about "more APIs should accept possibly uninitialized data, but they don't." Which is fair, but I don't know how big of a deal it really is. There is for sure desire to make this work with `std::io::Read`, and indeed, there are unstable APIs for that[1].

[1]: https://doc.rust-lang.org/std/io/trait.Read.html#method.read...

Re: How uv got so fast

#459

Post by dent9 »

zahlman wrote: post #237
baby_souffle wrote: post #157

Earlier quoted context omitted.

I largely agree but don't want to entirely discount the effect that using a compiled language had. At least in my limited experience, the selling point with the most traction is that you don't already need a working python install to get UV. And once you have UV, you can just go! If I had a dollar for every time I've helped somebody untangle the mess of python environment libraries created by an undocumented mix of p…

> the selling point with the most traction is that you don't already need a working python install to get UV. And once you have UV, you can just go! I still genuinely do not understand why this is a serious selling point. Linux systems commonly already provide (and heavily depend upon) a Python distribution which is perfectly suitable for creating virtual environments, and Python on Windows is provided by a tradition…

> I still genuinely do not understand why this is a serious selling point. Linux systems commonly already provide (and heavily depend upon) a Python distribution

Sounds like you've never actually used Python. You should never, ever be using the system Python for anything you need to run yourself. Don't even touch it. It's a great way to break your entire system. Many distros have stopped providing it at all, for good reason.

The first step every Python dev has to take on every single system they want to run their project on is to install their own sandboxed version of Python and it's libraries and it's library manager. Alternatively you pre build a docker container with it all packed inside which is the same basic thing.

Better option still is to simply ditch Python and switch to compiled languages that don't have this stupid problems.

Re: How uv got so fast

#460

Post by dent9 »

MrDrMcCoy wrote: post #453
whakim wrote: post #368

Earlier quoted context omitted.

> Most of the time you don't need a different Python version from the system one. Except for literally anytime you’re collaborating with anyone, ever? I can’t even begin to imagine working on a project where folks just use whatever python version their OS happens to ship with. Do you also just ship the latest version of whatever container because most of the time nothing has changed?

If you're writing Python tools to support OS operations in prod, you need to target the system Python. It's wildly impractical to deploy venvs for more than one or two apps, especially if they're relatively small. Developing in a local venv can help with that targeting, but there's no substitute for doing that directly on the OS you're deploying to.

This is why you DON'T write system tools in Python in the first place. Use a real language that compiles to a native self contained binary that doesn't need dependency installing. Or you use a container. This has been a solved problem for decades. Python users have been trying to drag the entire computing world backwards this whole time because their insistence on using a toy language invented to be the JavaScript of the server, as an actual production grade bare metal system language
Post reply on HN