Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A year of uv: pros, cons, and should you migrate

bitecode.dev

Post reply on HN
Print view View original

111–120 of 401 posts

Re: A year of uv: pros, cons, and should you migrate

#111

Post by gregmac »

woodruffw wrote: post #104
8n4vidtmkvmk wrote: post #86

Earlier quoted context omitted.

You know what we need? In both python and JS, and every other scripting language, we should be able to import packages from a url, but with a sha384 integrity check like exists in HTML. Not sure why they didn't adopt this into JS or Deno. Otherwise installing random scripts is a security risk

Python has fully-hashed requirements[1], which is what you'd use to assert the integrity of your dependencies. These work with both `pip` and `uv`. You can't use them to directly import the package, but that's more because "packages" aren't really part of Python's import machinery at all. (Note that hashes themselves don't make "random scripts" not a security risk, since asserting the hash of malware doesn't make it…

Good point, but it's still a very useful way to ensure it doesn't get swapped out underneath you.

Transitive dependencies are still a problem though. You kind of fall back to needing a lock file or specifying everything explicitly.

Re: A year of uv: pros, cons, and should you migrate

#112

Post by conradev »

d0mine wrote: post #81
conradev wrote: post #24

Earlier quoted context omitted.

uv should hard link files if they’re identical like Nix does If a package manager stores more than it needs to, it is a package manager problem.

What makes you think it doesn't?

This from above:

> I just used find to locate as many libtorch_cpu.dylib files as possible on my laptop and deleted 5.5GB of them

but maybe it wasn’t actually 5.5 GB!

Re: A year of uv: pros, cons, and should you migrate

#113

Post by zerkten »

sieve wrote: post #36
woodruffw wrote: post #34

Earlier quoted context omitted.

> Something to do with breakage in ABI perhaps. Was looking at the way python implements extensions the other day. Very weird. Yes, it's essentially that: CPython doesn't guarantee exact ABI stability between versions unless the extension (and its enclosing package) intentionally build against the stable ABI[1]. The courteous thing to do in the Python packaging ecosystem is to build "abi3" wheels that are stable and…

My use of python is somewhat recent. But the two languages that I have used a lot of - Java and JS - have interpreters that were heavily optimized over time. I wonder why that never happened with python and, instead, everyone continues to write their critical code in C/Rust. I am planning to shift some of my stuff to pypy (so a "fast" python exists, kind of). But some dependencies can be problematic, I have heard.

>> My use of python is somewhat recent. But the two languages that I have used a lot of - Java and JS - have interpreters that were heavily optimized over time. I wonder why that never happened with python and, instead, everyone continues to write their critical code in C/Rust.

Improving Python performance has been a topic as far back as 2008 when I attended my first PyCon. A quick detour on Python 3 because there is some historical revisionism because many online people weren't around in the earlier days.

Back then the big migration to Python 3 was in front of the community. The timeline concerns that popped up when Python really got steam in the industry between 2012 and 2015 weren't as huge a concern. You can refer to Guido's talks from PyCon 2008 and 2009 if they are available somewhere to get the vibe on the urgency. Python is impactful because it changes the language and platform while requiring a massive amount of effort.

Back to perf. Around 2008, there was a feeling that an alternative to CPython might be the future. Candidates included IronPython, Jython, and PyPy. Others like Unladen Swallow wanted to make major changes to CPython (https://peps.python.org/pep-3146/).

Removing the GIL was another direction people wanted to take because it seemed simpler in a way. This is a well researched area with David Beazley having many talks like this oldie (https://www.youtube.com/watch?v=ph374fJqFPE). The idea is much older (https://dabeaz.blogspot.com/2011/08/inside-look-at-gil-remov...).

All of these alternative implementations of Python from this time period have basically failed at the goal of replacing CPython. IronPython was a Python 2 implmentation and updating to Python 3 while trying grow to challenge CPython was impossible. Eventually, Microsoft lost interest and that was that. Similar things happened for the others.

GIL removal was a constant topic from 2008 until recently. Compatibility of extensions was a major concern causing inertia and the popularity meant even more C/C++/Rust code relying on a GIL. The option to disable (https://peps.python.org/pep-0703/) only happened because the groundwork was eventually done properly to help the community move.

The JVM has very clearly defined interfaces and specs similar to the CLR which make optimization viable. JS doesn't have the compatibility concerns.

That was just a rough overview but many of the stories of Python woes miss a lot of this context. Many discussions about perf over the years have descended into a GIL discussion without any data to show the GIL would change performance. People love to talk about it but turn out to be IO-bound when you profile code.

Re: A year of uv: pros, cons, and should you migrate

#114

Post by nabla9 »

barosl wrote: post #43

A very well written article! I admire the analysis done by the author regarding the difficulties of Python packaging. With the advent of uv, I'm finally feeling like Python packaging is solved. As mentioned in the article, being able to have inline dependencies in a single-file Python script and running it naturally is just beautiful. #!/usr/bin/env -S uv run # /// script # dependencies = ['requests', 'beautifulsoup4…

> by the author

And the author is?

Re: A year of uv: pros, cons, and should you migrate

#115

Post by ziml77 »

claytonjy wrote: post #52

A scenario for "don't use uv" I hope none of you encounter: many nvidia libraries not packaged up in something better like torch. Here's just one example, nemo2riva, the first in several steps to taking a trained NeMo model and making it deployable: https://github.com/nvidia-riva/nemo2riva?tab=readme-ov-file#... before you can install the package, you first have to install some other package whose only purpose is to…

Surely you can just manually add their index, right?

Re: A year of uv: pros, cons, and should you migrate

#116

Post by conradev »

Spivak wrote: post #105
conradev wrote: post #24

Earlier quoted context omitted.

uv should hard link files if they’re identical like Nix does If a package manager stores more than it needs to, it is a package manager problem.

You're about to be pleasantly surprised then. https://docs.astral.sh/uv/reference/settings/#link-mode It's even the default. Here's where it's implemented if you're curious https://github.com/astral-sh/uv/blob/f394f7245377b6368b9412d...

That is very cool! I would imagine that even if they didn’t implement explicit cloning, APFS would still clone the files.

`nix-store --optimize` is a little different because it looks for duplicates and hardlinks those files across all of the packages. I don’t know how much additional savings that would yield with a typical uv site-packages, but it guarantees no duplicate files.

Re: A year of uv: pros, cons, and should you migrate

#117

Post by lijok »

claytonjy wrote: post #52

A scenario for "don't use uv" I hope none of you encounter: many nvidia libraries not packaged up in something better like torch. Here's just one example, nemo2riva, the first in several steps to taking a trained NeMo model and making it deployable: https://github.com/nvidia-riva/nemo2riva?tab=readme-ov-file#... before you can install the package, you first have to install some other package whose only purpose is to…

Instead of installing nvidia-pyindex, use https://docs.astral.sh/uv/configuration/indexes/ to configure the index nvidia-pyindex points to.

Re: A year of uv: pros, cons, and should you migrate

#118

Post by aaravchen »

locusofself wrote: post #41

I've been mostly out of the python game for quite a while, but I never had that much issue with: pip install -r requirements.txt . Seems like a lot of people have tried their hand at various tooling, so there must be more to it than I am aware of.

A big thing that trips people up until they try to use a public project (from source) or an older project, is the concept of a dependencies file and a lock file.

The dependency file (what requirements.txt is supposed to be), just documents the things you depend on directly, and possibly known version constraints. A lock file captures the exact version of your direct and indirect dependencies at the moment in time it's generated. When you go to use the project, it will read the lock file, if it exists, and match those versions for anything listed directly or indirectly in the dependency file. It's like keeping a snapshot of the exact last-working dependency configuration. You can always tell it to update the lock file and it will try to recaclulate everything from latest that meets your dependency constraints in the dependency file, but if something doesn't work with that you'll presumably have your old lock file to fall back on _that will still work_.

It's a standard issue/pattern in all dependency managers, but it's only been getting attention for a handful of years with the focus on reproducibility for supply chain verification/security. It has the side effect of helping old projects keep working much longer though. Python has had multiple competing options for solutions, and only in the lat couple years did they pick a format winner.

Re: A year of uv: pros, cons, and should you migrate

#120

Post by aaravchen »

code_biologist wrote: post #13
belval wrote: post #7

> I had a friend who decided to not use uv, because the first time he used it, it was on a 15 years old codebase that had just been migrated to Python 3. It was standing on a pile of never cleaned up pip freeze exports, and uv could not make it work. This is my only gripe with uv, despite how the author decided to depict it, this really turns into a headache fast as soon as you have ~4-5 in-house packages. I don't th…

Did they run into a hard blocker, or was it just that using version overrides was possible but painful? I started looking seriously at uv/pdm once poetry made it entirely clear they didn't intend to support version overrides [1]. uv's support for overrides seems serviceable if unsophisticated [2][3]. [1] https://github.com/python-poetry/poetry/issues/697 [2] https://docs.astral.sh/uv/concepts/resolution/#dependency-o…

The linked poetry Issue is pretty understandable why they aren't going to support it. I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. Point blank, that's a packaging failure and the solution is, and always has been, to immediately yank the offending package. That said, the Python case has pretty limited ability to specify dependency package versions, which makes it nigh impossible to handle it downstream by blocklisting specific versions from an otherwise contiguous range.

Take for example the werkzeug package that released a breaking API regression in a patch release version. It didn't affect everyone, but notably did affect certain Flask(?) use cases that used werkzeug as a dependency. In a sane system, either werkzeug immediately removes the last released version as buggy (and optionally re-releases it as a non-backwards compatible SemVer change), or everyone starts looks for an alternative to non-compliant werkzeug. Pragmatically though, Python dependency specification syntax should have a way for Flask to specify, in a patch release of its own, that werkzeug up to the next minor version, _but excluding a specific range of patch versions_, is a dependency. Allowing them to monkey patch the problem in the short term.

It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though, which is what was requested from the poetry tool.

Post reply on HN