Live data from Hacker News

A year of uv: pros, cons, and should you migrate

bitecode.dev

371–380 of 401 posts

Re: A year of uv: pros, cons, and should you migrate

#371

Earlier quoted context omitted.

You know what we need? In both python and JS, and every other scripting language, we should be able to import packages from a url, but with a sha384 integrity check like exists in HTML. Not sure why they didn't adopt this into JS or Deno. Otherwise installing random scripts is a security risk

Deno and npm both store the hashes of all the dependencies you use in a lock file and verify them on future reinstalls.

The lockfile is good, but I'm talking about this inline dependency syntax,

  # dependencies = ['requests', 'beautifulsoup4']
And likewise, Deno can import by URL. Neither include an integrity hash. For JS, I'd suggest

    import * as goodlib from 'https://verysecure.com/notmalicious.mjs' with { integrity="sha384-xxx" }
which mirrors https://developer.mozilla.org/en-US/docs/Web/Security/Subres... and https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

The Python/UV thing will have to come up with some syntax, I don't know what. Not sure if there's a precedent for attributes.

Re: A year of uv: pros, cons, and should you migrate

#372

Earlier quoted context omitted.

You know what we need? In both python and JS, and every other scripting language, we should be able to import packages from a url, but with a sha384 integrity check like exists in HTML. Not sure why they didn't adopt this into JS or Deno. Otherwise installing random scripts is a security risk

Where do you initially get the magical sha384 hash that proves the integrity of the package the first time it's imported?

Same way we do in JS-land: https://developer.mozilla.org/en-US/docs/Web/Security/Subres...

tl;dr use `openssl` on command-line to compute the hash.

Ideally, any package repositories ought to publish the hash for your convenience.

This of course does nothing to prove that the package is safe to use, just that it won't change out from under your nose.

Re: A year of uv: pros, cons, and should you migrate

#373
post #245

Earlier quoted context omitted.

Have been using ZFS for the past thirteen years and all my workflows including backup are based on it. It just works.

Sure, I was just curious, since you mentioned not wanting to use ZFS without kernel support and BTRFS does have that. Being familiar with ZFS, I guess is a decent explanation.

When the topic of backups came up last year, I talked about my current solution: https://news.ycombinator.com/item?id=41042790. Someone suggested a workaround in the form of zfsbootmenu but I decided to stick to the simple way of doing things.

Re: A year of uv: pros, cons, and should you migrate

#374

> You don't even need to know there is a venv, or what activation means. > All those commands update the lock file automatically and transparently.... It's all taken care of. When is the python community going to realize that simple is the opposite of easy? I don't see how hiding these aspects is desirable at all; I want to know how my programming tools work! With all due respect to the author, I don't like the assum…

There is simplicity of interface and then of implementation. If you try uv you will find it is both convenient and easier to understand than competing solutions, because everything _just works_ and you find the proof of how it works waiting in your `git status`. You can be assisted in knowing that it just works because its install takes no time and no system setup. It is slick.

Re: A year of uv: pros, cons, and should you migrate

#375

Earlier quoted context omitted.

A big thing that trips people up until they try to use a public project (from source) or an older project, is the concept of a dependencies file and a lock file. The dependency file (what requirements.txt is supposed to be), just documents the things you depend on directly, and possibly known version constraints. A lock file captures the exact version of your direct and indirect dependencies at the moment in time it'…

Then why even have the dependency file? If the dependency file is wrong, and describes versions that are incompatible with the project, it should be fixed. Duplicating that information elsewhere is wrong. Lockfiles have a very obvious use case: Replicable builds across machines in CI. You want to ensure that all the builds in the farm are testing the same thing across multiple runs, and that new behaviors aren't intr…

>Then why even have the dependency file?

Horses for courses.

Dependency files - whether the project's requirements (or optional requirements, or in the future, other arbitrary dependency groups) in `pyproject.toml`, or a list in a `requirements.txt` file (the filename here is actually arbitrary) don't describe versions at all, in general. Their purpose is to describe what's needed to support the current code: its direct dependencies, with only as much restriction on versions as is required. The base assumption is that if a new version of a dependency comes out, it's still expected to work (unless a cap is set explicitly), and has a good chance of improving things in general (better UI, more performant, whatever). This is suitable for library development: when others will cite your code as a dependency, you avoid placing unnecessary restrictions on their environment.

Lockfiles are meant to describe the exact version of everything that should be in the environment to have exact reproducible behaviour (not just "working"), including transitive dependencies. The base assumption is that any change to anything in the environment introduces an unacceptable risk; this is the tested configuration. This is suitable for application development: your project is necessarily the end of the line, so you expect others to be maximally conservative in meeting your specific needs.

You could also take this as an application of Postel's Law.

>Lockfiles have a very obvious use case: Replicable builds across machines in CI.

There are others who'd like to replicate their builds: application developers who don't want to risk getting bug reports for problems that turn out to be caused by upstream updates.

> You should not use lockfiles as a "backup" to pyproject.toml. The version constraints in pyproject.toml should be correct. If you need to restrict to a single specific version, do so, "== 2.2.9" works fine.

In principle, if you need a lockfile, you aren't distributing a library package anyway. But the Python ecosystem is still geared around the idea that "applications" would be distributed the same way as libraries - as wheels on PyPI, which get set up in an environment, using the entry points specified in `pyproject.toml` to create executable wrappers. Pipx implements this (and rejects installation when no entry points are defined); but the installation will still ignore any `requirements.txt` file (again, the filename is arbitrary; but also, Pipx is delegating to Pip's ordinary library installation process, not passing `-r`).

You can pin every version in `pyproject.toml`. Your transitive dependencies still won't be pinned that way. You can explicitly pin those, if you've done the resolution. You still won't have hashes or any other supply-chain info in `pyproject.toml`, because there's nowhere to put it. (Previous suggestions of including actual lockfile data in `pyproject.toml` have been strongly rejected - IIRC, Hatch developer Ofek Lev was especially opposed to this.)

Perhaps in the post-PEP 751 future, this could change. PEP 751 specifies both a standard lockfile format (with all the sorts of metadata that various tools might want) and a standard filename (or at least filename pattern). A future version of Pipx could treat `pylock.toml` as the "compiled" version of the "source" dependencies in `pyproject.toml`, much like Pip (and other installers) treat `PKG-INFO` (in an sdist, or `METADATA` in a wheel) as the "compiled" version (dependency resolution notwithstanding!) of other metadata.

Re: A year of uv: pros, cons, and should you migrate

#376
post #220

Earlier quoted context omitted.

I create a .venv directory for each project(even for those test projects named pytest, djangotest). And each project has its own requirements file. Personally, Python packaging has never been a problem.

What do you do when you accidentally run pip install -r requirements.txt with the wrong .venv activated? If your answer is "delete the venv and recreate it", what do you do when your code now has a bunch of errors it didn't have before? If your answer is "ignore it", what do you do when you try to run the project on a new system and find half the imports are missing? None of these problems are insurmountable of cours…

>What do you do when you accidentally run pip install -r requirements.txt with the wrong .venv activated?

As someone with a similar approach (not using requirements.txt, but using all the basic tools and not using any kind of workflow tool or sophisticated package manager), I don't understand the question. I just have a workflow where this isn't feasible.

Why would the wrong venv be activated?

I activate a venv according to the project I'm currently working on. If the venv for my current code isn't active, it's because nothing is active. And I use my one global Pip through a wrapper, which (politely and tersely) bonks me if I don't have a virtual environment active. (Other users could rely on the distro bonking them, assuming Python>=3.11. But my global Pip is actually the Pipx-vendored one, so I protect myself from installing into its environment.)

You might as well be asking Poetry or uv users: "what do you do when you 'accidentally' manually copy another project's pyproject.toml over the current one and then try to update?" I'm pretty sure they won't be able to protect you from that.

>If your answer is "delete the venv and recreate it", what do you do when your code now has a bunch of errors it didn't have before?

If it did somehow happen, that would be the approach - but the code simply wouldn't have those errors. Because that venv has its own up-to-date listing of requirements; so when I recreated the venv, it would naturally just contain what it needs to. If the listing were somehow out of date, I would have to fix that anyway, and this would be a prompt to do so. Do tools like Poetry and uv scan my source code and somehow figure out what dependencies (and versions) I need? If not, I'm not any further behind here.

>And of course they become a lot harder when you try to work with someone else's project, or come back to a project from a couple of years ago and find it doesn't work.

I spent this morning exploring ways to install Pip 0.2 in a Python 2.7 virtual environment, "cleanly" (i.e. without directly editing/moving/copying stuff) starting from scratch with system Python 3.12. (It can't be done directly, for a variety of reasons; the simplest approach is to let a specific version of `virtualenv` make the environment with an "up-to-date" 20.3.4 Pip bootstrap, and then have that Pip downgrade itself.)

I can deal with someone else's (or past me's) requirements.txt being a little wonky.

Re: A year of uv: pros, cons, and should you migrate

#377

I've been mostly out of the python game for quite a while, but I never had that much issue with: pip install -r requirements.txt . Seems like a lot of people have tried their hand at various tooling, so there must be more to it than I am aware of.

I write internal tools using Python at work. These tools are often used by non-Python devs. I am so, so tired of adding a blurb on creating a venv. (Of course, the alternative—"install this software you've never heard of"—isn't fantastic either. But once they do have it, it'd be pretty neat to be able to tell them to just "uvx ".)

Is it not the same blurb every time that you could copy and paste?

Or you can make sure you have an entry point - probably a better UX for your coworkers anyway - and run them through a `pipx` install.

Or you could supply your own Bash script or whatever.

Or since you could use a simple packager like pex (https://docs.pex-tool.org/). (That one even allows you to embed a Python executable, if you need to and if you don't have to worry about different platforms.) Maybe even the standard library `zipapp` works for your needs.

Re: A year of uv: pros, cons, and should you migrate

#378

Earlier quoted context omitted.

You don't need to activate anything with uv, all commands do it in the venv automatically, and including uv run.

How exactly does `uv` determine which is "the" venv? Is it simply based on the working directory like the `direnv`/`autoenv`/etc. workflows others are describing here? It does seem like people have use cases for running code in a different environment vs. the one being actively used to develop the package.

It uses the content of the UV_PROJECT_ENVIRONMENT var env, which defaults to .venv. If you pass --active, it can also look whatever is in VIRTUAL_ENV and CONDA_PREFIX.

You can also force an env passing a complete path to --python

Re: A year of uv: pros, cons, and should you migrate

#379
post #343

The problem I have with uv is that it is not opinionated enough, or complete enough. It still needs a backend for building the package, and you still have a choice of backends. In other words, it is a nice frontend to hide the mess that is the Python packaging ecosystem, but the mess of an ecosystem is still there, and you still have to deal with it. You'll still have to go through hatchling's docs to figure out how…

>In other words, it is a nice frontend to hide the mess that is the Python packaging ecosystem, but the mess of an ecosystem is still there, and you still have to deal with it. You'll still have to go through hatchling's docs to figure out how to do x/y/z. You'll still have to switch from hatchling to flit/pdm/setuptools/... if you run into a limitation of hatchling. As a package author, you're never using uv, you're using uv+hatchling (or uv+something) and a big part of your pyproject.toml are not uv's configuration, it is hatchling configuration.

Strange. The main reason I'm not using uv or its competitors (or going back to Poetry, where I was using only a tiny fraction of the functionality) is that they're all too opinionated and all-in-one for me. If I had to write down a top 10 of reasons the Python packaging ecosystem is a "mess", and give detailed reasoning, probably at least 6 of them would be problems with Pip specifically (including things that appear to be problems in other tools, but which are really Pip's fault). And at least one more would be "Setuptools is a massive pile of backwards-compatibility wrappers that are mostly useless with the modern packaging flow, that didn't even directly make the wheels (relying on a separate dependency instead) until 70.1".

(The other reason I wouldn't go back to Poetry is because Masonry was a terrible, non-standards-compliant experience for me, and its installation procedures changed repeatedly over time, and you'd end up not being able to uninstall old versions cleanly.)

(Hatchling and Setuptools are build backends, yes; where you say "flit" I assume you mean flit-core, and similarly pdm-backend for PDM.)

If uv provided its own backend, there would still be a risk of running into limitations with that backend; and regardless you'd have to go through its docs to figure out how to do x/y/z with it.

Being able to experiment with different build backends was part of the explicit rationale for `pyproject.toml` in the first place. The authors of PEP 517 and 518 consciously expected to see competing backends pop up (and explicitly designed a system that would allow that competition, and allow for Pip etc. to know how to invoke every backend), and did not (per my understanding of Python Discourse forum discussion) consciously expect to see competing workflow tools pop up.

I'm making a build backend because I have my own opinion. I don't have any interest in making a workflow tool. I want people who like uv to be able to use my build backend. That's how the system was designed to work.

I don't want someone to integrate the baseline and make all the decisions for me. I want a better-quality baseline. Which is why I'm also making an installer/environment manager. `build` is a perfectly fine build frontend. I don't need or want a replacement.

Re: A year of uv: pros, cons, and should you migrate

#380

Earlier quoted context omitted.

The linked poetry Issue is pretty understandable why they aren't going to support it. I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. Point blank, that's a packaging failure and the solution is, and always has been, to immediately yank the offending package. That said, the Python case has pretty li…

I've honestly never heard of any dependency resolver that allows you to dynamically inject an override of a package's built in specification for an indirect dependency. npm and yarn both let you do it. PDM and uv think about it differently, but both allow overrides. It should never be on the end user to be specifying overrides of indirect dependency specifications at the top level though, which is what was requested…

>I just want to use Django package XYZ that says it's only compatible with Django 3.X on Django 4.

Indeed. One of the biggest recurring themes I've seen in Python packaging discussion is that package metadata can't be updated after the fact. When you publish something that depends on the just-released foolib N, you don't know if it will be compatible with foolib N+1 (you might not even be completely sure it will work with N.0.1) because it doesn't even exist yet so you can't possibly test it. In other languages, where the environment can contain multiple versions of the same library, it's common to assume it won't. For Python, there are lots of good reasons to assume it will (https://iscinumpy.dev/post/bound-version-constraints/), but people will blame you when you're wrong and the fix isn't straightforward. (AIUI, the best you can do is make a .post1 release with the updated metadata, and "yank" the existing release.)

Post reply on HN