Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uv: Running a script with dependencies

docs.astral.sh

Post reply on HN
Print view View original

141–150 of 174 posts

Re: Uv: Running a script with dependencies

#141

Post by zahlman »

sc077y wrote: post #131
slightwinder wrote: post #122

Earlier quoted context omitted.

> Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies, Be aware that uv will create a full copy of that environment for each script by default. Depending on your number of scripts, this could become wasteful really fast. There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default, or which disadvantages…

By default it will create hard links for python packages, so it won't consume any more memory (besides the small overhead of hard links).

As far as I can tell, this only applies to the wheel contents, not to the .pyc bytecode cache created by the interpreter. If you use the defaults, Python will just create per-environment copies on demand; if you precompile with `--compile-bytecode`, uv will put the results directly in the installed copy rather than caching them and hard-linking from there.

I plan on offering this kind of cached precompiled bytecode in PAPER, but I know that compiled bytecode includes absolute paths (used for displaying stack traces) that will presumably then refer to the cached copies. I'll want to test as best I can that this doesn't break anything more subtle.

Re: Uv: Running a script with dependencies

#142

Post by akshayka »

staplung wrote: post #48

Love this feature of UV. Here's a one-liner to launch jupyter notebook without even "installing" it: uv run --with jupyter jupyter notebook Everything is put into a temporary virtual environment that's cleaned up afterwards. Best thing is that if you run it from a project it will pick up those dependencies as well.

Very nice. Here's a one liner for marimo notebooks:

uvx marimo edit

A one liner with marimo that also respects (and records) inline script metadata using uv:

uvx marimo edit --sandbox my_notebook.py

https://docs.astral.sh/uv/guides/integration/marimo/

Re: Uv: Running a script with dependencies

#143

Post by Hackbraten »

zahlman wrote: post #140
Hackbraten wrote: post #133

Earlier quoted context omitted.

Unless it can't because you happen to have mounted your user cache directory from a different volume in an attempt to debloat your hourly backups.

... your hourly backups aren't also using the same hard-linking strategy?

They’re ZFS snapshots, so no. That’s why I’m forced to keep my cache directory in a different dataset.

Ironically, if my backup scheme were using hard links, then I could simply exclude the cache directory from backup, so I’d have no reason to do that mountpoint spiel, and uv’s hard links would work normally.

Re: Uv: Running a script with dependencies

#144

Post by zahlman »

slightwinder wrote: post #122
simonw wrote: post #4

The "declaring script dependencies" thing is incredibly useful: https://docs.astral.sh/uv/guides/scripts/#declaring-script-d... # /// script # dependencies = [ # "requests Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies, magically installed into a temporary virtual environment without you having to think about them at all. It's an implementation of Python PEP 723: h…

> Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies, Be aware that uv will create a full copy of that environment for each script by default. Depending on your number of scripts, this could become wasteful really fast. There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default, or which disadvantages…

>There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default

The hard-link strategy still saves disk space — although not as much (see other replies and discussion).

Possible reasons not to symlink that I can think of:

* stack traces in uncaught exceptions might be wonky (I haven't tried it)

* it presumably adds a tiny hit to imports from Python, but I find it hard to imagine anyone caring

* with the hard-linking strategy, you can purge the cache (perhaps accidentally?) without affecting existing environments

Re: Uv: Running a script with dependencies

#145

Post by zahlman »

Hackbraten wrote: post #143
zahlman wrote: post #140

Earlier quoted context omitted.

... your hourly backups aren't also using the same hard-linking strategy?

They’re ZFS snapshots, so no. That’s why I’m forced to keep my cache directory in a different dataset. Ironically, if my backup scheme were using hard links, then I could simply exclude the cache directory from backup, so I’d have no reason to do that mountpoint spiel, and uv’s hard links would work normally.

... ZFS snapshots don't produce copies of the underlying file data, do they? (Yeah, I'm still a dinosaur on ext4...)

Re: Uv: Running a script with dependencies

#146

Post by Hackbraten »

zahlman wrote: post #145
Hackbraten wrote: post #143

Earlier quoted context omitted.

They’re ZFS snapshots, so no. That’s why I’m forced to keep my cache directory in a different dataset. Ironically, if my backup scheme were using hard links, then I could simply exclude the cache directory from backup, so I’d have no reason to do that mountpoint spiel, and uv’s hard links would work normally.

... ZFS snapshots don't produce copies of the underlying file data, do they? (Yeah, I'm still a dinosaur on ext4...)

Nothing wrong with ext4! Having choices and preferences is a good thing.

You’re correct, ZFS snapshots don’t produce copies, at least not at the time they’re being created. They work a little like copy-on-write.

Re: Uv: Running a script with dependencies

#147

Post by zahlman »

sgarland wrote: post #107
tayloramurphy wrote: post #12

Earlier quoted context omitted.

I mean, who doesn't love requests?

Me, because nearly every time I see it used, it’s for a trivial request or two that can easily be handled with stdlib. If you need it, sure, it’s great, but let’s not encourage pulling in 3rd party libs unnecessarily.

Strong agree. Pip doesn't really need Requests' functionality as far as I can tell, but the vendored transitive dependencies represent a considerable fraction (something like a quarter IIRC) of Pip's bulk. And a big fraction of that bulk (and it's much the same for Rich) at startup, even if ultimately it turns out that no web requests need to be made. Which in turn is the main reason why Pip on my machine takes longer than the https://lawsofux.com/doherty-threshold/ to process `pip install` with no actual package specified. (A process that involves importing more than five hundred Python modules, of which almost a hundred are Requests and its dependencies.)

(Of the non-Requests imports, about two thirds of them occur before Pip even considers what's on the command line — which means that they will be repeated when you use the `--python` option. Of course, Requests isn't to blame for that, but it drives home the point about keeping dependencies under control.)

Re: Uv: Running a script with dependencies

#148

Post by zahlman »

trklausss wrote: post #92
simonw wrote: post #4

The "declaring script dependencies" thing is incredibly useful: https://docs.astral.sh/uv/guides/scripts/#declaring-script-d... # /// script # dependencies = [ # "requests Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies, magically installed into a temporary virtual environment without you having to think about them at all. It's an implementation of Python PEP 723: h…

My question is: if you already have to put the "import" there, why not have a PEP to specify the version in that same import statement, and let `uv` scan that dependency? It's something I never understood.

This is addressed in detail the relevant ecosystem-wide standard: https://peps.python.org/pep-0723/#why-not-infer-the-requirem...

My own take:

PEP 723 had a deliberate goal of not making extra work for core devs, and not requiring tools to parse Python code. Note that if you put inline comments on the import lines, that wouldn't affect core devs, but would still complicate matters for tools. Python's import statement is just another statement that occurs at runtime and can appear anywhere in the code.

Besides that, trying to associate version numbers with the imported module is a complete non-starter. Version numbers belong to the distributions[1], which may define zero or more top-level packages. The distribution name may be completely independent from the names of what is imported, and need not even be a valid identifier (there is a standard normalization process, but that still allows your distribution's name to start with a digit, for example).

[1]: You say "packages", but I'm trying to avoid the unfortunate overloading the term. PyPA recommends (https://packaging.python.org/en/latest/discussions/distribut...) "distribution package" for what you download from PyPI, and "import package" for what you import in the code; I think this is a bit unwieldy.

Re: Uv: Running a script with dependencies

#149

Post by zahlman »

freddie_mercury wrote: post #111
Bluestein wrote: post #108

Earlier quoted context omitted.

I am - while awesomely valuing uv - also with the part of the audience wondering/wishing this capability were a language feature .-

The PEP explains why it isn't part of the regular python syntax. uv and other tools would be forced to implement a full Python parser. And since the language changes they would need to update their parser when the language changes. This approach doesn't have that problem. Making it a "language feature" has no upside and lots of downside. As the PEP explains.

Beyond that, the needed name to download from PyPI doesn't necessarily have anything at all to do with the name used for an `import` statement. And a given PyPI download may satisfy multiple `import` statements. And it can even be possible to require a download from PyPI that doesn't install any importable code (it may legally be a meta-package that runs one-shot configuration code when "built from source").

Re: Uv: Running a script with dependencies

#150

Post by zahlman »

NeutralForest wrote: post #134
linhns wrote: post #77

Earlier quoted context omitted.

conda is a completely no-go for me. It’s for data scientists, and only data scientists who do not need to care much about production.

Sadly this isn't always true, if you have non-pure Python dependencies, you might still need to use conda since they package database drivers or complex C/C++ dependencies for example.

SciPy already provides pip-installable wheels for a wide variety of platforms: https://pypi.org/project/scipy/#files
Post reply on HN