Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uv: Running a script with dependencies

docs.astral.sh

Post reply on HN
Print view View original

151–160 of 174 posts

Re: Uv: Running a script with dependencies

#151

Post by zahlman »

do_not_redeem wrote: post #57
indigodaddy wrote: post #54

Earlier quoted context omitted.

But don't you have to only ever run it once to have the deps/venv for subsequent runs?

I don't know if uv garbage collects its cache, but it wouldn't surprise me. Otherwise disk usage would grow indefinitely.

If you indefinitely grow the collection of packages that you have in use in permanent virtual environments, then yes your disk usage will and should grow indefinitely anyway.

There's not much point recycling contents from a package cache unless you currently don't have a venv using that package and also don't reasonably expect to have one in the near future.

And I'm saying this as the guy complaining all the time about things like the size of Numpy wheels.

Re: Uv: Running a script with dependencies

#152

Post by zahlman »

Bluestein wrote: post #109
hoherd wrote: post #41

One gotcha I caught myself in with this technique is using it in a script that would remediate a situation where my home has lost internet and needed the router to be power cycled. When the internet is out, `uv` cannot download the dependencies specified in the script, and the script would fail. Thankfully I noticed this problem after writing it but before needing it to actually work, and refactored my setup to pre-i…

Here's a fun one (says he, in a panic): Will we get to a point where (through fault or, gasp, design) a given AI will generatively code a missing dependency, on the fly - perhaps as a "last ditch" effort? (I can imagine languages having official LLMs which would more or less "compress/know" enough of the language to be an ... ... import of last resort, of sorts, by virtue of which an approximation to the missing code…

I'm pretty sure we've already reached a point where people generatively code possibly-malicious packages to publish under names of non-existent packages commonly hallucinated by LLMs prompted by other devs.

Re: Uv: Running a script with dependencies

#153

Post by zahlman »

theesm wrote: post #99

I've been doing something similiar utilizing guix shell[0] setting my shebang to e.g.: #!/usr/bin/env -S guix shell python python-requests python-pandas -- python3 in scripts for including per-script dependencies. This is language agnostic as long as the interpreter and its dependencies are available as guix packages. I think there may be a similiar approach for utilizing nix shells that way as well. [0]: https://gui…

Do these environments reuse existing system-installed packages and hard-link the files?

Re: Uv: Running a script with dependencies

#154

Post by zahlman »

tame3902 wrote: post #81
alkh wrote: post #18

Earlier quoted context omitted.

Interesting, I guess it is indeed a BSD thing, cause this works for me with or without '-S' on Mac

There currently is a patch for adding '-S' to OpenBSD and in the discussion, the one who came originally up with it commented on how he added it to FreeBSD: "IIRC, the catalyst for it was that early FreeBSD (1990's?) did split up the words on the '#!' line because that seemed convenient. Years later, someone else noticed that this behavior did not match '#!' processing on any other unix, so they changed the behavior…

FWIW, GNU Coreutils env (as seen for example in my Linux Mint distribution) works the same way (no split by default, enabled by `-S`) and I definitely have used it locally to do things like `#!/usr/bin/env -S python -S -I`.

Re: Uv: Running a script with dependencies

#155

Post by zahlman »

satertek wrote: post #7

Why doesn't pip support PEP 723? I'm all for spreading the love of our lord and savior uv, but it should be necessary to have an official implementation.

It's completely out of scope for pip, which is purely about modifying the set of packages installed into non-transient environments.

Pipx is a wrapper that does more or less what you're looking for (including PEP 723 support), but it arbitrarily refuses to process top-level packages unless they specify an entry point (which makes them "applications" even with abstract dependencies).

I'm planning to support it in PAPER, which can roughly be described as my vision of what pip and pipx, taken together, should have been.

Re: Uv: Running a script with dependencies

#156

Post by skissane »

lenzm wrote: post #129
pjc50 wrote: post #93

Earlier quoted context omitted.

Semantic whitespace was bad enough, now we have semantic comment blocks? I'm mostly joking, but normally when people say language syntax they mean something outside a comment block.

Shebang lines are semantic comments too

Not necessarily-while many languages which accept shebangs use hash as a line comment introducer, others don’t, but nonetheless will accept a shebang at the start of a file only. e.g. some Lisp interpreters will ignore the first line of a source file if it starts with #!, but otherwise don’t accept # as comment syntax.

Re: Uv: Running a script with dependencies

#157

Post by crabbone »

simonw wrote: post #4

The "declaring script dependencies" thing is incredibly useful: https://docs.astral.sh/uv/guides/scripts/#declaring-script-d... # /// script # dependencies = [ # "requests Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies, magically installed into a temporary virtual environment without you having to think about them at all. It's an implementation of Python PEP 723: h…

> without you having to think about them at all.

Hahahahaha.

Oh. I'm rolling on the floor. Hahahahaha.

How do you never learn? No, honestly, how do you never learn this simple thing: it will break! I will bet my pension on that it will break, and perhaps not you, but some hundreds of developers will have to try to debug and try to figure out where the dependencies went and why they weren't installed correctly, or why something was missing and so on.

There will never be a situation that you don't have to think about something as important as dependencies at all.

Re: Uv: Running a script with dependencies

#158

Post by Bluestein »

zahlman wrote: post #152
Bluestein wrote: post #109

Earlier quoted context omitted.

Here's a fun one (says he, in a panic): Will we get to a point where (through fault or, gasp, design) a given AI will generatively code a missing dependency, on the fly - perhaps as a "last ditch" effort? (I can imagine languages having official LLMs which would more or less "compress/know" enough of the language to be an ... ... import of last resort, of sorts, by virtue of which an approximation to the missing code…

I'm pretty sure we've already reached a point where people generatively code possibly-malicious packages to publish under names of non-existent packages commonly hallucinated by LLMs prompted by other devs.

Oh my goodness: That is a truly truly scary thought.-

Re: Uv: Running a script with dependencies

#159

Post by nomel »

om8 wrote: post #66
nomel wrote: post #46

Earlier quoted context omitted.

Am I crazy for having my default interactive shell "/bin/env python" point to a virtual environment with all the random dependencies that my one off scripts need, so I can just run "python oneoneoff.py"? All of my one off scripts combined use maybe a dozen dependencies. If I need more, I just install them there. I use pyenv, so it's trivial to change the version of python that the interactive shell uses (default or t…

What would you do if some of the deps started to have conflicts in them? Also, what are your plans for migration when you'll need to move from one os version to another? Implicit solutions like yours have lower cost of entrance, but larger cost of support. uv python scripts just work if you set them up once

Note that the context here is specifically one off scripts.

> Also, what are your plans for migration when you'll need to move from one os version to another?

None of my one off python code is OS dependent. But, none of my professional production code is either, because it's rare to have OS specific python code (unless you're building your own libraries), so this concern is very confusing to me. But, to make sure I can reproduce my one off environment, I periodically pip freeze a requirements.txt.

Re: Uv: Running a script with dependencies

#160

Post by skissane »

zahlman wrote: post #149
freddie_mercury wrote: post #111

Earlier quoted context omitted.

The PEP explains why it isn't part of the regular python syntax. uv and other tools would be forced to implement a full Python parser. And since the language changes they would need to update their parser when the language changes. This approach doesn't have that problem. Making it a "language feature" has no upside and lots of downside. As the PEP explains.

Beyond that, the needed name to download from PyPI doesn't necessarily have anything at all to do with the name used for an `import` statement. And a given PyPI download may satisfy multiple `import` statements. And it can even be possible to require a download from PyPI that doesn't install any importable code (it may legally be a meta-package that runs one-shot configuration code when "built from source").

> Beyond that, the needed name to download from PyPI doesn't necessarily have anything at all to do with the name used for an `import` statement. And a given PyPI download may satisfy multiple `import` statements.

I think this is a design issue with PyPI though. It really should have some kind of index which goes from module names to packages which provide that module. (Maybe it already does but I don't know about it?)

Of course, that doesn't help if multiple packages provide the same module; but then if there was a process to reserve a module name – either one which no package is currently using, or if it is currently used only by a single package, give the owner of that package ownership of the module – and then the module name owner can bless a single package as the primary package for that module name.

Once that were done, it would be possible to implement the feature where "import X", if X can't be found locally, finds the primary package on PyPI which provides module X, installs it into the current virtualenv, and then loads it.

Obviously it shouldn't do this by default... maybe something like "from __future__ import auto_install" to enable it. And CPython might say the virtualenv configuration needs to nominate an external package installer (pip, pipx, poetry, uv, whatever) so CPython knows what to do in this case.

You could even build this feature initially as an extension installed from PyPI, and then later move it into the CPython core via a PEP. Just in the case of an extension, you couldn't use the "from __future__" syntax.

> it may legally be a meta-package that runs one-shot configuration code when "built from source"

True, but if Python were to provide this auto-install via "import X" feature, packages of that nature could be supported by including in them a dummy main module. All it would need would be an empty __init__.py. You could include some metadata in the __init__.py if you wished.

Once "import X" auto-install is supported, you could potentially extend the "import" syntax with metadata to specify you want to install a specific package (not the primary package for the module), and with specific versions. Maybe some syntax like:

    import foobarbaz ("foo-bar-baz>=3.0")
I doubt all this is going to happen any time soon, but maybe Python will eventually get there.
Post reply on HN