Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uv's killer feature is making ad-hoc environments easy

valatka.dev

Post reply on HN
Print view View original

291–300 of 428 posts

Re: Uv's killer feature is making ad-hoc environments easy

#291

Post by zahlman »

MrJohz wrote: post #137
mdaniel wrote: post #35

Earlier quoted context omitted.

Caveat: I'm a node outsider, only forced to interact with it But there are a shocking number of install instructions that offer $(npm i -g) and if one is using Homebrew or nvm or a similar "user writable" node distribution, it won't prompt for sudo password and will cheerfully mangle the "origin" node_modules So, it's the same story as with python: yes, but only if the user is disciplined Now ruby drives me fucking b…

It's worth noting that Node allows two packages to have the same dependency at different versions, which means that `npm i -g` is typically a lot safer than a global `pip install`, because each package will essentially create its own dependency tree, isolated from other packages. In practice, NPM has a deduplication process that makes this more complicated, and so you can run into issues (although I believe other pac…

>It's worth noting that Node allows two packages to have the same dependency at different versions

Yes. It does this because JavaScript enables it - the default import syntax uses a file path.

Python's default import syntax uses symbolic names. That allows you to do fun things like split a package across the filesystem, import from a zip file, and write custom importers, but doesn't offer a clean way to specify the version you want to import. So Pip doesn't try to install multiple versions either (which saves it the hassle of trying to namespace them). You could set up a system to make it work, but it'd be difficult and incredibly ugly.

Some other language ecosystems don't have this problem because the import is resolved at compile time instead.

Re: Uv's killer feature is making ad-hoc environments easy

#292

Post by andelink »

cyrialize wrote: post #192

For anyone that used rye, it's worth noting that the creator of rye recommends using uv. Also, rye is going to be continually updated to just interface with uv until rye can be entirely replaced by uv.

I believe they are from the same author, Charlie Marsh / Astral

Re: Uv's killer feature is making ad-hoc environments easy

#293

Post by zahlman »

fny wrote: post #153
mdaniel wrote: post #35

Earlier quoted context omitted.

Caveat: I'm a node outsider, only forced to interact with it But there are a shocking number of install instructions that offer $(npm i -g) and if one is using Homebrew or nvm or a similar "user writable" node distribution, it won't prompt for sudo password and will cheerfully mangle the "origin" node_modules So, it's the same story as with python: yes, but only if the user is disciplined Now ruby drives me fucking b…

Because you don’t need virtualenvs or ruby_modules. You can have however many versions of the same gem installed it’s simply referenced by a gemfile, so for Ruby version X you are guaranteed one copy of gem version Y and no duplicates. This whole installing the same dependencies a million times across different projects in Python and Node land is completely insane to me. Ruby has had the only sane package manager for…

In principle, venvs could hard-link the files from a common source, as long as the filesystem supports that. I'm planning to experiment with this for Paper. It's also possible to use .pth files (https://docs.python.org/3/library/site.html) to add additional folders to the current environment at startup. (I've heard some whispers that this causes a performance hit, but I haven't noticed. Python module imports are cached anyway.) Symlinks should work, too. (But I'm pretty sure Windows shortcuts would not. No idea about junctions.)

Re: Uv's killer feature is making ad-hoc environments easy

#294

Post by botanical76 »

zahlman wrote: post #206
nharada wrote: post #12

I really like uv, and it's the first package manager for a while where I haven't felt like it's a minor improvement on what I'm using but ultimately something better will come out a year or two later. I'd love if we standardized on it as a community as the de facto default, especially for new folks coming in. I personally now recommend it to nearly everyone, instead of the "welllll I use poetry but pyenv works or you…

(I will likely base a blog post in my packaging series off this comment later.) What people seem to miss about Pip is that it's by design, not a package manager. It's a package installer , only. Of course it doesn't handle the environment setup for you; it's not intended for that. And of course it doesn't keep track of what you've installed, or make lock files, or update your `pyproject.toml`, or... What it does do i…

Do you feel that Npm, mix, cargo went the wrong way, doing too much? It seems like their respective communities _love_ the standard tooling and all that it does. Or is Python fundamentally different?

Re: Uv's killer feature is making ad-hoc environments easy

#295

Post by imtringued »

zahlman wrote: post #278
phinnaeus wrote: post #199

Earlier quoted context omitted.

The lock file is for developers of the library, not consumers. Consumers just use the library’s dependency specification and then resolve their own dependency closure and then generate a lock file for that. If you, as a library developer, want to test against multiple versions of your dependencies, there are other tools for that. It doesn’t make lock files a bad idea in general.

As another library developer, of course I want to test against multiple versions. Or more accurately, I don't want to prevent my users from using different versions prematurely. My default expectation is that my code will work with a wide range of those versions, and if it doesn't I'll know - because I like to pay attention to other libraries' deprecations, just as I'd hope for my users to pay attention to mine. Lock…

>If I had a working environment and didn't want to risk breaking it right at the moment, I could just not upgrade it.

The point of a lockfile is to only upgrade when you want to upgrade. I hope you understand that.

Re: Uv's killer feature is making ad-hoc environments easy

#296

Post by zahlman »

whatever1 wrote: post #93
PaulHoule wrote: post #48

Earlier quoted context omitted.

Python has been cleaning up a number of really lethal problems like: (i) wrongly configured character encodings (suppose you incorporated somebody else's library that does a "print" and the input data contains some invalid characters that wind up getting printed; that "print" could crash a model trainer script that runs for three days if error handling is set wrong and you couldn't change it when the script was runni…

I don’t exactly remember the situation but a user created a python module named error.py. Then in their main code they imported the said error.py but unfortunately numpy library also has an error.py. So the user was getting very funky behavior.

Yep, happens all the time with the standard library. Nowadays, third-party libraries have this issue much less because they can use relative imports except for their dependencies. But the Python standard library, for historical reasons, isn't organized into a package, so it can't do that.

Here's a fun one (this was improved in 3.11, but some other names like `traceback.py` can still reproduce a similar problem):

  /tmp$ touch token.py
  /tmp$ py3.10
  Python 3.10.14 (main, Jun 24 2024, 03:37:47) [GCC 11.4.0] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  >>> help()
  Traceback (most recent call last):
    File "", line 1, in 
    File "/opt/python/standard/lib/python3.10/_sitebuiltins.py", line 102, in __call__
      import pydoc
    File "/opt/python/standard/lib/python3.10/pydoc.py", line 62, in 
      import inspect
    File "/opt/python/standard/lib/python3.10/inspect.py", line 43, in 
      import linecache
    File "/opt/python/standard/lib/python3.10/linecache.py", line 11, in 
      import tokenize
    File "/opt/python/standard/lib/python3.10/tokenize.py", line 36, in 
      from token import EXACT_TOKEN_TYPES
  ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/tmp/token.py)
Related Stack Overflow Q&A (featuring an answer from me): https://stackoverflow.com/questions/36250353

Re: Uv's killer feature is making ad-hoc environments easy

#297

Post by imtringued »

zahlman wrote: post #206
nharada wrote: post #12

I really like uv, and it's the first package manager for a while where I haven't felt like it's a minor improvement on what I'm using but ultimately something better will come out a year or two later. I'd love if we standardized on it as a community as the de facto default, especially for new folks coming in. I personally now recommend it to nearly everyone, instead of the "welllll I use poetry but pyenv works or you…

(I will likely base a blog post in my packaging series off this comment later.) What people seem to miss about Pip is that it's by design, not a package manager. It's a package installer , only. Of course it doesn't handle the environment setup for you; it's not intended for that. And of course it doesn't keep track of what you've installed, or make lock files, or update your `pyproject.toml`, or... What it does do i…

Virtualenv should have never existed in the first place. So you claiming that UV or whatever tool is doing too much, sounds to me like you're arguing based on "traditionalist" or "conservative" reasons rather than doing any technical thinking here.

Node.js's replacement for virtualenv is literally just a folder named "node_modules". Meanwhile python has an entire tool with strange ideosyncracies that you have to pay attention to otherwise pip does the wrong thing by default.

It is as if python is pretending to be a special snowflake where installing libraries into a folder is this super hyper mega overcomplicated thing that necessitates a whole dedicated tool just to manage, when in reality in other programming languages nobody is really thinking about that the fact that the libraries end up in their build folders. It just works.

So again you're pretending that this is such a big deal that it needs a whole other tool, when the problem in question is so trivial that another tool is adding mental overhead with regard to the microscopic problem at hand.

Re: Uv's killer feature is making ad-hoc environments easy

#298

Post by zahlman »

mdaniel wrote: post #123
PaulHoule wrote: post #97

Earlier quoted context omitted.

... it's tricky. In Java there's a cultural expectation that you name a package like package organization.dns.name.this.and.that; but real scalability in a module system requires that somebody else packages things up as package this.and.that; and you can make the system look at a particular wheel/jar/whatever and make it visible with a prefix you specify like package their.this.and.that; Programmers seem to hate rigo…

But let's all be real here: the ability of __init__.py to do FUCKING ANYTHING IT WANTS is insanity made manifest I am kind of iffy on golang's import (. "some/packge/for/side-effects") but at least it cannot suddenly mutate GOPATH[0]="/home/jimmy/lol/u/fucked" as one seems to be able to do on the regular with python I am acutely aware that is (programmer|package|organization|culture)-dependent but the very idea that…

It took me a while to realize that you do mean __init__.py running when a package imports.

But, you know, top-level code (which can do anything) runs when you import any Python module (the first time), and Python code doesn't have to be in a package to get imported. (The standard library depends on this.)

Re: Uv's killer feature is making ad-hoc environments easy

#299

Post by krick »

zahlman wrote: post #287
krick wrote: post #259

Ok, this must be a dumb question answered by the manual, but I still haven't got my hands on uv, so: but does it solve the opposite? I mean, I pretty much never want any "ad-hoc" environments, but I always end up with my .venv becoming an ad-hoc environment, because I install stuff while experimenting, not bothering to patch requirements.txt, pyproject.toml or anything of the sort. In fact, now I usually don't even b…

>installing stuff automatically changes pyprpject.toml (or whatever the standard will be with uv) pyproject.toml represents an inter-project standard and Charlie Marsh has committed to sticking with it, along with cooperating with future Python packaging PEPs. But while you can list transitive dependencies, specify exact versions etc. in pyproject.toml, it's not specifically designed as a lockfile - i.e., pyproject.t…

> uv isn't going to do things like analyzing your codebase

Sure, that's not what I meant (unless we call pyproject.toml a part of your codebase, which it kinda is, but that's probably not what you meant).

In fact, as far as I can tell from your answer, Python does move in the direction I'd like it to move, but it's unclear by how far it will miss and if how uv handles it is ergonomical.

As I've said, I think PHP's composer does a very good job here, and to clarify, this is how it works. There are 2 files: composer.json (≈pyproject.toml) and composer.lock (≈ PEP751) (also json). The former is kinda editable by hand, the latter you ideally never really touch. However, for the most part composer is smart enough that it edits both files for you (with some exceptions, of course), so every time I run `composer require your/awesomelib` it

1) checks the constraints in these files

2) finds latest appropriate version of your/awesomelib (5.0.14) and all its dependencies

3) writes "your/awesomelib": "^5.0"

4) writes "your/awesomelib": "5.0.14" and all its dependencies to composer.lock (with hashsums, commit ids and such)

It is a good practice to keep both inside of version control, so when I say "git diff tells me what I did last night" it means that I'll also see what I installed. If (as usual) most of it is some useless trash, I'll manually remove "your/awesomelib" from composer.json, run `composer install` and it will remove it and all its (now unneeded) dependencies. As the result, I never need to worry about bookkeeping, since composer does it for me, I just run `composer require ` and it does the rest (except for cases when is a proprietary repo on company's gitlab and such, then I'll need slightly more manual work).

That is, what I hope to see in Python one day (10 years later than every other lang did it) is declarative package management, except I don't want to have to modify pyproject.toml manually, I want my package manager do it for me, because it saves me 30 seconds of my life every time I install something. Which accumulates to a lot.

Re: Uv's killer feature is making ad-hoc environments easy

#300

Post by zahlman »

meitham wrote: post #256
worik wrote: post #130

Earlier quoted context omitted.

> Python has been cleaning up a number of really lethal problems like I wish they would stick to semantic versioning tho. I have used two projects that got stuck in incompatible changes in the 3.x Python. That is a fatal problem for Python. If a change in a minor version makes things stop working, it is very hard to recommend the system. A lot of work has gone down the drain, by this Python user, trying to work aroun…

I assume these breaking changes are in the stdlib and not in the python interpreter (the language), right? There was previous discussions about uncoupling the stdlib (python libraries) from the release and have them being released independently, but I can’t remember why that died off

> I assume these breaking changes are in the stdlib and not in the python interpreter (the language), right?

That's the usual case, but it can definitely also happen because of the language - see https://stackoverflow.com/questions/51337939 .

> There was previous discussions about uncoupling the stdlib (python libraries) from the release and have them being released independently, but I can’t remember why that died off

This sort of thing is mainly a social problem.

Post reply on HN