Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyenv – lets you easily switch between multiple versions of Python

github.com

Post reply on HN
Print view View original

131–140 of 341 posts

Re: Pyenv – lets you easily switch between multiple versions of Python

#131

Post by scaryclam »

latexr wrote: post #78
scaryclam wrote: post #70

Earlier quoted context omitted.

I'm not sure what you might mean by runtime, but a python venv does make sure you're using the same runtime. There are symlinks in the environments bin directory to the specific runtime. If you're plonking a python3.12 binary over the top of the python3.11 one, then yeah, you'll be using a different one, but that'd be an issue nomatter what you're using. Just don't uninstall your python binaries and you're fine. If y…

> but a python venv does make sure you're using the same runtime. It does not. > There are symlinks in the environments bin directory to the specific runtime. Precisely. They symlink to a path . Which means that if you have a certain version of Python at one location (let’s say /usr/bin/python3) and later update that (let’s say by upgrading macOS and the Xcode developer tools), the same virtual environment will point…

The binary installs aren't that generic in the machines I've used (mac and debian based, so covers a fair bit).

It's python3.11, not python3, and the python3 "executable" is, itself a symlink to the particular binary (in this example python3.11). Upgrading just changes the symlink, which wouldn't affect the venv, which isn't using python3, it's using python3.11.

I didn't introduce anything, I explained how the links are to the versioned binaries, which isn't what you're stating is happening.

Edit to add: as someone else also points out, you don't even have to use the symlinked versions, you can use --copies

Re: Pyenv – lets you easily switch between multiple versions of Python

#132

Post by sgarland »

mikemcquaid wrote: post #91
rcarmo wrote: post #28

Earlier quoted context omitted.

This is misleading. The older versions get cleaned out as brew does the upgrades (if not, then check your brew settings). My take on brew's Python is that it is there to make sure there is a uniform runtime for everything with a Python dependency, but I am not required to use it.

As. Homebrew maintainer: confirmed. You can also adjust your settings to pin versions or avoid cleanup, too. ‘brew pyenv-sync’ may be useful around this area, too.

If I pin a version of Python, isn't that going to wreck any tooling that depends on it? Unless you're saying have multiple versions of Python installed.

This is practically the only remaining annoyance I have with the Python ecosystem (relative imports aside). I use some tools, like Glances [0] whose formula relies on a much newer version (3.12) than the actual package requires (3.8) [1].

So when there's a Python update, all of those update as well. I thought I'd fixed this with pipx, but in a way that's worse, because the venvs it builds depend on a specific version of Python existing, which doesn't work well with brew always wanting to upgrade it.

I want a stable, system-level Python that I don't touch, don't add packages to, and which only exists as a dependency for anything that needs it. If an update would break a package I have installed (due to Python library deprecation, etc.), it should warn me before updating. Otherwise, I don't care, as long as any symlinks are taken care of.

Separately, I want a stable, user-level Python that I can do whatever I want to. Nothing updates it automatically. I can accomplish this by compiling Python and using `make altinstall`, but if there's a better way, I'd love to hear about it.

[0]: https://github.com/Homebrew/homebrew-core/blob/20e744191e74d...

[1]: https://github.com/nicolargo/glances

Re: Pyenv – lets you easily switch between multiple versions of Python

#133

Post by mega_dingus »

latexr wrote: post #78
scaryclam wrote: post #70

Earlier quoted context omitted.

I'm not sure what you might mean by runtime, but a python venv does make sure you're using the same runtime. There are symlinks in the environments bin directory to the specific runtime. If you're plonking a python3.12 binary over the top of the python3.11 one, then yeah, you'll be using a different one, but that'd be an issue nomatter what you're using. Just don't uninstall your python binaries and you're fine. If y…

> but a python venv does make sure you're using the same runtime. It does not. > There are symlinks in the environments bin directory to the specific runtime. Precisely. They symlink to a path . Which means that if you have a certain version of Python at one location (let’s say /usr/bin/python3) and later update that (let’s say by upgrading macOS and the Xcode developer tools), the same virtual environment will point…

This is not the way you're supposed to be using pyenv

You're supposed to install a specific version of python in a specific place, with a specific name. Say, /usr/local/python-3.10.6

Use pyenv to use that python. Control that by creating a `.python-version` file that says 3.10.6

You now have a project that uses 3.10.6. Unless, of course, somebody installs a different version in that path - at which point you've got bigger issues

Using pyenv to use `/usr/bin/python3` and hoping for the best misses the point

Re: Pyenv – lets you easily switch between multiple versions of Python

#134

Post by kzrdude »

synergy20 wrote: post #88

Just use the builtin "python -m venv" and life is good, why the others? I tried almost all of them, with various issues, now staying with the default venv, it's solid and get the job done.

The premise of controlling python version is that - you're sending the project to your friend / colleague and you want both of you to use the same python version so that you get the same behaviour.

And then the same for your friend the buildserver.

It's all about reproducibility.

Re: Pyenv – lets you easily switch between multiple versions of Python

#135

Post by hnfong »

wodenokoto wrote: post #43
TomSwirly wrote: post #24

This is a truly, truly terrible idea. It adds several failure modes, some subtle so you can go a long way in a state of error, just so beginners can type `python` instead of e.g. `python3.10`. Many developers, not just me, have a similar setup: we use virtual environments everywhere, and if you aren't in one, `python` doesn't even resolve to a symbol. If I want to write a quick script with no dependencies, I directly…

But how do I get a .venv with a new/different version of Python in your setup? And how do I add some kind of Python version requirement in my project?

python3.xx -m venv .

Re: Pyenv – lets you easily switch between multiple versions of Python

#136

Post by mwexler »

BiteCode_dev wrote: post #5

I have to warn again users that think they have found the silver bullet that pyenv comes with a big caveat: it compiles python on your machine. The number of possible modes of failure in this situation is huge. See also: "Why not tell people to "simply" use pyenv, poetry or anaconda" https://www.bitecode.dev/p/why-not-tell-people-to-simply-use I'm not saying pyenv is not a useful tool, but it is not a tool for beginn…

So, what is "the tool for beginners fighting with python packaging problems"?

That is, the pattern seems to be that someone mentions a solution, then a zillion responses as to why it sucks.

Is there any tool or pair that sucks least for most cases and beginners? I get that every case is different, but perhaps there are some useful starting points?

Re: Pyenv – lets you easily switch between multiple versions of Python

#137

Post by Shrezzing »

fbuilesv wrote: post #69
nlitened wrote: post #40

Tools you can use to make sure the Python program you wrote keeps working: requirements.txt, pip, pipenv, pyenv, virtualenv, pyenv-virtualenv, virtualenvwrapper, pyenv-virtualenvwrapper, venv, pyvenv, conda, miniconda, poetry, docker, nix. Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?

What package managers are people using in other languages to make sure that software "always works the same as when you first wrote it, without asterisks"? I'd like to understand how they solve the "package no longer exists in a central registry" problem.

Dotnet uses Nuget[1]. Packages in the system are immutable, & never changing. They can be unlisted, but never deleted (except in limited & extreme cases, like malware), which means even if a package maintainer stops publishing new versions to the repository, existing packages will continue to be publicly available for as long as Microsoft continues to exist.

[1] https://www.nuget.org/

Re: Pyenv – lets you easily switch between multiple versions of Python

#138

Post by jolux »

Plug for Mise (https://mise.jdx.dev/) which is like asdf (which is like pyenv, but for any language you might want all in one tool) but written in Rust and without shims (by default) such that it’s much faster at running the activated runtime. It installs runtimes in parallel and will even download the correct plugin if you don’t already have it installed. It can also replace direnv, which I haven’t done because I’m using it as a drop-in replacement for asdf with the .tool-versions files my team already has in every repo (it supports .python-version as well if you’ve been using pyenv). It works great as a “better asdf”/“better pyenv” for my purposes.

Re: Pyenv – lets you easily switch between multiple versions of Python

#139

Post by sumtechguy »

infecto wrote: post #65
nlitened wrote: post #40

Tools you can use to make sure the Python program you wrote keeps working: requirements.txt, pip, pipenv, pyenv, virtualenv, pyenv-virtualenv, virtualenvwrapper, pyenv-virtualenvwrapper, venv, pyvenv, conda, miniconda, poetry, docker, nix. Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?

I am always curious how many people, outside of those building code for third party clients, actually hit this problem? In the 10+ years of using Python I have never had a problem using the core tools. The ecosystem is far from perfect but it has never cause me a problem. Edit: Wow y'all are some sour people for voting down this question. I truly wonder how often people run into this problem compared to just complain…

I just hit it a few days ago. 3rd party script that I did not write. The dev has lost interest in it 3 years ago. It is written for 2.7.x (and it is a nontrivial amount of code). So my choices are some sort of pyenv thing (or one of them), or fix up the script myself so it runs on 3.12. I lucked out and someone else had already done the second thing. One project I worked on was 2.4.x. If they ever have to update that (which I hope they have) to something more recent I could see them doing it until they port it over. That is not even a lib or anything that is just the main prog.
Post reply on HN