I used to use this a lot when working on a Windows machine. Worked pretty well. But nowadays Nix solves the same problem in a fully general way, for all software rather than a single language. You can have whatever versions of whatever you want, side by side, without interfering with each other.
Pyenv – lets you easily switch between multiple versions of Python
21–30 of 341 posts
Re: Pyenv – lets you easily switch between multiple versions of Python
#22Is anyone is the AI/ML area finding success with anything other than conda, where installation of CUDA/CUDnn is required? Although I often have to pip install a lot of packages, I find conda's nvidia/pytorch/conda-forge channels are still by far the easiest way to get a deep learning stack up and running, and so I just stick with conda environments. I've tried poetry in the past but getting the NVidia deep learning s…
Pyenv just worked for me. I am actually using Fedora Silverblue and have GCC and the CUDA SDK available only inside a toolbox container. Therefore, I have to enter that toolbox to install things like FlashAttention.
Re: Pyenv – lets you easily switch between multiple versions of Python
#23I have to warn again users that think they have found the silver bullet that pyenv comes with a big caveat: it compiles python on your machine. The number of possible modes of failure in this situation is huge. See also: "Why not tell people to "simply" use pyenv, poetry or anaconda" https://www.bitecode.dev/p/why-not-tell-people-to-simply-use I'm not saying pyenv is not a useful tool, but it is not a tool for beginn…
> it compiles python on your machine What is the alternative? Every solution that I know of on Linux requires you to build Python on the machine: asdf, official Python downloads, etc.
These builds are an alternative: https://github.com/indygreg/python-build-standalone
Those are what Rye and hatch use.
Drawbacks: late availability of patch versions, various quirks from how they are built (missing readline, missing some build info that self-compiled C python modules might need.)
Re: Pyenv – lets you easily switch between multiple versions of Python
#24This is a truly, truly terrible idea. It adds several failure modes, some subtle so you can go a long way in a state of error, just so beginners can type `python` instead of e.g. `python3.10`.
Many developers, not just me, have a similar setup: we use virtual environments everywhere, and if you aren't in one, `python` doesn't even resolve to a symbol.
If I want to write a quick script with no dependencies, I directly call `python3.xx` on it. Otherwise, I create a virtualenv.
Yes, it's a bit harder for beginners, but from a huge amount of experience helping people who are starting up in programming, people have little issue in following a few more instructions. What demolishes beginners is getting into a bad state where nothing works and you don't know why.
Re: Pyenv – lets you easily switch between multiple versions of Python
#25I have to warn again users that think they have found the silver bullet that pyenv comes with a big caveat: it compiles python on your machine. The number of possible modes of failure in this situation is huge. See also: "Why not tell people to "simply" use pyenv, poetry or anaconda" https://www.bitecode.dev/p/why-not-tell-people-to-simply-use I'm not saying pyenv is not a useful tool, but it is not a tool for beginn…
Actually, it only builds it locally if it can't find a pre-packaged version for your system/arch. Admittedly that's most of the recent ones on a Mac, but there is a difference (I've been using pyenv for nearly ten years[1] now).
The big advantage for me is that I can match whatever runtime and standard library a target has (and yes, that's needed more times than not, even in this new age of Docker).
Additionally, you can build an _optimized_ Python. I have this set for my builds:
env PYTHON_CONFIGURE_OPTS='--enable-optimizations --with-lto' PYTHON_CFLAGS='-march=native -mtune=native' pyenv install 3.12.2
Re: Pyenv – lets you easily switch between multiple versions of Python
#26This is a truly, truly terrible idea. It adds several failure modes, some subtle so you can go a long way in a state of error, just so beginners can type `python` instead of e.g. `python3.10`. Many developers, not just me, have a similar setup: we use virtual environments everywhere, and if you aren't in one, `python` doesn't even resolve to a symbol. If I want to write a quick script with no dependencies, I directly…
though, with openinterpreter, you can just ask it to fix your python and it'll help you out of a weird situation that you as an newbie don't understand. Still, node's node_modules implementation isn't a bad one, at the expense of disk space, you don't run into the same problems as you do python.
Re: Pyenv – lets you easily switch between multiple versions of Python
#27I think the only problem of pyenv is that you can't upgrade an old virtualenv to a newer Python version.
Re: Pyenv – lets you easily switch between multiple versions of Python
#28Homebrew really bugs me. The devs want you to use the most current version of Python. Except the requirement is set at the package level and there's no enforcement. So you just keep getting python versions installed. I don't get why you just don't set a minimum version like every other manager. Or if you want to do this crazy thing, don't rely on package maintainers to enforce it. I just ends up in major bloat
This is misleading. The older versions get cleaned out as brew does the upgrades (if not, then check your brew settings). My take on brew's Python is that it is there to make sure there is a uniform runtime for everything with a Python dependency, but I am not required to use it.
Re: Pyenv – lets you easily switch between multiple versions of Python
#29I think the only problem of pyenv is that you can't upgrade an old virtualenv to a newer Python version.
Isn't that more of a limitation with virtualenv? I've been using virtualfish as my venv manager in the fish shell and it has a `vf upgrade` command for upgrading virtualenvs to newer versions, either by updating in-place or rebuilding the entire virtualenv. So far has worked nicely with my pyenv Python installs.
Re: Pyenv – lets you easily switch between multiple versions of Python
#30We use Pyenv successfully for developing the Flower open-source project. We use a few simple Bash scripts to manage virtual environments with different Python versions via pyenv and the pyenv-virtualenv plugin.
The main scripts are `venv-create.sh`, `venv-delete.sh` and `bootstrap.sh`. `venv-reset.sh` pulls these three scripts together to make reinstalling your venv a single command.
Here's the link if anyone is interested: https://github.com/adap/flower/tree/main/dev