Live data from Hacker News

Pyenv – lets you easily switch between multiple versions of Python

github.com

101–110 of 341 posts

Re: Pyenv – lets you easily switch between multiple versions of Python

#101
post #93

Just use the builtin "python -m venv" and life is good, why the others? I tried almost all of them, with various issues, now staying with the default venv, it's solid and get the job done.

Yeah, my thoughts exactly. I’ve never found myself in a situation where venv did not do exactly what I needed it to do.

The problem there is that by default a virtual environment version of Python is just a symbolic link to the actual binaries installed elsewhere on the machine. If you do an OS upgrade or do anything that increments the system Python version, it could cause problems in the environment.

Re: Pyenv – lets you easily switch between multiple versions of Python

#102
I'll toss in my (macOS) workflow too:

- Download whatever Python binaries I need from python.org

- virtualenv --python=$PYTHON_VERSION ~/.virtualenvs/$PROJECT_NAME

- pyproject.toml

- pip-compile --generate-hashes

There has been tons of churn in the Python project management space, and I feel like I've been blissfully unaware of all of it with this workflow. Can't recommend enough.

Why regular Python binaries and not pyenv? I've had enough pyenv snafus and meltdowns that I started looking for alternatives, and it turned out I could just install whatever Python binary I wanted and specify it wherever I wanted. What could be easier?

python -m venv is fine, I've just gotten used to virtualenv.

pyproject.toml is the future and almost everything supports it pretty well; it also has fewer implicit weirdnesses than setup.py or setup.cfg.

pip-tools are really simple, and they let you have hashed dependencies which are super important IMO.

Re: Pyenv – lets you easily switch between multiple versions of Python

#103
post #95

https://asdf-vm.com/ ASDF is better because it works with many more languages, other than only Python, like Rust, Go, Node, etc, and other tools, such as AWS/Google/Firebase/Azure CLIs.

I used to be all-in for pyenv and rbenv both but have loved using asdf so much more.

Re: Pyenv – lets you easily switch between multiple versions of Python

#104
This is the sort of tool that experienced devs all use, but nobody teaches or writes it into a tutorial on the larger language as a whole, because it feels like a side tool.

For now I'm not seeing a lot of reasons to use PyEnv over the `venv` module that ships with Python 3.3+ [1]. I'm sure there are some thing PyEnv does that `venv` doesn't, but the fact that `venv` ships with Python greatly simplifies things.

Basically my workflow is:

1. I'll create a Python project with, `mkdir dirname`, `cd dirname`, `git init`, `python3 -m venv .env`. This creates a hidden folder named `.env/` which contains the virtual environment. I'll then usually add `.env/` to my `.gitignore`, and also run `pip install --upgrade pip` and `pip install wheel`.

2. If I'm using an existing project, I'll do `git clone `projname`, `cd dirname`, `python3 -m venv .env`, `pip install -m requirements.txt` (requirements.txt is the idiomatic name for the dependencies list in Python projects).

3. I have the following lines in my `~/.bashrc` (hidden file that contains Bash settings):

    # Gets a directory named .env or .venv if it exists in the currend directory or any of its parents
    get_env() {
      if [ -d "$1/.env" ] ; then
        echo "$1/.env"
      else
        if [ -d "$1/.venv" ] ; then
          echo "$1/.venv"
        else
          if [ -d "$1/.." ] ; then
            get_env "$1/.."
          fi
        fi
      fi
    }

    get_absolute_path() {
      python3 -c "import os; print(os.path.realpath('$1'))"
    }

    on_prompt() {
      # Load a virtualenv environment if it exists in a file named .env
      env_folder=$(get_env $(pwd))

      if [ -d "$env_folder" ] ; then
        if [[ $VIRTUAL_ENV != $(get_absolute_path $env_folder) ]] ; then
          echo "Activating env '$env_folder'"
          source "$env_folder/bin/activate"
        fi
      else
        if [ -d "$VIRTUAL_ENV" ] ; then
          deactivate
        fi
      fi
    }

    # Call on_prompt() every time the command prompt executes
    PROMPT_COMMAND=on_prompt
What this does is when I `cd` or `pushd` into a directory or subdirectory of a directory that contains a `.env/` folder, it loads the virtual environment, and when I leave said directories, it exits the virtual environment.

4. When I install a dependency, I use `pip freeze` to get the dependency string, and I append that line to the file `requirements.txt`. For example, if I do `pip install django`, I get `Django==5.0.1` in my output from `pip freeze`, so I'll append that line to my `requirements.txt`.

5. To upgrade dependencies, I edit the version number in the `requirements.txt` file, and then run `pip install -m requirements.txt`. This makes sure that the requirements file stays up-to-date with my locally installed dependencies.

6. To get valid version numbers for a file, you can do `pip install ==`, which is basically asking pip to install an invalid version number. This causes it to list valid version numbers. For example, you can do `pip install django==` to view all available Django versions.

[1] https://docs.python.org/3/library/venv.html

Re: Pyenv – lets you easily switch between multiple versions of Python

#105
post #12

I have to warn again users that think they have found the silver bullet that pyenv comes with a big caveat: it compiles python on your machine. The number of possible modes of failure in this situation is huge. See also: "Why not tell people to "simply" use pyenv, poetry or anaconda" https://www.bitecode.dev/p/why-not-tell-people-to-simply-use I'm not saying pyenv is not a useful tool, but it is not a tool for beginn…

> it compiles python on your machine What is the alternative? Every solution that I know of on Linux requires you to build Python on the machine: asdf, official Python downloads, etc.

> Every solution that I know of on Linux requires you to build Python on the machine

Unless you need a Python that's not supported by your Linux distribution, you can just use what's available.

On macOS, MacPorts provides compiled versions for 3.2 all the way to 3.13, as well as 2.6 and 2.7. Right now, I have 3.8, 3.9, 3.10, 3.11, 3.12, and a 3.13 development build. The fact it's not Linux (or x86) might cause some frustration.

Re: Pyenv – lets you easily switch between multiple versions of Python

#106
post #12

Earlier quoted context omitted.

> it compiles python on your machine What is the alternative? Every solution that I know of on Linux requires you to build Python on the machine: asdf, official Python downloads, etc.

For now most people would do well to stick to python.org installers for mac and windows. For linux, official repos are ideal. If you really, really can't (which is different than wanting to), ubuntu deadsnake and red hat epl are the best second plan, while already more finicky. If you use something more exotic, you chose hardship, and you will have to be up to the task. Anything else will come with bigger caveats tha…

Even on Macs, you can use MacPorts. They provide a lot of versions pre-compiled (and everything is BSD-solid).

Re: Pyenv – lets you easily switch between multiple versions of Python

#107

Earlier quoted context omitted.

You didn’t miss Poetry, but I have to say up until I started using Poetry python in large projects was a pain in the tooling department to setup and maintain over longer periods of time. It’s no panacea, but feels more stable and usable (especially from onboarding new team members PoV) than other tooling I’ve tried

What does poetry deliver beyond what pyenv gets you? I haven't used either tool super extensively.

pyenv only manages python versions, while poetry manages dependencies and virtual environments. They are complimentary, but do not overlap.

Re: Pyenv – lets you easily switch between multiple versions of Python

#108
post #50

Tools you can use to make sure the Python program you wrote keeps working: requirements.txt, pip, pipenv, pyenv, virtualenv, pyenv-virtualenv, virtualenvwrapper, pyenv-virtualenvwrapper, venv, pyvenv, conda, miniconda, poetry, docker, nix. Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?

You've missed: pdm, uv, pip-tools, pipx, rye, and probably some others. Only pdm and poetry generate cross-platform lock files by default as far as I know, but there are a lot of people trying to solve this problem right now. It's not an easy problem to solve. Python's package management predates package managers from most other programming languages and Python itself predates Linux. There is a lot of baggage so chan…

Shameless plug: don't forget the wonderful pip-chill for simplifying gigantic requirements files (and for stripping out version numbers to make canaries easier to do).

Re: Pyenv – lets you easily switch between multiple versions of Python

#109
post #65

Tools you can use to make sure the Python program you wrote keeps working: requirements.txt, pip, pipenv, pyenv, virtualenv, pyenv-virtualenv, virtualenvwrapper, pyenv-virtualenvwrapper, venv, pyvenv, conda, miniconda, poetry, docker, nix. Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?

I am always curious how many people, outside of those building code for third party clients, actually hit this problem? In the 10+ years of using Python I have never had a problem using the core tools. The ecosystem is far from perfect but it has never cause me a problem. Edit: Wow y'all are some sour people for voting down this question. I truly wonder how often people run into this problem compared to just complain…

If I were writing applications only for myself it'd be fine, but I've definitely run into issues when writing code with others - teammates, open source repos, etc.

Re: Pyenv – lets you easily switch between multiple versions of Python

#110
post #61
post #53

Earlier quoted context omitted.

>The other part is OS state And that's the whole problem. "The same Python environment" doesn't mean much because it underspecifies the full runtime dependency chain.

It's inherit problem for all languages based on some runtime. Java, PHP, Ruby, whatever have the same issue. And TBH it's not very major issue to start with. No other solution besides fixed OS (containers, nix) could solve this problem.

The link posted solved this problem without containers
Post reply on HN