Pyenv – lets you easily switch between multiple versions of Python

One of the troubleshooting steps when "something mysterious" happens on developer's computer (eg. package installation fails for inexplicable reasons, Python "standard" library components missing or present when shouldn't be, incorrect component version etc.) was to remove pyenv.

This step was often met with resentment and arguments... but, in most cases I was able to win :)

----

Now, here's a larger point: some programs solve the problem by actually solving the problem, while other programs solve the problem by adding more code around the problem, which, usually, creates new problems while only partially solving the original problem.

An example of the former: fsck -- you run it, it looks at your filesystem, tries to fix it, if it's broken and then gets out of the way entirely. An example of the later: Kubernetes -- you start by having a problem of resource allocation / management and you end up with a problem of resource allocation / management compounded by problems with component version management, configuration management etc.

pyenv falls into the second category of programs. The problem it's trying to solve is: install and use multiple versions of Python. There's really no need for an extra helper program to solve this problem. Multiple versions of CPython can be installed and used together without the use of any extra tools. I.e. the solution to this "problem" is simply to learn how to do that.

Those who advocate for the use of pyenv and the likes usually make an argument for "simplicity". I.e. in their mind, not needing to know how to install multiple versions of Python is a bonus. Something that, potentially, saves them several hours of reading the documentation and perhaps saving them a tiny bit of typing when setting up Python initially.

I contend that this calculation is off because it doesn't account for the problems down the lane. In other words: pyenv helps until it doesn't, and then it becomes a liability. Debugging is always more difficult if you have more wrappers between you and your problem. So, while individual users will not face a lot of problems with "wrapper solutions", those who service such users will face such problems a lot more frequently. That's why, as an ops person, I really dislike "wrapper solutions" -- for me, they complicate the task, never help.

There aren't such tools. Python not being a standard and heavily reliant on the OS that runs it and on third-party components that are also not standard leaves you with no choice by to "be at the wheel" all the time. Virtually anything written in Python will go stale in a mater of few years. In other words, you need to constantly update and test as the environment changes just to stand still.

It's definitely a learning curve, but it turns out every conda user has been bit by the irreproducible tendencies of conda quite often. Nobody uses the conda env file, they just start an env and pip install things into it. They don't realize the base env has stuff, too, and conda envs are hierarchical rather than isolated. I know it's possible to use conda in an isolated and reproducible way, but have yet to meet someone that does so.

So it hasn't been hard to pitch poetry to these folks, and while many complain about the learning curve they appreciate the outcomes.

We're a pytorch shop, and torch mostly just works with pip or poetry these days, as long as you skip the versions the torch maintainers mispackaged. We rarely need anything higher-level that only conda could install.

We really like having more than two dependency groups as this allows us to keep research and production in the same repository. main, dev, research. Then researchers contribute to the core library of a project and keep research and production using the same code for running and evaluating models.