Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyenv – lets you easily switch between multiple versions of Python

github.com

Post reply on HN
Print view View original

121–130 of 341 posts

Re: Pyenv – lets you easily switch between multiple versions of Python

#121

Post by orf »

BiteCode_dev wrote: post #67
rcarmo wrote: post #25

Earlier quoted context omitted.

Actually, it only builds it locally if it can't find a pre-packaged version for your system/arch. Admittedly that's most of the recent ones on a Mac, but there is a difference (I've been using pyenv for nearly ten years[1] now). The big advantage for me is that I can match whatever runtime and standard library a target has (and yes, that's needed more times than not, even in this new age of Docker). Additionally, you…

That happens way more often than people want to admit, and I think it's not genuine to present the tool that way. E.G, I'm on Ubuntu 20.04 on an Dell XPS, a fairly standard machine, I'll get: pyenv install 3.9 -v /tmp/python-build.20240325124651.73089 ~ Downloading Python-3.9.19.tar.xz... -> https://www.python.org/ftp/python/3.9.19/Python-3.9.19.tar.xz ... LD_LIBRARY_PATH=/tmp/python-build.20240325124651.73089/Python…

I’ve quite literally never seen it download a pre-compiled Python - I didn’t actually know it had that functionality, and have been using it for probably a decade.

Re: Pyenv – lets you easily switch between multiple versions of Python

#122

Post by crabbone »

infecto wrote: post #65
nlitened wrote: post #40

Tools you can use to make sure the Python program you wrote keeps working: requirements.txt, pip, pipenv, pyenv, virtualenv, pyenv-virtualenv, virtualenvwrapper, pyenv-virtualenvwrapper, venv, pyvenv, conda, miniconda, poetry, docker, nix. Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?

I am always curious how many people, outside of those building code for third party clients, actually hit this problem? In the 10+ years of using Python I have never had a problem using the core tools. The ecosystem is far from perfect but it has never cause me a problem. Edit: Wow y'all are some sour people for voting down this question. I truly wonder how often people run into this problem compared to just complain…

Working in CI in a company that has couple dozens of Python packages: I'd say about once a week. There are some weeks with no incidents, and there are weeks when everything is broken for many days straight.

NB. The latest incident was Friday when I've discovered that some CI pipeline ran `setup.py install` that down the lane invoked easy_install, which doesn't have a policy of ignoring bizarre versions s.a. X.Y.Zrc1 or X.Y.Zb2 etc. It ran aground when it was trying to install scikit-learn which wanted NumPy >=X.Y.Z, but it already installed X.Y.Zb1, and it didn't realize that this version should be OK (also, it shouldn't have installed non-release versions anyways).

Re: Pyenv – lets you easily switch between multiple versions of Python

#123

Post by chrisfinazzo »

lb4r wrote: post #94
BiteCode_dev wrote: post #5

I have to warn again users that think they have found the silver bullet that pyenv comes with a big caveat: it compiles python on your machine. The number of possible modes of failure in this situation is huge. See also: "Why not tell people to "simply" use pyenv, poetry or anaconda" https://www.bitecode.dev/p/why-not-tell-people-to-simply-use I'm not saying pyenv is not a useful tool, but it is not a tool for beginn…

> See also: "Why not tell people to "simply" use pyenv, poetry or anaconda" > https://www.bitecode.dev/p/why-not-tell-people-to-simply-use Just curious: what are the downsides of poetry installed with pipx? The article mentions having to install poetry in another venv, but that's hardly an issue with pipx (you just add an 'x' after 'pip'), and installing pipx is as straight-forward as it can be.

I eventually landed on pipx as fighting with Pyenv and Anaconda - via Miniconda - was an exercise in frustration. There's some mucking about in `$HOME/.local`, but this is mostly self-contained and not a huge chore to keep running.

Coming from the Homebrew/Ruby ecosystem - Hey @mikemcquaid - installing a entirely separate package manager just to deal with a few projects felt like the wrong thing to do.

Occasionally, I have still needed to compile Python myself in order to get things to work, which isn't guaranteed not to blow up w/ `brew`, but this has become far less common of late.

Re: Pyenv – lets you easily switch between multiple versions of Python

#124

Post by claytonjy »

synergy20 wrote: post #88

Just use the builtin "python -m venv" and life is good, why the others? I tried almost all of them, with various issues, now staying with the default venv, it's solid and get the job done.

pyenv manages python versions, not virtual environments. venv won't help you grab python versions you don't have, and will happily install your requirements into a venv with the wrong python version.

Re: Pyenv – lets you easily switch between multiple versions of Python

#126

Post by abkfenris »

jphoward wrote: post #2

Is anyone is the AI/ML area finding success with anything other than conda, where installation of CUDA/CUDnn is required? Although I often have to pip install a lot of packages, I find conda's nvidia/pytorch/conda-forge channels are still by far the easiest way to get a deep learning stack up and running, and so I just stick with conda environments. I've tried poetry in the past but getting the NVidia deep learning s…

Have you tried https://pixi.sh/ ? It brings Cargo/NPM/Poetry like commands and lock files to the Conda ecosystem, and now can manage and lock PyPI dependencies alongside by using uv under the hood.

I haven't been using anything CUDA, but the scientific geospatial stack is often a similar mess to install, and it's been handling it really well.

Re: Pyenv – lets you easily switch between multiple versions of Python

#127

Post by rmnclmnt »

Y_Y wrote: post #47
rmnclmnt wrote: post #7

Earlier quoted context omitted.

For anything related to CUDA/CuDNN, use one of NVIDIA base Docker images. Then whether you use Conda / Pip / Poetry / Pipenv does not matter much. Not at all a Conda fan myself and avoid it like the plague

What's surprising to me is that this isn't better known. The only reliable solution I've found is to go with the pytorch or deepstream images from NGC. Conda is probably a good idea for noobs who need Cuda installed for them on windows, but otherwise I find it an endless source of finicky issues, especially for unsavvy ML scientists who are looking for a silver bullet for package management. This link shows which pac…

10 years ago, « Data Science » work past the experimental stage was performed by SWE with a knack for applied maths. So investing in tooling to do things properly was a given.

Nowadays, most DS people only want to do ML at the experimental stage only and get lost when things get on the engineering side of things. But for their defense, nowadays the bare minimum skills require to do programming, containerization, CI/CD, etc. More experienced and swiss army knife SWE/MLE have to educate the willing.

It was already the same 10 years ago with MATLAB dudes not wanting to get dirty with C/C++/ASM SIMD. The history repeats itself, only at a faster pace

Re: Pyenv – lets you easily switch between multiple versions of Python

#128

Post by lexicality »

p33p wrote: post #87
jpalomaki wrote: post #38

After some trial and error I have now settled on the following "stack": 1)I manage python versions with pyenv 2)For each new project I create a new virtualenv with venv "PYENV_VERSION=3.10 python -m venv .venv" 3)Then I start jump into the venv and initiate the project with Poetry ("poetry init -n") and manage dependencies with Poetry. If I'm keeping the projects under Dropbox, then I'll just add the .venv folder to…

Odd to me that you don’t let poetry create the venv? Why do this separately? Our flow is similar: pyenv (windows and linux), pipx, poetry. We’ve also defaulted poetry to utilize the current global version of Python and build the venv within the project folder.

Personally I've had problems with poetry managing virtualenvs in the past so I just don't let it touch them any more. Maybe it's better now, but I don't see any reason to risk it, given how often Python seems to like to ruin my day. I also don't like that by default it wants you to use `poetry run` to run things. Sure you can configure it not to, but it still annoys me

Re: Pyenv – lets you easily switch between multiple versions of Python

#129

Post by thangngoc89 »

maleldil wrote: post #112
thangngoc89 wrote: post #6

Earlier quoted context omitted.

I use poetry and direnv. Coming from node/npm, it feels natural for me to just do this. I have really no troubles of installing Pytorch with poetry

How are you installing Pytorch with CUDA with Poetry? I stopped using Poetry because it wouldn't automatically get the CUDA version; instead, it would install the CPU version. I migrated to PDM, which does the right thing.

Before CUDA 12.0 you have to specify a field in pyproject.toml like this

    [tool.poetry.dependencies]
    python = ">=3.10,
However, since CUDA 12.0 and Pytorch 2.1.0, just install like normal

    poetry add torch torchvision

Re: Pyenv – lets you easily switch between multiple versions of Python

#130

Post by llimllib »

antihero wrote: post #117
delduca wrote: post #95

https://asdf-vm.com/ ASDF is better because it works with many more languages, other than only Python, like Rust, Go, Node, etc, and other tools, such as AWS/Google/Firebase/Azure CLIs.

Yeah, ASDF is fantastic, I'm glad stuff like pyenv and rvm existed and paved the way for it but ASDF is the way to go nowadays.

asdf for python is a wrapper around pyenv, it didn't "pave the way", it's an important part of asdf.

(Also I like mise better currently: https://github.com/jdx/mise)

Post reply on HN