Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyenv – lets you easily switch between multiple versions of Python

github.com

Post reply on HN
Print view View original

291–300 of 341 posts

Re: Pyenv – lets you easily switch between multiple versions of Python

#291

Post by SOLAR_FIELDS »

Hackbraten wrote: post #247
rbanffy wrote: post #119

Earlier quoted context omitted.

If you are deploying to such an ancient OS, it's perhaps easier to have the whole OS packaged as a container or a VM and use that. It's not only different Python versions that might bite you.

What makes you think that GP’s comment is talking about older OSes? My understanding of their comment is that they’re talking about getting older Python interpreters to run on more modern OSes, modern enough that they don’t carry the older Python as a system package anymore. Hence, deadsnakes.

This is not about system packages or OSes, this is about your application needing Python 3.9.7 specifically, and locking to that, and 3.9.7 not being available in repositories anymore (3.9.7 is just an example). So normally you would either need to self host 3.9.7 somewhere or compile it from source on every new machine (which is terrible for CI but fine for local dev, a one off in local dev to build a Python version is nothing but paying 4 minutes every time on ci to compile your version of Python from source is a very angry amount of time for a lot of people).

Re: Pyenv – lets you easily switch between multiple versions of Python

#292

Post by mixmastamyk »

physicles wrote: post #287
mixmastamyk wrote: post #233

Earlier quoted context omitted.

Looks like what could be a shell alias to cd and activate? I always keep a few tabs open to my repos folder, so not sure it would help me much. Doesn't seem to support fish, though activate does.

I modified my bash prompt to detect and auto-activate Python environments, and show if one is currently active. Haven’t thought about activate in years. It’s great.

Yes, I did that on my work laptop with fish, since I only ever used it with the work project.

On my personal machine I don't bother with venvs, so also haven't thought about it in years. But was trying to figure out what the "workon" command did for GP.

Re: Pyenv – lets you easily switch between multiple versions of Python

#293

Post by philwelch »

SJetKaran wrote: post #289
philwelch wrote: post #151

Earlier quoted context omitted.

You could just give up and resort to using Docker.

Even at that, there are edge cases like when a python package like Tensorflow uses a instruction set that is not available on a Arm machine.

Yeah, personally I wouldn't develop on a different ISA than what I deployed to in production.

Re: Pyenv – lets you easily switch between multiple versions of Python

#294

Post by pxc »

speedgoose wrote: post #230
Cu3PO42 wrote: post #183

Earlier quoted context omitted.

This is not true. Nix actually solves the problem. If you package with Nix, you'll get the exact same version of Python with the exact same version of dependencies, including the exact same version of system libraries. Your build will work in a year just as it does today. You don't even need to keep any binary artifacts. Of course this doesn't come free: packaging with Nix may involve nontrivial effort for some proje…

How does nix solves the problem of Python dependencies not specifying their dependencies well or exhaustively? Do you have to find them out and fix them manually or can you generate a lock file and hope for the best?

> How does nix solves the problem of Python dependencies not specifying their dependencies well or exhaustively?

In Nix lingo, a package successfully building or running against an implicit or unmanaged dependency is called 'impurity'. To help keep builds 'pure', Nix builds everything in a sandbox and does various other tricks to ensure that a package being built can't/doesn't find any dependencies at build time that you don't explicitly tell Nix to include in the build environment.

(This is also increasingly how Linux distros build their packages, to solve the same problem.)

For the most part, if building (and running the test suite during the build) a Python package with Nix succeeds, you can be confident that you've got all the dependencies sorted out— even the ones upstream forgot to tell you about.

> Do you have to find them out and fix them manually or can you generate a lock file and hope for the best?

At install time, you can be confident that Nix will bring along all of the system-level dependencies your Python package needs. You don't have to find and fill any gaps at install time 10 years from now or whatever.

When you're writing your Nix package for the first time, you'll be doing a mix of generating the Nix code that defines your package from some upstream, Python-specific lockfile and making manual corrections when the build fails. Nix doesn't have any magic for figuring out dependencies that are left out of poetry.lock or whatever, or for disambiguating guaranteed-compatible exact versions from requirements.txt.

Re: Pyenv – lets you easily switch between multiple versions of Python

#295

Post by pxc »

fbuilesv wrote: post #69
nlitened wrote: post #40

Tools you can use to make sure the Python program you wrote keeps working: requirements.txt, pip, pipenv, pyenv, virtualenv, pyenv-virtualenv, virtualenvwrapper, pyenv-virtualenvwrapper, venv, pyvenv, conda, miniconda, poetry, docker, nix. Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?

What package managers are people using in other languages to make sure that software "always works the same as when you first wrote it, without asterisks"? I'd like to understand how they solve the "package no longer exists in a central registry" problem.

> I'd like to understand how they solve the "package no longer exists in a central registry" problem.

This is, of course, an infrastructure/maintenance issue as much as a package manager design issue. But in Nix's case, the public 'binary cache' (for Nixpkgs/NixOS) of build outputs includes not only final build outputs but also the source tarballs that go into them. As Nix disallows network access at build time, all dependencies are represented this way, including jar files or source tarballs, or whatever— Nix itself must be the one to fetch your dependencies. Consequently, everything you fetch from the Internet for your build is a kind of intermediary Nix build that can be cached using the usual Nix tools. The Nix community's public cache has a policy of retaining copies of upstream sources forever (there is recently talk of limiting storage of the final built packages to a retention period of only 2 years, but sources will continue to be retained indefinitely. So far the cache reaches back to its inception around a decade ago.)

Taken together, these things mean that when a project disappears entirely from GitHub or Maven Central or whatever, people building against old versions of it with Nix/Nixpkgs don't even notice. Nix just fetches those upstream sources from the public cache without even reaching out to that central repository from which those sources have been removed.

For private use cases where your project and its dependencies won't be mirrored to the public cache of Nixpkgs builds, you can achieve the same effect by running your own cache or paying a hosted service to do that.

For builds outside the Nix universe, you can make special arrangements for each type of package your various builds fetch, and mirroring those repos. Then configure your builds to pull from your mirrors instead of the main/public ones.

Re: Pyenv – lets you easily switch between multiple versions of Python

#296

Post by Capricorn2481 »

BiteCode_dev wrote: post #5

I have to warn again users that think they have found the silver bullet that pyenv comes with a big caveat: it compiles python on your machine. The number of possible modes of failure in this situation is huge. See also: "Why not tell people to "simply" use pyenv, poetry or anaconda" https://www.bitecode.dev/p/why-not-tell-people-to-simply-use I'm not saying pyenv is not a useful tool, but it is not a tool for beginn…

> People that love pyenv are so adamant at telling everybody they should use it. How it solved all their problems, and got rich and lost 5 pounds.

> It doesn’t support Windows. That’s game over right there, for half of the community.

What do you mean by this? I use pyenv on windows all the time.

Re: Pyenv – lets you easily switch between multiple versions of Python

#297

Post by Capricorn2481 »

camgunz wrote: post #279
Capricorn2481 wrote: post #273

Earlier quoted context omitted.

Does it handle OS differences?

Yep

How exactly? Isn't a requirements.txt file, generated from pip-compile, incapable of specifying OS stuff?

It's also my understanding that pip-tools does not let you make layered requirements files with pyproject.toml files, only with .in files

Re: Pyenv – lets you easily switch between multiple versions of Python

#298

Post by mikemcquaid »

godelski wrote: post #274
mikemcquaid wrote: post #91

Earlier quoted context omitted.

As. Homebrew maintainer: confirmed. You can also adjust your settings to pin versions or avoid cleanup, too. ‘brew pyenv-sync’ may be useful around this area, too.

Isn't it using the Python version in the formula?

Yes but there are multiple versioned formulae.

Re: Pyenv – lets you easily switch between multiple versions of Python

#299

Post by andreif »

neon5077 wrote: post #277

I really find it quite astounding just how much absolute nonsense python programmers are willing to put up with. Needing more than one, MAYBE two versions of your language installed is insanity. If I build a Python widget and send it to my non-programmer coworker, there is exactly zero chance it will work until I walk over and manually set up the correct language. Instead I use a language that natively compiles to an…

It’s totally up to your knowledge and experience to produce code that supports multiple versions. I had no issue with doing it. That’s what you pay for dynamic language that evolves rapidly.

Re: Pyenv – lets you easily switch between multiple versions of Python

#300

Post by andreif »

nlitened wrote: post #40

Tools you can use to make sure the Python program you wrote keeps working: requirements.txt, pip, pipenv, pyenv, virtualenv, pyenv-virtualenv, virtualenvwrapper, pyenv-virtualenvwrapper, venv, pyvenv, conda, miniconda, poetry, docker, nix. Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?

Dependencies are for the weak. You only need Python. Just support a range of versions if you intend to distribute.
Post reply on HN