Pyenv – lets you easily switch between multiple versions of Python • Hacker News Board

Re: Pyenv – lets you easily switch between multiple versions of Python

#271

Post by ris » Mon, Mar 25, 2024, 8:00 PM UTC

speedgoose wrote: post #230

Cu3PO42 wrote: post #183

Earlier quoted context omitted.

This is not true. Nix actually solves the problem. If you package with Nix, you'll get the exact same version of Python with the exact same version of dependencies, including the exact same version of system libraries. Your build will work in a year just as it does today. You don't even need to keep any binary artifacts. Of course this doesn't come free: packaging with Nix may involve nontrivial effort for some proje…

How does nix solves the problem of Python dependencies not specifying their dependencies well or exhaustively? Do you have to find them out and fix them manually or can you generate a lock file and hope for the best?

Ultimately the true bellwether of compatibility is the package's own test suite, which we try to get working (and integrated into the build process) for as many packages as possible. For packages which have poor/non-existent test suites, often a downstream package's test suite will expose compatibility problems (we've found bugs in zlib security patches using curl's test suite for instance).

nixpkgs maintainers are frequently the first to notify a project author of incompatibility with new versions of another package.

Re: Pyenv – lets you easily switch between multiple versions of Python

#272

Post by IshKebab » Mon, Mar 25, 2024, 8:04 PM UTC

notatallshaw wrote: post #165

IshKebab wrote: post #145

Earlier quoted context omitted.

There isn't one. Python distribution and packaging is just fundamentally horribly broken. I think his point was that you shouldn't pretend to users that just switching to pyenv is the solution.

> Python distribution and packaging is just fundamentally horribly broken It's clearly not because most people successfully use it fine. The problem of distribution and packaging is often a matter of user expectations vs. the actual problem. The user expectations is that Python is a high level language and will run the same across different machines regardless of OS and Hardware. The actual problem is Python is a glu…

> It's clearly not because most people successfully use it fine.

Well, no because it's perfectly possible to successfully use a horribly broken system. I use Python "successfully", it just meant I have spend probably literal weeks of my life fighting pip and virtualenv and relative imports and finding I need flags like `--config-settings editable_mode=compat`.

> The problem of distribution and packaging is often a matter of user expectations vs. the actual problem.

Ha yes, I expect it to work reliably and simply and it doesn't!

> The actual problem is Python is a glue language often depending on lots of libraries that are sensitive to how they were compiled and what hardware they are targeting.

That's completely irrelevant to the kind of problems I was talking about. I run into issues with Python failing to compile C libraries relatively rarely! Even compiling Python itself seems to work quite well (maybe not surprising since that's one of the only ways to get a new version on Linux).

It's all the packaging infrastructure that's a mess. Pip, virtualenv, setuptools, and also the module import system is a total disaster. You don't see questions like this for Go:

https://stackoverflow.com/questions/14132789/relative-import...

Re: Pyenv – lets you easily switch between multiple versions of Python

#273

Post by Capricorn2481 » Mon, Mar 25, 2024, 8:18 PM UTC

camgunz wrote: post #102

I'll toss in my (macOS) workflow too: - Download whatever Python binaries I need from python.org - virtualenv --python=$PYTHON_VERSION ~/.virtualenvs/$PROJECT_NAME - pyproject.toml - pip-compile --generate-hashes There has been tons of churn in the Python project management space, and I feel like I've been blissfully unaware of all of it with this workflow. Can't recommend enough. Why regular Python binaries and not…

Does it handle OS differences?

Re: Pyenv – lets you easily switch between multiple versions of Python

#274

Post by godelski » Mon, Mar 25, 2024, 8:24 PM UTC

mikemcquaid wrote: post #91

rcarmo wrote: post #28

Earlier quoted context omitted.

This is misleading. The older versions get cleaned out as brew does the upgrades (if not, then check your brew settings). My take on brew's Python is that it is there to make sure there is a uniform runtime for everything with a Python dependency, but I am not required to use it.

As. Homebrew maintainer: confirmed. You can also adjust your settings to pin versions or avoid cleanup, too. ‘brew pyenv-sync’ may be useful around this area, too.

Isn't it using the Python version in the formula?

Re: Pyenv – lets you easily switch between multiple versions of Python

#275

Post by recursive » Mon, Mar 25, 2024, 8:24 PM UTC

blowski wrote: post #3

OP - I'm curious why you've submitted this today, you seem experienced enough to guess this is not the first time you've used it. Has there been a major change or something happened in the community I need to know about? Or are you just spreading the love for a great piece of open source software?

I've used python lightly for a few decades now. I've never got far enough in to figure out how packages work. Once, at least 10 years ago, I briefly tried to use some published packages. I never figure out how to get it to work at all. Other than that, I've limited myself to what's available in the standard library. I've seen a lot of conflicting information about how python packages work, or should work. If there was a "one true way", I'd be interested. I don't want to have to compare a list of pros and cons. I want one officially blessed tool. I'll live with the constraints.

Re: Pyenv – lets you easily switch between multiple versions of Python

#276

Post by notatallshaw » Mon, Mar 25, 2024, 8:34 PM UTC

IshKebab wrote: post #272

notatallshaw wrote: post #165

Earlier quoted context omitted.

> Python distribution and packaging is just fundamentally horribly broken It's clearly not because most people successfully use it fine. The problem of distribution and packaging is often a matter of user expectations vs. the actual problem. The user expectations is that Python is a high level language and will run the same across different machines regardless of OS and Hardware. The actual problem is Python is a glu…

> It's clearly not because most people successfully use it fine. Well, no because it's perfectly possible to successfully use a horribly broken system. I use Python "successfully", it just meant I have spend probably literal weeks of my life fighting pip and virtualenv and relative imports and finding I need flags like `--config-settings editable_mode=compat`. > The problem of distribution and packaging is often a ma…

> It's all the packaging infrastructure that's a mess. Pip, virtualenv, setuptools, and also the module import system is a total disaster. You don't see questions like this for Go:

> https://stackoverflow.com/questions/14132789/relative-import...

I think you'll find the further you delve into it the less the problems are distinct, a lot of the issues with the module import system, pip, virtualenv, setuptools, etc. is because they are designed with having to support a vast range of things, from being depended on to interact with system libraries to downloading sdists and compiling arbitrary languages, etc.

Though the specific example you linked was largely solved with Python 3, there was a lot of confusion during the 2 to 3 transition because people had to support both behaviors, but most people don't have to think about Python 2 any more.

Re: Pyenv – lets you easily switch between multiple versions of Python

#277

Post by neon5077 » Mon, Mar 25, 2024, 8:35 PM UTC

I really find it quite astounding just how much absolute nonsense python programmers are willing to put up with.

Needing more than one, MAYBE two versions of your language installed is insanity. If I build a Python widget and send it to my non-programmer coworker, there is exactly zero chance it will work until I walk over and manually set up the correct language. Instead I use a language that natively compiles to an exe.

Python is pretty neat, but the concept of a program that just works anywhere is so utterly alien that I genuinely cannot find any practical use for it.

I can't just install a pip package to use globally, I have to set up some goddamn virtual environment because everything is so brittle that nothing works without the exact correct language version.

It's like NPM all over again. Dependency hell is so bad that your package manager is now a dependency so we need a package manager manager to manage installing the correct package manager version to then install the correct packages.

Every single time I've come up to some Python thing or other, I spend about fifteen minutes fucking with it before giving up and using a tool built in a sane language.

The fact that I can't just send any random person a program and expect it to run at all is just lunacy. And no, compiled Python does not count because that's even more brittle tooling on top of all the other bullshit I have to deal with. If it doesn't work out of the box, it's a bad tool and I have far better things to do with my very valuable time.

Re: Pyenv – lets you easily switch between multiple versions of Python

#278

Post by camgunz » Mon, Mar 25, 2024, 8:44 PM UTC

latexr wrote: post #219

camgunz wrote: post #96

Earlier quoted context omitted.

You can use --copies if you don't want symlinks [0]. [0]: https://docs.python.org/3/library/venv.html

Not with the macOS Python from the Xcode developer tools: > Error: This build of python cannot create venvs without using symlinks

Don't use that one

Re: Pyenv – lets you easily switch between multiple versions of Python

#279

Post by camgunz » Mon, Mar 25, 2024, 8:45 PM UTC

Capricorn2481 wrote: post #273

camgunz wrote: post #102

I'll toss in my (macOS) workflow too: - Download whatever Python binaries I need from python.org - virtualenv --python=$PYTHON_VERSION ~/.virtualenvs/$PROJECT_NAME - pyproject.toml - pip-compile --generate-hashes There has been tons of churn in the Python project management space, and I feel like I've been blissfully unaware of all of it with this workflow. Can't recommend enough. Why regular Python binaries and not…

Does it handle OS differences?

Yep

Re: Pyenv – lets you easily switch between multiple versions of Python

#280

Post by IshKebab » Mon, Mar 25, 2024, 8:47 PM UTC

notatallshaw wrote: post #276

IshKebab wrote: post #272

Earlier quoted context omitted.

> It's clearly not because most people successfully use it fine. Well, no because it's perfectly possible to successfully use a horribly broken system. I use Python "successfully", it just meant I have spend probably literal weeks of my life fighting pip and virtualenv and relative imports and finding I need flags like `--config-settings editable_mode=compat`. > The problem of distribution and packaging is often a ma…

> It's all the packaging infrastructure that's a mess. Pip, virtualenv, setuptools, and also the module import system is a total disaster. You don't see questions like this for Go: > https://stackoverflow.com/questions/14132789/relative-import... I think you'll find the further you delve into it the less the problems are distinct, a lot of the issues with the module import system, pip, virtualenv, setuptools, etc. is…

> Though the specific example you linked was largely solved with Python 3

I can assure you it absolutely was not.

> I think you'll find the further you delve into it the less the problems are distinct, a lot of the issues with the module import system, pip, virtualenv, setuptools, etc. is because they are designed with having to support a vast range of things, from being depended on to interact with system libraries to downloading sdists and compiling arbitrary languages, etc.

Not really. There are plenty of systems that have to "support a vast range of things" that aren't this bad.

In my opinion it's because the core Python devs didn't particularly care about the issue, never really tried to solve it, and as a result we have 10 incompatible half-baked third party solutions.

It's similar to the situation with C/C++ - worse in some ways, better in others (at least there is a de facto package registry in Python). In some ways it's because both languages are very old and predate the idea that packaging should be easy and reliable. That's fine, but please don't pretend that it is easy and reliable now.