Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Python virtual environments work

snarky.ca

Post reply on HN
Print view View original

221–230 of 293 posts

Re: How Python virtual environments work

#221

Post by mixmastamyk »

epgui wrote: post #198
mixmastamyk wrote: post #184

Earlier quoted context omitted.

Basically the point is to avoid the system python, which is not hard. One needs some sys-ad skills to understand what is going on however; unfortunately sounds like they are short supply. I don't do anything you mention, so there must be a simpler way.

It may not be hard to avoid system python, but that doesn't mean it's not also easy to mess everything up in 99 different barely-reproducible ways. I'm perfectly comfortable setting up python for projects I work on (with maybe 2-3 different methods), but that doesn't mean it's user-friendly or beginner-friendly. It doesn't mean the abstractions aren't taxing. I really agree with a lot of the criticism here.

"Mess everything up" is not a real state, it can always be fixed. However it might be faster, if you're using source control and short on time, to simply delete the folder(s), then re-clone and reinstall.

Beginner-friendly is to avoid all this complexity in the first place. I think it's a mistake to mention venvs to newbies until they are comfortable with python, paths, links, and environment vars.

Re: How Python virtual environments work

#222

Post by selcuka »

johtso wrote: post #214
whalesalad wrote: post #104

Earlier quoted context omitted.

I've been using Python since like 2006, so maybe I just have that generational knowledge and battlefront experience... but whenever I come into threads like this I really feel like an imposter or a fish out of water. Like, am I using the same Python that everyone else is using? I echo your stance - the less overhead and additional tooling the better. A simple requirements.txt file and pip is all I need.

Isn't pip + requirements.txt insufficient for repeatable deployments? You need to pin all dependencies not just your immediate project dependencies, unless you want some random downstream update to break your build. I guess you can do that by hand.. but don't you kind of need some kind of a lock file to stay safe/sane?

Not sure why repeatable deployments would be a problem. You can pin all dependencies by issuing a

    pip freeze > requirements.txt
if you want. The only catch is you should be using a similar architecture and Python version in both development and production.

This would also pin a few non-project dependencies such as `disttools` but that shouldn't be a problem.

Edit: TIL that pip constraints is a thing. See the comment posted by oblvious-earth for an even better approach.

Re: How Python virtual environments work

#223

Post by smeagull »

_coveredInBees wrote: post #67

I'm surprised at the number of people here complaining about venvs in Python. There are lots of warts when it comes to package management in Python, but the built-in venv support has been rock solid in Python 3 for a long time now. Most of the complaints here ironically are from people using a bunch of tooling in lieu of, or as a replacement for vanilla python venvs and then hitting issues associated with those tools…

My complaints stem from libraries/OSes requiring different tools. So conda is sometimes required, and pip is also sometimes required, and some provide documentation only for pipenv rather than venv. And then you've got Jupyter, which needs to be configured for each environment.

On top of that there are some large libraries that need to only be installed once per system because they're large, which you can do but does mess with dependency resolution, and god help you if you have multiple shadowing versions of the same library installed.

I wish it was simpler. I agree the underlying system is solid, but the fact that it doesn't solve some issues means we have multiple standards layered on top, which is itself a problem.

And great if you've been using vanilla venvs. Good for those that can. If I want hardware support for Apple's hardware I need to use fucking conda. Heaven help me if I want to combine that in a project with something that only uses pip.

Re: How Python virtual environments work

#224

Post by selcuka »

lmm wrote: post #178
dragonwriter wrote: post #109

Earlier quoted context omitted.

> With plain venv it’s hard to maintain multiple different Python versions on the same machine Ironically, given the usual software dev experience on Windows vs. Unixy systems, this is not a problem with the standard install on Windows with the py launcher.

It's not ironic at all; traditional Linux package management is actualy really bad . On a Linux system you can basically only have one version of anything installed at the same time.

I have multiple Python versions on both my Macbook M1 and my Linux laptop, with several dozen venvs. I honestly don't see why others are having so many issues.

Probably bad documentation/tutorials.

Re: How Python virtual environments work

#226

Post by int_19h »

zzzeek wrote: post #77
bobbylarrybobby wrote: post #64

Earlier quoted context omitted.

If it were really that simple, surely all these other solutions wouldn't exist?

it really is that simple conda exists because it deals with an entirely different package registry and is a whole distro on its own (I dont know why people need that either, my vague impression is that scence-y types want complete pushbutton installation, typing a command == fail, I guess, I dont know). poetry exists because it does some kind of automated version management thing (as did pipenv), that I'm sure is nic…

conda exists because it can install so much more stuff than pip without hassle, including things that aren't themselves Python packages, but which Python packages need. For example, if you ever tried to build a Python module that has some native code on Windows, you might appreciate this: https://anaconda.org/conda-forge/cxx-compiler.

Re: How Python virtual environments work

#227

Post by lmm »

selcuka wrote: post #224
lmm wrote: post #178

Earlier quoted context omitted.

It's not ironic at all; traditional Linux package management is actualy really bad . On a Linux system you can basically only have one version of anything installed at the same time.

I have multiple Python versions on both my Macbook M1 and my Linux laptop, with several dozen venvs. I honestly don't see why others are having so many issues. Probably bad documentation/tutorials.

Presumably they're not managed by your system package manager? Or you have one of the rare and relatively new distributions (certainly not typical Linux) that can handle that smoothly, like NixOS?

Re: How Python virtual environments work

#228

Post by riffraff »

Wowfunhappy wrote: post #202
TheRealPomax wrote: post #146

Earlier quoted context omitted.

Except when you try to move it, or copy it to a different location. This _almost_ made sense back when it was its own script, but it hasn't made sense for years, and the obstinacy to just sit down and fix this has been bafflingly remarkable. ("why not make everyone install their own venv and run pip install?" because, and here's the part that's going to blow your mind: because they shouldn't have to . The vast majori…

> Except when you try to move it, or copy it to a different location. Or just, y'know, rename the containing folder. Because last night I liked the name `foo` but this morning I realized I preferred `bar`, and I completely forgot that I had some python stuff inside and now it doesn't work and I have to recreate the whole venv!

Why does that break venv? I thought it'd be linking to things outside of itself but shouldn't be aware of where it is.

(Sorry, not a python expert)

Re: How Python virtual environments work

#229

Post by int_19h »

ggregoire wrote: post #113
sakex wrote: post #38

It feels like it is one of the reasons experienced devs are ditching Python for production systems. Besides horrendous performance and lousy semantics. The cost of setting up, maintaining the environment and onboarding people is just not worth it.

Experienced Python devs run their projects in docker, which solves the 3 issues you listed at the end.

It very much depends on what kind of project it is. Good luck shipping a desktop app that runs in Docker, for example. Not everything is web and ML.

Re: How Python virtual environments work

#230

Post by int_19h »

killjoywashere wrote: post #210

I didn’t realize venv was part of the standard library. If that’s the case, how is it that conda even exists? Anybody got a good history of this?

conda can install things other than Python packages. C++ compilers, for example, or native libraries that Python packages depend on.
Post reply on HN