Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Python virtual environments work

snarky.ca

Post reply on HN
Print view View original

71–80 of 293 posts

Re: How Python virtual environments work

#71

Post by adgjlsfhk1 »

cstrahan wrote: post #56
danielvaughn wrote: post #43

Earlier quoted context omitted.

I just began writing Python a few months ago. For years prior, I'd been a JS dev, and while NPM can be frustrating at times, I never encountered so many issues as I have in Python. It's crazy. I'm now curious whether there are languages out there that do have a really nice packaging system.

FWIW, I find Cargo to be one of the biggest reasons I like Rust so much — maybe even more than anything to do with Rust itself or safe code. I’ll often look for command line tools written in Rust, but not because of Rust fanboyism, but because I know I can just git clone the project and immediately start hacking on a new feature I need or a quick bug fix. In almost every other language I have to jump through one mill…

Yeah, one of Julia's best decisions was taking heavy inspiration from Rust for the package manager. Rust was 100% the first language to get dependency management right.

Re: How Python virtual environments work

#72

Post by zzzeek »

zelphirkalt wrote: post #70
birdstheword5 wrote: post #24

Earlier quoted context omitted.

It sounds mean to say it, but it's 100% true. I moved away from using python wherever I can. I've had colleagues struggle for days to install well used packages like pandas and numpy in conda.

My advice would be to not use Conda or other such "extra tooling". More trouble than benefit. Stick with venv and poetry.

poetry is "extra tooling"

Re: How Python virtual environments work

#73

Post by aflag »

_coveredInBees wrote: post #67

I'm surprised at the number of people here complaining about venvs in Python. There are lots of warts when it comes to package management in Python, but the built-in venv support has been rock solid in Python 3 for a long time now. Most of the complaints here ironically are from people using a bunch of tooling in lieu of, or as a replacement for vanilla python venvs and then hitting issues associated with those tools…

It's incredibly lacking in features. PyPI doesn't even properly index packages, making pip go into this dependency resolution he'll trying to find a set of versions that will work for you. It works for simple cases with few dependencies/not a lot of pinning. But if your needs are a bit more complex it certainly shows its rough edges.

I actually find it amazing that they python community puts up with that. But I suppose fixing it is not that pressing now the language is widely adopted. It's not going to be anyone's priority to mess with that. It's high risk low rewards sort of project.

Re: How Python virtual environments work

#74

Post by buildbot »

bobbylarrybobby wrote: post #64
buildbot wrote: post #63

I personally hate Conda with a firey passion - it does so much weird magic and ends up breaking things in non obvious ways. Python works best when you keep it really simple. Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

If it were really that simple, surely all these other solutions wouldn't exist?

Not saying Conda et. al don't solve problems, especially in specific cases, but they also add them in my opinion.

Re: How Python virtual environments work

#75

Post by Spivak »

bobbylarrybobby wrote: post #64
buildbot wrote: post #63

I personally hate Conda with a firey passion - it does so much weird magic and ends up breaking things in non obvious ways. Python works best when you keep it really simple. Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

If it were really that simple, surely all these other solutions wouldn't exist?

So there's two notions of simple.

Simple in the sense that it's actually simple, the software you need can be installed with pip install with precompiled binaries for your platform when necessary it supports Python 3.something+, and all it's dependencies are either >= version or the version is >= x.y Then there's simple as in the software is actually incredibly useful but is an absolute nightmare of complicated dependency trees where only specific pinned minor versions work together, you need multiple incompatible compiler toolchains and distro packages, it only works if you have CUDA, precompiled binaries exist for some but not all and if you use the precompiled binaries then it changes the dependency story, if you want jupyter support that's a whole different thing AHHHHHHHHHHHHH

In that case some people with more time than sanity said fuck it we'll make it work and conda was born. For me it's a lifesaver when you want to use a piece of software but I wouldn't ever dare deploy production software with it without it being strongly isolated from everything else.

Re: How Python virtual environments work

#76

Post by buildbot »

jstx1 wrote: post #66
buildbot wrote: post #63

I personally hate Conda with a firey passion - it does so much weird magic and ends up breaking things in non obvious ways. Python works best when you keep it really simple. Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

With plain venv it’s hard to maintain multiple different Python versions on the same machine; conda makes this much easier. Also on M1/M2 Macs some libraries (especially for ML) are only available through conda-forge.

Is it? I just install different version with brew, and choose which python3.X to create a venv with. The ML packages issue is much more of an issue, but now that the ARM transition is well underway more and more packages work via normal pip.

Re: How Python virtual environments work

#77

Post by zzzeek »

bobbylarrybobby wrote: post #64
buildbot wrote: post #63

I personally hate Conda with a firey passion - it does so much weird magic and ends up breaking things in non obvious ways. Python works best when you keep it really simple. Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

If it were really that simple, surely all these other solutions wouldn't exist?

it really is that simple

conda exists because it deals with an entirely different package registry and is a whole distro on its own (I dont know why people need that either, my vague impression is that scence-y types want complete pushbutton installation, typing a command == fail, I guess, I dont know).

poetry exists because it does some kind of automated version management thing (as did pipenv), that I'm sure is nice but not something I've ever needed, but the buzz got out and it took over, people who have never typed "virtualenv" use poetry and they have no idea why.

Re: How Python virtual environments work

#78

Post by Joker_vD »

dangerlibrary wrote: post #15
Max_Limelihood wrote: post #8

Answer: they don’t (Seriously, I’ve gotten so fed up with Python package management that I just use CondaPkg.jl, which uses Julia’s package manager to take care of Python packages. It is just so much cleaner and easier to use than anything in Python.)

I hate python package management - I really do. But I've never actually had a problem with virtual environments, and I think it's because I just use virtualenv directly (rather than conda or whatever else). I have these aliases in my .bashrc, and I can't remember the last time I had a major issue. alias venv='rm -rf ./venv && virtualenv venv && source ./venv/bin/activate' alias vact='source ./venv/bin/activate' alias…

> source ./venv/bin/activate

To this day I'm not quite sure why the venv developers decided that sourcing was a good idea; all it does can be effectively replaced with

    #!/bin/sh
    export VIRTUAL_ENV="path to venv"
    export PATH="$VIRTUAL_ENV/bin:$PATH"
    unset PYTHONHOME
    exec "$SHELL"
Just run this script to get into an "activated" shell. To deactivate, just press Ctrl+D. If you're really fancy, you can replace the last line with

    exec "${@:-$SHELL}"
to run a command directly in the activated environment (and then deactivate it immediately).

Re: How Python virtual environments work

#79

Post by zzzeek »

jstx1 wrote: post #66
buildbot wrote: post #63

I personally hate Conda with a firey passion - it does so much weird magic and ends up breaking things in non obvious ways. Python works best when you keep it really simple. Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

With plain venv it’s hard to maintain multiple different Python versions on the same machine; conda makes this much easier. Also on M1/M2 Macs some libraries (especially for ML) are only available through conda-forge.

I have like seven pythons on my machine, and I use virtualenv all over, there's no issue, what's the issue? I have to type the path of a specific interpreter when i make a virtualenv, is that the problem? I bet that's the problem

Re: How Python virtual environments work

#80

Post by pbronez »

buildbot wrote: post #63

I personally hate Conda with a firey passion - it does so much weird magic and ends up breaking things in non obvious ways. Python works best when you keep it really simple. Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

Until you want to use anything with a c extension..
Post reply on HN