You don't really need a virtualenv • Hacker News Board

Re: You don't really need a virtualenv

#131

Post by 0xbadcafebee » Fri, Feb 05, 2021, 1:53 PM UTC

collyw wrote: post #127

Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

We should have stuck with Perl. I never came across the same problems with CPAN.

CPAN I think only recently got virtualenv-like support; before it was much worse and a pain. Where CPAN modules shine are they're more stable, backwards compatible, and extensible. Modules whose last release was 15 years ago work fine today, and new modules are extended on them.

Re: You don't really need a virtualenv

#132

Post by paulie_a » Fri, Feb 05, 2021, 2:48 PM UTC

ajford wrote: post #110

smitty1e wrote: post #27

> things get tricky when it comes to nested venvs Never had such a requirement. Environments, like the interpreter itself, seem a singleton concept. I have used a Makefile that sources different Bash environment variables kept in files in etc/ within the venv to switch between, say, a Flask and a Gunicorn startup.

I honestly can't even come up with a valid reason for something like nested venvs. At that point it seems more like someone's doing something wrong than 'things get tricky'.

If anything it just simply seems like a dumb idea to do so. Virtualenv is simple and works really nicely. Why anyone would want to add unnecessary complexity is beyond me

Re: You don't really need a virtualenv

#133

Post by paulie_a » Fri, Feb 05, 2021, 2:53 PM UTC

Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

I cannot fathom where python is struggling is with that. Pip and a requirements file that can be pinned with version numbers

It's super easy, barely an inconvenience

Gradle is a hot mess and JavaScript is the just a complete shit show

Re: You don't really need a virtualenv

#134

Post by paulie_a » Fri, Feb 05, 2021, 2:58 PM UTC

hctaw wrote: post #102

nicoburns wrote: post #46

Earlier quoted context omitted.

Python should just do what JavaScript does and install all dependencies to a project-local folder.

I'm surprised this is getting downvoted. This is the only sane way to manage dependencies. They must be relative to the thing you care about. There are tradeoffs for sure, like the ridiculous sizes of those directories on development machines. But a giant SSD is cheaper than my time resolving dependency issues on user machines.

They essentially are. Not in the actual directory but in the virtual env and not cluttering up your project directory. This is superior to JavaScript in my opinion

No language should replicate anything JavaScript does when it comes to package management

Re: You don't really need a virtualenv

#135

Post by kcon » Fri, Feb 05, 2021, 4:32 PM UTC

Spiritus wrote: post #128

BiteCode_dev wrote: post #92

Earlier quoted context omitted.

First, so many tutorials give you terrible information. People don't know about "-m", the "py" command, are told to use sudo and not --user. The current situation is actually not that bad. With my students, the topic is quickly explained and mastered. But it's not a common occurrence. However, it is true that it's still not as easy as it could be. Some reasons. - Python is older than java. - Python is shipped as part…

The “py” command?

I think that is referring to the Python launcher utility for Windows: https://docs.python.org/3/using/windows.html#python-launcher...

Re: You don't really need a virtualenv

#136

Post by mixmastamyk » Fri, Feb 05, 2021, 7:09 PM UTC

dbsmith83 wrote: post #105

2fast4you wrote: post #96

Earlier quoted context omitted.

You could replace that with a tool like direnv. It’ll automatically load your variables for a project when you enter it’s directory. https://direnv.net/

Interesting, thanks!

Script, source, and systemd units are options as well of course.

Re: You don't really need a virtualenv

#137

Post by mixmastamyk » Fri, Feb 05, 2021, 7:13 PM UTC

rk06 wrote: post #76

imtringued wrote: post #34

Earlier quoted context omitted.

Last time I used python I just did pip install -r requirements.txt -t libs and added the folder to PYTHONPATH. I never bothered with virtualenv.

PYTHONPATH is also global! so all projects will still be referring to the same set of libs. how does that change anything at all?

That's an optional environment variable, it is not set unless you set it. Meaning you could change it to anything, per process. Also you can change sys.path. Not an issue.

Re: You don't really need a virtualenv

#138

Post by N1H1L » Fri, Feb 05, 2021, 8:19 PM UTC

Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

Tell me about it. I had cudatoolkit installed, and then followed every single PyTorch installation instructions, and then still it insisted that I should get the CPU version, not the GPU one.

I ultimately fixed this with going to the conda files list, and choosing exactly what I needed. Not straightforward at all.

Re: You don't really need a virtualenv

#139

Post by arbol » Sat, Feb 06, 2021, 9:18 AM UTC

blt wrote: post #123

I struggled with Python dependency hell often until I started using Conda. I wish it were faster, but it rarely surprises me.

Conda is so slow that I resorted to using it for env creation and then just pip install everything. I don't have 10+ minutes to wait for package to be installed, only to be told it can't resolve conflicts. Package version conflict is not such a big deal, just pip install a different version of the offending package!

Re: You don't really need a virtualenv

#140

Post by user5994461 » Sat, Feb 06, 2021, 1:09 PM UTC

danmur wrote: post #121

user5994461 wrote: post #116

Earlier quoted context omitted.

It's not optional. It's required for things to work properly. A basic example of something that would break if the environment is not activated is binaries, like mypy. You have a script that calls "mypy"? Gotta be in the virtual environment to invoke it, otherwise the executable might not be found, or worse, it might run another executable.

It's absolutely not required, I assure you. If you run the Python binary inside the venv you have the same environment as if you'd activated the venv, but only for that execution and not affecting your shell. Try making a venv, running python, and looking at sys.path.

I assure you 100%, it is required for everything to work, it's not just here for the show.

One of the thing that the virtual environment does is modify the PATH to lookup executables from the virtual environment first.

This is to ensure that when a script calls binaries like mypy/python, it's going to find the same binary from the environment it's currently running in.

Typically if the environment is not activated, python is going to lookup to /usr/bin/python and might be something else.

This is most noticeable when there are different versions of python on the systems and when having scripts that invoke themselves.