Live data from Hacker News

You don't really need a virtualenv

frostming.com

71–80 of 148 posts

Re: You don't really need a virtualenv

#71
post #6

As a Ruby and Python developer I often wonder why Ruby doesn't need a virtual env. What did Ruby get right that Python didn't?

People use RVM or rbenv to manage Ruby versions. The virtualenv analogue is Bundler, which provides something like requirements.txt and causes gems to be installed locally instead of globally. https://bundler.io/

Re: You don't really need a virtualenv

#72

Earlier quoted context omitted.

Everyone ends up using rbenv or rvm.

You don't need them to solve this problem, though. The basic facilities are baked into rubygems.

The basic facilities are baked into python tools too, you can direct installation of packages to arbitrary folders and then just add them to your PYTHONPATH.

But it's not as quick and clean as "python -m venv venv", so everyone uses that (and pyenv for different versions).

Re: You don't really need a virtualenv

#73
post #13
post #6

As a Ruby and Python developer I often wonder why Ruby doesn't need a virtual env. What did Ruby get right that Python didn't?

well, a lot of "system" utilities are implemented in Python and I'm not aware of many in Ruby (they must exist? I just checked to see what on my system depends on Ruby and it seems like the only things are a screenruler utility, some texlive packages I've never heard of and gnome-code-assistance, whatever that is). Pip installing random crap is a great way to completely break some Linux systems.

Running pip as root is a great way to self-compromise. Would you ever run npm as root...?

Re: You don't really need a virtualenv

#74

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

I am probably in the minority, but my experience is the complete opposite. I had more trouble trying to set up a gradle project in university than I ever had with Python.

Gradle is a nightmare.

Re: You don't really need a virtualenv

#75

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

I am probably in the minority, but my experience is the complete opposite. I had more trouble trying to set up a gradle project in university than I ever had with Python.

Probably not much of a minority, as gradle's design seems much more friendly to footguns than maven.

Re: You don't really need a virtualenv

#76
post #24

Earlier quoted context omitted.

Not sure about ruby, but most of the languages I know about, C#, javascript, php, etc don't need virtualenv because they install packages in a project locally (and globally if required). so envs of one project do not clash with another. while python install packages globally and only globally . so a virtualenv is needed to isolate different projects from one another

Last time I used python I just did pip install -r requirements.txt -t libs and added the folder to PYTHONPATH. I never bothered with virtualenv.

PYTHONPATH is also global! so all projects will still be referring to the same set of libs. how does that change anything at all?

Re: You don't really need a virtualenv

#77
post #62

Earlier quoted context omitted.

> It is not hard to run into missing .h files while trying to install these dependencies. They would be if python provided the .h files in the packages rather than expecting them to already exist on the system.

That's just dangerous band-aid. If there is a mismatch between bundled .h and installed system library, you'll get (in better case) linker error, or runtime problems.

Well I was assuming the actual library would be bundled too: not just the headers!

Re: You don't really need a virtualenv

#78

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

My ELI7 understanding is that python installs required modules (libraries) at the system level by default, and doesn't have good support for installing modules into local directories like node/C/java/etc. does, which leads to conflicts. Users have to find a set of modules that support all of the scripts on their system. The current workaround is to isolate each script's requirements, but there are a ton of competing tools to do this (virtualenv, pipenv, poetry, etc.), so it's hard to choose the "best" isolation solution.

One reason people like PEP 582 is that, by being something built into the interpreter, it will reduce a lot of the confusion over which isolation tool to use.

Part of the problem, which local installs won't completely solve, is that the python interpreter is still changing fast enough that modules are having trouble keeping up. E.g. tensorflow is still working on python3.9 support [1], and now we'll see python3.10 in the next couple weeks. Even with local installs we'll have to be careful to use the right interpreter.

Another piece of the problem, which is also kind of a good thing, is the huge size of the python module ecosystem. I've got python scripts with over a hundred requirements. It saves me from writing a lot of code, but resolving the requirements is a challenge. The pip package manager has recently added a dependency resolver, which has helped a little make sure everything can work together.

The only surefire ways I can think of to make this huge module ecosystem interoperate cleanly are (1) have every module support every other version of every other module and interpreter (insane amount of work, won't happen) or (2) have simpler interop interfaces like the unix shell convention of only passing strings between programs, but that would take away a lot of the ease-of-use of passing rich objects around between modules.

[1] https://github.com/tensorflow/tensorflow/issues/44485

Re: You don't really need a virtualenv

#79
post #38
post #24

Earlier quoted context omitted.

Not sure about ruby, but most of the languages I know about, C#, javascript, php, etc don't need virtualenv because they install packages in a project locally (and globally if required). so envs of one project do not clash with another. while python install packages globally and only globally . so a virtualenv is needed to isolate different projects from one another

What do you mean "python install[s] globally and only globally"? That's not true. You can instruct pip to install to any place you want.

the thing is python cli has no concept of locally scoped libraries. you can install libs wherever you want, but all python.exe will refer to the same global location.

this is why you need to activate virtualenv every time you open a new command line, so that it can set ENV variables for the current session and force python to use the locally scoped folders.

While in C# et all. the compiler knows to check files locally. and you don't need to activate (or deactivate) environment at all.

Re: You don't really need a virtualenv

#80
post #70

Earlier quoted context omitted.

JavaScript's package management is significantly better than Pythons: - There's one tool to use, and it comes pre-installed - You don't have to deal with virtualenvs or packages from different projects conflicting. - Native code dependencies "just work".

> You don't have to deal with virtualenvs or packages from different projects conflicting. That's only because you're basically assured there will be conflicts inside any single project . Every time I install anything via npm I get a boatload of warnings about insecure dependencies, which are effectively impossible to fix without breaking the whole mountain of hacks. There are good deployment stories out there. JS is…

> That's only because you're basically assured there will be conflicts inside any single project.

That's not an issue with the package management though, that's an issue with the quality of the packages themselves. The JS ecosystem does have issues there. But the tooling itself is good.

Post reply on HN