Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You don't really need a virtualenv

frostming.com

Post reply on HN
Print view View original

101–110 of 148 posts

Re: You don't really need a virtualenv

#101

Post by peterthehacker »

Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

> I had to set up Python projects for some machine learning classes in college and it was a complete mess.

Did you use conda (anaconda/miniconda) or pip?

Re: You don't really need a virtualenv

#102

Post by hctaw »

nicoburns wrote: post #46
mqus wrote: post #45

Earlier quoted context omitted.

I have the opinion that this is because "Deployment" is not a solved issue (or clearly defined and enforced) for Python. C has shared libraries and static compilation (bundling of dependencies) Java has .jar files which probably contain all your dependencies, go and Rust statically compile to something that has minimal dependencies and python has... no clue. Some packages you have to install via your package manager…

Python should just do what JavaScript does and install all dependencies to a project-local folder.

I'm surprised this is getting downvoted. This is the only sane way to manage dependencies. They must be relative to the thing you care about.

There are tradeoffs for sure, like the ridiculous sizes of those directories on development machines. But a giant SSD is cheaper than my time resolving dependency issues on user machines.

Re: You don't really need a virtualenv

#103

Post by hctaw »

HelloNurse wrote: post #58
nicoburns wrote: post #46

Earlier quoted context omitted.

Python should just do what JavaScript does and install all dependencies to a project-local folder.

You are assuming there is a "project", but Python code with massive dependencies can (and often should) be a bare script in an arbitrary folder that runs by invoking the Python interpreter on it, without "installing" anything except missing dependencies.

That's not a bare script then, it's a project with dependencies that need to be installed. This misunderstanding is how you fuck up a developer environment.

Re: You don't really need a virtualenv

#104

Post by michaericalribo »

swiftcoder wrote: post #65
michaericalribo wrote: post #2

> One day a minor release of Python is out It is decidedly not true that I want to update my venv for every minor version bump! I deploy to a cloud service w/ a specific version; my package manager is slow to update; I develop collaboratively using a shared container w/ a fixed version. Updating shared resources with every new release is not always realistic, and that makes it so that I do need (or at least want ) to…

> It is decidedly not true that I want to update my venv for every minor version bump! Until you run into a minor version bump that patches a critical security vulnerability, and then you suddenly may very much want to (and in some corporate environments, have a regulatory requirement to)

That’s not every minor bump, though, and is worth the extra effort!

Also, my use case is 100% non critical from a security standpoint, so I can afford to be careless... your point definitely stands in sensitive environments

Re: You don't really need a virtualenv

#105

Post by dbsmith83 »

2fast4you wrote: post #96
dbsmith83 wrote: post #91

Curious, what about environment variables? I like the ability to put my environment variables in my activate script. It seems like if you did things the way the author does, you would need all of your env variables to be global, which could be hazardous

You could replace that with a tool like direnv. It’ll automatically load your variables for a project when you enter it’s directory. https://direnv.net/

Interesting, thanks!

Re: You don't really need a virtualenv

#106

Post by pjfin123 »

mqus wrote: post #45
Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

I have the opinion that this is because "Deployment" is not a solved issue (or clearly defined and enforced) for Python. C has shared libraries and static compilation (bundling of dependencies) Java has .jar files which probably contain all your dependencies, go and Rust statically compile to something that has minimal dependencies and python has... no clue. Some packages you have to install via your package manager…

This video really helped me understand Python packaging of binaries:

https://youtu.be/02aAZ8u3wEQ

Re: You don't really need a virtualenv

#107

Post by BerislavLopac »

mqus wrote: post #45
Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

I have the opinion that this is because "Deployment" is not a solved issue (or clearly defined and enforced) for Python. C has shared libraries and static compilation (bundling of dependencies) Java has .jar files which probably contain all your dependencies, go and Rust statically compile to something that has minimal dependencies and python has... no clue. Some packages you have to install via your package manager…

Or to be precise, it is a solved issue -- several times over. After years of meandering, I think it is safe to say that deployment of Python libraries is pretty much resolved in one, general way: we have the wheel/PyPI/pip toolchain, and that's about it.

On the other hand, deployment of Python applications is still unresolved, and there is no single standard method to ensure both compatibility with and isolation from the underlying system in a universal way. There are different methods, from virtual environments (both standard and nonstandard ones like conda), to version managers like pyenv and asdf, to "jailers" like pipx and pyinstaller, to containers; but none of them has yet clearly won.

Re: You don't really need a virtualenv

#108

Post by wiredfool »

nicoburns wrote: post #86
wiredfool wrote: post #81

Earlier quoted context omitted.

At that point, you should just be installing a binary wheel. (Nb I’m a maintainer of a python project that’s difficult to build, and the number of issues we get that start: can’t install because foo.h can’t be found is very high, even though there’s other explanations closer to the end of the failed build)

Does that not happen by default when you try to install a package? And if so, why not?

There are several reasons that people don’t get the binary wheels, but a lot of the time it’s either pinned dependencies where they’re requesting an old version while running 3.9, that window between a new python release coming out and our quarterly release schedule, or an unsupported arch for binaries like Alpine, the m1 macs, random android.

Re: You don't really need a virtualenv

#109

Post by nicoburns »

wiredfool wrote: post #108
nicoburns wrote: post #86

Earlier quoted context omitted.

Does that not happen by default when you try to install a package? And if so, why not?

There are several reasons that people don’t get the binary wheels, but a lot of the time it’s either pinned dependencies where they’re requesting an old version while running 3.9, that window between a new python release coming out and our quarterly release schedule, or an unsupported arch for binaries like Alpine, the m1 macs, random android.

Ah I see, that makes sense. I believe Node modules typically fall back to building from source in those cases. Which can be a little slow, but generally works reliably.

Re: You don't really need a virtualenv

#110

Post by ajford »

smitty1e wrote: post #27

> things get tricky when it comes to nested venvs Never had such a requirement. Environments, like the interpreter itself, seem a singleton concept. I have used a Makefile that sources different Bash environment variables kept in files in etc/ within the venv to switch between, say, a Flask and a Gunicorn startup.

I honestly can't even come up with a valid reason for something like nested venvs. At that point it seems more like someone's doing something wrong than 'things get tricky'.
Post reply on HN