Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Python virtual environments work

snarky.ca

Post reply on HN
Print view View original

171–180 of 293 posts

Re: How Python virtual environments work

#171

Post by analog31 »

josteink wrote: post #90

All other languages: use whatever packages you like. You’ll be fine. Python: we’re going to force all packages from all projects and repos to be installed in a shared global environment, but since nobody actually wants that we will allow you to circumvent that by creating “virtual” environments you can maintain and have to deal with instead. Also remember to activate it before starting your editor or else lulz. And d…

I'm a long time Python user, and I don't defend virtual environments at all. I just don't use them. Granted, I'm a so called "scientific" programmer, and am not writing production code. I haven't run into the problems that are solved by virtual environments, nor have my colleagues. Sure, it means I'm probably living in a bubble, but it may be a bubble shared by a lot of people.

Python is the first language that I've used, where the user community is a major attraction, resulting in significant inertia. Replacing Python requires a new language and a new community. Also, the tools that helped build that community, such as Google and Stackoverflow, have (by some accounts) deteriorated.

If package management is that bad, then yeah, time to switch languages.

No high horse here. Like Sancho Panza, I have to be content with an ass. ;-)

Re: How Python virtual environments work

#172

Post by chpatrick »

coldtea wrote: post #141
whalesalad wrote: post #104

Earlier quoted context omitted.

I've been using Python since like 2006, so maybe I just have that generational knowledge and battlefront experience... but whenever I come into threads like this I really feel like an imposter or a fish out of water. Like, am I using the same Python that everyone else is using? I echo your stance - the less overhead and additional tooling the better. A simple requirements.txt file and pip is all I need.

Is it "generational knowledge and battlefront experience" or just "getting used to the (shitty) way things have always been" and Stockhold Syndrome?

It was pretty bad before but now it seems like there are a bunch of competing solutions each with their own quirks and problems. It feels like the JavaScript ecosystem.

Re: How Python virtual environments work

#173

Post by woodruffw »

crabbone wrote: post #134
whalesalad wrote: post #106

Earlier quoted context omitted.

I've been writing Python for a looong time. I have pushed out thousands and thousands of deployments across probably 40+ distinct Python codebases and only once or twice have I ever encountered a showstopper dependency resolution issue. At the end of the day you should want to have fine grained control over your deps and frankly there are many times where a decision cannot be automatically made by a package manager.…

> once or twice have I ever encountered a showstopper dependency resolution issue. Hahaha... (rolls on the floor) Do you want to know why that is? No seriously? I'm not laughing at you as much as I'm laughing at Python now, but hey, well, anyways, do you want to know why that happened to you? I know you don't. But I'll tell you anyways! Until quite recently, pip didn't give a rat's ass if the dependencies it installe…

I understand that you're being hyperbolic for rhetorical purposes, but I think you're overselling the problem with source distributions: most language package ecosystems boil down to the same "baseline" package representation as sdists do, and have the same basic "build it if no binary matches" approach. Many don't even provide built distributions; Rust and Go come to mind.

Python's problem isn't with source distributions as such, but with really bad metadata control (and excessively permissive approaches to metadata generation, like arbitrary Python setup files). Better metadata makes source-based language package management work just fine in every other language's ecosystem; much of the effort in Python packaging over the last ~8 years has been slowly turning Python in that direction.

Re: How Python virtual environments work

#174

Post by lmm »

Groxx wrote: post #96
aflag wrote: post #73

Earlier quoted context omitted.

It's incredibly lacking in features. PyPI doesn't even properly index packages, making pip go into this dependency resolution he'll trying to find a set of versions that will work for you. It works for simple cases with few dependencies/not a lot of pinning. But if your needs are a bit more complex it certainly shows its rough edges. I actually find it amazing that they python community puts up with that. But I suppo…

What does that have to do with venvs? I agree the packaging and distribution setup in python is an absolute mess, but that's entirely unrelated to venvs. It's like bringing up how python uses whitespace instead of curly-braces.

venvs are the recommended workaround for the fact that python packaging and distribution is a mess of global state. Lanugages with working packaging and distribution don't generally bother with anything venv-like.

Re: How Python virtual environments work

#175

Post by hot_gril »

Nadya wrote: post #170
hot_gril wrote: post #149

Earlier quoted context omitted.

I can't think of any package/dep systems I actually like other than npm. And they're even starting to screw that up with the `import` weirdness instead of the `require` that's been so simple and easy. Rust's system is probably the next best. ObjC/Swift packaging is a flaming disaster in practice, unless it's improved since I jumped that ship. Last time, I remember every single project having to rely on Cocoapods.

Weirdness? `require` was Node weirdness because Javascript lacked imports/exports at the time. The ES6 syntax is remarkably better, allows imports to be async, and doesn't need to run the code to see what should be assigned to `module.exports` allowing it to be statically analyzed which allows tree-shaking. Node's CJS syntax will only work for Node requiring that you transpile and bundle it for browsers. The ES6 synt…

Import is weird because there are several different ways to do the same thing, listed at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... and summarized under "there are four forms of import declarations." Four! And I always forget the different ways they work.

Sure tree-shaking and browser support are nice, but they didn't have to make the syntax this complicated to achieve that. Not an issue with other languages.

Re: How Python virtual environments work

#176

Post by harold_b »

cmcconomy wrote: post #17

My personal approach is: - use miniconda ONLY to create a folder structure to store packages and to specify a version of python (3.10 for example) - use jazzband/pip-tools' "pip-compile" to create a frozen/pinned manifest for all my dependencies - use pip install to actually install libraries (keeping things stock standard here) - wrap all the above in a Makefile so I am spared remembering all the esoteric commands I…

[dead]

Re: How Python virtual environments work

#177

Post by lmm »

buildbot wrote: post #63

I personally hate Conda with a firey passion - it does so much weird magic and ends up breaking things in non obvious ways. Python works best when you keep it really simple. Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

> Just a python -m venv per project, a requirements.txt, and you will basically never have issues.

As long as you always remember to run exactly the right two commands every time you work on any project and never run the wrong ones, or run project A's commands in project B's terminal. There's no undo BTW, if you ever do that you've permanently screwed up your setup for project B and there's no way to get back to what you had before (your best bet is destroying and recreating project B's venv, but that will probably leave you with different versions of dependencies installed from what you had before).

(And as others have said, that doesn't cover multiple versions of Python, or native dependencies. But even if your project is pure python with only pure python dependencies that work on all versions, venv+pip is very easy to mess up and impossible to fix when you do)

Re: How Python virtual environments work

#178

Post by lmm »

dragonwriter wrote: post #109
jstx1 wrote: post #66

Earlier quoted context omitted.

With plain venv it’s hard to maintain multiple different Python versions on the same machine; conda makes this much easier. Also on M1/M2 Macs some libraries (especially for ML) are only available through conda-forge.

> With plain venv it’s hard to maintain multiple different Python versions on the same machine Ironically, given the usual software dev experience on Windows vs. Unixy systems, this is not a problem with the standard install on Windows with the py launcher.

It's not ironic at all; traditional Linux package management is actualy really bad. On a Linux system you can basically only have one version of anything installed at the same time.

Re: How Python virtual environments work

#179

Post by lmm »

danielvaughn wrote: post #43
birdstheword5 wrote: post #24

Earlier quoted context omitted.

It sounds mean to say it, but it's 100% true. I moved away from using python wherever I can. I've had colleagues struggle for days to install well used packages like pandas and numpy in conda.

I just began writing Python a few months ago. For years prior, I'd been a JS dev, and while NPM can be frustrating at times, I never encountered so many issues as I have in Python. It's crazy. I'm now curious whether there are languages out there that do have a really nice packaging system.

The JVM (maven) has quietly had everything working really well for decades. You rarely hear much about it because it just works, and what you hear is mostly people hating on it because it wouldn't let them shoot themselves in the foot. Cargo works much the same way AIUI.

Re: How Python virtual environments work

#180

Post by danielvaughn »

hot_gril wrote: post #149
danielvaughn wrote: post #43

Earlier quoted context omitted.

I just began writing Python a few months ago. For years prior, I'd been a JS dev, and while NPM can be frustrating at times, I never encountered so many issues as I have in Python. It's crazy. I'm now curious whether there are languages out there that do have a really nice packaging system.

I can't think of any package/dep systems I actually like other than npm. And they're even starting to screw that up with the `import` weirdness instead of the `require` that's been so simple and easy. Rust's system is probably the next best. ObjC/Swift packaging is a flaming disaster in practice, unless it's improved since I jumped that ship. Last time, I remember every single project having to rely on Cocoapods.

I remember trying to use cocoapods back in 2015/2016, right around the time that Swift was technically available but not ready for production. I literally gave up trying to import packages, it was a shitshow.
Post reply on HN