Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Python virtual environments work

snarky.ca

Post reply on HN
Print view View original

241–250 of 293 posts

Re: How Python virtual environments work

#241

Post by bombolo »

MrJohz wrote: post #237
chpatrick wrote: post #172

Earlier quoted context omitted.

It was pretty bad before but now it seems like there are a bunch of competing solutions each with their own quirks and problems. It feels like the JavaScript ecosystem.

Ironically, the Javascript ecosystem is far better than the Python ecosystem when it comes to packaging and dependencies. NPM just does the right thing by default: you define dependencies in one place, and they are automatically fixed unless you choose to update them. Combine that with stuff like workspaces and scripts, and you basically have everything you need for the vast majority of use cases. Yes, there's also o…

> they are automatically fixed unless you choose to update them

That's a good way to never get vulnerabilities fixed.

It hardly seems like "the right thing" to me.

Re: How Python virtual environments work

#242

Post by bombolo »

kermatt wrote: post #213
anongraddebt wrote: post #151

Earlier quoted context omitted.

Twice bricking my laptop’s ability to do python development because of venv + symlink bs was the catalyst I needed to go all-in on remote dev environments. I don’t drive python daily, but my other projects thank Python for that.

How do you brick a machine with venvs?

He runs pip as root and doesn't use venvs.

Re: How Python virtual environments work

#243

Post by bombolo »

jayvius wrote: post #117
zzzeek wrote: post #77

Earlier quoted context omitted.

it really is that simple conda exists because it deals with an entirely different package registry and is a whole distro on its own (I dont know why people need that either, my vague impression is that scence-y types want complete pushbutton installation, typing a command == fail, I guess, I dont know). poetry exists because it does some kind of automated version management thing (as did pipenv), that I'm sure is nic…

> (I dont know why people need that either, my vague impression is that scence-y types want complete pushbutton installation, typing a command == fail, I guess, I dont know). Essentially, yes, and justifiably so. Try installing science-y Python packages on Windows written in C. When conda was created in 2012 this meant installing Visual Studio 2005 (for Python 2.7) which was hard to find on Microsoft's own website ev…

> on Windows

Ah, I found your bug!

Re: How Python virtual environments work

#244

Post by MrJohz »

bombolo wrote: post #241
MrJohz wrote: post #237

Earlier quoted context omitted.

Ironically, the Javascript ecosystem is far better than the Python ecosystem when it comes to packaging and dependencies. NPM just does the right thing by default: you define dependencies in one place, and they are automatically fixed unless you choose to update them. Combine that with stuff like workspaces and scripts, and you basically have everything you need for the vast majority of use cases. Yes, there's also o…

> they are automatically fixed unless you choose to update them That's a good way to never get vulnerabilities fixed. It hardly seems like "the right thing" to me.

I mean, a project needs regular care and maintenance, however you organise it. If you're never scheduling time to maintain your dependencies, you're going to be in trouble either way. But at least if you lock your dependencies, you know what will actually get installed, and you can find the buggy or insecure versions.

We found a bug on a Python project I worked on recently that only seemed to happen on certain machines. We couldn't reproduce it in a dev environment, and one machine the was affected suddenly stopped being affected after a while. It turns out the issue was a buggy dependency: one particular build of the project happened to have picked up the buggy version, but later builds used the fixed version and so didn't have a problem. So we'd only see the bug depending on which build the machine had last used, and if someone put a different build on there, it would reset that completely. On our development machines, we used slightly different builds that just happened but to have been affected.

Pinning dependencies wouldn't necessarily have prevented the bug in the first place - sometimes you just have buggy dependencies - but the debugging process would have gone much more quickly and smoothly with a consistent build environment. We could also have been much more confident that the bug wouldn't accidentally come back.

Re: How Python virtual environments work

#245

Post by anongraddebt »

kermatt wrote: post #213
anongraddebt wrote: post #151

Earlier quoted context omitted.

Twice bricking my laptop’s ability to do python development because of venv + symlink bs was the catalyst I needed to go all-in on remote dev environments. I don’t drive python daily, but my other projects thank Python for that.

How do you brick a machine with venvs?

By debugging homebrew issues during Moneterey updates.

I didn’t brick the machine, just the ability to setup a typical python venv.

Re: How Python virtual environments work

#246

Post by Doxin »

coldtea wrote: post #141
whalesalad wrote: post #104

Earlier quoted context omitted.

I've been using Python since like 2006, so maybe I just have that generational knowledge and battlefront experience... but whenever I come into threads like this I really feel like an imposter or a fish out of water. Like, am I using the same Python that everyone else is using? I echo your stance - the less overhead and additional tooling the better. A simple requirements.txt file and pip is all I need.

Is it "generational knowledge and battlefront experience" or just "getting used to the (shitty) way things have always been" and Stockhold Syndrome?

or the third option: did the whole packaging nonsense actually get kinda alright lately?

Re: How Python virtual environments work

#247

Post by coldtea »

Doxin wrote: post #246
coldtea wrote: post #141

Earlier quoted context omitted.

Is it "generational knowledge and battlefront experience" or just "getting used to the (shitty) way things have always been" and Stockhold Syndrome?

or the third option: did the whole packaging nonsense actually get kinda alright lately?

There's a PEP to get a part of it right [1] - at least the installation of dependencies and the need for virtualenv side, but atm the packaging nonsense is still as bad as it always has been.

https://peps.python.org/pep-0582/

Sample comment from its discussion:

>> Are pip maintainers on board with this? > Personally, no. I like the idea in principle, but in practice, as you say, it seems like a pretty major change in behaviour and something I’d expect to be thrashed out in far more detail before assuming it’ll “just happen”.

As if the several half-arsed official solutions already existing around packaging (the several ways to build and create packages) had deep thinking and design behind them...

Re: How Python virtual environments work

#248

Post by Doxin »

riffraff wrote: post #228
Wowfunhappy wrote: post #202

Earlier quoted context omitted.

> Except when you try to move it, or copy it to a different location. Or just, y'know, rename the containing folder. Because last night I liked the name `foo` but this morning I realized I preferred `bar`, and I completely forgot that I had some python stuff inside and now it doesn't work and I have to recreate the whole venv!

Why does that break venv? I thought it'd be linking to things outside of itself but shouldn't be aware of where it is. (Sorry, not a python expert)

When creating the venv it hardcodes some paths so the python interpreter knows where to find its modules and the likes.

That said, re-creating a venv shouldn't be hard and if it is you're doing something wrong in your development setup.

Re: How Python virtual environments work

#249

Post by coldtea »

chpatrick wrote: post #172
coldtea wrote: post #141

Earlier quoted context omitted.

Is it "generational knowledge and battlefront experience" or just "getting used to the (shitty) way things have always been" and Stockhold Syndrome?

It was pretty bad before but now it seems like there are a bunch of competing solutions each with their own quirks and problems. It feels like the JavaScript ecosystem.

The Javascript ecosystem for other things, like frameworks, sure.

But when it comes to packages and "virtual envs" the Javascript ecosystem is leaps and bounds better.

Re: How Python virtual environments work

#250

Post by coldtea »

bombolo wrote: post #241
MrJohz wrote: post #237

Earlier quoted context omitted.

Ironically, the Javascript ecosystem is far better than the Python ecosystem when it comes to packaging and dependencies. NPM just does the right thing by default: you define dependencies in one place, and they are automatically fixed unless you choose to update them. Combine that with stuff like workspaces and scripts, and you basically have everything you need for the vast majority of use cases. Yes, there's also o…

> they are automatically fixed unless you choose to update them That's a good way to never get vulnerabilities fixed. It hardly seems like "the right thing" to me.

It's 200% is "the right thing".

Updating packages should be strictly left to the developer's discretion. That schedule is up to the developer using the packages, not upstream.

Not to mention that dependencies updating themselves whenever they like to "fix vulnerabilities" is a sure-fire way to break your program and introduce breakage and vulnerabilities in behavior...

Post reply on HN