Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You don't really need a virtualenv

frostming.com

Post reply on HN
Print view View original

121–130 of 148 posts

Re: You don't really need a virtualenv

#121

Post by danmur »

user5994461 wrote: post #116
danmur wrote: post #12

Earlier quoted context omitted.

Activating the venv is completely optional. I never do it, but some people seem to like it.

It's not optional. It's required for things to work properly. A basic example of something that would break if the environment is not activated is binaries, like mypy. You have a script that calls "mypy"? Gotta be in the virtual environment to invoke it, otherwise the executable might not be found, or worse, it might run another executable.

It's absolutely not required, I assure you. If you run the Python binary inside the venv you have the same environment as if you'd activated the venv, but only for that execution and not affecting your shell. Try making a venv, running python, and looking at sys.path.

Re: You don't really need a virtualenv

#122

Post by BiteCode_dev »

frostming wrote: post #119
BiteCode_dev wrote: post #95

My first attempt, and I get "AttributeError: module 'toml.decoder' has no attribute 'TomlPreserveCommentDecoder'" So careful, far from stable. I reported it ( https://github.com/frostming/pdm/issues/247 ) because the project is great and I want it to succeed, but I won't put it in prod any time soon. I prefer my package manager to be stable than fancy. It's also why I don't use poetry and so on in training. There is…

> So careful, far from stable. True, bugs can't be exposed by a few users. People have vastly different environment setup for their Python development. System python, in-venv, pyenv, asdf, homebrew, etc. So I post it here to seek for help testing. The sematic versioning may be a bit misleading, it is just because there was a big breaking change to support PEP 621, and I marked it as "Alpha" stage.

I have a lot of respect for the way you react to this comment, and how quickly you fixed the bug.

Thank you for working on such a useful project, and so seriously.

I immediately encountered a second different error, and I'll also take the time to craft a good bug report for it.

Re: You don't really need a virtualenv

#124

Post by wiredfool »

nicoburns wrote: post #109
wiredfool wrote: post #108

Earlier quoted context omitted.

There are several reasons that people don’t get the binary wheels, but a lot of the time it’s either pinned dependencies where they’re requesting an old version while running 3.9, that window between a new python release coming out and our quarterly release schedule, or an unsupported arch for binaries like Alpine, the m1 macs, random android.

Ah I see, that makes sense. I believe Node modules typically fall back to building from source in those cases. Which can be a little slow, but generally works reliably.

I can see that, but we rely on a bunch of system libraries (libjpeg/zlib, + optional libtiff/freetype/etc, and python headers as well). Pretty much by definition, if we don't provide binaries, it's not going to be that simple.

I'd say maybe 25-50% of developer linux workstations have the appropriate packages and will compile out of the box, and if they don't, it's almost always a one line command to install them with a package manager. We've got instructions or docker images for testing for most major linux distros.

MacOS is probably in the 5-10% range for successfully compiling out of the box, some of the headers ship with MacOS, but there are a couple steps to go through. M1 is still a WIP in the python packaging community.

Windows... 0% compile out of the box. You have to set it up, download dependencies and so on. There are at any one time a handful of people who can compile for the windows platform, and a smaller handful of people who can debug problems.

Now, to be fair, I think this project is probably in the most complicated 1% of python projects from a packaging point of view. There are probably more complicated ones, and the only two complications I'm aware of that we don't have to deal with are 1) The chicken and egg of if we break an update we lose the ability to update all the other packages including our own (pip) and 2) GPU drivers.

Re: You don't really need a virtualenv

#125

Post by esyir »

beojan wrote: post #120
ajford wrote: post #110

Earlier quoted context omitted.

I honestly can't even come up with a valid reason for something like nested venvs. At that point it seems more like someone's doing something wrong than 'things get tricky'.

If you have unlimited disk space, sure. Otherwise, it would be really nice to be able to layer venvs so you don't need to have multiple copies of a package installed.

Isn't that basically docker images.

Re: You don't really need a virtualenv

#126

Post by collyw »

beojan wrote: post #120
ajford wrote: post #110

Earlier quoted context omitted.

I honestly can't even come up with a valid reason for something like nested venvs. At that point it seems more like someone's doing something wrong than 'things get tricky'.

If you have unlimited disk space, sure. Otherwise, it would be really nice to be able to layer venvs so you don't need to have multiple copies of a package installed.

My virtualenv for work comes in at 847Mb (actually a lot more than I expected). I have a 2Tb disk attached to my machine. For all intents and purposes, we do have unlimited disk space.

Re: You don't really need a virtualenv

#127

Post by collyw »

Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

We should have stuck with Perl. I never came across the same problems with CPAN.

Re: You don't really need a virtualenv

#128

Post by Spiritus »

BiteCode_dev wrote: post #92
Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

First, so many tutorials give you terrible information. People don't know about "-m", the "py" command, are told to use sudo and not --user. The current situation is actually not that bad. With my students, the topic is quickly explained and mastered. But it's not a common occurrence. However, it is true that it's still not as easy as it could be. Some reasons. - Python is older than java. - Python is shipped as part…

The “py” command?

Re: You don't really need a virtualenv

#129

Post by teitoklien »

smitty1e wrote: post #117
raverbashing wrote: post #54

Earlier quoted context omitted.

Yes you're using nested venvs you're really making it harder on yourself. Way way harder on yourself.

I've used a nested git repository, with the child directory in the .gitignore of the parent. I know that there is some thing about installing git checkouts with pip, but that has not been important to me yet.

Why not just use git submodules feature ?

Re: You don't really need a virtualenv

#130

Post by Dryondristica »

Traubenfuchs wrote: post #43

Could someone ELI7 why python is struggling so hard with dependencies and dependency management when in the Java world this is a solved problem and just works with maven, which is working so damn well it last got updated in November 2019, and the biggest fight going is whether you should use gradle as client instead of maven to access the same dependency ecosystem? I had to set up Python projects for some machine lea…

People always criticise Python for dependency management, but I've never found it to be that bad. virtualenv with requirements.txt has been completely fine in my experience with medium-sized projects. If necessary you can pin versions with a separate version-locked requirements.txt, and populate that from a base requirements file.
Post reply on HN