Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Python Development Environment, 2020 Edition

jacobian.org

Post reply on HN
Print view View original

211–220 of 241 posts

Re: My Python Development Environment, 2020 Edition

#211

Post by axelf4 »

analog31 wrote: post #42
marmada wrote: post #31

Does anyone else think this reflects badly on Python? The fact that the author has to use a bunch of different tools to manage Python versions/projects is intimidating. I don't say this out of negativity for the sake of negativity. Earlier today, I was trying to resurrect an old Python project that was using pipenv. "pipenv install" gave me an error about accepting 1 argument, but 3 were provided. Then I switched to…

Granted, I'm just a scientific programmer, but my workplace has a full blown software team maintaining a multi million line codebase. That codebase is rebuilt every night, and as I understand it, you're not allowed to submit a change that breaks anything. And they have people whose job is to keep their tools working. What people casually think of as "Python" is really a huge dynamic ecosystem of packages. Imagine tha…

> you're going to hit some bumps if you haul an old code out of the vault and try to fire it up on a new system.

Most package managers have lockfiles that allow for some degree of determinism. Of course if a library introduces breaking changes you're going to have to rewrite, but only when explicitly upgrading the dependency.

Re: My Python Development Environment, 2020 Edition

#212

Post by ssully »

skocznymroczny wrote: post #208
TheChaplain wrote: post #158

Earlier quoted context omitted.

I never had any issue myself, but I guess that is because I use the standard tools: python3 -m venv /tmp/foo /tmp/foo/bin/pip -U pip wheel /tmp/foo/bin/pip -r requirements.txt I understand some might not like it, but really, it's simple and it works.

Until you have to deploy on a machine without internet access, and suddenly pip -r requirements is not enough, especially if you don't have a local pip mirror.

I develop a project that gets deployed to some users with locked down boxes (tight permissions, no internet), and it's really not that bad. You just download the dependencies using `pip download package_name` and bundle them with your project. Your install is basically the same; `pip install -r requirements.txt --no-index --find-links /path/localdeps/`.

It's not as nice as just doing a regular pip -r, but it works and isn't that much effort.

Re: My Python Development Environment, 2020 Edition

#213

Post by ssully »

coldtea wrote: post #185
roel_v wrote: post #181

Earlier quoted context omitted.

Until now I just deal with that by always explicitly specifying the whole path to the python executable. Granted my needs are very run-of-the-mill, but wouldn't that suffice for many people? I've tried several times over the last few months to get into all this python environment/package stuff, but it all feels like yak shaving... much like the hours and hours I spend customizing vim years ago, which were fun (at tha…

> Granted my needs are very run-of-the-mill, but wouldn't that suffice for many people? For individual people perhaps. For companies with older projects, newer projects, greenfield stuff, etc, no.

I maintain a mix of old and new python projects (sadly, still working on migrating some older 2.7 stuff to 3) and my setup is the same as TheChaplain. I just keep a separate venv for each project and have it setup appropriately in VSCode. With VSCode, I don't even have to think about which venv I am working with outside of when I initially set the interpreter.

Re: My Python Development Environment, 2020 Edition

#214

Post by ScoutOrgo »

Grue3 wrote: post #151
Evidlo wrote: post #37

For those new or unfamiliar with python, I think the best solution is the simplest: pip and virtualenv

I would actually recommend conda, it solves the "multiple version" issue and is compatible with virtualenv/pip so you can keep using them as usual.

Miniconda makes this really simple and their doc on environments is easy to read/understand. The benefits of conda really shine when trying to install a package with external dependencies to an environment.

The one thing I thought was neat here was pipx. I do have a few CLIs set up in my default conda env and haven't run into any dependency problems yet, but have occasionally tried to use them while another env is activated. Having a separate env automatically created for the entry points is a nice value add.

Re: My Python Development Environment, 2020 Edition

#216

Post by pvg »

fock wrote: post #150
pvg wrote: post #105

Earlier quoted context omitted.

For years, I have used Virtualenv Isn't this just a variant of what the original comment is critiquing, though? In order to sanely use Python and external dependencies you need some highly stateful, magical tool that does 'environments' for you. The conceptual load of this is quite high. Adding docker only makes it higher - now you're not merely 'virtualizing' a single language runtime but an entire OS as well - just…

if the conceptual load of having a large collection of binaries separately from your project (and ready for reuse!) is that high, you probably should use something else. I wonder how C/C++ people just manage this hardship... (and for all npm-friends: it's exactly the same "conceptual load" which arises from having node 8 and 12 around. except that these will be just incompatible by default, so noone bothers)

I don't understand the point you're making, where does a 'large collection of binaries' come in? The thing I'm talking about is right in the name 'virtual environment'. Or are you simply answering the GP's comment about this being a potential wart in Python with the suggestion that people should just use something else?

Re: My Python Development Environment, 2020 Edition

#218

Post by wp381640 »

jacobian wrote: post #202
wp381640 wrote: post #111

If you're going to be using pyenv + poetry you should be aware of #571 that causes issues with activating the virtualenv https://github.com/sdispater/poetry/issues/571 the OP himself has a fix for this in his own dotfiles repo: https://github.com/jacobian/dotfiles/commit/e7889c5954daacfe...

Ha, I'd forgotten about that. Thanks for the reminder. (Though, how'd you find that? Mildly creepy that you know more about my dotfiles than I do!)

I landed on that issue a while ago and your pull request was linked so I ripped your solution and added it to my own dotfiles :)

Re: My Python Development Environment, 2020 Edition

#219

Post by yrro »

AdrienLemaire wrote: post #15
tedivm wrote: post #7

Earlier quoted context omitted.

With pipx when you install things they go into isolated environments. With pip you're just installing things globally. This difference is important due to dependencies- if you have two different CLI tools you want to install but they have conflicting dependencies then pip is going to put at least one of them into an unusable state, while pipx will allow them to both coexist on the same system.

I haven't used pipx, but as far as I understand, pipx = pip + venv. If your pip executable is in a virtualenv, the "globally installed" is locally installed. pipx, poetry, pipenv and co are still nice wrappers to have, I suppose. It just feel less useful now that most of my projects are dockerized.

pipx looks nice. Is there any way to persuade it to install 'wheel' before it installs the desired package?

That way 'pipx install foo' can download and install wheels rather than downloading source distributions and building/installing them...

Re: My Python Development Environment, 2020 Edition

#220

Post by anonu »

There are two things that I find a bit elusive with Python:

1. Highlight to run 2. Remoting into a kernel

Both features are somewhat related. I want to be able to fire up a Python Kernel on a remote server. I want to be able to connect to it easily (not having to ssh tunnel over 6 different ports). I want connect my IDE to it and easily send commands and view data objects remotely. Spyder does all this but its not great. You have to run a custom kernel to be able to view variables locally.

Finally, I want to be able to connect to a Nameko or Flask instance as I would any remote kernel and hot-swap code out as needed.

Post reply on HN