Live data from Hacker News

My Python Development Environment, 2020 Edition

jacobian.org

111–120 of 241 posts

Re: My Python Development Environment, 2020 Edition

#111
If you're going to be using pyenv + poetry you should be aware of #571 that causes issues with activating the virtualenv

https://github.com/sdispater/poetry/issues/571

the OP himself has a fix for this in his own dotfiles repo:

https://github.com/jacobian/dotfiles/commit/e7889c5954daacfe...

Re: My Python Development Environment, 2020 Edition

#112
post #73

Earlier quoted context omitted.

Yes. As someone who has never dove deep into python, but has had some contact with it: the package manager ecosystem is the #1 thing keeping me away from it. npm sucks and all, but at least it just works and doesn't get in my way as much.

What does npm do that python can’t? I’m curious.

npm is equivalent to combining pip and virtualenv into a single tool. This gives better ergonomics when switching between projects since you never have to "activate" your environment, it's always activated when standing in the project directory.

Re: My Python Development Environment, 2020 Edition

#113
post #112
post #73

Earlier quoted context omitted.

What does npm do that python can’t? I’m curious.

npm is equivalent to combining pip and virtualenv into a single tool. This gives better ergonomics when switching between projects since you never have to "activate" your environment, it's always activated when standing in the project directory.

That does sound convenient. I wonder if the virtualenv aspect is relevant though, i.e. do people really deploy npm apps outside of a container/isolation layer?

I imagine if you're deploying docker, you probably should be developing in docker (e.g. using PyCharm's remote interpreter/docker interpreter integration).

Re: My Python Development Environment, 2020 Edition

#114
post #98

Earlier quoted context omitted.

To be fair, he says in the article that his requirements are somewhat different from most Python developers, to wit: * "I need to develop against multiple Python versions - various Python 3 versions (3.6, 3.7, 3.8, mostly), PyPy, and occasionally Python 2.7 (less and less often, thankfully)." * "I work on many projects simultaneously, each with different sets of dependencies, so some sort of virtual environment or is…

It's kind of shocking to hear these two quoted as "different than most" requirements – as a Ruby developer, this sounds like exactly a thing that any engineer supporting production systems would need routinely. RVM and bundler are standard developer tools and I would never question the need for supporting multiple versions on the same machine, unless in a very well-defined scenario where RVM was unneeded (like in a c…

Bundler is really nice and much better than virtualenv/pipenv/etc. I find this sadly ironic as I actually consider most of the Python ecosystem to have much more breadth and quality in terms of libraries; it’s just a pain in the ass to manage those dependencies once you actually want to use them.

Re: My Python Development Environment, 2020 Edition

#115
post #98

Earlier quoted context omitted.

To be fair, he says in the article that his requirements are somewhat different from most Python developers, to wit: * "I need to develop against multiple Python versions - various Python 3 versions (3.6, 3.7, 3.8, mostly), PyPy, and occasionally Python 2.7 (less and less often, thankfully)." * "I work on many projects simultaneously, each with different sets of dependencies, so some sort of virtual environment or is…

It's kind of shocking to hear these two quoted as "different than most" requirements – as a Ruby developer, this sounds like exactly a thing that any engineer supporting production systems would need routinely. RVM and bundler are standard developer tools and I would never question the need for supporting multiple versions on the same machine, unless in a very well-defined scenario where RVM was unneeded (like in a c…

I think contemporary best practices start with the production environment and work backwards towards creating a development environment as close to the production environment as possible.

These days, most productions environments are effectively isolated containers. If your production environment is a container, you probably should develop in a container as well. In that case you don't need much tooling for isolating an application's dependencies from other applications.

The tooling that you need is to build a python application, which means (1) get the dependencies (2) copy over some source code (or invoke the C-compiler if build a CPython extension) (3) run tests. Python's builtin setuptools does that fine. It didn't strike me as amazingly simple, but it's not amazingly complex either. pip is essentially a convenience wrapper for setuptools, i.e. pip is to setuptools as apt is to dpkg.

Basically, I believe that because of docker isolation is a irrelevant criterion by which to judge a language/ecosystem.

Re: My Python Development Environment, 2020 Edition

#116
post #107

Earlier quoted context omitted.

Pyenv is for installing multiple versions of python. Virtualenvs are a layer beneath that. It’s super useful for maintaining static versions of python, like 2.7, 3.6 and 3.7 when you have many projects that have different python requirements.

I understand how this was needed historically, when using the official installer might overwrite the Python you already had installed. But as far as I know, you can download an installer for a new version, run it, and it doesn't touch your previous installation. For example I've had 3.7 on my macOS system for a while, installed from the official installer, not through Homebrew. I just installed 3.8, which pointed my…

Sometimes scripts will want to invoke Python as ‘python’ and it’s easier to use pyenv to set $PATH overrides.

Re: My Python Development Environment, 2020 Edition

#117

Earlier quoted context omitted.

> There is also usually no "system Java", so there is nothing to break along those lines. Oh, but there is :( `jenv shell 1.8` is muscle memory for me now.

You can have multiple JVMs or JDKs installed, and therefore the need to change environment variables depending on your use cases, but I was referring to Java being part of the operating system in the same way that Python is part of some operating systems, for example several Linux distributions (Fedora, RHEL, and practically all derivatives).

Isn't it still just a package on Red Hat distros? A base system package, granted, because some system tools are written in Python.

But in any case, it just becomes one more version of Python to consider. If you're already dealing with multiple versions, what difference does it make?

Re: My Python Development Environment, 2020 Edition

#118
post #31

Does anyone else think this reflects badly on Python? The fact that the author has to use a bunch of different tools to manage Python versions/projects is intimidating. I don't say this out of negativity for the sake of negativity. Earlier today, I was trying to resurrect an old Python project that was using pipenv. "pipenv install" gave me an error about accepting 1 argument, but 3 were provided. Then I switched to…

Yes. As someone who has never dove deep into python, but has had some contact with it: the package manager ecosystem is the #1 thing keeping me away from it. npm sucks and all, but at least it just works and doesn't get in my way as much.

Anecdotally, I've had more Node packages with native code in them fail to build for me when installed via npm, than Python packages with native code fail when installed via pip. That whole node-gyp thing is a huge mess.

Re: My Python Development Environment, 2020 Edition

#119
post #27

> Although Docker meets all these requirements, I don't really like using it. I find it slow, frustrating, and overkill for my purposes. How so? I've been using Docker for development for years now and haven't experienced this EXCEPT with some slowness I experienced with Docker Compose upon upgrading to MacOS Catalina (which turned out to be bug with PyInstaller, not Docker or Docker Compose). This is on a Mac, btw;…

The build step for installing or upgrading a package can be a killer with nontrivial projects.

It seems like long builds are either (a) necessary or (b) user error. (a) If you have a tree of dependencies and you change the root, you should rebuild everything that depends on it to make sure it's still compatible. (b) if you placed your application into one of the initial Dockerfile layers, but then you're installing dependencies that don't depend on you, it's user error.

What's the situation where your application needs to go first in the Dockerfile, and then you need to put a bunch of stuff that doesn't depend on your application?

Re: My Python Development Environment, 2020 Edition

#120

I’ve been manually deploying my projects for years. Can anyone comment on the Docker learning & troubleshooting story for python?

Docker + setuptools/pip + python is great for development and production. Docker is definitely worth learning, and is pretty easy to learn.
Post reply on HN