Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Python Development Environment, 2018 Edition

jacobian.org

Post reply on HN
Print view View original

201–210 of 224 posts

Re: My Python Development Environment, 2018 Edition

#201

Post by CMCDragonkai »

nextos wrote: post #128
thepratt wrote: post #89

Earlier quoted context omitted.

I'm a big fan apart from how they version. Depending on when you install the version will point at a different commit hash. In our case, all of our Haskell builds failed on all but 1 machine (day apart in resolutions). How nixos handles environments meant our large package builds would kill the entire os, or underlying lib linking just failed - never figured out why, but it wasn't worth investing more time.

This is one of the usability problems they have. There are some simple solutions to this, but it should be more straightforward.

Pinning is the answer to these usability problems.

Re: My Python Development Environment, 2018 Edition

#202

Post by collyw »

This would have been useful if he compared them to virtualenv, which as far as I am aware is still the "standard" way of managing python environments.

I tried virtualenv wrapper a while ago, and that was basically just another set of commands to do the same thing as virtualenv. Having already learned virtualenv command that gave me no real advantage. Are any of these tools mentioned any different?

Re: My Python Development Environment, 2018 Edition

#203

Post by speedplane »

The first requirement in the article:

> I need to develop against multiple Python versions, including 2.7, various Python 3 versions (3.5 and 3.6, mostly), and PyPy.

The article states that this is an "unusual" setup, but unfortunately it is all too common. The Python 2/3 split has created such a large and sad schism in the development community, has wasted countless developer hours, and has held back the language itself incalculably. A travesty.

Re: My Python Development Environment, 2018 Edition

#204

Post by speedplane »

falcolas wrote: post #141
amenod wrote: post #55

Earlier quoted context omitted.

This, me too. I never understood the need for virtualenv and similar. Do people really encounter trouble with conflicting packages that often? I try to write scripts so they run on different versions of python anyway, unless there is a very specific reason why that is not possible; and even then you can run python versions in parallel on a Debian/Ubuntu box, with different pip installs for each of them. As for produc…

I've found that the library dependency issues appear in direct proportion to the number 3rd party libraries you use. If you're mostly developing against the standard library with a handful other stable libraries, it's not an issue. If you're developing with a requirements.txt (or pipfile) with 10 or more entries, you'll start running into conflicts on a regular basis. It can also occur as projects age, especially wit…

I think pretty much everyone would "prefer to use the standard library where possible". But try building a modern web app with that. No one uses libraries because they want to, they use b/c they need to.

Re: My Python Development Environment, 2018 Edition

#205

Post by speedplane »

collyw wrote: post #202

This would have been useful if he compared them to virtualenv, which as far as I am aware is still the "standard" way of managing python environments. I tried virtualenv wrapper a while ago, and that was basically just another set of commands to do the same thing as virtualenv. Having already learned virtualenv command that gave me no real advantage. Are any of these tools mentioned any different?

Does virtualenv handle different python versions? My understanding was that it just handled different sets of 3rd party packages.

Re: My Python Development Environment, 2018 Edition

#206

Post by vivan »

speedplane wrote: post #205
collyw wrote: post #202

This would have been useful if he compared them to virtualenv, which as far as I am aware is still the "standard" way of managing python environments. I tried virtualenv wrapper a while ago, and that was basically just another set of commands to do the same thing as virtualenv. Having already learned virtualenv command that gave me no real advantage. Are any of these tools mentioned any different?

Does virtualenv handle different python versions? My understanding was that it just handled different sets of 3rd party packages.

Yes, you can make virtualenvs with different Python interpreters.

Re: My Python Development Environment, 2018 Edition

#207

Post by wirrbel »

pwang wrote: post #176
wirrbel wrote: post #157

Earlier quoted context omitted.

As if that still was a problem, now that we have wheels, and especially manylinux wheels.

See Myth #6 here: http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-mis...

Valid points. Still most of them are not issues in my daily work.

And by the way Peter, thank you for the amazing work you do.

Re: My Python Development Environment, 2018 Edition

#208

Post by joshmaker »

wirrbel wrote: post #156
Lxr wrote: post #117

Earlier quoted context omitted.

To further elaborate on 2, it solves the problem of maintaining loose version ranges in your requirements.txt file, but keeping the versions pinned when you deploy. For example if you put `foo>=2` in your requirements.txt, this is dangerous without some way of pinning e.g. `foo==2.18.2` and running your tests against that before you deploy. But you obviously don't want to manually edit requirements.txt with minor ver…

Don't forget that usually you'll start to sort your requirements into dev requirements and production requirements which makes these packaging scripts much more complicated. https://github.com/jazzband/pip-tools would be what I used before pipenv came to be.

Two features I miss from pip-tools:

1. `pip-sync`, An easy way to ensure my local environment actually matches my defined requirements. I guess the pipenv version of this would be `pipenv uninstall --all && pipenv install` which isn't quite as elegant, but perhaps good enough.

2. The ability to create more than two requirement sets. For my projects it's often handy to three sets of requirements:

• Normal production requirements end users will need to run the app

• CI requirements needed for testing, but not running the app in production (Selenium, Flake8, etc)

• Local debugging tools (ipython, ipdb)

I could include my local debugging tools in the `--dev` requirements, but then I'm unnecessarily making my CI builds slower by adding requirements for packages that should never actually be referenced in committed code. Alternatively, I could leave them out of my dependencies entirely, but then I have to remember to reinstall them every time I sync my local env to ensure it matches the piplock file.

Re: My Python Development Environment, 2018 Edition

#209

Post by nickjj »

axitanull wrote: post #173
nickjj wrote: post #113

I'm a big fan of using Docker because for real world web app development, your app is often more than just getting Python and a virtualenv set up. Earlier this week I wrote about the pains of setting up a Python development experience without Docker, and then compared it to Docker as well. If anyone is curious, that's located at https://nickjanetakis.com/blog/setting-up-a-python-developme... . By the way, I would say…

>These are pretty big Flask apps too, which have thousands of lines of code, dozens of packages, tons of assets and require running Celery, Postgres, Redis, etc.. I was trying to create a new project last month trying to put all of these together. One thing that I was stuck in was putting these together: - non-single-thread-Flask - Redis pub-sub, subscribe to listen to a channel - and websocket (such as socket.io) up…

In the past, when I needed websockets in a Flask app I used Faye (backed by Redis) for the websocket back-end.

It was really easy to set up with Docker as it just becomes another container in your Compose file.

Nowadays I would just use Pusher.

Re: My Python Development Environment, 2018 Edition

#210

Post by cat199 »

AlexCoventry wrote: post #143
Spiritus wrote: post #131

Earlier quoted context omitted.

My problem with using Docker (only) is that it doesn't translate well to editors. Like, using jedi-vim[1] with a virtualenv constructed by a Docker container doesn't work at all. Unless I actually run vim itself inside said container. So unless your dependences build on macOS (like in my case), everything goes out the door. [1] https://github.com/davidhalter/jedi-vim

Emacs has pretty nice integrations with inferior docker processes. I'm getting them via spacemacs, so I'm not sure what specific package provides it, but I assume it's elpy.

what kinds of interactions? (like: what operations, etc)

using elpy here with a native install and can use e.g. tramp for remote editing/python sessions in most cases (even across machines), but determining docker mount points to edit in-machine not so much..

I suppose one can simply map a local code directory into a runtime environment, but this also makes e.g. interacting with an in-container interpreter a bit bothersome (not so bad actually, but have to set a separate interpreter path to something like 'docker exec -it ctid python'

would like to track this down I suppose

Post reply on HN