Live data from Hacker News

My Python Development Environment, 2020 Edition

jacobian.org

221–230 of 241 posts

Re: My Python Development Environment, 2020 Edition

#221
post #176

Earlier quoted context omitted.

Right, so if I'm understanding correctly, I'd need to pin the whole nix-pkgs system to the particular git revision which contains the version I need?

That's it, also you can cherry pick different software from different git revisions.

Do you have any public examples of doing this? Or docs?

Closest I found was to pin the nix-channel entirely, but not different components to different versions.

Re: My Python Development Environment, 2020 Edition

#222

Earlier quoted context omitted.

It doesn't have to be this way, here's a simpler alternative: https://news.ycombinator.com/item?id=21513044

I don't understand... That looks very similar to the other blog post.

You'll find it's a much simpler setup. Thanks for the input however.

Re: My Python Development Environment, 2020 Edition

#223
post #216
post #150

Earlier quoted context omitted.

if the conceptual load of having a large collection of binaries separately from your project (and ready for reuse!) is that high, you probably should use something else. I wonder how C/C++ people just manage this hardship... (and for all npm-friends: it's exactly the same "conceptual load" which arises from having node 8 and 12 around. except that these will be just incompatible by default, so noone bothers)

I don't understand the point you're making, where does a 'large collection of binaries' come in? The thing I'm talking about is right in the name 'virtual environment'. Or are you simply answering the GP's comment about this being a potential wart in Python with the suggestion that people should just use something else?

the point is that the classic virtualenv does not take care of different base distributions (3.5,3.6,3.7...). You have to install them yourselves and switch them in your PATH before setting up a virtualenv. if this is too much "conceptual load", I wish the parent good luck with finding something not having this problem ;). If there are binaries involved, there is the additional problem that you also depend on compiler features and C-libs, which I wish everyone good luck in replicating in their CI for any language (containerization and NixOS/guix/spack are approaches to that problem)

Re: My Python Development Environment, 2020 Edition

#224

Earlier quoted context omitted.

You should try Dockerizing your project. Then other people just have to type docker run and it’ll work everywhere.

Well in sceintific projects you are working in HPC platforms which docker doesn't exists and there are many reasons for it (it is not just technical).

and nowadays most HPC centers run on spack or easybuild anyways. I adopted easybuild 2 years ago, since then switched to spack and see that our local HPC center is also using it for their new modules...

Re: My Python Development Environment, 2020 Edition

#225
post #137
post #67

python -m venv venv source venv/bin/activate pip install -U pip pip install whatever # deactivate no need any third-party tools, venv is built-in the above steps always worked for me out of the box.

But how about replacing all of these commands with two words? poetry install

which is another layer of abstraction and dependency that I do not really need, e.g poetry no longer maintained, poetry(or whatever) has an urgent bugfix,etc

Re: My Python Development Environment, 2020 Edition

#226
post #67

python -m venv venv source venv/bin/activate pip install -U pip pip install whatever # deactivate no need any third-party tools, venv is built-in the above steps always worked for me out of the box.

Do you not consider pip third-party? It does not come with the standard Python distribution. Although `ensurepip` does.

No, pip has been available in standard Python distributions since 3.4.[1] Distributions that don't come with pip (e.g. some Linux distro packages) are non-standard.

[1] https://docs.python.org/3/whatsnew/3.4.html#whatsnew-pep-453

Re: My Python Development Environment, 2020 Edition

#227
post #96
post #39

I have never understood the need for all the different tools surrounding Python packaging, development environments, or things like Pipenv. For years, I have used Virtualenv and a script to create a virtual environment in my project folder. It's as simple as a node_modules folder, the confusion around it is puzzling to me. Nowadays, using setuptools to create packages is really easy too, there's a great tutorial on t…

Last time I looked, it was very tedious to set up `pip` to be secure and pin your dependencies to hashes. Without this, a compromise of a library's pypa account would allow them to execute arbitrary code on your system, assuming you didn't notice the change. You can use `pip-tools` to get something like a Gemfile/package.json, but there are a few restrictions that are suboptimal. So Pipenv/Poetry are the current best…

What do you find restrictive about pip-tools? It's been working excellent for me.

Re: My Python Development Environment, 2020 Edition

#228
post #73

Earlier quoted context omitted.

Yes. As someone who has never dove deep into python, but has had some contact with it: the package manager ecosystem is the #1 thing keeping me away from it. npm sucks and all, but at least it just works and doesn't get in my way as much.

What does npm do that python can’t? I’m curious.

[deleted]

Re: My Python Development Environment, 2020 Edition

#229
post #8

Earlier quoted context omitted.

Being in the data science community myself, I prefer straight venv + pip to conda. It’s simpler for me to manage errors. I only use conda when I have to.

Yeah I don't know a single person who chooses to use conda.

On the other hand, almost everyone I know who does scientific python in grad school loves miniconda...
Post reply on HN