Live data from Hacker News

Please do not use Python for tooling

borud.no

21–30 of 84 posts

Re: Please do not use Python for tooling

#21
post #9

"In a pinch, even Java provides a better alternative as you have the ability to build all-in-one-jar files that contain all the dependencies. Not fashionable, but objectively a far less brittle option." Really? In a pinch, use a language that's 90% boilerplate and relies on a mastery of the IDE? It's hard to take this seriously.

You don't like people being hard on Python fine. Don't do the same for Java. Modern Java doesn't have nearly as much boilerplate and is nearly as concise as python.

Relying on IDE Python can be great and so can Java. Different strokes. Different use cases.

Re: Please do not use Python for tooling

#22
"Tooling" can mean so much, and different things to different people.

Python definitely has its strengths, and dominates certain domains -- but no one language is all-purpose, so in my book a warning against using one thing for all the things is decent advice.

Re: Please do not use Python for tooling

#23
I don't think the type system particularly matters. You can make something work without assigning a type like "int" to variables. Python doesn't let you do things like "foo" + 3 (while Node is happy to), so even though you're not naming your types, the types are still static.

One unexpected downside of static languages are dealing with web services written in dynamic languages. As an example, I was writing a Slack app in Go a couple years ago. Some response gets unmarshalled into an easy-to-use struct, but contrary to the documentation, the server has no interest in returning data that can always be unmarshaled into that struct. Sometimes instead of a list of 1 element, the value will just be that one element. There is no type "[]Foo|Foo" in Go, so now you have to write a custom unmarshaling function (or say "fuck it all!" and use map[string]any, at which point you're just writing Javascript). This doesn't cause problems for people using Python or Javascript because neither of those care what's in a dictionary, but statically typed languages do, and you'll have to write extra code to work around that.

I agree that it's annoying to require the users and tool author to have the same runtime and packages installed, though. Python is an incomprehensibly large can of worms here. A python package's dependencies are architecture/platform dependent. To install C packages, you need the exact same C compiler that was used to build Python. Some popular packages have an indirect dependency on a Fortran compiler. It's a pretty big nightmare. You could bundle the runtime with the application like Go does, but the runtime is pretty big and the language is too dynamic to remove the parts you aren't actually going to use in advance, so it's not as easy as a sell.

An option there is that if you're using Python for internal tooling, you bless a particular version once a year, install it on every workstation, and say "sorry, you can't use a different version of Python". That will alleviate a lot of the author's problems, but obviously it's exceedingly politically unpopular.

Anyway, there probably isn't an objective truth here, just personal and organizational preferences here. I agree with the author in that whenever I write some tooling in Python, I regret it almost immediately. But it's working for people.

Re: Please do not use Python for tooling

#24

Also don't use C or C++, i wasted so much time trying to compile stuff (no offence if you're a C dev). Also no Java please, how do I even install JVM on my system? Node and JavaScript, Ruby, Perl, Bash are out too of course. Bash sometimes works but what if I'm using a different shell?

Write Bourne shell (/bin/sh), it's POSIX so on every useful machine.

Re: Please do not use Python for tooling

#25
The article is kind of hyperbolic, but I feel like it echoes how I am beginning to feel about Python.

Just last week, I spent a whole afternoon getting a particular repository of Python code running on my laptop, even with the help of virtualenv and pyenv. requirements.txt doesn't tell me which version of Python the code was developed with, and several libraries are only available on certain versions, so I have to play the guessing game first of all. Then, some of the modules don't have binaries available for M1, and I can't build them from source because I don't have x, y, and z tools installed. Then there's always some issue with PYTHONPATH. I ended up having to build the whole Ubuntu docker container and develop inside that.

I love Python, it's not always like this, and it's certainly not only Python, but that experience is something I dread anyway coming in to every new Python project. It feels like DLL hell all over again. I have had a much better experience personally with C# and Rust, but admittedly I had much more solo control over those projects.

Re: Please do not use Python for tooling

#26
> If you use a language that can produce binaries the job of ensuring you have all the dependencies in all the right versions is a one-time job: it happens at build time.

This is not true for most language toolchains by default as far as I know. Most languages don't produce fully self-contained binaries. The developer have to do extra work to create self-contained binaries, otherwise the users still have to work to get the dependencies necessary.

What makes tooling different from system software in my view is that you're using software on a per-project basis, updating it as the project evolves. The developer of the tooling and the user of the tool (another developer) both have responsibilities. They have to agree on a common platform that the tools can target. The tool author is responsible for documenting explicitly any prerequisites and setup steps and they must make sure the tool doesn't implicitly depend on anything more. The user must make sure they've set up an environment for the project that meets the tools needs.

I've personally found that Python works reasonably well for tooling if I as a tool author follow a few guidelines. I require the users only to have a shell, python3, and python3-venv or miniconda installed on their base system. I provide a setup/activation script that creates/activates a virtualenv or a Conda env in the project directory and makes sure that the packages in the also-included requirements.txt or environment.yaml are installed before the tooling is run.

Since the scripts are provided as part of the tools, the tool author becomes responsible for automating the creation of a working environment for the user of the tool. This process can be automated and reproducible based on a frozen requirements.txt or similar, so most of the brittleness can be eliminated by the tool author.

I don't think any other tool implementation language would provide huge benefits to the users. They would usually still need to install some system-wide prerequisites and use some kind of per-project activation script.

The reason I like Python as a tool author is because it's better than writing shell scripts, and it's still easy to include as source with any kind of project. The standard library -- with the argparse, subprocess, urllib, shutil, etc. modules -- is good enough that for simpler tools no external dependencies (nor any activation script nor requirements.txt) are needed, but familiar for many developers.

Re: Please do not use Python for tooling

#27

I don't think the type system particularly matters. You can make something work without assigning a type like "int" to variables. Python doesn't let you do things like "foo" + 3 (while Node is happy to), so even though you're not naming your types, the types are still static. One unexpected downside of static languages are dealing with web services written in dynamic languages. As an example, I was writing a Slack ap…

> To install C packages, you need the exact same C compiler that was used to build Python.

Is this true? I have never encountered this problem.

My experience is that, since CPython is written in C, you only need to have any C compiler for your platform, because the C compilers will be ABI-compatible. I am not sure about Windows, because Windows has some funny issues with C runtime compatibility. But I have upgraded my Python installation and C compiler separately before without issue.

Re: Please do not use Python for tooling

#28
post #9

"In a pinch, even Java provides a better alternative as you have the ability to build all-in-one-jar files that contain all the dependencies. Not fashionable, but objectively a far less brittle option." Really? In a pinch, use a language that's 90% boilerplate and relies on a mastery of the IDE? It's hard to take this seriously.

You don't like people being hard on Python fine. Don't do the same for Java. Modern Java doesn't have nearly as much boilerplate and is nearly as concise as python. Relying on IDE Python can be great and so can Java. Different strokes. Different use cases.

I don't mind people being hard on Python, but the idea of Java over Python "in a pinch" is lost on me, but what do I know...

Re: Please do not use Python for tooling

#29

I don't think the type system particularly matters. You can make something work without assigning a type like "int" to variables. Python doesn't let you do things like "foo" + 3 (while Node is happy to), so even though you're not naming your types, the types are still static. One unexpected downside of static languages are dealing with web services written in dynamic languages. As an example, I was writing a Slack ap…

> even though you're not naming your types, the types are still static.

Python is strongly typed, but definitely not statically.

    >>> a = "foo"
    >>> type(a)
    
    >>> a = 3
    >>> type(a)
    
The variable `a` has different types at different parts of the program. That's not static.

Re: Please do not use Python for tooling

#30

I don't think the type system particularly matters. You can make something work without assigning a type like "int" to variables. Python doesn't let you do things like "foo" + 3 (while Node is happy to), so even though you're not naming your types, the types are still static. One unexpected downside of static languages are dealing with web services written in dynamic languages. As an example, I was writing a Slack ap…

> You could bundle the runtime with the application like Go does, but the runtime is pretty big and the language is too dynamic to remove the parts you aren't actually going to use in advance, so it's not as easy as a sell.

It's not so big as to be prohibitive though, especially in a dev environment... if this became the norm, I would be a lot happier with using python-based tooling than I am today.

Post reply on HN