Ask HN: Let's build Checkstyle for Bash?

Some context to my statements: I'm a SWE that works on infrastructure services and tooling, primarily on Linux but also Unixes.

It depends on what I'm doing but usually Go if an application requires any longevity at all.

I use Python or Java for things that require more dynamic typing. The main usecase here is big data or to prototype applications.

Bash I usually reserve for very small tasks. When I bootstrap services in containers, you'll usually find some Bash. A whole CLI written in Bash? I would not approve that PR. Bash lacks variable scoping for the most part, the ways in which you can implement it are hacky and non-obvious. Bash's syntax and the way it behaves varies by system, even installer scripts have to account for this. As I'm typing this, I realize the Bash code I write generally follows the same testing and constraints as the Makefiles I write.

I don't think I really understand the requirement for "a single document". This departs from programming practices I've built up over the years. When I come across large multi-thousand LOC documents (in any language, much less Bash) I'm usually concerned.

I've always been confused that in ${current_year} people are still directly writing huge + complex shell scripts.

Compare/contrast Javascript, another language we're "stuck with": while you can still directly write it, many people don't, instead using transpilers to target Javascript as an "object code" from some other, stricter language (e.g. TypeScript, ClojureScript, arbitrary native languages via Emscripten / WASM compiler targets, etc.) Via these languages/compilers, everybody who wants compile-time strictness can have it. And you can even get runtime strictness that Javascript itself doesn't have, as the checked semantics of these languages will often be "lowered" into the resulting Javascript by generating explicit assertions (usually this is on by default, unless you ask for an optimized build.) But the output is still just "Javascript", that any browser can run.

Why hasn't a similar thing happened for POSIX-standard Bourne shell? Why are we seemingly "satisfied" with writing + maintaining 5000-line install.sh scripts in our codebases, with a bunch of extra mental overhead / tooling required to keep the code sane and clean; when we could just be writing in a sane and clean language to begin with?

Is it just that nobody's bothered to create such a language/compiler? Do I need to be the one to step up, here?

Much of the time that people are writing shell scripts, they're writing them not because they prefer shell syntax to that of some other language, but rather because they're creating a script that needs to be widely disseminated/deployed to all sorts of machines with unpredictable toolchain availability.

This is why a large fraction of the shell scripts that exist in the world still hold to Bourne shell syntax, rather than using any of the syntax extensions from its descendant shells: Bourne shell (or at least, something at /bin/sh that interprets Bourne-shell syntax) is part of the POSIX standard. So you can expect any POSIX system — no matter how weird — to be able to run (Bourne) shell scripts. You can run them on Alpine. You can run them on Busybox. You can run them on your NAS. You can run them on your router. You can run them in your initramfs, on your Kubernetes nodes, on your Mosix cluster, on your smart TV, on your watch, whatever. They run on Windows; they run on macOS; they (obviously) run on Linux and BSD. New Bourne shell scripts written today, run on 40-year-old operating systems nobody even remembers.

(If you think about it, the whole way that GNU autotools does its job, is by relying on the portability of POSIX-compliant Bourne shell syntax — i.e. the ./configure script — to bootstrap out enough of an understanding of whatever system it lands on, to find the tools required to then probe the system for the availability + capabilities of its compilation toolchain.)

For a small example of the type of script I'm talking about — see e.g. the script you download+run when you run the command-line on https://rustup.rs: https://github.com/hsivonen/rustup.rs/blob/master/rustup-ini....

For a much better example (that I sadly can't link to), see the install.sh for VMWare Workstation for Linux.

There's absolutely no benefit that these scripts get from being developed directly in POSIX-compiliant Bourne shell syntax, rather than being developed in something that compiles to said syntax; any more than programs for your PC would benefit from being developed directly in ASM, rather than in something that compiles to it.

The niche for such a language would be extremely narrow compared to existing better-than-shell-script alternatives. Wasn't that perl's selling point? Compiling to bash doesn't seem like that much of a better option than shipping (or just requiring implicitly) a $BETTER_LANGUAGE runtime, and writing in that language. (I imagine someone might respond with some carefully constructed scenario where you can only use a shell script, but I'd expect that sort of scenario to be rare.)

My go-to is either Python or Ruby when shell scripts start to get unwieldy - lately it's been Ruby but I'm mentioning Python because I have experience with the type-checking options for Python, and you mentioned that as a benefit. I'm sure someone will point out that type-checking exists for Ruby too (Sorbet, others?); I've just never tried it.

I don’t think many are happy writing that much bash. I certainly wouldn’t be! I think this problem is completely solved by other languages though.

Bash is popular because it’s available (nearly) everywhere, is very portable, and has a massive “standard library.” (Technically, not a standard library, but you can basically guarantee that things like sed, curl, rsync etc. will be available or easy to get anywhere.) And its constructs are very well fitted to smaller scripts and automations.

You loose those benefits with a compilation step. Nearly all the scripts I interact with are just smaller automations put into a “bin” folder in the repo. These scripts are used for the build steps of other programs. So what would you write for automations related to building a compiled bash project? Probably bash lol

If you’re writing thousands of lines, you can just use any other language, right? Pretty much any other scripting language can execute commands in bash anyways. To my knowledge, stepping into a different language once you need more refined flow control and data structures is pretty normal. Afaik many languages can either produce output which can be run directly on the CLI (rust, C++, C), and others can be executed if you have the language runtime installed (JS via node, python, etc.)