Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to do things safely in Bash (2018)

github.com

Post reply on HN
Print view View original

51–60 of 98 posts

Re: How to do things safely in Bash (2018)

#51

Post by marcosdumay »

rectang wrote: post #47
marcosdumay wrote: post #30

Earlier quoted context omitted.

Hum, no those people are probably writing the same kind of script you do. Their Python versions are indeed longer and more verbose. I always recommend Python anyway if you care about edge cases (what is not always). That longer script handles them on the obvious way, while the short and more readable shell script does something absolutely crazy every time a detail is different from planed. And if you try to correctly…

Which of these is simpler? Rust: fn add(a: i32, b: i32) -> i32 { a + b } Python: def add(a, b): return a + b Arguably the Python function is "simpler" because it occupies fewer characters. And yet, the Rust function is more tightly specified; it does less. It can't throw exceptions, unlike the Python function. Invalid programs where you pass the wrong type to the Rust function won't even compile, whereas the Python f…

Oh, I'm really not comparing it with Rust. The comparison is between a general language and a shell. If it's Python, Rust, PHP, Java, or whatever is of secondary relevance.

I've done some ops work in C, Python, and C#. I'm also waiting for a good problem to try some Haskell shell monad. All the same old language pros and cons apply.

Re: How to do things safely in Bash (2018)

#52

Post by stormbrew »

rectang wrote: post #47
marcosdumay wrote: post #30

Earlier quoted context omitted.

Hum, no those people are probably writing the same kind of script you do. Their Python versions are indeed longer and more verbose. I always recommend Python anyway if you care about edge cases (what is not always). That longer script handles them on the obvious way, while the short and more readable shell script does something absolutely crazy every time a detail is different from planed. And if you try to correctly…

Which of these is simpler? Rust: fn add(a: i32, b: i32) -> i32 { a + b } Python: def add(a, b): return a + b Arguably the Python function is "simpler" because it occupies fewer characters. And yet, the Rust function is more tightly specified; it does less. It can't throw exceptions, unlike the Python function. Invalid programs where you pass the wrong type to the Rust function won't even compile, whereas the Python f…

Perhaps sometimes. But properly handling errors around process management in python is not simple either, even with libraries like subprocess. My experience is that when python scripts doing a lot of process management fail they fail in far more complex and hard to diagnose ways than shell scripts do (especially bash scripts with `set -e -o pipefail`).

And like, `subprocess.run()` defaults its check argument (that will make it throw an exception on non-zero exit code) to false, which means that to get that same kind of behaviour as `set -e` in python means making sure you always pass check=False to every invocation, afaik.

Sometimes added complexity can introduce classes of errors.

Re: How to do things safely in Bash (2018)

#53

Post by stormbrew »

dragonwriter wrote: post #32
stormbrew wrote: post #14

Earlier quoted context omitted.

This, and I mean very specifically this with python always as the go-to replacement, comes up every single time anyone even utters the words "shell" or "bash" and every time all I can think is that the people saying it aren't writing the same kinds of shell scripts I ever see, use, or wind up writing. And every time I've ever seen something that's allegedly "a shell script replaced with a python script" it's way more…

> I just wonder, really, what "a shell script" even is to you if not something where you need to "shell out to external processes"? A replacement for a shell script may not itself be a shell script. A fairly typical python-as-shell-replacement I have that (among other things) creates/manipulates test resources in AWS corresponding to git feature branches, shells out less than it would if it was a typical shell langua…

I mean I'd agree that's a good candidate for replacement; once you start doing anything more complicated than maybe idempotent curls to an external service I think it ceases to be a 'shell script'.

This may sound a little like "no true scotsman" I guess, but "what is the scotsman" is kind of the key issue that I'm saying gets glossed over when people say blanket things like "never write shell, always use python."

Re: How to do things safely in Bash (2018)

#54

Post by stormbrew »

kubanczyk wrote: post #40
blabitty wrote: post #18

Earlier quoted context omitted.

I wonder too. Things like file and directory manipulations are way more complicated in native Python and I find the exceptions harder to troubleshoot than native commands. I'm not even going to get into pythons version and dependency hell, suffice it to say I have more confidence in a standard version of bash and standard utilities generally being present on a modern Linux system. What I've seen from pythonistas as r…

Here is a novel idea. I am allowed to call 'mv' or 'cp' binaries from bash. I am also allowed to call 'mv' or 'cp' binaries from python with subprocess! (Edit: bad example binaries, 'df' or 'tar' would be probably better.) I must choose not to install bash libraries for a bash script beyond builtins. I can choose not to install python libraries beyond builtins. It becomes a really simple tradeoff. Nobody forces you t…

It's worth noting that there are still a shocking number of production machines in the world that don't have stable installations of python3.x yet.

Re: How to do things safely in Bash (2018)

#55

Post by tediousdemise »

PowerShell is cross platform now and it’s quite good. I would encourage people who’ve never tried it to give it a shot.

Like 99% of other people who never used PoSh, I thought it was just a Windows shovelware replacement for Command Prompt. I was mistaken—the syntax is actually a lot simpler than bash, but it’s just as capable.

One of the cooler features about PoSh is that you can leverage Visual Basic/C#/.NET from it, and you can script GUIs (like Python’s tkinter library).

Re: How to do things safely in Bash (2018)

#56

Post by greggyb »

nunez wrote: post #28

If you're gonna Bash, definitely use shellcheck for everything. If you're writing Bash in vim, I'd highly recommend installing Syntastic and enabling the checker below: ``` let g:syntastic_sh_checkers = ["shellcheck", "-e", "SC1090"] ```

Note that HN doesn't support markdown, but does have its own limited markup.

Specifically, backticks are simply displayed, rather than marking a code snippet or block.

You can indent by four spaces for a code block as below:

    This is a preformatted section preceded by
    four spaces in the comment editor.

Re: How to do things safely in Bash (2018)

#57

Post by rectang »

marcosdumay wrote: post #51
rectang wrote: post #47

Earlier quoted context omitted.

Which of these is simpler? Rust: fn add(a: i32, b: i32) -> i32 { a + b } Python: def add(a, b): return a + b Arguably the Python function is "simpler" because it occupies fewer characters. And yet, the Rust function is more tightly specified; it does less. It can't throw exceptions, unlike the Python function. Invalid programs where you pass the wrong type to the Rust function won't even compile, whereas the Python f…

Oh, I'm really not comparing it with Rust. The comparison is between a general language and a shell. If it's Python, Rust, PHP, Java, or whatever is of secondary relevance. I've done some ops work in C, Python, and C#. I'm also waiting for a good problem to try some Haskell shell monad. All the same old language pros and cons apply.

I'm just making a general argument about what "simple" means. A well-specified program may or may not be superficially simpler, but it will exhibit simpler behavior.

Re: How to do things safely in Bash (2018)

#58

Post by kaliszad »

yeowMeng wrote: post #7
pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

> The [python] scripts are much faster... Pipelines are performant; many algorithms can be expressed as pipelines. Bash has many built-in string manipulation mechanisms which do not require a sub shell. Why do people who subscribe to the “avoid shell scripts at all costs” ideology feel so passionately about prescribing to others? I like python, but if I am in a rush it’s shell.

Actually, the one thing that I as a systems engineer really like about Clojure is the threading macro (-> some-data some-function-applied-on-some-data some-otherfunction) or (->> ...) for inserting not as a second, but last argument. That is like a pipeline but usually more readable to me even though I am proficient with the shell (and I must admit even PowerShell, which really is more comparable to Perl with a bit of SQL and other flavours here and there than bash).

Sysadmins/ system engineers might love Babashka: https://github.com/babashka/babashka which is a large subset of Clojure + some frequently used libraries as a native GraalVM image. It is portable, has very fast startup and if the script becomes a larger program, you can easily switch to ClojureScript + Node.js (e.g. for still very fast startup) or Clojure (on the normal JVM) or perhaps build your own GraalVM image. You might also just open a REPL and run it as a single session but that is rather unique in the sysadmin/ systems engineer space, where most things are launched on schedule e.g. each 5 minutes by a script and in case the startup time is somewhat long, it might dominate the execution time.

Btw. babashka seems to be about twice as fast to start on my Debian: time bb -e '(+ 1 1)' executes on average in about 11 ms vs time python3 -c 'print(1 + 1)' executes on average in about 23 ms

Re: How to do things safely in Bash (2018)

#59

Post by dragonwriter »

stormbrew wrote: post #53
dragonwriter wrote: post #32

Earlier quoted context omitted.

> I just wonder, really, what "a shell script" even is to you if not something where you need to "shell out to external processes"? A replacement for a shell script may not itself be a shell script. A fairly typical python-as-shell-replacement I have that (among other things) creates/manipulates test resources in AWS corresponding to git feature branches, shells out less than it would if it was a typical shell langua…

I mean I'd agree that's a good candidate for replacement; once you start doing anything more complicated than maybe idempotent curls to an external service I think it ceases to be a 'shell script'. This may sound a little like "no true scotsman" I guess, but "what is the scotsman" is kind of the key issue that I'm saying gets glossed over when people say blanket things like "never write shell, always use python."

I’m not sure I’d agree with defining the scope of things for which an actual shell language is ideal as inherently “shell scripts” (for a few reasons, including the evolving variety of shell languages, the fuzzy boundaries of that set, etc.), I think that it is probably the case that the choice between bash (or powershell or any other shell language) and Python isn’s as simple as “always use Python” and that there is plenty of space for discussion around which tasks each is appropriate for, with the answer differing based on which shell languages you are looking at.

Re: How to do things safely in Bash (2018)

#60

Post by lukeschlather »

stormbrew wrote: post #14
pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

This, and I mean very specifically this with python always as the go-to replacement, comes up every single time anyone even utters the words "shell" or "bash" and every time all I can think is that the people saying it aren't writing the same kinds of shell scripts I ever see, use, or wind up writing. And every time I've ever seen something that's allegedly "a shell script replaced with a python script" it's way more…

I agree, but I use Powershell rather than Python. (I've actually written Powershell scripts that implement pieces in Python.) Because gluing together bits of Powershell and Python is still better than trying to write it in bash.
Post reply on HN