Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe ways to do things in bash

github.com

Post reply on HN
Print view View original

141–150 of 255 posts

Re: Safe ways to do things in bash

#141

Post by frou_dh »

Aelius wrote: post #29

Interesting stuff, but given bash's - Relative unportability - Poor noncompliant sh interpreter - Poor performance (can be 4x slower than POSIX sh shells like dash for certain tasks) I personally think bash is always the wrong choice. Use POSIX sh (or a real language). POSIX sh is 98% the same thing, there's no good reason to even use bash over sh in the vast majority of cases. It's just this blight that won't go awa…

I agree with you, but there's one fly in the ointment. POSIX Shell does not support local variables! Writing functions without being able to have local variables? That's pretty damn gross.

So I use the `local` keyword, and now my scripts aren't strictly POSIX Shell any more.

Re: Safe ways to do things in bash

#142

Post by [deleted] »

oblio wrote: post #44
dozzie wrote: post #41

Earlier quoted context omitted.

> That's not general POSIX, right? I seem to recall it's Bash-specific? (P.S. I think you forgot quotes?) Wrong. (And no, I did not.) POSIX, or rather SUS (I've never had an access to POSIX), mandates ${foo%...} syntax and its three cousins. And assignment is not subject to word splitting for variable expansion.

Regarding POSIX: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3...

[deleted]

Re: Safe ways to do things in bash

#143

Post by coandco »

geofft wrote: post #129
foxylad wrote: post #126

Earlier quoted context omitted.

> without excessive verbosity If no-one (including you, unless you dream in bash) is ever going to modify or extend the script, OK. Otherwise python's readability/maintainability trumps excessive verbosity every time.

By "excessive verbosity" I do mean poor readability and poor maintainability. If you're invoking 5 commands and most of your Python code is wiring up the commands to each other right, just write 5 lines of shell, don't make me pull up the subprocess docs to see if your 30 lines of Python are doing the same thing and how to make a small change without risking pipes deadlocking. Python is readable and maintainable when…

I'd recommend checking out Plumbum (https://plumbum.readthedocs.io) -- at the very least, it has a solid base for easily setting up pipelines, input/output redirection, and signal handling.

Re: Safe ways to do things in bash

#144

Post by Groxx »

Overwhelmingly a great resource, though I will nitpick at this one:

>Globbing is easier to use correctly than find.

For very simple purposes or small file trees, sure. Outside that: find is incredibly more powerful, and worth using in many cases. If nothing else, learning to use `find . -path ignore_this_path -prune -o *.ext -print` can change e.g. a Go project script from visibly-slow to instant. (e.g. the latest place I used this went from 1-5+ (hot vs cold) seconds to 5-50ms)

Re: Safe ways to do things in bash

#145

Post by mar77i »

Xcelerate wrote: post #78

I always think — when a programming/scripting language requires this much bizarre knowledge just to write basic code that performs basic tasks, perhaps it is time for that language to be retired. I really don't understand why bash still exists. I've switched to fish and am much happier with the change.

As a CS history lesson I think bash is extremely well fit. The UNIX shell started off at some point, and in the meanwhile it became possible to use newlines and spaces and unicode, and to have that tool still proving its usefulness after all these years is not only incredible, I'd even go as far as say that the learning of bash makes you more aware of general weirdnesses whenever you attempt to use similar patterns in your own programming. CSV needs quotes, SQL needs quotes. Streaming data and delimiting it correctly isn't just a bash "problem domain". It's pretty much universal, really.

For a recent example of a related problem: https://bugs.chromium.org/p/chromium/issues/detail?id=533361

Re: Safe ways to do things in bash

#146

Post by JepZ »

While I agree with the guide, there is one thing I was missing while writing bash scripts with 'set -e' and that was some kind of stack tracing. So I added a nice trap function to my personal bash script template. Be aware, that this version does not include all best practices described in the guide.

  #!/usr/bin/env bash
  #--------------------------------------------
  # Default Bash Script Header
  set -eu
  trap stacktrace EXIT
  function stacktrace {
          if [ $? != 0 ]; then
                  echo -e "\nThe command '$BASH_COMMAND' triggerd a stacktrace:"
                  for i in $(seq 1 $((${#FUNCNAME[@]} - 2))); do j=$(($i+1)); echo -e "\t${BASH_SOURCE[$i]}: ${FUNCNAME[$i]}() called in ${BASH_SOURCE[$j]}:${BASH_LINENO[$i]}"; done
          fi
  }
  
  SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
  #--------------------------------------------

Re: Safe ways to do things in bash

#147

Post by emmelaich »

freedomben wrote: post #69

I've written a ridiculous amount of shell script in my day, especially when doing "devops" before we had a term like "devops" to describe it. I've fallen in almost every pit bash has. With that background, here is my opinion. 1. This article contains excellent advice and should be starred for later retrieval. 2. Having basic scripting skills will make you a way better programmer. Many times I've done huge refactors a…

I probably have a similar history but I've come to the belief that you should not use Bash if you're doing anything fancy or long (>10 lines).

Just use Perl, Python or Ruby. One or all of them is installed on every machine you are likely to use.

Re: Safe ways to do things in bash

#148

Post by jwilk »

jackewiehose wrote: post #35
jwilk wrote: post #5

> POSIX mandates /bin/sh Nope. On the contrary, it says: Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh Source: http://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.... A pedantically-compliant shell script should not have shebang at all.

> it says: Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh It also recommends a script: "Installation time script to install correct POSIX shell pathname". But I wonder how they execute this script. So this is all crap. They should remove that and instead put in something like "you have to test your script on the platform where you deploy.". The artic…

> But I wonder how they execute this script.

As I mentioned earlier, the trick is to have no shebang.

If the first line of a file of shell commands starts with the characters "#!", the results are unspecified.

[...]

If the execve() function fails due to an error equivalent to the [ENOEXEC] error [...], the shell shall execute a command equivalent to having a shell invoked with the pathname resulting from the search as its first operand, with any remaining arguments passed to the new shell, except that the value of "$0" in the new shell may be set to the command name.

Source: http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu...

Re: Safe ways to do things in bash

#149

Post by jwilk »

__blockcipher__ wrote: post #62
nodesocket wrote: post #3

Highly recommend shellcheck[1]. There is a SublimeLinter plugin[2] that automatically checks your shell scripts as you code them. It generally makes best practice suggestions including quoting. [1] https://github.com/koalaman/shellcheck [2] https://github.com/SublimeLinter/SublimeLinter-shellcheck

Seconding this. Shellcheck is fucking amazing and whenever I visit the wiki page for a specific issue it's extremely descriptive and helps you understand why the flagged behavior is a problem. Some of the tips are obscure stuff that I never would have realized, like using `command -v` instead of `which` in my (arch) linux install script because command -v is more portable. And then you have the classic stuff like pri…

I'm all for "command -v", but do you know which OSes don't have "which" by default?

Re: Safe ways to do things in bash

#150

Post by laumars »

freedomben wrote: post #121
antris wrote: post #113

Earlier quoted context omitted.

What is your opinion on using other languages to augment bash? Awk, perl etc.

I personally consider awk, sed, grep, cut, etc. to be "part" of bash. I don't think I ever write a script without invoking those venerable tools at least once. I often have many pipes, such as some_command \ | grep -E 'some.*regex$' \ | sed -e 's/erase_text//g' \ | sed -e 's/erase_more//g' \ | awk '{ print $2 }' \ | cut -d ':' -f 1 I used to reach for Perl one-liners a lot, and still do sometimes when I need a gross…

You might already be aware of this, but you can chain multiple sed functions together into one command. eg

    sed -e 's/erase_text//g; s/erase_more//g'
But I appreciate your example there is more for illustrative purposes rather than an actual pipeline you have in production.
Post reply on HN