Live data from Hacker News

Bash Pitfalls

bash.cumulonim.biz

11–20 of 55 posts

Re: Bash Pitfalls

#11
post #3

Earlier quoted context omitted.

Thanks. I don't understand why people submit links with content that can't be accessed by more than a few users simultaneously.

How would one know this in advance?

How would one know in advance that submitting to HN would result in traffic consisting of more than a handful of users?

Re: Bash Pitfalls

#12
Maybe some controversial advice: Go ahead, fall in these pits.

I write my fair share of shell scripts and I've hit practically every one of these snags in the past. However, for the majority of tasks I perform with bash, I genuinely don't care if I support spaces in filenames, or if I throw away a little efficiency with a few extra sub-shells, or if I can't test numbers vs strings or have a weird notion of booleans.

Your scripts are going to have bugs. The important question is: What happens when they fail?

Are your scripts idempotent? Are they audit-able? Interruptible? Do you have backups before performing destructive operations? How do you verify that they did the right job?

For example, if your shell scripts operate only on files under version control, you can simply run a diff before committing. Rather than spent a bunch of time tracking down a word expansion bug, you can simply rename that one file that failed to not include a space in its name.

Re: Bash Pitfalls

#13

Maybe some controversial advice: Go ahead, fall in these pits. I write my fair share of shell scripts and I've hit practically every one of these snags in the past. However, for the majority of tasks I perform with bash, I genuinely don't care if I support spaces in filenames, or if I throw away a little efficiency with a few extra sub-shells, or if I can't test numbers vs strings or have a weird notion of booleans.…

I totally agree. Sometimes the strength of making a 'quick bash script', is that you are making a quick bash script. I have made some pretty strong, well tested projects in bash before including one that was a big part of an open-source qmail project.

Sometimes though you just need to get stuff done with the least amount of fuss, without worrying about the extreme edge-cases which the majority of the webpage attached to this story talk about. Heck, it's probably the majority of bash work I'd say that ends up like that.

Re: Bash Pitfalls

#14
I find it sad and amusing that we're writing a ton of mission-critical code in this language that has an incredible number of obscure quirks. Yes, most of these pitfalls are directly connected to the semantics of Unix, but I wish someone made a concerted effort to get rid of them in an otherwise evolutionary way.

Re: Bash Pitfalls

#15
post #13

Maybe some controversial advice: Go ahead, fall in these pits. I write my fair share of shell scripts and I've hit practically every one of these snags in the past. However, for the majority of tasks I perform with bash, I genuinely don't care if I support spaces in filenames, or if I throw away a little efficiency with a few extra sub-shells, or if I can't test numbers vs strings or have a weird notion of booleans.…

I totally agree. Sometimes the strength of making a 'quick bash script', is that you are making a quick bash script. I have made some pretty strong, well tested projects in bash before including one that was a big part of an open-source qmail project. Sometimes though you just need to get stuff done with the least amount of fuss, without worrying about the extreme edge-cases which the majority of the webpage attached…

On the other hand, if you examine the given examples, you'll see that very often the "correct" way isn't really longer or harder to type. If you make a point of sticking to the correct way, it will eventually become automatic, and there'll be no fuss and worry. This might end up saving some sorry ass later on.

And having learned the correct way, you'll instantly see it when a script you review is doing something in a way that will eventually bite someone.

There's no downsides to learning and doing things right. Of course it does take some extra time and effort at the start, as everything..

Sure, there are extreme examples that can be really hard to handle portably and safely if you're doing something more complicated (embedded newlines in filenames come to mind). So in the end some corner cutting is often inevitable :-)

Re: Bash Pitfalls

#16
The following does not work on files with spaces according to the article:

  for i in $(ls *.mp3); do
    some command $i
  done
So does that mean, that "for" will do something per word of the output of $, rather than per line of output of it?

What to do if I want to do something for every line? What for example if I really want the output of ls, find (or any other command you can put int he $()) and loop through that line per line, even if some output has spaces?

Thanks.

Re: Bash Pitfalls

#18
post #7

The Unix shell may be a highly powerful interactive programming environment, but it's sure hard to think of anything that comes anywhere close to sucking as badly. With the shell and the standard Unix commands, some things that are hard in other languages are easy, and most of the things that are easy in other languages are hard to impossible... I'd love to see a clean slate replacement for the shell that still feels…

> I'd love to see a clean slate replacement for the shell that still feels Unix-like and retains most of its existing benefits.

Have you looked at scsh, the Scheme Configurable Shell? It's a nice clean language with a REPL (well, it's just scheme) that has syntax for all the usual things you'd want in a unix shell -- pipelines, redirections, environment, signals, etc.

http://www.scsh.net/docu/html/man-Z-H-1.html

Re: Bash Pitfalls

#19

The following does not work on files with spaces according to the article: for i in $(ls *.mp3); do some command $i done So does that mean, that "for" will do something per word of the output of $, rather than per line of output of it? What to do if I want to do something for every line? What for example if I really want the output of ls, find (or any other command you can put int he $()) and loop through that line p…

There are a couple of ways you can do it. Don't do this with `ls` though; there are better ways to get that info in scripts.

    # Loops over lines, in a subshell
    prints_lines | while read -r line; do
      some_command "$line"
    done

    # Loops over lines, in the current shell
    while read -r line; do
      some_command "$line"
    done 

Re: Bash Pitfalls

#20

The following does not work on files with spaces according to the article: for i in $(ls *.mp3); do some command $i done So does that mean, that "for" will do something per word of the output of $, rather than per line of output of it? What to do if I want to do something for every line? What for example if I really want the output of ls, find (or any other command you can put int he $()) and loop through that line p…

> So does that mean, that "for" will do something per word of the output of $, rather than per line of output of it?

Correct. The argument to "for" is a list of words.

> What to do if I want to do something for every line?

Use a while loop.

    find /some/dir/ -type f |
    while read -r line; do
       ; # something with $line
    done
PS. You should almost always use `find` instead of `ls` in shell scripts. Given a pattern, `ls` will exit non-zero if nothing matches it, and you should be treating non-zero exits like you would exceptions in other languages.
Post reply on HN