Live data from Hacker News

Safe ways to do things in bash

github.com

111–120 of 255 posts

Re: Safe ways to do things in bash

#111

I've written a ridiculous amount of shell script in my day, especially when doing "devops" before we had a term like "devops" to describe it. I've fallen in almost every pit bash has. With that background, here is my opinion. 1. This article contains excellent advice and should be starred for later retrieval. 2. Having basic scripting skills will make you a way better programmer. Many times I've done huge refactors a…

> 6. You will not regret getting really good at shell script. You'll have to take my word for it now because you don't know what you're missing.

Stories! C'mon, it was a long Monday :-)

Re: Safe ways to do things in bash

#112
post #31

Earlier quoted context omitted.

I've worked on systems where env was installed as /bin/env and not as /usr/bin/env . (I think it was SunOS 4.) For that matter, under Termux on Android it's /data/data/com.termux/files/usr/bin/env (but termux has a hack to make normal shebangs work).

Termux rewrites scripts via termux-fix-shebang.

It now also does rewriting on the fly for certain shebang lines, as part of its C library, so unmodified scripts can run.

Re: Safe ways to do things in bash

#113

I've written a ridiculous amount of shell script in my day, especially when doing "devops" before we had a term like "devops" to describe it. I've fallen in almost every pit bash has. With that background, here is my opinion. 1. This article contains excellent advice and should be starred for later retrieval. 2. Having basic scripting skills will make you a way better programmer. Many times I've done huge refactors a…

What is your opinion on using other languages to augment bash? Awk, perl etc.

Re: Safe ways to do things in bash

#114
post #96

The advice to double-quote everything is an interesting way to circumvent "detailed knowledge may be required". I wish it noted that you can't quote regular expressions, though: $ cat - > foo #!/bin/bash a="BCD" [[ $a =~ .C. ]] && echo 1 [[ $a =~ ".C." ]] && echo 2 ^D $ bash foo 1 $

this is intentional, since you can store your regular expression in a variable: $ a=BCD; b=.C.; [[ $a =~ $b ]]; echo $?; [[ $a =~ "$b" ]]; echo $? otherwise, interpolating variables in regular expressions as text would require other syntax (more confusing). also, "cat -" is redundant, use "cat". this behavior is specified by POSIX.

I use “cat -“ so that my code makes more sense to people. I want STDIN declared somehow, and the dash is effective. Technically I shouldn’t have bothered with the cat at all in an HN code snippet, but it was a courtesy to provide a familiar environment for the block I wanted to convey. It worked so well that you linted it! I really appreciate the thought.

Re: Safe ways to do things in bash

#115

Earlier quoted context omitted.

My personal experience would disagree, and I feel that is throwing caution to the wind. As any software person knows, you really can't tell what's going to happen to the code/scripts you put out, not committing it to source control is a dangerous game to play. If you have a simple shell script sitting on a server doing some basic task, why would you not have it under source control where it can be viewed by future te…

At that point, you care enough to put it under source control, and it shouldn't be a shell script. That's the entire point of the comment you replied to

It's that arbitrary distinction that I am responding to. I don't agree that you should reserve source control for the complex.

A shell script can be the appropriate tool for many important tasks.

Re: Safe ways to do things in bash

#116

Recently I've replaced most of my bash scripting with the python library invoke. It's written by the same people who wrote the python ssh scripting framework fabric. http://www.pyinvoke.org/ Works great!!

I'll have to try that one. I've done a few things with: Python process launching http://amoffat.github.com/sh

Be careful with sh library! It runs the programs under tty by default, so you get random effects like ascii color sequences and truncated git output. There is an option to change this, but it not default, and it is easy to forget it.

Re: Safe ways to do things in bash

#117
post #95

Earlier quoted context omitted.

Re 6: I am an /extremely/ mediocre developer but I have crafted some real 99th percentile bash skills (within my company, not globally) and I completely self-sustain myself on just that. (You're wondering how many people write bash where I am, and the answer is somewhere between 8 and 15 thousand people). It's funny how many things that seem kind of incredible for a pure bash solution, looking back over the past 21 y…

> It's funny how many things that seem kind of incredible for a pure bash solution, ... My go to example of “Bash can do whaaat?!” is the source for xip.io. A custom DNS server written in a handful of lines of Bash! https://github.com/basecamp/xip-pdns/blob/master/bin/xip-pdn...

I have made a lot of DNS products and doing the same thing happening here is a few lines of code in most any language. This code, while entertaining, cannot handle very simple DNS packets because of its "compression encoding.

Re: Safe ways to do things in bash

#118
post #8

From the article: > Should I use curly braces? Bad: some_command $arg1 $arg2 $arg3 Extra bad (cargo culting unnecessary braces): some_command ${arg1} ${arg2} ${arg3} Correct: some_command "${arg1}" "${arg2}" "${arg3}" Better: some_command "$arg1" "$arg2" "$arg3" > In the "extra bad" and "correct" examples, braces compete with quotes under the limits of tolerable verbosity. > Shellharden will rewrite all these variant…

> You are source controlling your shell scripts right? My personal heuristic for shell scripts are that if I care enough about them to put them under source control, they shouldn't be a shell script.

I have seen entirely too many Python scripts that are poor reimplementations of shell scripts that don't get the edge cases right, have more security vulnerabilities, etc. than a shell script a fifth of the size. There are things you simply can't do in other languages without excessive verbosity. (One that came up at my workplace recently is using Use the right tool for the job, and then put it in version control (and add tests, too, by the way). Shell is the right language for tasks that primarily involve running lots of subprocesses, whether they are simple or complex. If a small part of your task needs functionality that can't be done well in shell, fortunately, shell is very good at running subprocesses, and it's a perfectly reasonable approach to do something like this:

    foo () {
        python3 -c 'import sys, foo; print(foo.bar(sys.argv[1:]))' "$@"
    }

    a="$(foo "$baz" "$quux")"
I regularly do this with the requests and json modules in particular, because being an HTTP client or a JSON parser is not a thing shell is good at. (For the specific problem of manipulating JSON, jq is another fine option if you have it installed.)

Re: Safe ways to do things in bash

#119

Earlier quoted context omitted.

At that point, you care enough to put it under source control, and it shouldn't be a shell script. That's the entire point of the comment you replied to

It's that arbitrary distinction that I am responding to. I don't agree that you should reserve source control for the complex. A shell script can be the appropriate tool for many important tasks.

I agree that simpler scripts deserve to be in source control too. I read it to mean what I also would say, myself: if it's anything but the most trivial of scripts (so, rather _programs_), they shouldn't be in BASH. Some people in this discussion are clearly very well versed in BASH. Great. For average developers, though, it's hard to build (good) programs in BASH, and the rabbit hole swallows them. Every time.

Re: Safe ways to do things in bash

#120
post #9

Highly recommend shellcheck[1]. There is a SublimeLinter plugin[2] that automatically checks your shell scripts as you code them. It generally makes best practice suggestions including quoting. [1] https://github.com/koalaman/shellcheck [2] https://github.com/SublimeLinter/SublimeLinter-shellcheck

Not only does it make suggestions, almost all of the 'error codes' have extensive documentation on why something is wrong and often multiple alternative solutions for each use case (eg: https://github.com/koalaman/shellcheck/wiki/SC2086 ). I learned more bash from Shellcheck than all tutorials and references combined.

Same. I was a bit cocky when I first tried out shell check because I had been doing bash for years. Shell check flagged something I'd been using for a while and after reading the docs I realized shell check's suggestio was much cleaner and just-as-safe way to do what I was doing. Really impressive piece of software.
Post reply on HN