Earlier quoted context omitted.
As someone who has to occasionally modify 100+ line bash scripts written by Coworkers from Christmas Past which matched your spec in terms of what they had to do, please please just use Python (or similar). Yes, you will have a few extra lines but it will be vastly more readable and maintainable. And yes, I know I will get the standard the person who wrote the script did a bad job but at some point it should be okay…
I think this might help others, but ShellCheck[0] is a good place to start to help eliminate poor shell scripting. And I would make an argument though that even large shell scripts in bash have their place. I often write scripts in either Node or Python, but only when I need things bash is bad about (any sort of proper data structure beyond strings or arrays). But there are just so many things bash makes insanely eas…
Things I Wish I'd Known About Bash
201–210 of 272 posts
Re: Things I Wish I'd Known About Bash
#202Earlier quoted context omitted.
> "The Perl erasure in this HN thread is startling." "Erasure" to me implies some active effort to remove Perl from discourse. I don't see anything like that here: indeed, there are a number of positive mentions, and no negative ones I see. Granted, Python and Ruby are both mentioned more often, but none of those is at Perl's expense. Am I misunderstanding what you mean by 'erasure'?
This is what Perl is designed to be. Perl unlike the others is almost certain to be on any Unix or Linux installation. Several commenters leaving out Perl in discussions of the next step up from bash scripts is truly strange. I suppose being ignored beats the typical herp-derp anti-Perl bigotry, but I’d prefer all-around civility.
> "Several commenters leaving out Perl in discussions of the next step up from bash scripts is truly odd."
I'm having a hard time following you here. Do you think that they're doing so for any other reason that Perl is no longer their go-to tool? There are communities where Perl is still used: PostgreSQL for example uses Perl for some of its scripting, as well as its build farm tool, in particular because of its portability on older systems.
That said, from what I've seen over the past 10 years or so, Perl hasn't had much of a presence in areas where a lot of computer work in tech is being done. For example, in cloud computing, or scientific computing, or machine learning, or web frameworks. Please don't read this to mean that Perl couldn't be or isn't being used in these cases or wouldn't be a better fit. (As an aside, I think Perl missed out a lot while a large portion of the community was focused on Perl 6: there's only so much energy in a community, and that absorbed on Perl 6 wasn't focusing on evangelism. But that's not something I'm interested in litigating here.) Or that there isn't something a bit frustrating in seeing the wheel reinvented time and time again. And so many examples on the web use bash as a common denominator. This puts Perl further out of mind if it's not already part of your everyday workflow. And how many developers today have come of age without seeing Perl in their everyday environments?
Consider the current forum. What's the percentage of front-page posts that are about Perl or tools where Perl is a part of the tool chain? It would be understandable for the people who frequent HN to not view Perl as their go-to. I don't consider it uncivil for people to neglect to mention some other language when it's not something they'd actually think of reaching for. It seems the solution would be to share examples of where Perl provides advantages, both in the comments here and in submissions to HN.
Re: Things I Wish I'd Known About Bash
#203The trickiest part of Bash I know is that "$(command)" results in the truncation of the output of the command before the newline. It's both handy and damning depending on what you're trying to do.
I don't think that's correct, do you have an example? This works for me:
$ var=$(echo $'a\nb')
$ echo "$var"
a
bRe: Things I Wish I'd Known About Bash
#204For me, the biggest gotcha in bash is whether or not a sub-process/shell will be invoked, which can affect things like mutable variables and the number of open file handles. For example: COUNT=0 someCommand | while read -r LINE do COUNT=$(( COUNT + 1 )) done echo "$COUNT" This will always print `0`, since the `COUNT=` line will be run in a sub-process due to the pipe, and hence it can't mutate the outer-process's `CO…
I think I've run into the first issue you describe, and I'm having a hell of a time trying to understand it. Would you mind taking a look at my example and helping me out? Consider the following: cd /tmp/ echo -e "hello world\nhello world\n:)" >> hello.txt cat hello.txt Outputs: hello world hello world :) Then running bash -c 'sed s/"hello"/"hiiii"/ hello.txt | tee hello.txt' cat hello.txt yields hiiii world hiiii wo…
The `>>` operator in use is /appending/ to the file.
Also, as mentioned in a sibling comment, your hello.txt is both an input and output.
Re: Things I Wish I'd Known About Bash
#205The trickiest part of Bash I know is that "$(command)" results in the truncation of the output of the command before the newline. It's both handy and damning depending on what you're trying to do.
I don't think that's correct, do you have an example? This works for me: $ var=$(echo $'a\nb') $ echo "$var" a b
Oh, I meant before the trailing newlines, sorry for being unclear. Try using a\nb\n\n\n instead of a\nb and observing that the output doesn't change.
Re: Things I Wish I'd Known About Bash
#206If you "set -e", you probably also want to "set pipefail". By default, a pipeline returns the return value of the last element. pipefail means that if any element of the pipeline fails, then the pipeline as a whole will fail. I discovered this the hard way when I had:
make run-asan-test | c++filt
And even if the tests failed, the script would succeed.
Re: Things I Wish I'd Known About Bash
#207The thing I wished I had learned earlier is "quick and dirty assertions". If you write lots of functions in Bash, you quickly end up getting tripped up by cases where an argument is omitted and the function does something totally batshit given the missing (empty string) argument. Now, the canonical way to handle this is to put validators on your input, (and make sure those validators don't crash with cryptic errors if someone calling your function is using "set -u") like so:
function() myfunc {
local foo="${1:-}"
if [ -z "$foo" ]; then
echo "Invalid first parameter!" >&2
return 127
fi
...
}
Instead, the quick and dirty way is to just "assert" via [parameter expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Par...):
function myfunc() {
local foo="${1:?First parameter must be provided}"
...
}
...and while we're on the subject of "set -e", I think that the ["unofficial Bash strict mode"](http://redsymbol.net/articles/unofficial-bash-strict-mode/) (putting "set -euo pipefail" and "IFS=$'\n\t'" at the top of your scripts) has been a bigger bug-prevention/rapid development aide to me than anything else. To be clear, I think it's a means of detecting some kinds of bugs. I've read Wooledge and others' objections to those patterns, especially "set -e", and agree with the point that this does not make your programs objectively safer and shouldn't be counted on as a crutch. Then again, neither does a linter, but it still helps you detect and avoid some kinds of bugs, so why not use it?
Re: Things I Wish I'd Known About Bash
#208> !$ - I use this dozens of times a day. It repeats the last argument of the last command. Press ESC then full stop instead. Less key presses.
Fewer key presses, but doesn't work if you're using vi bindings.
ESC _ or M-_ does work in vi mode though (and does the same thing as ESC ./M-.). Search for "yank-last-arg" in the bash manual.
Re: Things I Wish I'd Known About Bash
#209Using readline is a great thing to know about too. My favourite little-known readline command is operate-and-get-next: https://www.gnu.org/software/bash/manual/html_node/Miscellan... You can use it to search back in history with C-r and then execute that command with C-o and keep pressing C-o to execute the commands that followed that one in history. Very helpful for executing a whole block of history. For some reaso…
Another obscure readline feature is that ~/.inputrc accepts key sequences bound to arbitrary (quoted) macros, including macros that contain more key sequences.
# a basic macro that types "foo" bound to META+"f"
"\ef": "foo"
# -k
"\ek": shell-kill-word
# -y
"\C-y": yank
# -RIGHT
"\e[c": shell-forward-word
# -LEFT
"\e[d": shell-backward-word
# -LEFT
"\e[D": "\e[d\ek\e[d^Y"
# -RIGHT
"\e[C": "\e[d\ek\e[c^Y"
# ^ ^ ^ ^
# | | | > yank
# | | > shell-forward-word
# | > shell-kill-word
# > shell-backward-word
Re: Things I Wish I'd Known About Bash
#210For me, the biggest gotcha in bash is whether or not a sub-process/shell will be invoked, which can affect things like mutable variables and the number of open file handles. For example: COUNT=0 someCommand | while read -r LINE do COUNT=$(( COUNT + 1 )) done echo "$COUNT" This will always print `0`, since the `COUNT=` line will be run in a sub-process due to the pipe, and hence it can't mutate the outer-process's `CO…
let COUNT=COUNT+1
is all you need