Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pure Bash Bible (2018)

github.com

Post reply on HN
Print view View original

81–90 of 106 posts

Re: Pure Bash Bible (2018)

#81

Post by ftaghn »

dredmorbius wrote: post #31
lnxg33k1 wrote: post #9

I bought it just few days ago find it helpful but also i wish there was more context / explanation around some black magic syntax

Definitely this. If only to note, say, the section(s) of the Bash manpage or GNU Info Manual in which topics are discussed. Taking the first-listed example, the '%%', '##', etc., substitutions are covered in Parameter Expansion . I tend to go first to manpages, though I'm well aware that GNU deprecates these (exceedingly unwisely IMO) in favour of the Info Doc format (and its dedicated single-purpose reader). I'll no…

People all too often forget that there is often a lot of extra documentation hidden in /usr/share/doc/ when you install -doc packages, at least on deb systems, and not just info and manpages. For example, the bash manual you cite is in /usr/share/doc/bash/bashref.pdf along with its html version as bashref.html. There is also an /example/ folder filled with commented programs and functions to serve as reference. And a README that suggests looking for "Advanced Bash-Scripting Guide" which is a massive online book filled with all sorts of esoteric knowledge. You can also find things like the release changelogs and one that details incompatibilities from previous versions.

While there are exceptions, most packages have useful goodies in there to check first before one has to think about looking things up online.

Re: Pure Bash Bible (2018)

#82

Post by 0xbadcafebee »

kjuulh wrote: post #37

I like bash, zsh and other shell languages as well. However, unless you're really experienced, they shouldn't be the product you ship, whether that is internally or externally to you or your company. Shell languages is the final layer of configuration and customization you apply to the setup, it should almost never be the foundation of what you build. Especially for a company, unless you've got really, really skilled…

Most commercial products that include an installer on *NIX are written in shell. No other method allows you to have a universal installer on every platform that will always work.

> Especially for a company, unless you've got really, really skilled engineers on the scripts, it becomes a complete mess. And even then it can become almost impossible to grok.

Dude. It's really not that hard a language. Is amateur shell script ugly? verbose? redundant? bug-riddled? Absolutely. But it works despite that, and has fewer dependencies than other solutions, so in practice, it's fine.

That said, use whatever you know. Wanna use Python? Use it. Wanna use a compiled language? Do it. Just make something. The biggest difficulties you will have in producing a product will not be the language.

You don't have to avoid shell just because people on HN (who've probably never read the man page) keep repeating that it's terrible. With Shellcheck, almost anyone can write good scripts now. If you can write a concise, bug-free script that works, then do it.

Re: Pure Bash Bible (2018)

#83

Post by rollcat »

cutler wrote: post #52
rollcat wrote: post #34

Earlier quoted context omitted.

I would argue reaching for the shell to do your scripting is never a good idea, unless your problem is trivial to begin with (e.g. fits in less than a hundred lines of extremely readable and straightforward code). My rule of thumb is: "can I fit it in a one-liner?" - then I refactor it until it's readable, or otherwise use something else. Once you need to do fancy array manipulation, hash tables, data structures, etc…

For 1-liners there are only 2 real non-shell contenders - Perl and Ruby. Yes you can do them with Python but it ain't pretty.

One-liners are meant to be thrown away, so it does not matter if it's pretty or not. If you care enough to keep it, you should consider rewriting it for clarity. That last step does not require using any particular language; e.g. my PS1 started off as an increasingly convoluted one-liner, until I rewrote it in Go.

Re: Pure Bash Bible (2018)

#84

Post by ftaghn »

balou23 wrote: post #78
ftaghn wrote: post #28

Earlier quoted context omitted.

> and slower. I like to write posix sh scripts for the sake of portability and, funnily enough, future proofing as I don't like having to maintain stuff against changes that break compatibility, which is something bash does (archlinux is still on an older bash even though debian stable has the latest, because bash 5.2 broke some of archlinux's own scripts. This is why you should not write bash scripts.), but bash bei…

I had the misfortune of having to maintain an 18'000 line bash 3 script. Array handling broke in many subtle ways between bash 3 and 4. > If your scripts do that much work that your shell's speed matters, you should reconsider writing shell scripts and start thinking about using something like perl, python or ruby. Usually yes. But there is a small subset of use cases which need a really portable solution. Once I wou…

> I had the misfortune of having to maintain an 18'000 line bash 3 script

My hats off to you, I do not think I have the fortitude to stomach.. this.

> run it with busybox with the "exec prefers applets" option

> edit: it's actually the "run 'nofork' applets directly" option. Can give quite a speed boost compared to bash if you have to call "external" utilities like grep/head/etc.

Nifty trick, I had no idea about that, thanks for sharing this.

Re: Pure Bash Bible (2018)

#85

Post by inferiorhuman »

0xbadcafebee wrote: post #82
kjuulh wrote: post #37

I like bash, zsh and other shell languages as well. However, unless you're really experienced, they shouldn't be the product you ship, whether that is internally or externally to you or your company. Shell languages is the final layer of configuration and customization you apply to the setup, it should almost never be the foundation of what you build. Especially for a company, unless you've got really, really skilled…

Most commercial products that include an installer on *NIX are written in shell. No other method allows you to have a universal installer on every platform that will always work. > Especially for a company, unless you've got really, really skilled engineers on the scripts, it becomes a complete mess. And even then it can become almost impossible to grok. Dude. It's really not that hard a language. Is amateur shell sc…

  No other method allows you to have a universal installer on every
  platform that will always work.
No shell will guarantee that either. Last week I spent a few hours debugging an installer (bash script) that was failing with a rather puzzling error message (variable not defined). Non-Linux, bash 5.x. Worked on nearly everything under the sun including OSX with its ancient GPLv2 bash.

Turns out on one non-Linux operating systems, one of the utilities used for string manipulation, while POSIX compliant, behaved differently than the same utility in nearly every other operating system.

   has fewer dependencies than other solutions
Wrong, wrong, wrong. Nearly everything you do in a shell script is an external dependency. And that's the biggest problem with shell scripts. The actual language is so narrowly scoped that you're dependent upon external utilities that are not so easy to upgrade.

Count how many programs are in /bin. Those are your dependencies. Unlike e.g. python or ruby dependencies, you'll have a much harder time upgrading /bin. Having to fork a new process for trivial tasks is also one of the reasons why shell scripts are so slow.

Sure, bash has builtin versions of some of those utilities like [, but not every system has bash.

Re: Pure Bash Bible (2018)

#86

Post by inferiorhuman »

jjice wrote: post #56

I always hear people talk about writing portable, POSIX compliant shell scripts so they'll work in more environments (assuming those environments are needed, of course), but how often do you run into an environment where you don't have a fairly complete version of Bash available? Genuine open question, I'd be really interested to know what that situation is. Lean container images, embedded, older systems? I'm really…

The BSDs all come with some non-bash POSIX shell in /bin/sh, and while bash is available via ports it won't be installed at /bin/bash.

The solution is, do as little with shell scripts as possible to bootstrap a more workable environment.

Re: Pure Bash Bible (2018)

#87

Post by JNRowe »

nerdponx wrote: post #47
kjuulh wrote: post #37

I like bash, zsh and other shell languages as well. However, unless you're really experienced, they shouldn't be the product you ship, whether that is internally or externally to you or your company. Shell languages is the final layer of configuration and customization you apply to the setup, it should almost never be the foundation of what you build. Especially for a company, unless you've got really, really skilled…

At least you've discovered Zsh and hopefully its significant improvements over Bash. The only Bash benefit is readarray/mapfile, Z Shell is better in just about every other way.

Unless I'm misreading your comment, zsh does provide similar functionality via zsh/mapfile¹. It, and a lot of other cool stuff, is documented in zshmodules(1).

¹ https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#The-...

Re: Pure Bash Bible (2018)

#88

Post by JNRowe »

softwaredoug wrote: post #24
bhaak wrote: post #14

Depending on your job description it is completely fine to not write shell scripts. For anything bigger than a screenful of lines or something that would need proper unit testing or something that other people who are likely to be less versed in shell scripting need to maintain, think twice about using shell scripts. Unixoid machines can use anything as scripts and using a language that breathes the programming langu…

I typically agree, but I don't think its as simple as a "screenful of lines" criteria. I find it useful when most of your work is wiring other command line tools together. When that's most of your work, shell scripting feels like it has the best affordances. Anything that gets too fancy beyond this, with anything that resembles an algorithm or computation, you'll feel real pain :) Bash doesn't even support floating p…

There was a really awful and amazing floating point implementation posted here¹ a couple of years ago, with a few interesting comments too². I remember it in part because I was surprised there wasn't more discussion.

¹ https://github.com/clarity20/shellmath

² https://news.ycombinator.com/item?id=26250743

Re: Pure Bash Bible (2018)

#89

Post by dredmorbius »

ftaghn wrote: post #81
dredmorbius wrote: post #31

Earlier quoted context omitted.

Definitely this. If only to note, say, the section(s) of the Bash manpage or GNU Info Manual in which topics are discussed. Taking the first-listed example, the '%%', '##', etc., substitutions are covered in Parameter Expansion . I tend to go first to manpages, though I'm well aware that GNU deprecates these (exceedingly unwisely IMO) in favour of the Info Doc format (and its dedicated single-purpose reader). I'll no…

People all too often forget that there is often a lot of extra documentation hidden in /usr/share/doc/ when you install -doc packages, at least on deb systems, and not just info and manpages. For example, the bash manual you cite is in /usr/share/doc/bash/bashref.pdf along with its html version as bashref.html. There is also an /example/ folder filled with commented programs and functions to serve as reference. And a…

On Debian, install the dwww package (https://packages.debian.org/bookworm/dwww>), and point your browser at https://localhost/dwww/>.

What you'll find is both a set of pages & hierarchy of documentation as well as search (via swish++) through that space. And yes, there is an absolute trove of information available:

- Manpages

- Info pages

- Default package information

- Supplementary package documents, if installed

- HOWTOs from the Linux Documentation Project.

- Debian-specific documentation (installation, administration, security, policy, and other manuals)

- RFCs (divided into several groups, for manageability / relevance)

- Resources such as the Linux Gazette

There's also a commandline utility, dwww, which will invoke your browser of choice. Point that at a terminal-based browser (w3m, lynx, links, elinks, etc.) and you can quickly scan through docs from your terminal.

Re: Pure Bash Bible (2018)

#90

Post by da39a3ee »

javier_e06 wrote: post #30

After years writing bash scripts I stumbled with this line... set -e Which forces the script to exit on first error. This line be in the bible as Genesis 1:1

No. I agree, when I first encountered -e I thought it sounded like a good idea. But it's not too long before the problems rear their heads.

  #!/bin/bash
  set -e
  grep nomatchforthis
  echo "Won't get here!"
Post reply on HN