Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pure Bash Bible

github.com

Post reply on HN
Print view View original

51–60 of 258 posts

Re: Pure Bash Bible

#51

Post by masklinn »

harry8 wrote: post #36
masklinn wrote: post #28

Earlier quoted context omitted.

I'm not sure what that's supposed to do, but subprocess provides these facilities, although definitely in a more verbose way. f = Popen('ls', stdout=PIPE).stdout f.read() f.close() alternatively given this exact behaviour: run('ls', stdout=PIPE).stdout

yeah sure subprocess.check_output(). If you ever dealt with much perl you know how much more pleasant and easy launching processes was in that language - like shell. Python is great, I'm not on the "bash python" wagon. Launching processes, getting the output, munging it and shoving it at another process is more fiddly and less natural in python. If you want to use that as evidence that I'm a garbage developer go ahea…

> yeah sure subprocess.check_output()

It's deprecated and does something quite different.

> If you ever dealt with much perl you know how much more pleasant and easy launching processes was in that language - like shell.

I'm sure it is, but you're missing my point.

> If you want to use that as evidence that I'm a garbage developer go ahead

Well that escalated quickly.

> open(SRC, "output_generator|");

> open(WC, "|wc -l);

>

> And you do whatever you want with those filehandles.

Again python does roughly the same, just with more overhead: a trailing pipe is an "stdout=PIPE", an input is a "input=", and you access / forward stdout explicitly:

    src = Popen('output_generator', stdout=PIPE)
    wc = Popen(['wc', '-l'], input=src.stdout)
And as the sibling notes, for shell replacements you can use the sh library to lower the syntactic overhead of popen.

Re: Pure Bash Bible

#52

Post by dyanaraps »

acqq wrote: post #41
dyanaraps wrote: post #18

Earlier quoted context omitted.

I understand your concerns about bash features and POSIX compatibility. The bash bible was written specifically to document the shell extensions bash implements. My focus for the past few months has been writing a Linux distribution (and its package manager/tooling) in POSIX sh. I've learned a lot of tricks and I'm very tempted to write a second "bible" with snippets that are supported in all POSIX shells. (I created…

> I've learned a lot of tricks and I'm very tempted to write a second "bible" with snippets that are supported in all POSIX shells. That could be potentially even more interesting than a bash-specific one (as it is harder to get it right -- bash can be figured out out of the single reference, anything "portable" has many dependencies).

Off the top of my head, a few notable things I've learned:

- Safely working with "string lists" (list="el el el el").

    - Filtering out duplicate items.

    - Reversing the list.

    - etc.
- Using `case` to do sub-string matching (using globbing).

- Using `set -- el el el` to create an "array" (only one array at a time!).

- `read -r` is still powerful in POSIX `sh` for getting data out of files. `while read -r` even more so.

- POSIX `sh` has `set -e` and friends so you can exit on errors etc.

- Ternary operators still exist for arithmetic (`$(($# > 0 ? 1 : 0))`).

- Each POSIX `sh` shell has a set of quirks you need to account for. What works in one POSIX `sh` shell may not in another. (I found a set of differences between `ash`/`dash` in my testing).

POSIX `sh` is a very simple language compared to `bash` and all of its extensions so there won't be as many snippets but there's some gold to be found.

Re: Pure Bash Bible

#53

Post by jgtrosh »

dyanaraps wrote: post #23
jigglesniggle wrote: post #4

While this is interesting I see doing anything but launching programs with simple text-substituted arguments as too much for bash or sh. Run shellcheck on some of your own code, or the code of even a simple project to see how hard it is to really use bash. Why I think people gravitate towards it is because languages such as python add too much pomp to launching a shell process. A language like perl is usually easier…

The shell is my favorite language and I really don't know why. It's definitely possible to write shell code which properly passes shellcheck's linter though it's an uphill battle to learn the ins and outs and _why_ X is wrong when Y is right. I even managed to write a full TUI file manager in bash! https://github.com/dylanaraps/fff I full understand that there are times when the shell should not be used and when othe…

Have you read Bash Pitfalls[1]? Do you write truly correct Bash/POSIX code? Do you still love it?

[1]: https://mywiki.wooledge.org/BashPitfalls

Re: Pure Bash Bible

#54

Post by lostmsu »

jigglesniggle wrote: post #21
danielecook wrote: post #16

Earlier quoted context omitted.

I don’t think it’s pomp. Once I learned Unix pipes and the tools for manipulating data (sed awk cut etc), it just became much faster and easier than writing python scripts that do the same thing. You can literally connect the output of one process to the input of another with a single character. It’s much more complex in python. It also tends to be very portable.

I think you're reading my comment the wrong way: I meant to say that doing e.g. piping in Python is a lot of pointless work (pomp), as you agree. This is perhaps one big benefit but not one that is exclusive to a sh-like language. Instead I would like to see a language with strong flow control or metaprogramming capabilities take on processes as a first class citizen. Perl is probably the closest but still has some w…

So PowerShell then?

Re: Pure Bash Bible

#55

Post by tannhaeuser »

dyanaraps wrote: post #18
tannhaeuser wrote: post #9

Didn't realize at first that "pure" refers to features available in bash without calling out to external processes, when I would've thought purism in this context should refer to avoiding bashisms and writing portable (ksh, POSIX shell) scripts.

I understand your concerns about bash features and POSIX compatibility. The bash bible was written specifically to document the shell extensions bash implements. My focus for the past few months has been writing a Linux distribution (and its package manager/tooling) in POSIX sh. I've learned a lot of tricks and I'm very tempted to write a second "bible" with snippets that are supported in all POSIX shells. (I created…

You can add me to those interested in a pure POSIX shell bible. Whenever I'm about to write a moderately large script, I'm actively avoiding bashisms as they don't buy me much yet make my script unportable. But, given you've spent so much time on this subject, are there any bashisms that are truly essential and you don't want to live without?

Re: Pure Bash Bible

#56

Post by dyanaraps »

0xdeadb00f wrote: post #49
dyanaraps wrote: post #18

Earlier quoted context omitted.

I understand your concerns about bash features and POSIX compatibility. The bash bible was written specifically to document the shell extensions bash implements. My focus for the past few months has been writing a Linux distribution (and its package manager/tooling) in POSIX sh. I've learned a lot of tricks and I'm very tempted to write a second "bible" with snippets that are supported in all POSIX shells. (I created…

I'd be very interested in a POSIX sh version of this bash bible.

I've started working on it here: https://github.com/dylanaraps/pure-sh-bible

Re: Pure Bash Bible

#57

Post by dyanaraps »

acqq wrote: post #41
dyanaraps wrote: post #18

Earlier quoted context omitted.

I understand your concerns about bash features and POSIX compatibility. The bash bible was written specifically to document the shell extensions bash implements. My focus for the past few months has been writing a Linux distribution (and its package manager/tooling) in POSIX sh. I've learned a lot of tricks and I'm very tempted to write a second "bible" with snippets that are supported in all POSIX shells. (I created…

> I've learned a lot of tricks and I'm very tempted to write a second "bible" with snippets that are supported in all POSIX shells. That could be potentially even more interesting than a bash-specific one (as it is harder to get it right -- bash can be figured out out of the single reference, anything "portable" has many dependencies).

I've started working on it here: https://github.com/dylanaraps/pure-sh-bible

Re: Pure Bash Bible

#58

Post by kamaal »

masklinn wrote: post #47
kamaal wrote: post #33

Earlier quoted context omitted.

Doesn't scale well when you have to stitch more than two commands with pipes, which is a very common use case for Unix CLI utilities.

Don't see why it "doesn't scale well", the overhead is roughly constant: use the stdout of one program as the input of the next.

That's not as easy as

    cat something | grep "this" | cut -f 1 | sed -e 's/.../.../'
You end up writing too much code, it's very verbose. Some times symbols are what you want. In fact the biggest progress in the growth of Math happened when they tossed out doing math with words and bought in symbols.

Re: Pure Bash Bible

#59

Post by dyanaraps »

jgtrosh wrote: post #53
dyanaraps wrote: post #23

Earlier quoted context omitted.

The shell is my favorite language and I really don't know why. It's definitely possible to write shell code which properly passes shellcheck's linter though it's an uphill battle to learn the ins and outs and _why_ X is wrong when Y is right. I even managed to write a full TUI file manager in bash! https://github.com/dylanaraps/fff I full understand that there are times when the shell should not be used and when othe…

Have you read Bash Pitfalls[1]? Do you write truly correct Bash/POSIX code? Do you still love it? [1]: https://mywiki.wooledge.org/BashPitfalls

> Have you read Bash Pitfalls[1]?

I've read pretty much everything I could get my hands on regarding the shell (including the mentioned link) and I still love it.

> Do you write truly correct Bash/POSIX code?

If we define correct as passing shellcheck, avoiding all pitfalls and maintaining compatibility (POSIX sh not bash), then yes, I like to think so. :)

> Do you still love it?

Oh yeah! I've been writing a ton of POSIX sh as of late. My latest project being a Linux distribution: https://getkiss.org/

(hello from Firefox in KISS!)

Re: Pure Bash Bible

#60

Post by dyanaraps »

tannhaeuser wrote: post #55
dyanaraps wrote: post #18

Earlier quoted context omitted.

I understand your concerns about bash features and POSIX compatibility. The bash bible was written specifically to document the shell extensions bash implements. My focus for the past few months has been writing a Linux distribution (and its package manager/tooling) in POSIX sh. I've learned a lot of tricks and I'm very tempted to write a second "bible" with snippets that are supported in all POSIX shells. (I created…

You can add me to those interested in a pure POSIX shell bible. Whenever I'm about to write a moderately large script, I'm actively avoiding bashisms as they don't buy me much yet make my script unportable. But, given you've spent so much time on this subject, are there any bashisms that are truly essential and you don't want to live without?

> You can add me to those interested in a pure POSIX shell bible.

I've started working on it here: https://github.com/dylanaraps/pure-sh-bible

> are there any bashisms that are truly essential and you don't want to live without?

The only thing I'd say I miss when writing POSIX `sh` is arrays.

I work around this by using 'set -- 1 2 3 4' to mimic an array using the argument list. The limitation here though is that you're limited to one "array" at a time.

The other alternative I make use of is to use "string lists" (list="1 2 3 4") with word splitting.

This can be made safe if the following is correct:

- Globbing is disabled.

- You control the input data and can safely make assumptions (no spaces or new lines in elements).

While it's something that'd be nice to have, there are ways to work around it.

EDIT: One more thing would be "${var:0:1}" to grab individual characters from strings (or ranges of characters from strings).

Post reply on HN