Live data from Hacker News

Shell script best practices, from a decade of scripting things

sharats.me

431–440 of 500 posts

Re: Shell script best practices, from a decade of scripting things

#431

> [[ ]] is a bash builtin, and is more powerful than [ ] or test. Agreed on the powerful bit, however [[ ]] is not a "builtin" (whereas [ and test are builtins in bash), it's a reserved word which is more similar to if and while. That why [[ ]] can break some rules that builtins cannot, such as `[[ 1 = 1 && 2 = 2 ]]` (vs `[ 1 = 1 ] && [ 2 = 2]` or `[ 1 = 1 -a 2 = 2 ]`, -a being deprecated). Builtins should be conside…

Thanks. Didn't know the word builtin had a specific meaning in bash, which, seems obvious now in hindsight. Should be fixed soon.

Re: Shell script best practices, from a decade of scripting things

#432

This guy shells!

Hey Peter! It's so humbling to see you check out my blog!

Your articles on awk and sed were a huge inspiration to me around 2008-09, and I super-looked up to you. Never have I imagined you would check out my blog one day!

Thank you for all your work dude! Stay awesome.

Re: Shell script best practices, from a decade of scripting things

#434

Earlier quoted context omitted.

> Which works not just to preserve the previous statement from internal inconsistency It doesn't. You now have 4 numbers.

0, 1, 3, 4 and infinity - there's four numbers in this industry. Five There's five numbers in this industry 0, 1, 3, 4, 5 and infinity Wait, I'll come in again

0, 1, 7, and indeterminate, IME.

The 7 being for design. If there are more than 7 boxes on the whiteboard, try again.

Re: Shell script best practices, from a decade of scripting things

#435
post #351

Earlier quoted context omitted.

> I started using inline 'python -c' more often than the python repl now as it stores the command in shell history and it is then one fzf search away. Do you not have a ~/.python_history? The exact same search functions are available on the REPL. Ctrl-R, type your bit, bam.

Exact same - can I use fzf gistory search using Ctrl+R like I can in shell?

I've just started installing ipython on pretty much every python environment I set up on personal laptops, but there is repl history even without ipython: https://stackoverflow.com/a/7008316/1170550

Re: Shell script best practices, from a decade of scripting things

#437
post #216

My biggest complaint about "idiomatic" shell scripting is the use of the [ and [[ operators. It gives the illusion that [ or [[ are part of the shell syntax when actually they're just programs / builtins / functions which communicate with the rest of the script the same way (most) other things interact -- setting exit status. Specifically this means if .. then .. fi works with any program not just [ [[ operators. Tra…

Nice tip!

Re: Shell script best practices, from a decade of scripting things

#438

Earlier quoted context omitted.

new to 'rq', it's not in active development, any other alternatives? it seems doing a lot other than convert structured data to json.

Not sure what it is doing more...I'm referring to this rq: https://github.com/dflemstr/rq#format-support-status It converts to/from the listed formats. There is also `jc` (written in Python) with the added benefit that it converts output of many common unix utilities to json. So you would not need to parse `ip` for example. https://github.com/kellyjonbrazil/jc#parsers

Also look at `yq` - https://github.com/mikefarah/yq

This is a wrapper to jq that also supports yaml and other file formats.

Re: Shell script best practices, from a decade of scripting things

#439
post #206

Earlier quoted context omitted.

I noticed that I became so much more quick after taking 1 hour to properly learn awk. Yes, it literally takes about 1 hour.

The only thing I use AWK for is getting at columns from output, (possibly processing or conditionally doing something on each) what would be the next big use-case?

I have used it to extract a table to restore from a MySQL database dump.

Re: Shell script best practices, from a decade of scripting things

#440
post #352

Earlier quoted context omitted.

> What is the Shell paradigm? I would argue that it's line-oriented pipelines. Which python can do realitively well, by using the `subprocess` module. Here is an example including a https://porkmail.org/era/unix/award (useless use of cat) finding all title lines in README.md and uppercasing them with `tr` import subprocess as sp cat = sp.Popen( ["cat", "README.md"], stdout=sp.PIPE, ) grep = sp.Popen( ["grep", "#"], s…

> But on the other side of that coin its alot easier in python to do a complex regular expression (maybe depending on a command line argument) on one of those, using the result in an HTTP request via the `requests` module, packing the results into a digram rendered in PNG and sending it via email. Doesn't sound so bad. A quick argument parser, a call out to grep or sed, pipe to curl, then to graphviz I guess (I don't…

It's certainly possible, but here comes the fun; How read/maintain/extend-able is the solution? How well does it handle errors, assist the user? Add checking if all the programs are installed and useful error messages into the mix. Then the API does a tiny change and now we need a `jq` between curl and graphviz, and maybe we'd need an option for that case as well, and so on, and so on, ...

Bash scripts have a nasty tendency to grow, sometimes in ways that are disproportional to the bit of extra functionality that is suddenly required. Very quickly, a small quick'n dirty solution can blow up to a compost-heap ... no less dirty, but now instead of a clean-wipe, I'd need a shovel to get through it.

I think my handle speaks for itself as to how much I like bash. But I have had the pleasure of getting handed over bash scripts, hundreds of lines long, with the error description being "it no longer works, could you have a look at it? and the original author both unreachable and apparently having string feelings against comments.

And in many of these cases, it took me less time to code a clean solution in Python or Go, than it took me to grok what the hell that script was actually doing.

Post reply on HN