Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shell script best practices, from a decade of scripting things

sharats.me

Post reply on HN
Print view View original

451–460 of 500 posts

Re: Shell script best practices, from a decade of scripting things

#451

Post by GoblinSlayer »

sophacles wrote: post #416
ilyt wrote: post #213

Earlier quoted context omitted.

> Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. It is "opinion" based on debugging scripts made by people (which might be "you but few years ago") that don't know the full extent of death-traps tha…

You say this as if it wasn't extremely common to find giant python monstrosities that can be replaced by a handful of lines of shell. TBF the shell code often is not just cleaner and easier to follow, but also faster. It's possible to use the wrong tool for the job in any language - including language choice itself. Dismissing a programming language because it's not shell and dismissing shell because it's not a prora…

Bash is a good tool if the script is short enough, but if you have to write more than 10 lines, then use a real language.

Re: Shell script best practices, from a decade of scripting things

#452

Post by ronyclau »

AtlasBarfed wrote: post #391
xelxebar wrote: post #182

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

Hard Disagree. Bash programming: - no standard unit testing - how do you debug except with printlns? Fail. - each line usually takes a minimum of 10 minutes to debug unless you've done bash scripting for... ten years - basic constructs like the arg array are broken once you have special chars and spaces and want to pass those args to other commands. and UNICODE? Ha. - standard library is nil, you're dependent on a ho…

> - basic constructs like the arg array are broken once you have special chars and spaces and want to pass those args to other commands. and UNICODE? Ha.

Any example with this? The following works reasonably well for me.

  args=(-a --b 'arg with space' "一 二 三")
  someprog "${args[@]}"

Re: Shell script best practices, from a decade of scripting things

#453

Post by GoblinSlayer »

bitofhope wrote: post #197
xelxebar wrote: post #182

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

There are workloads where shell scripts are the so-called right tool for a job . All too often I see people writing scripts in "proper" languages and calling os.system() on every other line. Shell scripts are good for gluing programs together. It's fine to use them for that.

I wrote such a program, that runs other programs for heavy lifting but also parses text which you can't possibly do in bash.

Re: Shell script best practices, from a decade of scripting things

#454

Post by GoblinSlayer »

chubot wrote: post #298
blueflow wrote: post #285

Earlier quoted context omitted.

POSIX shell is the standard, not bash.

That ship has sailed, because busybox ash and dash continually implement some of bash features and semantics, which come from ksh. And OSH implements almost all of bash That is, The posix shell spec is missing a lot of reality. It’s not very actively maintained, unfortunately The canonical example is not having local vars, which basically every shell supports

That's just feature creep, eventually they will implement all features of bash, zsh, csh, fish, elvish, oil, HolyC and all other shells that emerge in the meantime.

Re: Shell script best practices, from a decade of scripting things

#455

Post by sargstuff »

mmh0000 wrote: post #383
ketanmaheshwari wrote: post #234

Earlier quoted context omitted.

Awk is awesome but saying it literally takes 1 hour to properly learn it is a bit overselling.

What? The awk manual is only 827 highly technical pages[1]. If you can't read and interalize that in an hour, I suspect you're a much worse programmer than the OP. [1] https://www.gnu.org/software/gawk/manual/gawk.html For the sarcasm impared among us, everything above this, but possibly including this sentence is sarcasm.

Think the more relevant script equivalent of 'Everything in this statement is false.' is 'All output must return false to have true side effects.'

The quick one ~ true ~ fix was ! or #! without the 1024k copyright.

s-expression notation avoids the issue with (."contents")

MS Windows interpretation is much more terse & colorful.

Re: Shell script best practices, from a decade of scripting things

#456

Post by sargstuff »

qwertywert_ wrote: post #355
j0hnyl wrote: post #206

Earlier quoted context omitted.

I noticed that I became so much more quick after taking 1 hour to properly learn awk. Yes, it literally takes about 1 hour.

The only thing I use AWK for is getting at columns from output, (possibly processing or conditionally doing something on each) what would be the next big use-case?

awk autonoma theory and oop the results. add unicode for extra tics!

Scripted cholmskey grammar ( https://en.wikipedia.org/wiki/Universal_grammar ) to unleash the power of regular expressions.

Re: Shell script best practices, from a decade of scripting things

#457

Post by sargstuff »

humanrebar wrote: post #214
snidane wrote: post #204

Earlier quoted context omitted.

Shell and SQL make you 10x productive over any alternative. Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores , while somebody comfortable is shell writes a parallelized one liner, rips through GBs of data, and delivers the answer in 15 minutes. What Python is to Java, Shell is to Python. It sp…

For simple scripting tasks, yes. I have had the opposite experience for more critical software engineering tasks (as in, coding integrated over time and people). Language aside, the ecosystem and culture do not afford enough in way of testing, dependency management, feature flags, static analysis, legibility, and so on. The reason people say to keep shell programs short is because of these problems, it needs to be po…

awk great for things that will be used over several decades. (where hardware / OS started with nolonger exists at end of multi-decade project, but data from start to end still has to be used)

Re: Shell script best practices, from a decade of scripting things

#458

Post by heresie-dabord »

HyperSane wrote: post #299
heresie-dabord wrote: post #189

Earlier quoted context omitted.

"Primitive" seems like the wrong word to apply. A modern Python programmer (a common example) might think that anything not OOP is archaic. But OOP is just one design pattern. And... it can easily lead to over-engineering and delight in the pattern per se. Bash, awk, grep et alii don't do OOP. But they are close to the data and are powerful. Compaints about the notation of these tools (e.g. "line noise") are becoming…

""Primitive" seems like the wrong word to apply" Painful, ugly, unpleasant?

I guess you would admit that beauty is subjective. Beauty in tooling is an even more rarefied discussion.

Code that we might deem "beautiful" may not even compile. ^_^

Does the tool do what it needs to do without getting fussy, gobbling RAM, and requiring a small army of maintainers to feed the monster? There is most definitely a place in technology for that kind of sanity.

There is something to be said for a design that doesn't bring with it a brigade of Opinions-As-A-Service proponents.

Any non-trivial notation is going to require learning. There is absolute value in learning. I think of many flexible and effective notations in tooling that has literally been the foundation of modern computing.

Maybe we are in the age of the Mono-Notation Luxury Coder. ^_^

Re: Shell script best practices, from a decade of scripting things

#459

Post by [deleted] »

justanotherbody wrote: post #316
irrational wrote: post #313

Earlier quoted context omitted.

> while somebody comfortable is shell writes a parallelized one liner Do you have an example of this? I didn’t even know you could make sql calls in scripts.

mysql, psql etc. let you issue sql from the command line I don't do much sql in bash scripts but I do keep some wrapper scripts that let me run queries from stdin to databases in my environment

[deleted]

Re: Shell script best practices, from a decade of scripting things

#460

Post by sargstuff »

justanotherbody wrote: post #316
irrational wrote: post #313

Earlier quoted context omitted.

> while somebody comfortable is shell writes a parallelized one liner Do you have an example of this? I didn’t even know you could make sql calls in scripts.

mysql, psql etc. let you issue sql from the command line I don't do much sql in bash scripts but I do keep some wrapper scripts that let me run queries from stdin to databases in my environment

WASM Gawk wrapper script in a web browser with relevant information about schema grammar / template file would allow for alternate display formats beyond cli text (aka html, latex, "database report output", cvs, etc )
Post reply on HN