Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shell script best practices, from a decade of scripting things

sharats.me

Post reply on HN
Print view View original

401–410 of 500 posts

Re: Shell script best practices, from a decade of scripting things

#401

Post by meken »

irrational wrote: post #313
snidane wrote: post #204

Earlier quoted context omitted.

Shell and SQL make you 10x productive over any alternative. Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores , while somebody comfortable is shell writes a parallelized one liner, rips through GBs of data, and delivers the answer in 15 minutes. What Python is to Java, Shell is to Python. It sp…

> while somebody comfortable is shell writes a parallelized one liner Do you have an example of this? I didn’t even know you could make sql calls in scripts.

I don’t have an example, but this article comes to mind and you may be able to find an example in it

https://adamdrake.com/command-line-tools-can-be-235x-faster-...

Re: Shell script best practices, from a decade of scripting things

#403

Post by jstanley »

Master Foo once said to a visiting programmer: “There is more Unix-nature in one line of shell script than there is in ten thousand lines of C.”

The programmer, who was very proud of his mastery of C, said: “How can this be? C is the language in which the very kernel of Unix is implemented!”

Master Foo replied: “That is so. Nevertheless, there is more Unix-nature in one line of shell script than there is in ten thousand lines of C.”

The programmer grew distressed. “But through the C language we experience the enlightenment of the Patriarch Ritchie! We become as one with the operating system and the machine, reaping matchless performance!”

Master Foo replied: “All that you say is true. But there is still more Unix-nature in one line of shell script than there is in ten thousand lines of C.”

The programmer scoffed at Master Foo and rose to depart. But Master Foo nodded to his student Nubi, who wrote a line of shell script on a nearby whiteboard, and said: “Master programmer, consider this pipeline. Implemented in pure C, would it not span ten thousand lines?”

The programmer muttered through his beard, contemplating what Nubi had written. Finally he agreed that it was so.

“And how many hours would you require to implement and debug that C program?” asked Nubi.

“Many,” admitted the visiting programmer. “But only a fool would spend the time to do that when so many more worthy tasks await him.”

“And who better understands the Unix-nature?” Master Foo asked. “Is it he who writes the ten thousand lines, or he who, perceiving the emptiness of the task, gains merit by not coding?”

Upon hearing this, the programmer was enlightened.

(https://catb.org/~esr/writings/unix-koans/)

Re: Shell script best practices, from a decade of scripting things

#404

Post by snidane »

chungy wrote: post #351
snidane wrote: post #204

Earlier quoted context omitted.

Shell and SQL make you 10x productive over any alternative. Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores , while somebody comfortable is shell writes a parallelized one liner, rips through GBs of data, and delivers the answer in 15 minutes. What Python is to Java, Shell is to Python. It sp…

> I started using inline 'python -c' more often than the python repl now as it stores the command in shell history and it is then one fzf search away. Do you not have a ~/.python_history? The exact same search functions are available on the REPL. Ctrl-R, type your bit, bam.

Exact same - can I use fzf gistory search using Ctrl+R like I can in shell?

Re: Shell script best practices, from a decade of scripting things

#405

Post by orwin »

Beltalowda wrote: post #154
orwin wrote: post #125

Earlier quoted context omitted.

I feel like powershell hides too much to be used regularly. I have a dozen of small shellscripts and aliases to do basically what PS help me to do when i work on windows (and some), but at least i know how it work behind. I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data l…

> I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data loaded frontend i think). PS i don't understand what it wants me to do. Bit off-topic, but I worked with ExtJS around the same time, and I found it one of the most confusing development experiences I ever had. "It's so ea…

Yeah, exactly, powershell is easy, but not simple!

ExtJS (on netbeans with windows server 2004) was always my worst development experience, it was my first internship too. That's probably the reason why it took so long (and a bag of money) for me to try JS, an IDE and developing on windows again (I only used Windows for CTFs and AndroidStudio).

Re: Shell script best practices, from a decade of scripting things

#407

Post by Aachen »

kleer001 wrote: post #341
ghostoftiber wrote: post #268

Instead of implementing a -h or --help, consider using some code like "if nothing else matches, display the help". The asterisk is for this purpose. while getopts :hvr:e: opt do case $opt in v) verbose=true ;; e) option_e="$OPTARG" ;; r) option_r="$option_r $OPTARG" ;; h) usage exit 1 ;; \*) echo "Invalid option: -$OPTARG" >&2 usage # call some echos to display docs or something... exit 2 ;; esac done

Why not both (or all three)? That's what I do. When I get to a new command I find it a bit anti-social when it takes effort to find the help.

I find it really annoying when I typo an argument and now my shell scrollback is pooped full of help text and you first have to scroll up to find the actual error message (like "invalid choice for --mode" or whatever). Don't remember the most recent offender, but it's typically ancient software that is not in widespread use that does this. Often C or Perl (maybe because those languages are also the oldest).

Running without any arguments? Yes, that should output info in most cases, identical to -(-)h(elp) or even /? and /h(elp) if you're feeling Windowsey that day. Outputting your full usage info, especially when spanning more than half a terminal in full screen on a modern resolution, when "nothing matches"? Please no.

Re: Shell script best practices, from a decade of scripting things

#408

Post by Aachen »

rethab wrote: post #97
bluetomcat wrote: post #46

Use the shell only if your script is mostly about calling other programs and filtering and redirecting their output. That's what the syntax of these languages is optimised for. As soon as you need any data manipulation (i.e. arrays, computation, etc.) it becomes a pain and Python is the much better fit.

AWK is just fine for data manipulation. And unlike python, you don't need to worry about whether it's installed and in what version.

Isn't awk that thing that has different versions with entirely different features and syntaxes? I think one of them ships with mac and the other with, uh, everything I've ever used anyway (maybe not *bsd or something). Or was that sed?

I only ever use it on my own systems so awk works fine for me and I use it regularly, but iirc it's not true that you don't have to worry about versions.

Python is where I don't worry about versions. Everyone's got python3 by now (word got round) and most basics, like print(), works just fine in 2.7 (the main py2 backwards compatibility thing I run into is bytes vs unicode strings; if the script needs to work with raw bytes, you'll just need any python3 version). The issue I run into is with Windows people not having python installed, and (worse) not being able to install it in 30 seconds with one command, but that would be the same with awk.

Re: Shell script best practices, from a decade of scripting things

#409

Post by Aachen »

ndsipa_pomu wrote: post #121

I think BASH scripting is the opposite of riding a bike - you end up re-learning it almost every time you need to do it

Then you haven't learned it, or you need it no more than once a year for 15 minutes maybe?

My girlfriend complained about Firefox aalllways needing updates every time she starts it. Yeah, because she used Chrome most of the time, if you start Firefox once every other month, of course that's going to happen every time. This sounds like a similar issue: the software may not be the friendliest, but you can't really expect another outcome if you never use it because you don't like it because you never use it.

Re: Shell script best practices, from a decade of scripting things

#410

Post by travisjungroth »

Beltalowda wrote: post #314
monknomo wrote: post #293

Earlier quoted context omitted.

if you want `fish_config` opens up an easy editor for changing all the colors to whatever you find quiet and soothing. You have a level of control over things popping up too

Quite a few things can't be disabled; for example AFAIK it doesn't offer a way to disable the autocomplete altogether, or the "fuzzy" matching. I really dislike these things. Fish is a great shell, but very opinionated which is great if your preferences align with that, and not-so-great if they don't. Which is fine because it makes the project better for those who do want these things, and not every project needs to…

This comment is so reasonable I’m getting a contact high of pragmatism.
Post reply on HN