Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unix Commands I Wish I’d Discovered Years Earlier

spin.atomicobject.com

Post reply on HN
Print view View original

171–180 of 259 posts

Re: Unix Commands I Wish I’d Discovered Years Earlier

#171

Post by csense »

My favorite is the parallel jobs feature of xargs. For example, say you want to run a script you wrote called process-video.sh to do some processing on all the video files in a directory (extracting audio to MP3, converting format, etc.). You want to use all 8 of your cores. You could write a Makefile and run it with -j9, or you can do this:

   find . -name "*.flv" | xargs -n 1 -P 9 ./process-video.sh
This immediately forks 9 instances of process-video.sh on the first 9 .flv files in the current directory, then starts a new instance whenever a running instance completes, so 9 instances are always in flight. (I usually set to number of cores plus one for CPU-bound tasks, hence 9 for my i7 with eight cores [1].)

If you add -print0 to the find command and -0 to the xargs command, it uses null-terminated filenames (which does the right thing when filenames contain whitespace).

[1] Logical cores. Most i7's have four physical cores which become eight logical cores through the magic of hyperthreading.

Re: Unix Commands I Wish I’d Discovered Years Earlier

#175

Post by jasomill »

shabble wrote: post #123
aeon10 wrote: post #122

Earlier quoted context omitted.

can you please explain how this works?

it's quite a common pattern in commandline apps to check if the stdout is a tty or not, and do things differently if so. So when you pipe it through `cat', it does the no-fancy-included output, much like the colouring that `ls' will apply interactively, but not when piped/redirected.

ls | cat is also useful because it forces single-column output.

Re: Unix Commands I Wish I’d Discovered Years Earlier

#177

Post by caw »

jasomill wrote: post #175
shabble wrote: post #123

Earlier quoted context omitted.

it's quite a common pattern in commandline apps to check if the stdout is a tty or not, and do things differently if so. So when you pipe it through `cat', it does the no-fancy-included output, much like the colouring that `ls' will apply interactively, but not when piped/redirected.

ls | cat is also useful because it forces single-column output.

Isn't that the same as ls -1?

Re: Unix Commands I Wish I’d Discovered Years Earlier

#179

Post by jasomill »

ilikepi wrote: post #144
shabble wrote: post #134

Earlier quoted context omitted.

I've been told before that `od' is the most portable, although I prefer xxd in virtually all situations.

I've known about 'od', but I've always used 'hexdump' instead. Amusingly, I just noticed that they are literally the same (hard-linked) binary on OS X 10.6, even though the arguments differ somewhat. The man pages states that 'od' is part of POSIX, which aligns with your comment about portability.

This is also a common UNIXism: programs that behave differently depending on what they're called as. Another example is gzip (gunzip = gzip -d, gzcat = gzip -dc).

Re: Unix Commands I Wish I’d Discovered Years Earlier

#180

Post by thret »

csense wrote: post #171

My favorite is the parallel jobs feature of xargs. For example, say you want to run a script you wrote called process-video.sh to do some processing on all the video files in a directory (extracting audio to MP3, converting format, etc.). You want to use all 8 of your cores. You could write a Makefile and run it with -j9, or you can do this: find . -name "*.flv" | xargs -n 1 -P 9 ./process-video.sh This immediately f…

May I ask, what do you gain by setting it to +1 cores?
Post reply on HN