Live data from Hacker News

GNU Parallel – The command line power tool

slideshare.net

11–20 of 29 posts

Re: GNU Parallel – The command line power tool

#11
post #5

i use GNU parallel exclusively in place of xargs simply because it has --dry-run.

wouldn't an "echo" just do the same? $ mkdir a $ cd a $ touch b c d e $ find -type f | xargs echo rm rm ./b ./d ./e ./c $ ls b c d e $ find -type f | xargs rm $ ls $

What if your command has a pipe in it? Then putting echo in front won't work, because the command after the pipe is still executed. The dry run option always works, and doesn't require editing the command itself.

Re: GNU Parallel – The command line power tool

#12

i use GNU parallel exclusively in place of xargs simply because it has --dry-run.

Nice. I just had to check to see if xargs has a similar feature. It doesn't, though the --no-run-if-empty and --verbose options are both handy. I believe I've used xargs with "echo " to proof output before committing it. You can simply re-run that piped to bash (or your shell of choice) to execute commands if you wish (say, if parallel isn't available). E.g., echo foo bar baz | xargs -n1 -t echo ls | bash ... will ex…

Yes, you can use echo, but that won't work if the command is something like "ls DIR | wc -l".

Re: GNU Parallel – The command line power tool

#13
post #10

Wow, I must have reinvented this particular wheel at least five times.

This is exactly what I feel like every time I see an article/presentation about (maybe even really small) tools that just get the job done but I didn't know about and didn't even think about looking for although I can think of so many cases where they would've been incredibly useful.

Re: GNU Parallel – The command line power tool

#15
post #14

Anybody have a link to a version viewable without proprietary plugins? "Flash Player 9 (or above) is needed to view presentations"

This looks very similar but is actually a year newer than the slides on slideshare:

http://www.luga.de/Angebote/Vortraege/GNU_Parallel_LIT_2011/...

Re: GNU Parallel – The command line power tool

#17
post #7

I use parallel all the time for embarrasingly parallel scientific computations on a cluster. It is very easy to use and elegant, and it's one of the programs I'm most grateful for. Recently the developers fixed a major bug for me, that child jobs on other nodes would not be killed when parallel was killed. This was the only thing stopping me from recommending it to my labmates, now there's no reason not to use it!

Remember to show them 'parallel --bibtex'

Re: GNU Parallel – The command line power tool

#19
post #9
post #5

Earlier quoted context omitted.

wouldn't an "echo" just do the same? $ mkdir a $ cd a $ touch b c d e $ find -type f | xargs echo rm rm ./b ./d ./e ./c $ ls b c d e $ find -type f | xargs rm $ ls $

One advantage of "parallel --dry-run" over "xargs echo" is that the former quotes its output: $ touch 'Ham Jam Spam' $ touch 'J.R. "Bob" Dobbs' $ find . -type f -print0 | parallel -n1 -0 --dry-run echo echo ./Ham' 'Jam' 'Spam echo ./J.R.\ \"Bob\"\ Dobbs $ find . -type f -print0 | xargs -n1 -0 echo echo echo ./Ham Jam Spam echo ./J.R. "Bob" Dobbs For my own "dry runs", though, I've always preferred passing the command…

Quoting is nice. That's better than bash -x debug mode too.

Re: GNU Parallel – The command line power tool

#20
post #9
post #5

Earlier quoted context omitted.

wouldn't an "echo" just do the same? $ mkdir a $ cd a $ touch b c d e $ find -type f | xargs echo rm rm ./b ./d ./e ./c $ ls b c d e $ find -type f | xargs rm $ ls $

One advantage of "parallel --dry-run" over "xargs echo" is that the former quotes its output: $ touch 'Ham Jam Spam' $ touch 'J.R. "Bob" Dobbs' $ find . -type f -print0 | parallel -n1 -0 --dry-run echo echo ./Ham' 'Jam' 'Spam echo ./J.R.\ \"Bob\"\ Dobbs $ find . -type f -print0 | xargs -n1 -0 echo echo echo ./Ham Jam Spam echo ./J.R. "Bob" Dobbs For my own "dry runs", though, I've always preferred passing the command…

Why does the parallel example look weird? It put the single quotes in all the wrong places, and completely around Jam. It should have printed

  echo './Ham
  Jam
  Spam'
but your output looks different. As an example of how it should look, try this:

  $ find . -type f -print0 | xargs -n1 -0 perl -e'print "\"$_\" " for (@ARGV)'
  "./Ham
  Jam
  Spam"
Post reply on HN