In what ways is this better than make -j?
parallel is intended to work on arbitrary commands.
GNU Parallel - build and execute command lines from standard input in parallel
11–20 of 37 posts
Re: GNU Parallel - build and execute command lines from standard input in parallel
#12Re: GNU Parallel - build and execute command lines from standard input in parallel
#13On first impressions I like this much more than ppss. The distributed setup is much easier and the documentation is more thorough.
Re: GNU Parallel - build and execute command lines from standard input in parallel
#14It seems like 90% of the uses for this can be taken care of with xargs: echo "file1 file2" | xargs -P 2 gzip
Probably, although it doesn't seem to be the focus of xargs.
And the version of xargs that is included with Solaris 10 doesn't have the -P option. In which case, installing gnu parallel is a slightly easier option than installing a different version of xargs.
Re: GNU Parallel - build and execute command lines from standard input in parallel
#15In what ways is this better than make -j?
make requires a Makefile, whereas one can pass parameters directly to parallel.
There also seems to be a few more options revolving around job success/failure and how to react -- a) ignore failed jobs and report how many at the end, b) cleanly exit as soon as a job fails and c) stop all jobs as soon as one fails.
Re: GNU Parallel - build and execute command lines from standard input in parallel
#16In what ways is this better than make -j?
make requires a Makefile, whereas one can pass parameters directly to parallel. There also seems to be a few more options revolving around job success/failure and how to react -- a) ignore failed jobs and report how many at the end, b) cleanly exit as soon as a job fails and c) stop all jobs as soon as one fails.
Those (a) (b) and (c) points sound like strengths of make, to me.
Re: GNU Parallel - build and execute command lines from standard input in parallel
#17I'll express here what I feel while reading any man pages..
Examples should be at the top!
(10 years of frustration in that one line message :p)
Re: GNU Parallel - build and execute command lines from standard input in parallel
#18Would it work on a PS3 running ubuntu?
should work if you install moreutils package in ubuntu (lynx)
Re: GNU Parallel - build and execute command lines from standard input in parallel
#19This is very cool, but a little opaque at first read... To get a quicker and digestible intro, watch the video introduction (linked on the page): http://www.youtube.com/watch?v=OpaiGYxkSuQ
Well, the example(at least the first one) in that video is bit skewed. First, he runs gzip and then immediately runs 'parallel gzip' without dropping disk caches. So in the later case the bottleneck would be CPU rather than disk IO(everything read from disk cache in the RAM). IMO I expect for the work that is IO bound we won't see any significant improvement using parallel or anything similar.