21–30 of 135 posts
This is only tangentially related, but after all the posts here the last few days about thought terminating cliches, I can’t help but reflect on the “X considered harmful” title cliche
do_something | ( while read -r v; do
. . .
done )
Never can remember all the -I stuff around xargs
I tend to reach for gnu parallel instead of xargs - https://www.gnu.org/software/parallel/parallel_alternatives.... parallel is probably on the complex side but its also been actively developed, bugfixed and had a lot of road miles from large computing users.
I mention it here: https://www.oilshell.org/blog/2021/08/xargs.html#xargs-p-aut... What does it do that xargs and shell can't? (honest question)
Today I appreciated Powershell
I was happy to read that the author comes to the same conclusion and proposes an `each` builtin (albeit only for the Oil shell)! Like that there is no need to learn another mini language as pointed out.
I tend to reach for gnu parallel instead of xargs - https://www.gnu.org/software/parallel/parallel_alternatives.... parallel is probably on the complex side but its also been actively developed, bugfixed and had a lot of road miles from large computing users.
I mention it here: https://www.oilshell.org/blog/2021/08/xargs.html#xargs-p-aut... What does it do that xargs and shell can't? (honest question)
If you use `xargs -P`, all processes share the same stdout and output may be mixed arbitrarily between them. (If the program being executed uses line buffering, lines usually won't be mixed together from multiple invocations, but they can be if they're long enough).
In contrast, `parallel` by default doesn't mix together output from different commands at all, instead buffering the entire output until the command exits and then printing it.
With `--line-buffer` the unit of atomicity can be weakened from an entire command output to individual lines of output, reducing latency.
Alternately, with `--keep-order`, `parallel` can ensure the outputs are printed in the same order as the corresponding inputs, which makes the output deterministic if the program is deterministic. Without that you'll get results in an arbitrary order.
These aren't technically things that xargs and shell can't do; you could reimplement the same behavior by hand with the shell. But by the same token, there isn't anything xargs can do that the shell can't do alone; you could always use the shell to manually split up the input and invoke subprocesses. It's just a question of how much you want to reimplement by hand.
Earlier quoted context omitted.
I mention it here: https://www.oilshell.org/blog/2021/08/xargs.html#xargs-p-aut... What does it do that xargs and shell can't? (honest question)
Resumption, error reporting and much better progress monitoring.
For others that didn't know about it, see the examples here: https://www.gnu.org/software/parallel/parallel_tutorial.html...
Here's another surprising feature: https://www.gnu.org/software/parallel/parallel_tutorial.html...
Earlier quoted context omitted.
I mention it here: https://www.oilshell.org/blog/2021/08/xargs.html#xargs-p-aut... What does it do that xargs and shell can't? (honest question)
Remote execution.