i found nothing in this blog post useful other than string slicing/splitting.
and you forgot put the ultimate flag of this program that is -r.
An Opinionated Guide to Xargs
121–130 of 135 posts
Re: An Opinionated Guide to Xargs
#122Earlier quoted context omitted.
These days bash and/or zsh are available nearly every place I care about, so I find POSIX compliance to be much less relevant.
No, process substitution must be provided by the kernel/syslibs, it is not feature of bash. For example there is bash on AIX, but process substitution is not possible because the OS do not support it.
ksh93 depends exclusively on the kernel implementation of /dev/fd devices. I just checked `cat Bash uses /dev/fd when available, but also appears to have an internal implementation which silently creates named pipes and cleans them up. In Bash 5.0.18 on AIX, fake process substitution works just fine, in my testing.
Re: An Opinionated Guide to Xargs
#123Earlier quoted context omitted.
I usually write zsh scripts and I think there’s a shell option in zsh that allows the loop at the end of the pipe to modify variables in the enclosing body: I remember at least one occasion where I was surprised about this discrepancy between shells.
Interesting! Indeed, Greg's BashFAQ notes it too: https://mywiki.wooledge.org/BashFAQ/024 >Different shells exhibit different behaviors in this situation: >- BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator (' '). >- BASH, Yash and PDKsh-derived shells create a new process only if the loo…
Check out bash lastpipe option.
Re: An Opinionated Guide to Xargs
#124I tend to reach for gnu parallel instead of xargs - https://www.gnu.org/software/parallel/parallel_alternatives.... parallel is probably on the complex side but its also been actively developed, bugfixed and had a lot of road miles from large computing users.
I mention it here: https://www.oilshell.org/blog/2021/08/xargs.html#xargs-p-aut... What does it do that xargs and shell can't? (honest question)
> What does it do that xargs and shell can't? (honest question)
For me, an essential feature of GNU parallel is that it is semantically equivalent to "sh". Imagine that you write a file that contains a long list of commands. You can pipe that file to "sh" to run the commands, or pipe it to "parallel" to do the same, but faster. If you are building the list of commands on the fly, then you can use xargs with a slightly different syntax. But somehow using "sh" or "parallel" gives a certain peace of mind due to its straightforward semantics. I never used any argument of GNU parallel apart from -j
My usage pattern: to build a list of commands explicitly then run it (possibly teeing the list into a temporary file to inspect it):
for i in one two three; do
printf "echo $i\n"
done |sh # or |parallelRe: An Opinionated Guide to Xargs
#125Earlier quoted context omitted.
No, process substitution must be provided by the kernel/syslibs, it is not feature of bash. For example there is bash on AIX, but process substitution is not possible because the OS do not support it.
ksh93 depends exclusively on the kernel implementation of /dev/fd devices. I just checked `cat Bash uses /dev/fd when available, but also appears to have an internal implementation which silently creates named pipes and cleans them up. In Bash 5.0.18 on AIX, fake process substitution works just fine, in my testing.
Yes, you are right. Bash 5 on AIX 7.2 works with process substitution. Thanks for the advise!
Re: An Opinionated Guide to Xargs
#126Wanting verbose logging from xargs, years ago I wrote a script called `el` (edit lines) that basically does `xargs -0` with logging. https://github.com/westurner/dotfiles/blob/develop/scripts/e... It turns out that e.g. -print0 and -0 are the only safe way: line endings aren't escaped: find . -type f -print0 | el -0 --each -x echo GNU Parallel is a much better tool: https://en.wikipedia.org/wiki/GNU_parallel
(author here) Hm I don't see either of these points because: GNU xargs has --verbose which logs every command. Does that not do what you want? (Maybe I should mention its existence in the post) xargs -P can do everything GNU parallel do, which I mention in the post. Any counterexamples? GNU parallel is a very ugly DSL IMO, and I don't see what it adds. -- edit: Logging can also be done with by recursively invoking sh…
In your examples you fail to put 'xargs -P' in the middle of a pipeline: You only put it at the end.
In other words:
some command | xargs -P other command | third command
UNIX is great in that you can pipe commands together, but due to the interleaving issue 'xargs -P' fails here. It does not live up to the UNIX philosophy. Which is probably why you unconsciously only use it at the end of a pipeline.
You can find a different counterexample on https://unix.stackexchange.com/questions/405552/using-xargs-... I will be impressed if you can implement that using xargs. Especially if you can make it more clean than the paralel version.
Re: An Opinionated Guide to Xargs
#127In Bash (not every shell supports this) functions can be exported, which enables this nice pattern with xargs:
myfunc() {
printf " %s" "I got these arguments:" "$@" $'\n'
}
export -f myfunc
seq 6 | xargs -n2 bash -c 'myfunc "$@"' "$0"Re: An Opinionated Guide to Xargs
#128I find that using the example of `rm` to discuss whether to pick `find -exec` or `find | xargs` rather strange, given the existence of `find -delete`. Maybe pick a different example operation to automate.
Re: An Opinionated Guide to Xargs
#129Note that the suggested: rm $(ls | grep foo) will not work if you have file names that contain spaces. Shell programming is planted thick with landmines like this.
Using "ls" is also problematic, because it will outright skip large classes of characters.
Re: An Opinionated Guide to Xargs
#130I tend to reach for gnu parallel instead of xargs - https://www.gnu.org/software/parallel/parallel_alternatives.... parallel is probably on the complex side but its also been actively developed, bugfixed and had a lot of road miles from large computing users.
I mention it here: https://www.oilshell.org/blog/2021/08/xargs.html#xargs-p-aut... What does it do that xargs and shell can't? (honest question)
I use GNU Parallel for long-running jobs for its --eta option. If a job will take days or longer it's useful to know that early in the process. You might want to cancel it and try something else, and if you want to proceed with the long job you can make plans around when your data will be ready.