Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Opinionated Guide to Xargs

oilshell.org

Post reply on HN
Print view View original

71–80 of 135 posts

Re: An Opinionated Guide to Xargs

#71

Post by JNRowe »

HMH wrote: post #26

I always wonder why something like xargs is not a shell built-in. It's such a common pattern, but I dread formulating the correct incantation every time. I was happy to read that the author comes to the same conclusion and proposes an `each` builtin (albeit only for the Oil shell)! Like that there is no need to learn another mini language as pointed out.

If you're a zsh user it offers a version of something like xargs in zargs¹. As the documentation shows it can be really quite powerful in part because of zsh's excellent globbing facilities, and I think without that support it wouldn't be all that useful as a built-in.

I'd also perhaps argue that the reason we don't want xargs to be a built-in is precisely because of zargs and the point in your second paragraph. If it was built-in it would no doubt be obscenely different in each shell, and five decades later a standard that no one follows would eventually specify its behaviour ;)

¹ https://zsh.sourceforge.io/Doc/Release/User-Contributions.ht... - search for "zargs", it has no anchor. Sorry.

Re: An Opinionated Guide to Xargs

#72

Post by fiddlerwoaroof »

chriswarbo wrote: post #66
aaaaaaaaaaab wrote: post #47

Earlier quoted context omitted.

Some additional tips: 1. You don't need the parentheses. 2. If you use process substitution [1] instead of a pipe, you will stay in the same process and can modify variables of the enclosing scope: i=0 while read -r v; do ... i=$(( i + 1)) done The drawback is that this way `do_something` has to come after `done`, but that's bash for you ¯\_(ツ)_/¯ [1] https://www.gnu.org/software/bash/manual/html_node/Process-S...

I use this exact pattern a lot. One thing to consider is that in the process substitution version, do_something can't modify the enclosing variables. The vast majority of the time I want to modify variables in the loop body and not the generating process, but it's worth keeping in mind. One common pattern I use this for is running a bunch of checks/tests, e.g. EXIT_CODE=0 while read -r F do do_check "$F" || EXIT_CODE…

I usually write zsh scripts and I think there’s a shell option in zsh that allows the loop at the end of the pipe to modify variables in the enclosing body: I remember at least one occasion where I was surprised about this discrepancy between shells.

Re: An Opinionated Guide to Xargs

#73

Post by JNRowe »

entire-name wrote: post #60
fiddlerwoaroof wrote: post #55

Earlier quoted context omitted.

Yeah, although I use the parentheses mostly because I like how it reads. And that process substitution trick is important too. I think the redirection can come first, though (not at a computer to test):

Redirection like this doesn't seem to work if it comes first on GNU bash 5.0.17(1)-release. For documentation purposes, this is the exact thing I tried to run: $ Maybe there is another way...

One way which isn't great, but an option nonetheless… The zsh parser is happy with that form:

    $ zsh -c '
My position isn't that it is a good reason to switch shells, but if you're using it anyway then it is an option.

Re: An Opinionated Guide to Xargs

#74

Post by fl0wenol »

derriz wrote: post #57
aaaaaaaaaaab wrote: post #48

I would recommend using -0 instead of -d, as the latter is not supported on BSD (and macOS) xargs: do_something | tr \\n \\0 | xargs -0 ...

I wish this was the default behavior of xargs (the 'tr \\n \\0 | xargs -0' bit). I don't know why xargs splits on spaces and tabs as well as newlines by default and doesn't even have a flag to just split on lines. Ok filenames can theoretically have newlines in them but I'd be happy to deal with that weird case. I can't recall ever having encountered it in years of using bash on various systems. Shell pipes would the…

xargs defaults to all whitespace because it was designed to get around the problem of short argv lengths (like, I'm talking 4k or less on older Unix-y systems, sometimes as low as 255 bytes).

So the defaults went with principle of least surprise, pretending it's like a very long args list that you could theoretically enter at the shell, including quotes.

You could, for example, edit the args list in vi and line split / indent as you please but not impact the end result.

Re: An Opinionated Guide to Xargs

#75

Post by aaaaaaaaaaab »

fiddlerwoaroof wrote: post #72
chriswarbo wrote: post #66

Earlier quoted context omitted.

I use this exact pattern a lot. One thing to consider is that in the process substitution version, do_something can't modify the enclosing variables. The vast majority of the time I want to modify variables in the loop body and not the generating process, but it's worth keeping in mind. One common pattern I use this for is running a bunch of checks/tests, e.g. EXIT_CODE=0 while read -r F do do_check "$F" || EXIT_CODE…

I usually write zsh scripts and I think there’s a shell option in zsh that allows the loop at the end of the pipe to modify variables in the enclosing body: I remember at least one occasion where I was surprised about this discrepancy between shells.

Interesting! Indeed, Greg's BashFAQ notes it too: https://mywiki.wooledge.org/BashFAQ/024

>Different shells exhibit different behaviors in this situation:

>- BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator ('').

>- BASH, Yash and PDKsh-derived shells create a new process only if the loop is part of a pipeline.

>- KornShell and Zsh creates it only if the loop is part of a pipeline, but not if the loop is the last part of it. The read example above actually works in ksh88, ksh93, zsh! (but not MKsh or other PDKsh-derived shells)

>- POSIX specifies the bash behaviour, but as an extension allows any or all of the parts of the pipeline to run without a subshell (thus permitting the KornShell behaviour, as well).

Re: An Opinionated Guide to Xargs

#76

Post by fiddlerwoaroof »

JNRowe wrote: post #73
entire-name wrote: post #60

Earlier quoted context omitted.

Redirection like this doesn't seem to work if it comes first on GNU bash 5.0.17(1)-release. For documentation purposes, this is the exact thing I tried to run: $ Maybe there is another way...

One way which isn't great, but an option nonetheless… The zsh parser is happy with that form: $ zsh -c ' My position isn't that it is a good reason to switch shells, but if you're using it anyway then it is an option.

I’ve always preferred zsh and, as I’ve slowly adopted nix, I’ve slowly stopped writing bash in favor of zsh

Re: An Opinionated Guide to Xargs

#77

Post by grawlinson »

orhmeh09 wrote: post #68
WhatIsDukkha wrote: post #15

I tend to reach for gnu parallel instead of xargs - https://www.gnu.org/software/parallel/parallel_alternatives.... parallel is probably on the complex side but its also been actively developed, bugfixed and had a lot of road miles from large computing users.

The nagware prompts of parallel are so objectionable that I will do a lot of things to avoid using it at all. So pretentious!

Seems like some distributions patch out the nagware. I know Arch Linux does[0].

[0]: https://github.com/archlinux/svntogit-community/tree/package...

Re: An Opinionated Guide to Xargs

#78

Post by [deleted] »

fiddlerwoaroof wrote: post #55
aaaaaaaaaaab wrote: post #47

Earlier quoted context omitted.

Some additional tips: 1. You don't need the parentheses. 2. If you use process substitution [1] instead of a pipe, you will stay in the same process and can modify variables of the enclosing scope: i=0 while read -r v; do ... i=$(( i + 1)) done The drawback is that this way `do_something` has to come after `done`, but that's bash for you ¯\_(ツ)_/¯ [1] https://www.gnu.org/software/bash/manual/html_node/Process-S...

Yeah, although I use the parentheses mostly because I like how it reads. And that process substitution trick is important too. I think the redirection can come first, though (not at a computer to test):

[deleted]

Re: An Opinionated Guide to Xargs

#79

Post by zeroimpl »

Zababa wrote: post #5
legobmw99 wrote: post #2

This is only tangentially related, but after all the posts here the last few days about thought terminating cliches, I can’t help but reflect on the “X considered harmful” title cliche

I've been thinking about titles, and it's hard to make a good one that doesn't look like a total cliché. "X considered harmful", "an opinionated guide to X", some kind of joke or reference, what could be a collection of tags (X, Y and Z), "things I have learned doing X", etc.

I specifically clicked on this topic because of the word “opinionated”. As I already know how to use xargs, I was curious what kind of non-conventional or controversial opinion the author might have.
Post reply on HN