Obligatory XKCD: https://xkcd.com/927/
along with: I love standards- Theres soany to choose from!
I wrote an bash enumerator because I was sick of xargs
41–50 of 202 posts
Re: I wrote an bash enumerator because I was sick of xargs
#42Not sure I see the point of this. Remembering a bunch of options on one tool is no better than a bunch of tools/constructs? Especially if the latter are useful elsewhere. And it can't be used for scripts unless you start shipping it alongside, which… nah. Just go with foo | while read X; do bar "$X"; done
I use "| while read" as well, because it works well in a pipeline and handles embedded spaces. It doesn't handle embedded newlines, but in practice, real files have embedded spaces, while embedded newlines only happen in test cases and exploits. (You can do actual NUL-delimited reads with `-d ''`, but for a quick command-line operation that's generally not necessary, and if you're going to be that careful you probably also need `-r`.)
Re: I wrote an bash enumerator because I was sick of xargs
#43Earlier quoted context omitted.
Xargs is fine, but openbsd has a -J option and every time I read the man page to figure out how to use it I read the -I and -J options and my brain glazes over. https://man.openbsd.org/xargs Other brain glazing obsd wierdness is it's two argument cd command, a cryptid I am unable to wrap my head around. https://man.openbsd.org/ksh#cd~2
I highly recommend memorizing the POSIX version of every *NIX command and sticking to them. 60% of the time, they work every time. ( https://pubs.opengroup.org/onlinepubs/9799919799/idx/utiliti... )
I highly recommend using whatever capabilities are convenient of any utility you're running, and not caring about what other systems do unless you're actually trying to write a portable shell script.
Re: I wrote an bash enumerator because I was sick of xargs
#44On the topic of xargs replacements, I love gnu parallel. The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs. Parallel has an option for almost everything, it's almost too much. But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0] The gnu parallel book and reading mate…
I have used echo as a sort of poor mans equivalent for a safe check of a pipeline, removing the echo when I felt the rest of the pipeline was working correctly. shell stuff | xargs -n 1 -I % echo real command and % args I have also been known to write scripts where instead of executing the critical parts it prints them. Then a dry run is script and the real run is script | sh
Love that idea, thanks for sharing
Re: I wrote an bash enumerator because I was sick of xargs
#45On the topic of xargs replacements, I love gnu parallel. The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs. Parallel has an option for almost everything, it's almost too much. But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0] The gnu parallel book and reading mate…
I have used echo as a sort of poor mans equivalent for a safe check of a pipeline, removing the echo when I felt the rest of the pipeline was working correctly. shell stuff | xargs -n 1 -I % echo real command and % args I have also been known to write scripts where instead of executing the critical parts it prints them. Then a dry run is script and the real run is script | sh
Shouldn't you use echo instead of cat?
Re: I wrote an bash enumerator because I was sick of xargs
#46Earlier quoted context omitted.
the example as given does not accumulate, so that is what i worked with. find -exec command '{}' '+' accumulates file arguments too. the only advantage of xargs is that you can tell it how many arguments to accumulate,̶ ̶t̶h̶e̶ ̶d̶o̶w̶n̶s̶i̶d̶e̶ ̶o̶f̶ ̶x̶a̶r̶g̶s̶ ̶i̶s̶ ̶t̶h̶a̶t̶ ̶y̶o̶u̶ ̶h̶a̶v̶e̶ ̶t̶o̶ ̶s̶p̶e̶c̶i̶f̶y̶ ̶t̶h̶e̶ ̶n̶u̶m̶b̶e̶r̶,̶ ̶w̶h̶e̶r̶e̶a̶s̶ ̶f̶i̶n̶d̶ ̶j̶u̶s̶t̶ ̶f̶i̶t̶s̶ ̶a̶s̶ ̶m̶a̶n̶y̶ ̶a̶s̶ ̶i̶t̶ ̶c…
xargs will fit as many as it can in 128 KiB or the system limit, which is smaller, so in practice, it's almost always pretty similar default packing
you are right, i forgot to check the default. fixed my comment.
Re: I wrote an bash enumerator because I was sick of xargs
#47Earlier quoted context omitted.
I have used echo as a sort of poor mans equivalent for a safe check of a pipeline, removing the echo when I felt the rest of the pipeline was working correctly. shell stuff | xargs -n 1 -I % echo real command and % args I have also been known to write scripts where instead of executing the critical parts it prints them. Then a dry run is script and the real run is script | sh
Shouldn't you use echo instead of cat?
yes echo, total brain fart there. corrected.
Re: I wrote an bash enumerator because I was sick of xargs
#48Not sure I see the point of this. Remembering a bunch of options on one tool is no better than a bunch of tools/constructs? Especially if the latter are useful elsewhere. And it can't be used for scripts unless you start shipping it alongside, which… nah. Just go with foo | while read X; do bar "$X"; done
Depending on the actual command, this can be far slower and less efficient than xargs. You're creating a separate process for each invocation of bar when a lot of commands will take many targets for a single invocation.
Try this with find and grep vs xargs. There's a big difference.
Re: I wrote an bash enumerator because I was sick of xargs
#49Not sure I see the point of this. Remembering a bunch of options on one tool is no better than a bunch of tools/constructs? Especially if the latter are useful elsewhere. And it can't be used for scripts unless you start shipping it alongside, which… nah. Just go with foo | while read X; do bar "$X"; done
Bash while loops are pretty readable and the above would be a nice to iterate over lines if it wasn't for that gnarly pipe, which is a common source of errors in this construct. Remember that a pipe starts a new shell. So:
grep stuff file.txt | while read key value ; do [ "$key" = "target" ] && found="$value" ; done
while read key value ; do [ "$key" = "target" ] && found="$value" ; done Re: I wrote an bash enumerator because I was sick of xargs
#50Literally just use xargs with -I {} and quotation marks?
That won't save you from weird file names, but null termination will.