Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I wrote an bash enumerator because I was sick of xargs

numerlab.org

Post reply on HN
Print view View original

121–130 of 202 posts

Re: I wrote an bash enumerator because I was sick of xargs

#121

Post by tom_ »

I never liked xargs either. My replacement is kind of like "xargs -n 1 -d '\n' -J '{}'", which experience has taught me is almost always what I want. It consumes all of its input before starting, so it knows how many files it has to process, and it can print progress to stderr and/or the terminal title as it goes. It can run each command via the shell if you want. It has --dry-run and --keep-going. It can read the file names from a file rather than stdin. I have a few more ideas for things it could do, but I haven't needed to add them yet.

(For enumerating files, I use find or dir/b/s, possibly combined with grep, then pipe the result in.)

People moaning about avoiding non-basic use of xargs and bash (and inadvertently demonstrating in many cases why some of us think that xargs sucks) miss a large part of the point, which is that it's nice to have a tool that does exactly what you want, and works in a way that's convenient for you, and isn't so widely used that you have to worry about modifying it. If you find the tool doesn't work the way you like, you can just change it. You don't have to be answerable to anybody else. I think tptacek's quite good essay could be relevant: https://sockpuppet.org/blog/2026/05/12/emacsification/

(You don't have to use LLMs for this! I wrote my program by hand, it's only like 250 lines of Python, and you could write one too. But, whichever barriers to entry prevent you from creating your own tools for yourself, using an LLM would probably lower at least some of them.)

Re: I wrote an bash enumerator because I was sick of xargs

#122

Post by giov4 »

giov4 wrote: post #118

I lol'd, but do you think this is valid? https://github.com/wallach-game/bashumerate/pull/1/files to me looks like a right solution, even if I was also tired of writing for loops by hand usually

On a second thought I think that the effort in trying to find a shell uncomfortability problem and trying to solve it with a "wrapped workaround" still represents positive effort and should be supported and fostered without jumping directly in 90's like rtfm mode.

especially in the case of a younger profile, learning and putting dev effort, even if vibed or redundant, its still effort and a learning activity!

https://github.com/wallach-game

so I support this even if the result and problem solving can be less significant

Re: I wrote an bash enumerator because I was sick of xargs

#124

Post by Dibby053 »

giov4 wrote: post #118

I lol'd, but do you think this is valid? https://github.com/wallach-game/bashumerate/pull/1/files to me looks like a right solution, even if I was also tired of writing for loops by hand usually

Some of these are wrong, as standard shell globs don't traverse directories. For instance the first example should be:

    ls **/*.sh
I think that would work in Bash just as well as using find.

But yeah, even in POSIX it feels redundant when we already have find, seq, grep and whatnot.

Re: I wrote an bash enumerator because I was sick of xargs

#125

Post by Dibby053 »

giov4 wrote: post #122
giov4 wrote: post #118

I lol'd, but do you think this is valid? https://github.com/wallach-game/bashumerate/pull/1/files to me looks like a right solution, even if I was also tired of writing for loops by hand usually

On a second thought I think that the effort in trying to find a shell uncomfortability problem and trying to solve it with a "wrapped workaround" still represents positive effort and should be supported and fostered without jumping directly in 90's like rtfm mode. especially in the case of a younger profile, learning and putting dev effort, even if vibed or redundant, its still effort and a learning activity! https:/…

It's possible the tool is useless and a consequence of not reading the manual, but it could just be the tool is nice but the examples are too simplistic.

In any case, toxicity aside, publishing a command line project and having someone dismiss the examples with shell oneliners is also a valuable lesson, and I don't think it will make him a worse developer.

Re: I wrote an bash enumerator because I was sick of xargs

#126

Post by wongarsu »

Dibby053 wrote: post #124
giov4 wrote: post #118

I lol'd, but do you think this is valid? https://github.com/wallach-game/bashumerate/pull/1/files to me looks like a right solution, even if I was also tired of writing for loops by hand usually

Some of these are wrong, as standard shell globs don't traverse directories. For instance the first example should be: ls **/*.sh I think that would work in Bash just as well as using find. But yeah, even in POSIX it feels redundant when we already have find, seq, grep and whatnot.

I frequently have cases where `ls **/*.png` fails because the argument list is too long. Just happened yesterday with an rm I wanted to run.

Find works, but the syntax is arcane. Not bad, but unlike any other common cli tool, which makes it more difficult to remember if you haven't needed it in a while

Re: I wrote an bash enumerator because I was sick of xargs

#127

Post by GuB-42 »

IdiotSavage wrote: post #93

In this thread: bash experts with arcane knowledge, unintentionally demonstrating how awful bash is. The obvious solution would be to use something more sane, like PowerShell or nushell, but instead old experts will always defend the skills they have honed for years, while criticizing anything that's different.

Maybe it is out of habit, but I never managed to get into PowerShell, in fact, I am not at easy with the Microsoft way of doing things, with few exceptions. Too much UNIX I guess. Nushell seems to be based on the PowerShell philosophy of using structured data and not text, not my thing.

I really like the UNIX way of using text I/O, it has it flaws but it works for me. But that being said, I still hate bash and all its family. It has so many footguns it is an entire armory at this point, mostly related to spaces and escaping.

Something Perl-like could be a saner replacement. It is already a bit shell-like, it doesn't struggle with escaping the way bash does, and it has very powerful text processing abilities that go well with traditional UNIX tools.

Re: I wrote an bash enumerator because I was sick of xargs

#128

Post by MrDOS »

ykonstant wrote: post #116
jsrcout wrote: post #104

Earlier quoted context omitted.

For tools that don't support -0, you can add the NULs yourself without much fuss. It's a one-liner in awk/perl/sed. Very handy.

If you do not have -0 options in your xargs/find, I wonder if you will have those extensions in your awk/sed. Perl is a different story.

I think the parent was referring to using awk/sed to do the equivalent of:

    find ... | tr '\n' '\0' | xargs -0 ...
I.e., a blind replacement without the tool having any particular semantic understanding of what it's translating.

That still requires your xargs to have -0 support, though, and I'd be surprised to have that without the corresponding option on find. But I've done this before when feeding xargs from something other than find.

Re: I wrote an bash enumerator because I was sick of xargs

#129

Post by ndsipa_pomu »

IdiotSavage wrote: post #93

In this thread: bash experts with arcane knowledge, unintentionally demonstrating how awful bash is. The obvious solution would be to use something more sane, like PowerShell or nushell, but instead old experts will always defend the skills they have honed for years, while criticizing anything that's different.

Yes, Bash is filled with footguns and is awful due to that. However, it's still incredibly useful as it's everywhere and has a far greater lifespan than almost anything else. You can write scripts in PowerShell or nushell, but then find that twenty years later they're no longer usable or you find a twenty year old machine that won't have PowerShell/nushell installed.

It's not so much about defending arcane scripting skills, but that Bash functions as a lowest common denominator and is useful because of that. If you want something that works reliably over decades, then it's best not to go for an "improved" shell as it may not still be around.

I like to think of Bash script writing as the opposite of riding a bike - you have to relearn it almost every time you write a script.

Re: I wrote an bash enumerator because I was sick of xargs

#130

Post by MrDOS »

dataflow wrote: post #105
darrenf wrote: post #88

Earlier quoted context omitted.

> parallel: Warning: Finding the maximal command line length. This may take up to 1 minute. > So I wait a few seconds [snip] This warning is only ever printed if running in Cygwin, not Linux or macOS or elsewhere. Cygwin is notoriously slow. # This is slow on Cygwin, so give Cygwin users a warning if($^O eq "cygwin" or $^O eq "msys") { ::warning("Finding the maximal command line length. ". "This may take up to 1 minu…

This is on MSYS2, yes, and that excuses absolutely nothing, because this shouldn't be happening in the first place for speed to be even relevant. At the risk of repeating myself thrice: the messages are confusing, the Ctrl+C handling is just utterly broken, the citation message adds to the confusion while being frankly obnoxious, and all of the delays and outputs are unnecessary in the first place as proven by litera…

> the messages are confusing > while being frankly obnoxious

Try reading some time. It's pretty great.

> the Ctrl+C handling is just utterly broken

That's your terminal, not parallel.

> and all of the delays

Your terminal is adding the delay, not parallel. parallel is just warning you about your broken terminal. You're shooting the messenger here.

Post reply on HN