Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I wrote an bash enumerator because I was sick of xargs

numerlab.org

Post reply on HN
Print view View original

91–100 of 202 posts

Re: I wrote an bash enumerator because I was sick of xargs

#91

Post by zxexz »

I don’t want to be a downer here, but the structure of this repo and the verbosity+language of the docs feel 80% vibecoded. Not that that’s wrong; I just feel kinda gullible for even clicking on this.

One uses xargs or parallel only a few times before they remember some of the quirks that but them. And then they become cautious. And then it’s muscle memory. And if it’s not an often occurrence, they learn to check the man page.

Anyways, in a world of “vibecoding” why add another “tool” to the mix when the LLMs have been trained on all our stackoverflow-posted grievances to begin with?

Re: I wrote an bash enumerator because I was sick of xargs

#92

Post by dundarious »

If you want a shell to interact with the results, you can of course just use a (sub)shell.

    ls -1 ./*.sh | xargs -rd\\n sh -c 'for i in "$@" ; do ... ; done' sh
1. not strictly necessary to use -1 as I believe all common ls detect !isatty(stdout) and produce line-by-line output anyway.

2. xargs -r just doesn't run the command if there's no input, also not strictly necessary but I'm addicted to using it because it's the sensible default to me.

3. xargs -d\\n makes it collect fields as full lines, which is what you typically want, unless you're able to generate NULs.

4. use whatever shell you want of course, but I don't use bashisms, etc., by default, /bin/sh is fine for me, even if it's dash.

5. the trailing "sh" at the end is due to a quirk of `sh -c` usage, where $0 is the first non option argument, so `printf %s\\n 1 2 3 | xargs -rd\\n sh -c 'for i in "$@" ; do printf "%s " "$i" ; done ; echo'` (note the lack of trailing "sh") would only print "2 3 " as $0 is not included in "$@" ($0 is 1, $1 is 2, $2 is 3). It's very easy to just always give the shell name itself manually as $0 instead of trying to ingest "$0" into your logic.

Of course, you could `find . -maxdepth 1 -type f -name '*.sh' -print0 | xargs -r0 ...` instead, depending on what you're up to, that may be the easiest. It's definitely the simplest -- as long as your xargs has -0 support.

Re: I wrote an bash enumerator because I was sick of xargs

#93

Post by IdiotSavage »

In this thread: bash experts with arcane knowledge, unintentionally demonstrating how awful bash is.

The obvious solution would be to use something more sane, like PowerShell or nushell, but instead old experts will always defend the skills they have honed for years, while criticizing anything that's different.

Re: I wrote an bash enumerator because I was sick of xargs

#94

Post by t43562 »

eqvinox wrote: post #7

Not sure I see the point of this. Remembering a bunch of options on one tool is no better than a bunch of tools/constructs? Especially if the latter are useful elsewhere. And it can't be used for scripts unless you start shipping it alongside, which… nah. Just go with foo | while read X; do bar "$X"; done

While FTW! I just wondered whether it could be simplified because I get a bit tired of typing out my own belt and braces version of it:

  { # code that generates one item/filename per line of output }  | { while read ITEM; do # do something with "$ITEM"; done; }
So I just tried this and it seems to work for a trivial case although you need 1 escape:

  enum() {
    local source=$1; shift; local action=$1; shift  
    { eval "$source" ; } | { while read ITEM; do eval "$action"; done; }
  }

  $ enum "find /tmp" "echo found: \$ITEM"

  found: /tmp/.XIM-unix
  found: /tmp/.ICE-unix

Re: I wrote an bash enumerator because I was sick of xargs

#95

Post by brokenmachine »

dataflow wrote: post #14
tester457 wrote: post #9

On the topic of xargs replacements, I love gnu parallel. The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs. Parallel has an option for almost everything, it's almost too much. But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0] The gnu parallel book and reading mate…

Oh man, parallel is awful. I have to rant about this because I literally tried it again today morning. Every single damn time I try parallel and decide to give it another chance, something ends up not working or causing a problem. I can never get it to just do what I want and get out of the way. Today I foolishly thought maybe I was the one who was holding it wrong every single time in the past, so I copy pasted anot…

I'm not new to command line tools, but I somehow managed to delete ~10k pictures while trying to use parallel to resize them.

I forget the details but it was some kind of surprisingly weird foot-gun behavior.

Luckily I had a backup, but it really has made me scared to try using parallel again.

Re: I wrote an bash enumerator because I was sick of xargs

#97

Post by [deleted] »

dataflow wrote: post #80
nvme0n1p1 wrote: post #33

Earlier quoted context omitted.

> the app just hung there trying to figure out how long its command line can be? That's common on linux. Many tools read from stdin if a file path isn't given: cat, xargs, base64, cksum, etc. The citation thing is a little silly, I'll give you that one.

> That's common on linux. Many tools read from stdin if a file path isn't given: cat, xargs, base64, cksum, etc. No. I did pipe to stdin. It's not my first time using Linux... Here's a command line I ran right now , and the output I see: $ echo "http://www.example.com" | parallel -k -j 8 curl -s "{}" Academic tradition requires you to cite works you base your article on. [...more nonsense...] To silence this citation…

[deleted]

Re: I wrote an bash enumerator because I was sick of xargs

#98

Post by kfsone »

Careful - you start from the POSIX terminal spec, you think maybe it would be interesting having objects instead of raw text streams, and next thing you've reinvented powershell...

(*35 years living and loving sh/ksh/bash/dash etc, only tried pwsh so I could write some comparisons and slag off MS a bit; now it's my default shell on everything)

Re: I wrote an bash enumerator because I was sick of xargs

#99

Post by account42 »

> Consistent syntax — same {} placeholder for files, lines, ranges, or lists

Inconsistent syntax native bash methods so its an additional syntax to learn.

> Template mode — single-quoted commands work as shell templates: enumerate -f '*' -- 'cat {} | head -5'

Passing commands a a single string is BAD. Now you have to think about escaping and quoting. What does {} get replaced with if the enumerant contains unsafe characters? Can it be used as part of a larger argument or only on its own? Who knows, it's not bash. Compared to a bash loop its also always a subshell with all the implications that has - even find can be piped into a normal bash loop.

> Filters — --include and --exclude with glob patterns

A fraction of what find or native loops provide. And since this can't replace them in general enumerate is an additional thing to learn on top.

> Extensible — drop a file in lib/enumerators/ to add custom sources

To extend bash loops you don't even need root access, you just add the code to the loop.

Re: I wrote an bash enumerator because I was sick of xargs

#100

Post by lobofta »

IdiotSavage wrote: post #93

In this thread: bash experts with arcane knowledge, unintentionally demonstrating how awful bash is. The obvious solution would be to use something more sane, like PowerShell or nushell, but instead old experts will always defend the skills they have honed for years, while criticizing anything that's different.

Switched to Nushell and I am not looking back. I don't see any major reason why we should keep dragging Bash into the twenty first century. Nushell is the first time I feel like I can write complex systems operations in a shell without having to spend either a ton of time in the docs or being at the mercy of an LLM. It is godsend.
Post reply on HN