Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I wrote an bash enumerator because I was sick of xargs

numerlab.org

Post reply on HN
Print view View original

11–20 of 202 posts

Re: I wrote an bash enumerator because I was sick of xargs

#11

Post by quotemstr »

tester457 wrote: post #9

On the topic of xargs replacements, I love gnu parallel. The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs. Parallel has an option for almost everything, it's almost too much. But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0] The gnu parallel book and reading mate…

I love GNU Parallel too. I love how it's free software and how I can patch out the obnoxious citation notice.

Re: I wrote an bash enumerator because I was sick of xargs

#12

Post by somat »

tester457 wrote: post #9

On the topic of xargs replacements, I love gnu parallel. The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs. Parallel has an option for almost everything, it's almost too much. But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0] The gnu parallel book and reading mate…

I have used echo as a sort of poor mans equivalent for a safe check of a pipeline, removing the echo when I felt the rest of the pipeline was working correctly.

  shell stuff | xargs -n 1 -I % echo real command and % args
I have also been known to write scripts where instead of executing the critical parts it prints them. Then a dry run is

  script
and the real run is

  script | sh

Re: I wrote an bash enumerator because I was sick of xargs

#13

Post by 0xbadcafebee »

somat wrote: post #10
ggm wrote: post #4

find -print0 | xargs -0 -I {} "the {} iterated command"

Xargs is fine, but openbsd has a -J option and every time I read the man page to figure out how to use it I read the -I and -J options and my brain glazes over. https://man.openbsd.org/xargs Other brain glazing obsd wierdness is it's two argument cd command, a cryptid I am unable to wrap my head around. https://man.openbsd.org/ksh#cd~2

I highly recommend memorizing the POSIX version of every *NIX command and sticking to them. 60% of the time, they work every time. (https://pubs.opengroup.org/onlinepubs/9799919799/idx/utiliti...)

Re: I wrote an bash enumerator because I was sick of xargs

#14

Post by dataflow »

tester457 wrote: post #9

On the topic of xargs replacements, I love gnu parallel. The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs. Parallel has an option for almost everything, it's almost too much. But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0] The gnu parallel book and reading mate…

Oh man, parallel is awful. I have to rant about this because I literally tried it again today morning.

Every single damn time I try parallel and decide to give it another chance, something ends up not working or causing a problem. I can never get it to just do what I want and get out of the way.

Today I foolishly thought maybe I was the one who was holding it wrong every single time in the past, so I copy pasted another command that was supposed to work, and thought surely this would be straightforward. Boy was I wrong. I got some manifesto about academic citations and plagiarism, which confused the hell out of me. After I wasted time trying to figure out how to turn off that nonsense, the app just hung there trying to figure out how long its command line can be? Literally doing nothing? What the hell? I killed it but then my terminal didn't close because every time I did this apparently some perl command was spawned in the background blocked on nothing. Why the hell was perl even relevant? Nothing I wrote used Perl. Just run the darn commands I asked in parallel, is that so hard?

Re: I wrote an bash enumerator because I was sick of xargs

#18

Post by 0xbadcafebee »

I wanted something simpler — one consistent way to iterate over anything.

That's not actually simpler though. Simple is removing everything unnecessary. You took commands which could already do what you wanted, and added an extra program which calls them in specific ways. This will add bugs and maintenance headaches, not be portable, etc. This is added complexity.

The reason you made this script is not because you wanted simpler, you wanted easier. There's nothing wrong with that, and I'll grant you it probably is, especially for those unaccustomed to these commands. But easier != simpler. Often you'll find that simple is hard and easy is complexity deferred.

Re: I wrote an bash enumerator because I was sick of xargs

#19

Post by ggm »

0xbadcafebee wrote: post #13
somat wrote: post #10

Earlier quoted context omitted.

Xargs is fine, but openbsd has a -J option and every time I read the man page to figure out how to use it I read the -I and -J options and my brain glazes over. https://man.openbsd.org/xargs Other brain glazing obsd wierdness is it's two argument cd command, a cryptid I am unable to wrap my head around. https://man.openbsd.org/ksh#cd~2

I highly recommend memorizing the POSIX version of every *NIX command and sticking to them. 60% of the time, they work every time. ( https://pubs.opengroup.org/onlinepubs/9799919799/idx/utiliti... )

BSD vs linux with a standards committee inbetween.

Re: I wrote an bash enumerator because I was sick of xargs

#20

Post by gumby »

em-bee wrote: post #15
ggm wrote: post #4

find -print0 | xargs -0 -I {} "the {} iterated command"

why would you ever pipe find into xargs instead of calling -exec? find -exec the '{}' iterated command ';'

Because xargs is faster. Exec will invoke the command once per matching file (which is sometimes what you want, of course)!

While xargs will accumulate a bunch of file names, then when it as n names will invoke the command with those names, while continuing to accumulate names until n is reached or the pipe closes.

The size of n depends on the system, but is usually at least a thousand.

Post reply on HN