41–50 of 76 posts
Earlier quoted context omitted.
I don't care what GNU thinks, but it's simply not scalable. Imagine a world where every utility has its own irritating nag message that needs to be turned off.
Don't need to imagine, I lived in that world where most utilities I used were shareware with a nag screen on startup...
If you want to execute commands on multiple remote hosts over SSH, just run a tmux session and launch a tab on each host in a loop and execute them. It's far easier to follow the output and individually deal with prompts.
True, but for a large number of hosts, going manually through prompts doesn't scale anymore. Sometimes you just want to check whether all hosts have the same configuration file, or something like that. That's where GNU Parallel can save your gluteus maximus: cat hosts.txt | parallel --quote --timeout=10 ssh {} 'echo {} $(md5sum ~/.config/file)' The above command would take a list of hostnames from the hosts.txt file,…
command_to_test && exit
in the swarm of tmux tabs and let only the anomalies stay open.
Earlier quoted context omitted.
I don't understand how GNU finds this acceptable...
I don't care what GNU thinks, but it's simply not scalable. Imagine a world where every utility has its own irritating nag message that needs to be turned off.
How so?
A lot of software requires you to configure it before the first run, and we regard that as scalable.
A lot of software requires you to pay for it before the first run (most Microsoft server software comes to mind), yet we regard that as scalable. You can also pay for gnu paralell: https://git.savannah.gnu.org/cgit/parallel.git/tree/doc/cita...
Is it because you insist that you get software for free (zero cost in gnu speak)? Because that is really not what the free software movement is all about.
Earlier quoted context omitted.
The comparison is not very fair to modern day xargs. `nproc` is a relatively standard utility (coreutils). So, xargs -P$(nproc) gets you core (or core-proportional) parallelism. Grouping output/Making a safe parallel grep is also easy-ish with `--process-slot-var=slot` and sending to `tmpOut.$slot`. Jobs on remote computers can be done similarly with any kind of `arrayVar[$slot]` setup where `arrayVar` has a bunch of…
Perl has retained stable backwards compatibility and no breaking changes for 20+ years. What's wrong with Perl?
but still miles better than other such languages, and esp. if it would have been written in C, just as the incompatible moreutils counterpart.
If you want to execute commands on multiple remote hosts over SSH, just run a tmux session and launch a tab on each host in a loop and execute them. It's far easier to follow the output and individually deal with prompts.
True, but for a large number of hosts, going manually through prompts doesn't scale anymore. Sometimes you just want to check whether all hosts have the same configuration file, or something like that. That's where GNU Parallel can save your gluteus maximus: cat hosts.txt | parallel --quote --timeout=10 ssh {} 'echo {} $(md5sum ~/.config/file)' The above command would take a list of hostnames from the hosts.txt file,…
cat hosts.txt | parallel --quote --timeout=10 ssh {} 'echo {} $(md5sum ~/.config/file)'
parallel --slf hosts.txt --timeout=10 --nonall --tag md5sum .config/fileIf you want to execute commands on multiple remote hosts over SSH, just run a tmux session and launch a tab on each host in a loop and execute them. It's far easier to follow the output and individually deal with prompts.
parallel --tmux ...Earlier quoted context omitted.
Perl has retained stable backwards compatibility and no breaking changes for 20+ years. What's wrong with Perl?
Lots of unstable breakages over the last year's. automake got broken by an unnecessary deprecation, signatures got broken, encodings, and dozens more. but still miles better than other such languages, and esp. if it would have been written in C, just as the incompatible moreutils counterpart.
parallel -k --tag --argsep -- {} echo ::: 1 -- parallel-*
It's nice, but the citation bit strikes me as very non-free: > If you use --will-cite in scripts to be run by others you are making it harder for others to see the citation notice. The development of GNU parallel is indirectly financed through citations, so if your users do not know they should cite then you are making it harder to finance development. However, if you pay 10000 EUR, you have done your part to finance…
I was trying to use parallel properly for the first time last week and the nag screen put me off and made me choose an alternative. I ended up with the moreutils one and it appears to work just as well for my simple needs.
Also: https://git.savannah.gnu.org/cgit/parallel.git/tree/doc/cita...
It's nice, but the citation bit strikes me as very non-free: > If you use --will-cite in scripts to be run by others you are making it harder for others to see the citation notice. The development of GNU parallel is indirectly financed through citations, so if your users do not know they should cite then you are making it harder to finance development. However, if you pay 10000 EUR, you have done your part to finance…
We may need a standard directory and behavior for naggy utilities of this type. Maybe something like: for util in /usr/annoying/bin/*; do touch ~/.$(basename $util)/stfu done Of course I don't like the idea of having to create a prefs file for every obnoxious utility, but that's the way parallel is currently operating. Maybe all such utilities could be required to use standard nag library with a global setting.
https://git.savannah.gnu.org/cgit/parallel.git/tree/doc/cita...
I think many free software developers would rejoice if you cracked that problem.
This can be a good entry into the "why" for many people that use xargs now - https://www.gnu.org/software/parallel/parallel_alternatives.... It has a lot of features that can feel excessive at first glance but if you have felt some pain in building jobs, most of it is pretty sensible and much better than rollyourown.
The comparison is not very fair to modern day xargs. `nproc` is a relatively standard utility (coreutils). So, xargs -P$(nproc) gets you core (or core-proportional) parallelism. Grouping output/Making a safe parallel grep is also easy-ish with `--process-slot-var=slot` and sending to `tmpOut.$slot`. Jobs on remote computers can be done similarly with any kind of `arrayVar[$slot]` setup where `arrayVar` has a bunch of…
I am curious how you come to that conclusion.
> `nproc` is a relatively standard utility (coreutils). So, xargs -P$(nproc) gets you core (or core-proportional) parallelism.
I follow you on this point. A bit harder on remote systems, but definitely doable.
> Grouping output/Making a safe parallel grep is also easy-ish with `--process-slot-var=slot` and sending to `tmpOut.$slot`.
I tried spending 5 minutes on coding this, but the details seem to be very hard to get right: composed commands, grouping stderr, combined with not leaving tmp files behind if killed and allowing for the total output to be bigger than the free space on /tmp. I could not do it.
Could you consider spending 5 minutes on showing in code how you would do it?
> Jobs on remote computers can be done similarly with any kind of `arrayVar[$slot]` setup where `arrayVar` has a bunch of `ssh` targets, possibly duplicates if you want to run >1 job per host. (In pure POSIX sh you could use eval and $1, $2 positional args with shell arithmetic..)
This one seemed even harder to me: It was completely unclear how you would make sure that a given number of jobs were constantly running. And how would you need to quote data, so an eval would not cause "foo space space bar" turn into "foo space bar". And how you would kill remote jobs, if the local script was killed.
If you believe this is simple, could you spend 5 minutes on showing the rest of us how you would do it in actual working code? Because it seems the devil is really in the detail.
> Last I looked at the source for GNU parallel it looked like mountains upon mountains of Perl I would rather not depend upon, personally, but to each his own.
Personally, I would take production tested code over home-made untested code any day - no matter the language in which it was written.