Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Design of GNU Parallel (2015)

gnu.org

Post reply on HN
Print view View original

21–30 of 75 posts

Re: Design of GNU Parallel (2015)

#21

Post by anthk »

InfamousRece wrote: post #18
anthk wrote: post #14

Parallel, vidir to edit directories with nvi/vim, moreutils, detox to scrap out any non-typeable char... These are a must have today.

moreutils have its own parallel utility that I actually prefer to Gnu parallel.

No problems, they almost work the same I think. Oh, another bunch of small tools to help yourself:

    - entr. It runs a command on file/directory changes.
    - spt. Simple pomodoro technique. A good timer to help yourself to work and take rests.
    - herbe. It works great as a notifier for spt. Add "play" from sox to write a script to both
   notify and play a sound in parallel.
    - sox/ffmpeg/imagemagick. Audio, video and image production and conversion on the CLI. A must have.
    - catdoc/antiword/odt2txt/wordgrinder+sc-im+gnuplot. Word/Excel/Libreoffice files reading and editing on the terminal. Gnuplot with help with sc-im. This can be a beast over SSH. With Gnuplot compiled with sixel support (and XTerm) you can do magic.
- iomenu

     - cat bookmarks.txt | iomenu | xargs firefox. Pick from a list of items (one per line) and choose. I think it has fuzzy-finding matches.
I have several more. Simple battery meter (sbm), grabc to grab a color from the screen, pointtools+catpoint to do "presentations" over a terminal, nncp-go+yggdrasil for ad-hoc networking and secure encrypted backups between devices...

Re: Design of GNU Parallel (2015)

#22

Post by vrnvu »

michalc wrote: post #9

I've never been sure if it's too much of a hack, but I've used GNU parallel in Docker containers as a quick and easy way of getting multiple processes running for web applications. And with the `--halt now,done=1` option (that I think is relatively recent?) it means that if any of the parallel processes exit, parallel would exit itself, the whole container will shut down, and external orchestration would start anothe…

Cool tip thanks for sharing! I love letting process crash *when possible* on failures so the OS restart them for me versus trying to handle it manually at process level.

Re: Design of GNU Parallel (2015)

#23

Post by mhh__ »

a2800276 wrote: post #7

Wait what: `parallel` is a Perl script!? [1] I would have thought it's black magic with assembler optimisations for MIPS and special considerations for HP-UX... This is such a lovely and interesting writeup, it's wonderful that people take their time to share so generously! [1] : an 11k loc petal script, you can read along here: https://github.com/gitGNU/gnu_parallel/blob/master/src/paral...

assembly optimizations for starting processes?

Re: Design of GNU Parallel (2015)

#24

Post by ajsnigrutin »

ketzu wrote: post #16

This was quite interesting to look through! Perl 5.8.0 is over 20 years old ( https://dev.perl.org/perl5/news/2002/07/18/580ann/ ) while centOS 3.9 was released in 2007! At the same time it seems not-that-old and ancient. My personal anecdote with gnu parallel was running into it while working in academia. It worked well and saved me some time, but I felt that it was unreasonable of a tool to ask for a citation to pa…

Yep, that's the great thing about perl... take a 20 year old script and it still works today. In comparison, if they used python, they'd be using python 2.2.

Re: Design of GNU Parallel (2015)

#27

Post by krylon »

NortySpock wrote: post #4

I found GNU parallel useful when I wanted to queue up transcoding of flac files to mp3 on my Raspberry Pi. A few ffmpeg flags plus a list of files meant I could easily just saturate one job per core with a one-line bash command.

I like to use ts(1) for that. http://vicerveza.homeunix.net/~viric/soft/ts/

Re: Design of GNU Parallel (2015)

#28

Post by hexane360 »

RhysU wrote: post #19
chungy wrote: post #6

Earlier quoted context omitted.

Thankfully both Debian and Arch patch out the citation nonsense.

It is "nonsense" because...? A) You don't understand. Please read the "Citation notice" section in the article. B) You understand but don't use GNU Parallel. C) You understand and use GNU Parallel in a non-academic setting and find the hassle of supplying --no-notice to be onerous vs the effort to write/maintain your own tool. D) You understand and use GNU Parallel in an academic setting and have cited Ole or plan to…

It's nonsense because the standard in academic settings is to cite works which contribute scientifically to the current work, not merely utilities. If I publish a paper on a command line tool for parallel processing, inspired by features from GNU parallel, I would cite GNU parallel. But if I'm doing (for instance) computational biology work, I'm not going to cite: - the Linux kernel - Python - Matlab - GNU parallel - RFC 793 - Every other program I use

Asking for citations is fine. But GNU parallel wants to treat it like a requirement of using the software, without making it a condition of the copyright: "== Is the citation notice compatible with GPLv3? ==

Yes. The wording has been cleared by Richard M. Stallman to be compatible with GPLv3. This is because the citation notice is not part of the license, but part of academic tradition."

This is disingenuous, because citing every tool you use in preparing a scientific work is not part of academic tradition. And the statement that "If you pay 10000 EUR you should feel free to use GNU Parallel without citing." doesn't make any sense in the "academic tradition" framing. If Ole thinks citations are required by academic tradition, that shouldn't change if I pay him enough money.

"If you disagree with Richard M. Stallman's interpretation and feel the citation notice does not adhere to GPLv3, you should treat the software as if it is not available under GPLv3. And since GPLv3 is the only thing that would give you the right to change it, you would not be allowed to change the software.

In other words: If you want to remove the citation notice to make the software compliant with your interpretation of GPLv3, you first have to accept that the software is already compliant with GPLv3, because nothing else gives you the right to change it. And if you accept this, you do not need to change it to make it compliant."

And this is legal nonsense. If I release something under a license, and then break that license, that doesn't nullify the original license. Claiming otherwise would allow me to un-copyleft someone else's code.

Re: Design of GNU Parallel (2015)

#29

Post by remram »

mhh__ wrote: post #23
a2800276 wrote: post #7

Wait what: `parallel` is a Perl script!? [1] I would have thought it's black magic with assembler optimisations for MIPS and special considerations for HP-UX... This is such a lovely and interesting writeup, it's wonderful that people take their time to share so generously! [1] : an 11k loc petal script, you can read along here: https://github.com/gitGNU/gnu_parallel/blob/master/src/paral...

assembly optimizations for starting processes?

Maybe for reading the input, splitting it, and assembling the possibly-very-long argument lists passed to the processes.

Re: Design of GNU Parallel (2015)

#30

Post by BooneJS »

Before GNU Parallel I used to use Ruby's workers and job queue to keep ${N} cores busy with work. It sorta worked like GNU parallel but was quite basic. I've since switched to using GNU Parallel. Stable code I don't have to write doesn't have to be maintained... not to mention it has more features than I normally supported.
Post reply on HN