Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GNU Parallel Cheat Sheet [pdf]

gnu.org

Post reply on HN
Print view View original

31–40 of 64 posts

Re: GNU Parallel Cheat Sheet [pdf]

#31

Post by mechanical_jane »

bonoboTP wrote: post #11

Ah, the rare case of nagware in GNU. From the man page: "--citation Print the BibTeX entry for GNU parallel and silence citation notice. If it is impossible for you to run --bibtex you can use --will-cite. If you use --will-cite in scripts to be run by others you are making it harder for others to see the citation notice. The development of GNU parallel is indirectly financed through citations, so if your users do no…

Do you feel this is covered in the FAQ? http://git.savannah.gnu.org/cgit/parallel.git/plain/doc/cita...

Re: GNU Parallel Cheat Sheet [pdf]

#32

Post by rurban »

LeoPanthera wrote: post #28
bonoboTP wrote: post #11

Ah, the rare case of nagware in GNU. From the man page: "--citation Print the BibTeX entry for GNU parallel and silence citation notice. If it is impossible for you to run --bibtex you can use --will-cite. If you use --will-cite in scripts to be run by others you are making it harder for others to see the citation notice. The development of GNU parallel is indirectly financed through citations, so if your users do no…

It would be trivial to fork parallel. If people cared enough, a fork would appear and be adopted. That's the beauty of free software. If you don't like it you can change it.

It was already forked in its early perl days, and thus it's very hard to use it properly as build tool, as the non-GNU version has an entirely different argument syntax. eg on macOS or BSD. You really have to probe for the GNU version (the more popular and newer one, with this awkward citation and begging), but for longer tasks it speeds up processing immensely. There's no need for Hadoop when you can use parallel. I'm processing hundreds of log files in one of my build-steps (similar to pgo, profile guided optimization), and with parallel it needs 30s, without 3min. This makes a difference.

Re: GNU Parallel Cheat Sheet [pdf]

#33

Post by snazz »

mechanical_jane wrote: post #30
bonoboTP wrote: post #22

Earlier quoted context omitted.

They can do experimentations and distribute their work through their channels then. But GNU tools (and FOSS by extension) are so popular because of their no-nonsense philosophy of here it is do with it whatever you want. Run it anywhere and any way you please. Citing it or not is an issue of academic practice/considerations (whether its use was a significant part of the research etc.). Mandating it through nag messag…

Are your questions answered in the FAQ? http://git.savannah.gnu.org/cgit/parallel.git/plain/doc/cita...

The FAQ includes a lot of support for what seems like the wrong hill to die on.

Here’s my thought process:

- the GNU Parallel author(s) want/wants people to use and contribute to it.

- they think that most users are academics who write papers and that potential users will find the project after reading the citation, which may or may not be true

- they include a nagware message that “reminds” users to cite the software

- despite the message being controversial and being the subject of the #1 comment in an otherwise unrelated HN thread about the software in general, an FAQ is written to back up the existence of this message

This brings me to the question of whether the inclusion of this message acts more as a deterrent to potential contributors and users. I agree with the motivation, but the means feels petty and undercuts the original goal.

Re: GNU Parallel Cheat Sheet [pdf]

#34

Post by e12e »

gcommer wrote: post #10
mruts wrote: post #3

I've never used GNU Parallel. But could someone explain to me the value add vs GNU xargs -P/--max-procs? From the examples at the top, it seems like those could be achieved with xargs.

parallel is like xargs++; for simple cases it does the same thing as xargs, but it also has many more advanced features such as: - Splitting input lines into multiple fields and building more complex commands from them - Running jobs on remote nodes - Pausing/resuming batch jobs (--joblog) - ETA and progress bars - Passing data to programs on stdin and generally many, many other ways of distributing and collecting da…

OT: I got curious, and this also works:

PAGER="wc -l" man xargs

(although my man page for xargs is just 211 lines)

Re: GNU Parallel Cheat Sheet [pdf]

#35

Post by mechanical_jane »

snazz wrote: post #33
mechanical_jane wrote: post #30

Earlier quoted context omitted.

Are your questions answered in the FAQ? http://git.savannah.gnu.org/cgit/parallel.git/plain/doc/cita...

The FAQ includes a lot of support for what seems like the wrong hill to die on. Here’s my thought process: - the GNU Parallel author(s) want/wants people to use and contribute to it. - they think that most users are academics who write papers and that potential users will find the project after reading the citation, which may or may not be true - they include a nagware message that “reminds” users to cite the softwar…

I enjoy following your thought process. I cannot make that fit with the content of the FAQ:

"In other words: It is preferable having fewer users, who all know they should cite, over having many users, who do not know they should cite.

If the goal had been to get more users, then the license would have been public domain.

...

The citation notice is about (indirect) funding - nothing else."

Does that fit with your assumption that "the GNU Parallel author(s) want/wants people to use and contribute to it"?

Re: GNU Parallel Cheat Sheet [pdf]

#36

Post by srean »

mruts wrote: post #3

I've never used GNU Parallel. But could someone explain to me the value add vs GNU xargs -P/--max-procs? From the examples at the top, it seems like those could be achieved with xargs.

To get an exhaustive answer you really need to go through the command line options as described in its man pages. xargs is good for simple stuff as long as you avoid some gotchas,but this really does a whole lot more. A lot more than anyone can do justice in a comment. AWK with gnu parallel is surprisingly potent combination.

Re: GNU Parallel Cheat Sheet [pdf]

#37

Post by magissima »

e12e wrote: post #34
gcommer wrote: post #10

Earlier quoted context omitted.

parallel is like xargs++; for simple cases it does the same thing as xargs, but it also has many more advanced features such as: - Splitting input lines into multiple fields and building more complex commands from them - Running jobs on remote nodes - Pausing/resuming batch jobs (--joblog) - ETA and progress bars - Passing data to programs on stdin and generally many, many other ways of distributing and collecting da…

OT: I got curious, and this also works: PAGER="wc -l" man xargs (although my man page for xargs is just 211 lines)

Try changing your terminal's width and running it again ;)

Re: GNU Parallel Cheat Sheet [pdf]

#38

Post by srean »

bonoboTP wrote: post #24
hprotagonist wrote: post #13

Still often the simplest way to get parallel computation in python, sadly.

The multiprocessing module is pretty good in Python.

Not in my experience. Has edge cases, especially on windows. Its understandable though, if you look under the covers there is a huge amount of complexity there.

Re: GNU Parallel Cheat Sheet [pdf]

#39

Post by e12e »

magissima wrote: post #37
e12e wrote: post #34

Earlier quoted context omitted.

OT: I got curious, and this also works: PAGER="wc -l" man xargs (although my man page for xargs is just 211 lines)

Try changing your terminal's width and running it again ;)

Ah.

MANWIDTH=80 PAGER="wc -l" man xargs 292

Re: GNU Parallel Cheat Sheet [pdf]

#40

Post by snazz »

mechanical_jane wrote: post #35
snazz wrote: post #33

Earlier quoted context omitted.

The FAQ includes a lot of support for what seems like the wrong hill to die on. Here’s my thought process: - the GNU Parallel author(s) want/wants people to use and contribute to it. - they think that most users are academics who write papers and that potential users will find the project after reading the citation, which may or may not be true - they include a nagware message that “reminds” users to cite the softwar…

I enjoy following your thought process. I cannot make that fit with the content of the FAQ: "In other words: It is preferable having fewer users, who all know they should cite, over having many users, who do not know they should cite. If the goal had been to get more users, then the license would have been public domain. ... The citation notice is about (indirect) funding - nothing else." Does that fit with your assu…

What kind of “funding” is referenced in the FAQ? Is there some kind of organization that I am not aware of that pays the author(s) for citations in papers? How is the “long term survival” impacted by whether the author receives citations?

I’m confused as to how “[not including citations] would not have been sustainable in the long term” unless either citations become money at some point or the author is motivated sufficiently by citations to the extent that they would otherwise not work on the project.

If you are an author or are involved in the project, please know that this isn’t intended to be an attack, I’m just interested as to why a project would do something that seems counterintuitive (at least from my point of view).

Post reply on HN