Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GNU Parallel, where have you been all my life?

alexplescan.com

Post reply on HN
Print view View original

181–190 of 277 posts

Re: GNU Parallel, where have you been all my life?

#181

Post by dahart »

memefrog wrote: post #174
dahart wrote: post #150

Earlier quoted context omitted.

Yes kind-of… in this case Parallel’s notice is not a modification of the license at all, and Stallman is the person who ruled on this question and confirmed this to be true. The GPL doesn’t prevent authors from including a notice, and having a notice doesn’t conflict with the terms of the GPL. I feel like the whole problem here is that the legality of Parallel’s notice, and the separation of the notice from the GPL,…

>in this case Parallel’s notice is not a modification of the license at all This is a question of law that only a court can answer.

That’s not true. The language of Parallel’s citation notice, while confusing to some users, does not impose any legal requirements and is not part of the license. Neither the notice nor the license claim otherwise. RMS, and more importantly, Ole Tange, agree that Parallel’s notice is not legally binding, and intended to write it that way, and there is a publicly visible history of this intention and agreement.

Re: GNU Parallel, where have you been all my life?

#182

Post by akerl_ »

aidenn0 wrote: post #168
akerl_ wrote: post #78

Earlier quoted context omitted.

Can you back that up? If I put the GPL in my software and add a file next to it that says "Also you can't use this software if you make more than $100k/year", I've pretty clearly added an additional clause that's incompatible with the GPL.

If I say "Please don't use this software if you make more than $100k/year" I haven't added an additional clause, just communicated a desire. I'm as annoyed by parallel's citation nag (particularly since I don't plan on ever publishing a scientific paper), but it does not impose extra requirements.

I think I agree with you: if you put in requests that have no mandate or attachment to "or this changes your rights under the license", there's no issue.

But I was responding to the comment upthread "You can add any message you want into your GPL program". If you add a message that says a user of the software must do something / must agree to additional terms / etc, that additional text is not compatible with the GPL. I'm not a lawyer, so I have no idea whether the result would be that the restriction doesn't count and the software is GPL'd, or that the software isn't viably GPL'd because the GPL+clause isn't a valid license for somebody to use.

Re: GNU Parallel, where have you been all my life?

#183

Post by tomberek »

toastal wrote: post #96

I use this with Nix all the time. Great utility.

Especially with the remote SSH features one needs a way to ensure everything needed for your process is on the target machine; Nix makes this easy.

Nix + GDAL + GNUParallel + autoscaling groups === massive geospatial data processing pipeline

Re: GNU Parallel, where have you been all my life?

#184

Post by brabel »

jooz wrote: post #179

I try to use it last week to run 10 instances of curl against a webserver. I was expecting something simple as 'parallel -j10 curl https://whatever ' but couldnt find the right syntax in less time that took me to prepare a dirty shell script that did the same.

If you want a simple load testing tool for HTTP, use wrk2[1].

    wrk -t2 -c100 -d30s -R2000 http://127.0.0.1:8080/index.html
> This runs a benchmark for 30 seconds, using 2 threads, keeping 100 HTTP connections open, and a constant throughput of 2000 requests per second (total, across all connections combined).

Some distros include `ab`[2] which is also good, but wrk2 improves on it (and on wrk version 1) in multiple ways, so that's what I use myself.

[1] https://github.com/giltene/wrk2

[2] https://httpd.apache.org/docs/2.2/programs/ab.html

Re: GNU Parallel, where have you been all my life?

#185

Post by bee_rider »

foldr wrote: post #101
dahart wrote: post #81

Earlier quoted context omitted.

> Is the author still adding the "cite me or pay 10000€" notice to the output? And calling that GPL? IIRC the citation notice was cleared by Stallman as GPL compatible. I’d be surprised if anyone’s paid, I assumed that’s rhetoric to imply the value of a citation, or lack of citation, for anyone publishing scientific works. > These are just some of the reasons I’ll never use parallel. Hey I’ve actually ranted on HN be…

> it’s not tradition to write citations for tools used to conduct research Academics seem to have a very blinkered attitude to this. I wrote some software that was popular for a while in a niche field, and people were forever asking me to waste my time by 'publishing' the manual in some pointless journal so that they could cite something and give me credit. Writing useful software counts for less in that world than p…

That is a weird request, a source doesn’t need to be published in a journal to be cite-able. On the other hand, if you put a bibtex snippet on your site that indicates how you’d like to be cited, that is super helpful.

Re: GNU Parallel, where have you been all my life?

#186

Post by cb321 »

titzer wrote: post #146

I didn't know about this, and reading through the comments, I found out that xargs can also do batching and parallelism (nice!). However, it appears that if you pipe the output of an xargs-parallel command into another utility, it jumbles the output of the multiple subprocesses, whereas GNU parallel does not. I was a little put off by the annoying/scary citation issue mentioned by another commenter, so I am not sure…

You can probably do something that creates as many FIFOs as you have parallelism and just be careful about emitting whole records like https://github.com/c-blake/bu/blob/main/doc/funnel.md . That one's Nim, but the meat is only like 50 lines and easily ported to C like your progress tool. ( EDIT: and it will also probably be drastically lower overhead than `parallel` which has over 70X worse time overhead and 10X the RAM overhead of tools written in fast, native-compiled languages: https://github.com/c-blake/bu/blob/main/tests/strench.sh )

Re: GNU Parallel, where have you been all my life?

#187

Post by dahart »

memefrog wrote: post #173
dahart wrote: post #165

Earlier quoted context omitted.

I’m not sure I understand what you’re asking. Why is the intention of the author of the license in question? In this case, Stallman simply clarified that Parallel’s notice did not count as a legal requirement and does not conflict with the GPL. His opinion wasn’t necessary, but since he wrote the license, it is authoritative. In this case, the question wasn’t brought to court, it was simply a clarifying discussion, a…

>His opinion wasn’t necessary, but since he wrote the license, it is authoritative. No it isn't. Licences, like most legal documents, are construed objectively. The subjective intention of the author is totally irrelevant to the meaning.

You might have misunderstood what I said. It’s not up for debate whether RMS’s opinions or intent on the GPL have affected industry practice; that’s a fact of history. His statements on the GPL are authoritative in the sense that they may have prevented the courts from examining this question.

Re: GNU Parallel, where have you been all my life?

#188

Post by shakow »

Eiim wrote: post #121
dahart wrote: post #81

Earlier quoted context omitted.

> Is the author still adding the "cite me or pay 10000€" notice to the output? And calling that GPL? IIRC the citation notice was cleared by Stallman as GPL compatible. I’d be surprised if anyone’s paid, I assumed that’s rhetoric to imply the value of a citation, or lack of citation, for anyone publishing scientific works. > These are just some of the reasons I’ll never use parallel. Hey I’ve actually ranted on HN be…

Citing tools is maybe not tradition, but it's gaining popularity. R, for example, has 320,000 cites. ( https://scholar.google.com/scholar?cites=1605575084878280829... )

> R, for example, has 320,000

Which is, to be frank, ridiculous compared to the number of papers it enabled.

Re: GNU Parallel, where have you been all my life?

#189

Post by shakow »

rlpb wrote: post #128
dahart wrote: post #81

Earlier quoted context omitted.

> Is the author still adding the "cite me or pay 10000€" notice to the output? And calling that GPL? IIRC the citation notice was cleared by Stallman as GPL compatible. I’d be surprised if anyone’s paid, I assumed that’s rhetoric to imply the value of a citation, or lack of citation, for anyone publishing scientific works. > These are just some of the reasons I’ll never use parallel. Hey I’ve actually ranted on HN be…

> and the citation notice is a one-time thing you can silence permanently This doesn't scale. Imagine if all the software you used nagged you and had their own individual methods to silence them. I don't think this would be reasonable. What makes this particular software so special?

> This doesn't scale.

--will-cite

Re: GNU Parallel, where have you been all my life?

#190

Post by colejohnson66 »

lysium wrote: post #147
coliveira wrote: post #127

Earlier quoted context omitted.

What you mention is the main reason why shell script is not a decent language to write long programs. It is full of inconsistencies, and since it depends on other commands, you have to learn the quirks of each command you use. Moreover, good luck if you need to debug this. Shell should only be used for small scripts that are easy to debug.

Do you recommend any good alternative when your shell program gets too large? Honest question, as I’m struggling to leave the shell environment once the program gets too large. I could use Perl, but $? and the likes get quickly out of hand. Python’s support for pipes was difficult last time I used it, but that may have changed. What would you recommend?

Powershell uses proper objects instead of stringly nonsense.
Post reply on HN