One thing I've used parallel before is to add the ability to add straightforward retry mechanisms, and it was great! https://www.jvt.me/posts/2022/04/28/shell-queue/
GNU Parallel, where have you been all my life?
91–100 of 277 posts
Re: GNU Parallel, where have you been all my life?
#92Looks like a great tool!
Thanks for the link to the book: https://zenodo.org/record/1146014
Re: GNU Parallel, where have you been all my life?
#93I'm using task spooler a lot for parallel background processing. What I like the most it the ability to add further tasks to the queue after processing has already started. https://manpages.ubuntu.com/manpages/xenial/man1/tsp.1.html
from that man page, there is a name clash with "ts" from moreutils
I installed task-spooler just now, because I’ve been wanting something like this for a long time.
It looks like the actual name of the task-spooler command on Debian after install is “tsp”, not “ts”. So no collision :)
Now it just remains to be seen if the package by default allows the tasks to continue to run after I log out, or if systemd will annoyingly kill the tasks after I disconnect from ssh the same way systemd annoyingly kills my “screen” sessions when I disconnect ssh, and there is some cumbersome thing you have to do on each of your systems to have systemd not kill “screen” :(
Re: GNU Parallel, where have you been all my life?
#94Is the author still adding the "cite me or pay 10000€" notice to the output? And calling that GPL? And still answering every xargs Stackoverflow question with "you should use GNU Parallel" instead of answering the question? That really gets old quickly when googling for xarg answers. These are just some of the reasons I'll never use parallel. xargs is perfectly fine for most usecases, and it can do everything I need…
Here it’s discussed on the gnu mailing list in 2013:
https://lists.gnu.org/archive/html/parallel/2013-11/msg00006...
Re: GNU Parallel, where have you been all my life?
#95Earlier quoted context omitted.
> Where you get the "or pay 10000€" part from? Most likely from the manpage: If you use --will-cite in scripts to be run by others you are making it harder for others to see the citation notice. The development of GNU parallel is indirectly financed through citations, so if your users do not know they should cite then you are making it harder to finance development. However, if you pay 10000 EUR, you have done your p…
Again, it's not part of the license nor are you forced to select between "cite GNU parallels or pay 10000 EUR". You're free to use it however you want since the software is GPL, including ignoring any of the output from using the tool if you so chose to.
I answered where that comes from, not if it is part of the license (it isn't).
Re: GNU Parallel, where have you been all my life?
#96I use this with Nix all the time. Great utility.
Re: GNU Parallel, where have you been all my life?
#97Is the author still adding the "cite me or pay 10000€" notice to the output? And calling that GPL? And still answering every xargs Stackoverflow question with "you should use GNU Parallel" instead of answering the question? That really gets old quickly when googling for xarg answers. These are just some of the reasons I'll never use parallel. xargs is perfectly fine for most usecases, and it can do everything I need…
> Is the author still adding the "cite me or pay 10000€" notice to the output? And calling that GPL? IIRC the citation notice was cleared by Stallman as GPL compatible. I’d be surprised if anyone’s paid, I assumed that’s rhetoric to imply the value of a citation, or lack of citation, for anyone publishing scientific works. > These are just some of the reasons I’ll never use parallel. Hey I’ve actually ranted on HN be…
> it’s not tradition to write citations for tools used to conduct research.
Thia is true, but it also makes it very hard for academics and PhD students who mainly write software over papers. They get no citations and eventually have to leave academia.
If we had a better practice of citing central software we use - at least the academic software that wants to be cited - we could have a more flourishing ecosystem of such software funded by the universities.
Re: GNU Parallel, where have you been all my life?
#98I have wanted to parallelize my .zshrc file for a while – all those environment setup scripts for nvm, pyenv, starship, etc really makes the startup time noticably slow. Does anyone know how to do this?
Re: GNU Parallel, where have you been all my life?
#99Another reminder that you shouldn't use Bash to write scripts. E.g. in Python this would all be very easy to do. Just start a bunch of threads and e.g. invoke subprocess.run() from them.
subprocess.run is likely to be significantly slower than a low-level dedicated utility like parallel, and adding a lot of flakyness and overhead. I'm a big pythonaro but one should always use the best tool for the job.
Re: GNU Parallel, where have you been all my life?
#100Earlier quoted context omitted.
> Is the author still adding the "cite me or pay 10000€" notice to the output? And calling that GPL? IIRC the citation notice was cleared by Stallman as GPL compatible. I’d be surprised if anyone’s paid, I assumed that’s rhetoric to imply the value of a citation, or lack of citation, for anyone publishing scientific works. > These are just some of the reasons I’ll never use parallel. Hey I’ve actually ranted on HN be…
> it’s not tradition to write citations for tools used to conduct research. Thia is true, but it also makes it very hard for academics and PhD students who mainly write software over papers. They get no citations and eventually have to leave academia. If we had a better practice of citing central software we use - at least the academic software that wants to be cited - we could have a more flourishing ecosystem of su…
I can understand that, and I can understand Ole’s request for citations - Googling him it looks like he is (or has been) employed by a university.
The good news is that the new ‘tradition’ these days for academic software is to open-source all the software written for a paper or academic project, so practically everything done is visible on github & arxiv.