I would like to contribute my computing resources.
Is there a SETI-like project to train LLM on libgen, scihub and the likes?
1–9 of 9 posts
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#2By "SETI" I assume you mean the SETI@Home distributed computing project.
There's a two-way market where you can rent out your GPU here: https://vast.ai/ If you want to donate your GPU you could set your price to $0.00 I guess.
Yandex and Hugging Face trained a couple nets using distributed GPUs. Their lessons learned paper is here: https://arxiv.org/abs/2106.10207 Takeaway: using consumer GPUs on the other end of consumer internet connections means you end up very I/O limited. 50 megabit upload on a cable modem doesn't compare well with 50000 megabit NVswitch links in a proper GPU cluster. As far as I can tell they are no longer doing distributed training.
And it's widely assumed the big nets are trained on copyrighted material through libgen and scihub. So if your question is "is there an open source net that's trained on pirated books" the answer is "Yes, and it's called Llama/Mistral/Falcon etc etc"
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#3I believe most modern LLMs already use them.
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#4it's much more thrilling to walk into a bookstore and steal a book the old fashioned way
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#5There are a lot of scanned books on the Internet Archive, though the OCR quality is pretty bad, and now there are the possible copyright issues of training on any still-in-copyright works.
I'd expect any SETI-like project on libgen/sci-hub would have to be somewhat hush hush. It wouldn't surprise me if it's already being done by somebody evil, who will use the results for their own nefarious purposes and not release them.
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#6it's much more thrilling to walk into a bookstore and steal a book the old fashioned way
When you steal a book from a bookstore, that copy of the book disappears. The bookstore no longer has it.
When you download a file from libgen, does libgen no longer have that file? Does the author lose their own copy of their book?
What exactly are you stealing (making you have something and depriving someone else of the thing).
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#7we're working on distributed computing for training LLMs with Socket runtime (https://github.com/socketsupply/socket).
Our ultimate goal is to provide a web browser-compatible environment (for quickly building UIs) with a state of the art P2P protocol — A DTN design that is delay, fault, and partition tolerant, with comprehensive and complete NAT traversal, without any server or cloud infrastructure requirements from the user — lowering the technical and monetary barriers for distributing and parallelizing work beyond the cloud. Our native plugin system was designed to easily support projects like llama.cpp which makes it simple to load any LLM and expose JavaScript APIs over it.
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#8There is Petals [0] which is basically what you're asking about: "Run large language models at home, BitTorrent‑style."
It doesn't include training from scratch, but it supports fine-tuning and inference powered by a swarm of GPUs.
Re: Is there a SETI-like project to train LLM on libgen, scihub and the likes?
#9From the title, I thought you meant is there a project that tries to search for intelligence within LLM trained on books