Will it be possible to do LLM Pool Training?

Inspired by Crypto Pool Mining: would it be possible to train LLMs as a pool of volunteers? I feel like that would be a good way to have frontier models that are open and don’t rely on big corporations with data centers to train and release them…

Datacenter-scale GPGPU compute is extremely efficient compared to what you would achieve with a network of independently operated distributed heterogeneous devices spread across the world. Just imagine the overhead for ensuring that a node doesn't poison the gradients. And that's just one very small part of the problem.