Decoupled DiLoCo: Resilient, Distributed AI Training at Scale
deepmind.google