Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch. - Arthur Douillard; Yanislav Donchev; Keith Rush; Satyen Kale; Zachary Charles; Zachary Garrett; Gabriel Teston; Dave Lacey; Ross McIlroy; Jiajun Shen; Alexandre Ramé; Arthur Szlam; et al. (CoRR, 2025)

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo. - Zachary Charles; Gabriel Teston; Lucio Dery; Keith Rush; Nova Fallen; Zachary Garrett; Arthur Szlam; Arthur Douillard (CoRR, 2025)