Hinted Networks. - Joel Lamy-Poirier; Anqi Xu (CoRR, 2018)

Layered gradient accumulation and modular pipeline parallelism: fast and efficient training of large language models. - Joel Lamy-Poirier (CoRR, 2021)

Breadth-First Pipeline Parallelism. - Joel Lamy-Poirier (MLSys, 2023)

SantaCoder: don't reach for the stars! - Loubna Ben Allal; Raymond Li; Denis Kocetkov; Chenghao Mou; Christopher Akiki; Carlos Muñoz Ferrandis; Niklas Muennighoff; Mayank Mishra; Alex Gu; Manan Dey; Logesh Kumar Umapathi; Carolyn Jane Anderson; et al. (CoRR, 2023)

StarCoder: may the source be with you! - Raymond Li; Loubna Ben Allal; Yangtian Zi; Niklas Muennighoff; Denis Kocetkov; Chenghao Mou; Marc Marone; Christopher Akiki; Jia Li 015; Jenny Chim; Qian Liu 012; Evgenii Zheltonozhskii; et al. (Trans. Mach. Learn. Res., 2023)

StarCoder 2 and The Stack v2: The Next Generation. - Anton Lozhkov; Raymond Li; Loubna Ben Allal; Federico Cassano; Joel Lamy-Poirier; Nouamane Tazi; Ao Tang; Dmytro Pykhtar; Jiawei Liu 004; Yuxiang Wei 003; Tianyang Liu 003; Max Tian; et al. (CoRR, 2024)

Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training. - Oleksiy Ostapenko; Charles Guille-Escuret; Luke Kumar; Max Tian; Denis Kocetkov; Gopeshh Subbaraj; Raymond Li; Joel Lamy-Poirier; Sébastien Paquet; Torsten Scholak (CoRR, 2025)