The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset. - Hugo Laurençon; Lucile Saulnier; Thomas Wang; Christopher Akiki; Albert Villanova del Moral; Teven Le Scao; Leandro von Werra; Chenghao Mou; Eduardo González Ponferrada; Huu Nguyen; Jörg Frohberg; Mario Sasko; et al. (NeurIPS, 2022)

SantaCoder: don't reach for the stars! - Loubna Ben Allal; Raymond Li; Denis Kocetkov; Chenghao Mou; Christopher Akiki; Carlos Muñoz Ferrandis; Niklas Muennighoff; Mayank Mishra; Alex Gu; Manan Dey; Logesh Kumar Umapathi; Carolyn Jane Anderson; et al. (CoRR, 2023)

The BigCode Project Governance Card. - Sean Hughes; Harm de Vries; Jennifer Robinson; Carlos Muñoz Ferrandis; Loubna Ben Allal; Leandro von Werra; Jennifer Ding; Sébastien Paquet; Yacine Jernite (CoRR, 2023)

The Stack: 3 TB of permissively licensed source code. - Denis Kocetkov; Raymond Li; Loubna Ben Allal; Jia Li 015; Chenghao Mou; Yacine Jernite; Margaret Mitchell; Carlos Muñoz Ferrandis; Sean Hughes; Thomas Wolf 008; Dzmitry Bahdanau; Leandro von Werra; et al. (Trans. Mach. Learn. Res., 2023)

StarCoder: may the source be with you! - Raymond Li; Loubna Ben Allal; Yangtian Zi; Niklas Muennighoff; Denis Kocetkov; Chenghao Mou; Marc Marone; Christopher Akiki; Jia Li 015; Jenny Chim; Qian Liu 012; Evgenii Zheltonozhskii; et al. (Trans. Mach. Learn. Res., 2023)

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations. - Alexander Hägele; Elie Bakouch; Atli Kosson; Loubna Ben Allal; Leandro von Werra; Martin Jaggi (NeurIPS, 2024)

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale. - Guilherme Penedo; Hynek Kydlícek; Loubna Ben Allal; Anton Lozhkov; Margaret Mitchell; Colin A. Raffel; Leandro von Werra; Thomas Wolf 008 (NeurIPS, 2024)

StarCoder 2 and The Stack v2: The Next Generation. - Anton Lozhkov; Raymond Li; Loubna Ben Allal; Federico Cassano; Joel Lamy-Poirier; Nouamane Tazi; Ao Tang; Dmytro Pykhtar; Jiawei Liu 004; Yuxiang Wei 003; Tianyang Liu 003; Max Tian; et al. (CoRR, 2024)

SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model. - Loubna Ben Allal; Anton Lozhkov; Elie Bakouch; Gabriel Martín Blázquez; Guilherme Penedo; Lewis Tunstall; Andrés Marafioti; Hynek Kydlícek; Agustín Piqueres Lajarín; Vaibhav Srivastav; Joshua Lochner; Caleb Fahlgren; et al. (CoRR, 2025)

SmolVLM: Redefining small and efficient multimodal models. - Andrés Marafioti; Orr Zohar; Miquel Farré; Merve Noyan; Elie Bakouch; Pedro Cuenca 002; Cyril Zakka; Loubna Ben Allal; Anton Lozhkov; Nouamane Tazi; Vaibhav Srivastav; Joshua Lochner; et al. (CoRR, 2025)

The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text. - Nikhil Kandpal; Brian Lester; Colin Raffel; Sebastian Majstorovic; Stella Biderman; Baber Abbasi; Luca Soldaini; Enrico Shippole; A. Feder Cooper; Aviya Skowron; John Kirchenbauer; Shayne Longpre; et al. (CoRR, 2025)