In-training Matrix Factorization for Parameter-frugal Neural Machine Translation. - Zachary Kaden; Teven Le Scao; Raphaël Olivier (CoRR, 2019)

Transformers: State-of-the-Art Natural Language Processing. - Thomas Wolf 008; Lysandre Debut; Victor Sanh; Julien Chaumond; Clement Delangue; Anthony Moi; Pierric Cistac; Tim Rault; Rémi Louf; Morgan Funtowicz; Joe Davison; Sam Shleifer; et al. (EMNLP (Demos), 2020)

Neural Differential Equations for Single Image Super-resolution. - Teven Le Scao (CoRR, 2020)

Datasets: A Community Library for Natural Language Processing. - Quentin Lhoest; Albert Villanova del Moral; Yacine Jernite; Abhishek Thakur; Patrick von Platen; Suraj Patil; Julien Chaumond; Mariama Drame; Julien Plu; Lewis Tunstall; Joe Davison; Mario Sasko; et al. (EMNLP (Demos), 2021)

How many data points is a prompt worth? - Teven Le Scao; Alexander M. Rush (NAACL-HLT, 2021)

Multitask Prompted Training Enables Zero-Shot Task Generalization. - Victor Sanh; Albert Webson; Colin Raffel; Stephen H. Bach; Lintang Sutawika; Zaid Alyafeai; Antoine Chaffin; Arnaud Stiegler; Teven Le Scao; Arun Raja; Manan Dey; M. Saiful Bari; et al. (CoRR, 2021)

What Language Model to Train if You Have One Million GPU Hours? - Teven Le Scao; Thomas Wang; Daniel Hesslow; Lucile Saulnier; Stas Bekman; M. Saiful Bari; Stella Biderman; Hady Elsahar; Niklas Muennighoff; Jason Phang; Ofir Press; Colin Raffel; et al. (EMNLP (Findings), 2022)

Multitask Prompted Training Enables Zero-Shot Task Generalization. - Victor Sanh; Albert Webson; Colin Raffel; Stephen H. Bach; Lintang Sutawika; Zaid Alyafeai; Antoine Chaffin; Arnaud Stiegler; Arun Raja; Manan Dey; M Saiful Bari; Canwen Xu; et al. (ICLR, 2022)

What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (ICML, 2022)

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset. - Hugo Laurençon; Lucile Saulnier; Thomas Wang; Christopher Akiki; Albert Villanova del Moral; Teven Le Scao; Leandro von Werra; Chenghao Mou; Eduardo González Ponferrada; Huu Nguyen; Jörg Frohberg; Mario Sasko; et al. (NeurIPS, 2022)

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (CoRR, 2022)

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. - Teven Le Scao; Angela Fan; Christopher Akiki; Ellie Pavlick; Suzana Ilic; Daniel Hesslow; Roman Castagné; Alexandra Sasha Luccioni; François Yvon; Matthias Gallé; Jonathan Tow; Alexander M. Rush; et al. (CoRR, 2022)

Crosslingual Generalization through Multitask Finetuning. - Niklas Muennighoff; Thomas Wang; Lintang Sutawika; Adam Roberts; Stella Biderman; Teven Le Scao; M. Saiful Bari; Sheng Shen 001; Zheng Xin Yong; Hailey Schoelkopf; Xiangru Tang; Dragomir Radev; et al. (ACL (1), 2023)

FinGPT: Large Generative Models for a Small Language. - Risto Luukkonen; Ville Komulainen; Jouni Luoma; Anni Eskelinen; Jenna Kanerva; Hanna-Mari Kupari; Filip Ginter; Veronika Laippala; Niklas Muennighoff; Aleksandra Piktus; Thomas Wang; Nouamane Tazi; et al. (EMNLP, 2023)

Joint Representations of Text and Knowledge Graphs for Retrieval and Evaluation. - Teven Le Scao; Claire Gardent (IJCNLP (Findings), 2023)

Scaling Data-Constrained Language Models. - Niklas Muennighoff; Alexander M. Rush; Boaz Barak; Teven Le Scao; Nouamane Tazi; Aleksandra Piktus; Sampo Pyysalo; Thomas Wolf 008; Colin A. Raffel (NeurIPS, 2023)

Mistral 7B. - Albert Q. Jiang; Alexandre Sablayrolles; Arthur Mensch; Chris Bamford 001; Devendra Singh Chaplot; Diego de Las Casas; Florian Bressand; Gianna Lengyel; Guillaume Lample; Lucile Saulnier; Lélio Renard Lavaud; Marie-Anne Lachaux; et al. (CoRR, 2023)

Mixtral of Experts. - Albert Q. Jiang; Alexandre Sablayrolles; Antoine Roux; Arthur Mensch; Blanche Savary; Chris Bamford 001; Devendra Singh Chaplot; Diego de Las Casas; Emma Bou Hanna; Florian Bressand; Gianna Lengyel; Guillaume Bour; et al. (CoRR, 2024)

Pixtral 12B. - Pravesh Agrawal; Szymon Antoniak; Emma Bou Hanna; Baptiste Bout; Devendra Singh Chaplot; Jessica Chudnovsky; Diogo Costa; Baudouin De Monicault; Saurabh Garg; Théophile Gervet; Soham Ghosh; Amélie Héliou; et al. (CoRR, 2024)

Magistral. - Abhinav Rastogi; Albert Q. Jiang; Andy Lo; Gabrielle Berrada; Guillaume Lample; Jason Rute; Joep Barmentlo; Karmesh Yadav; Kartik Khandelwal; Khyathi Raghavi Chandu; Léonard Blier; Lucile Saulnier; et al. (CoRR, 2025)

Voxtral. - Alexander H. Liu; Andy Ehrenberg; Andy Lo; Clément Denoix; Corentin Barreau; Guillaume Lample; Jean-Malo Delignon; Khyathi Raghavi Chandu; Patrick von Platen; Pavankumar Reddy Muddireddy; Sanchit Gandhi; Soham Ghosh; et al. (CoRR, 2025)