Few-shot Learning with Multilingual Language Models. - Xi Victoria Lin; Todor Mihaylov; Mikel Artetxe; Tianlu Wang; Shuohui Chen; Daniel Simig; Myle Ott; Naman Goyal 001; Shruti Bhosale; Jingfei Du; Ramakanth Pasunuru; Sam Shleifer; et al. (CoRR, 2021)

Efficient Large Scale Language Modeling with Mixtures of Experts. - Mikel Artetxe; Shruti Bhosale; Naman Goyal 001; Todor Mihaylov; Myle Ott; Sam Shleifer; Xi Victoria Lin; Jingfei Du; Srinivasan Iyer 001; Ramakanth Pasunuru; Giri Anantharaman; Xian Li 003; et al. (CoRR, 2021)

Efficient Large Scale Language Modeling with Mixtures of Experts. - Mikel Artetxe; Shruti Bhosale; Naman Goyal 001; Todor Mihaylov; Myle Ott; Sam Shleifer; Xi Victoria Lin; Jingfei Du; Srinivasan Iyer 001; Ramakanth Pasunuru; Giridharan Anantharaman; Xian Li 003; et al. (EMNLP, 2022)

Few-shot Learning with Multilingual Generative Language Models. - Xi Victoria Lin; Todor Mihaylov; Mikel Artetxe; Tianlu Wang; Shuohui Chen; Daniel Simig; Myle Ott; Naman Goyal 001; Shruti Bhosale; Jingfei Du; Ramakanth Pasunuru; Sam Shleifer; et al. (EMNLP, 2022)

OPT: Open Pre-trained Transformer Language Models. - Susan Zhang; Stephen Roller; Naman Goyal 001; Mikel Artetxe; Moya Chen; Shuohui Chen; Christopher Dewan; Mona T. Diab; Xian Li 003; Xi Victoria Lin; Todor Mihaylov; Myle Ott; et al. (CoRR, 2022)

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization. - Srinivasan Iyer 001; Xi Victoria Lin; Ramakanth Pasunuru; Todor Mihaylov; Daniel Simig; Ping Yu; Kurt Shuster 001; Tianlu Wang; Qing Liu; Punit Singh Koura; Xian Li 003; Brian O'Horo; et al. (CoRR, 2022)

A Theory on Adam Instability in Large-Scale Machine Learning. - Igor Molybog; Peter Albert; Moya Chen; Zachary DeVito; David Esiobu; Naman Goyal 001; Punit Singh Koura; Sharan Narang; Andrew Poulton; Ruan Silva; Binh Tang; Diana Liskovich; et al. (CoRR, 2023)

Llama 2: Open Foundation and Fine-Tuned Chat Models. - Hugo Touvron; Louis Martin; Kevin Stone; Peter Albert; Amjad Almahairi; Yasmine Babaei; Nikolay Bashlykov; Soumya Batra; Prajjwal Bhargava; Shruti Bhosale; Dan Bikel; Lukas Blecher; et al. (CoRR, 2023)

Optimizing Pretraining Data Mixtures with LLM-Estimated Utility. - William Held; Bhargavi Paranjape; Punit Singh Koura; Mike Lewis; Frank Zhang; Todor Mihaylov (CoRR, 2025)

BTS: Harmonizing Specialized Experts into a Generalist LLM. - Qizhen Zhang 002; Prajjwal Bhargava; Chloe Bi; Chris X. Cai; Jakob N. Foerster; Jeremy Fu; Punit Singh Koura; Ruan Silva; Sheng Shen 016; Emily Dinan; Suchin Gururangan; Mike Lewis (CoRR, 2025)

Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks. - Rylan Schaeffer; Punit Singh Koura; Binh Tang; Ranjan Subramanian; Aaditya K. Singh; Todor Mihaylov; Prajjwal Bhargava; Lovish Madaan; Niladri S. Chatterji; Vedanuj Goswami; Sergey Edunov; Dieuwke Hupkes; et al. (CoRR, 2025)