The Hateful Memes Challenge: Competition Report. - Douwe Kiela; Hamed Firooz; Aravind Mohan; Vedanuj Goswami; Amanpreet Singh; Casey A. Fitzpatrick; Peter Bull; Greg Lipstein; Tony Nelli; Ron Zhu; Niklas Muennighoff; Riza Velioglu; et al. (NeurIPS (Competition and Demos), 2020)

Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes. - Niklas Muennighoff (CoRR, 2020)

Diagnosing the Impact of AI on Radiology in China. - Niklas Muennighoff (CoRR, 2021)

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation. - Kaustubh D. Dhole; Varun Gangal; Sebastian Gehrmann; Aadesh Gupta; Zhenhao Li; Saad Mahamood; Abinaya Mahendiran; Simon Mille; Ashish Srivastava; Samson Tan; Tongshuang Wu; Jascha Sohl-Dickstein; et al. (CoRR, 2021)

What Language Model to Train if You Have One Million GPU Hours? - Teven Le Scao; Thomas Wang; Daniel Hesslow; Lucile Saulnier; Stas Bekman; M. Saiful Bari; Stella Biderman; Hady Elsahar; Niklas Muennighoff; Jason Phang; Ofir Press; Colin Raffel; et al. (EMNLP (Findings), 2022)

SGPT: GPT Sentence Embeddings for Semantic Search. - Niklas Muennighoff (CoRR, 2022)

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. - Teven Le Scao; Angela Fan; Christopher Akiki; Ellie Pavlick; Suzana Ilic; Daniel Hesslow; Roman Castagné; Alexandra Sasha Luccioni; François Yvon; Matthias Gallé; Jonathan Tow; Alexander M. Rush; et al. (CoRR, 2022)

BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting. - Zheng Xin Yong; Hailey Schoelkopf; Niklas Muennighoff; Alham Fikri Aji; David Ifeoluwa Adelani; Khalid Almubarak; M. Saiful Bari; Lintang Sutawika; Jungo Kasai; Ahmed Baruwa; Genta Indra Winata; Stella Biderman; et al. (CoRR, 2022)

Crosslingual Generalization through Multitask Finetuning. - Niklas Muennighoff; Thomas Wang; Lintang Sutawika; Adam Roberts; Stella Biderman; Teven Le Scao; M. Saiful Bari; Sheng Shen 001; Zheng Xin Yong; Hailey Schoelkopf; Xiangru Tang; Dragomir Radev; et al. (ACL (1), 2023)

BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting. - Zheng Xin Yong; Hailey Schoelkopf; Niklas Muennighoff; Alham Fikri Aji; David Ifeoluwa Adelani; Khalid Almubarak; M. Saiful Bari; Lintang Sutawika; Jungo Kasai; Ahmed Baruwa; Genta Indra Winata; Stella Biderman; et al. (ACL (1), 2023)

MTEB: Massive Text Embedding Benchmark. - Niklas Muennighoff; Nouamane Tazi; Loïc Magne; Nils Reimers 003 (EACL, 2023)

FinGPT: Large Generative Models for a Small Language. - Risto Luukkonen; Ville Komulainen; Jouni Luoma; Anni Eskelinen; Jenna Kanerva; Hanna-Mari Kupari; Filip Ginter; Veronika Laippala; Niklas Muennighoff; Aleksandra Piktus; Thomas Wang; Nouamane Tazi; et al. (EMNLP, 2023)

Scaling Data-Constrained Language Models. - Niklas Muennighoff; Alexander M. Rush; Boaz Barak; Teven Le Scao; Nouamane Tazi; Aleksandra Piktus; Sampo Pyysalo; Thomas Wolf 008; Colin A. Raffel (NeurIPS, 2023)

SantaCoder: don't reach for the stars! - Loubna Ben Allal; Raymond Li; Denis Kocetkov; Chenghao Mou; Christopher Akiki; Carlos Muñoz Ferrandis; Niklas Muennighoff; Mayank Mishra; Alex Gu; Manan Dey; Logesh Kumar Umapathi; Carolyn Jane Anderson; et al. (CoRR, 2023)

C-Pack: Packaged Resources To Advance General Chinese Embedding. - Shitao Xiao; Zheng Liu 011; Peitian Zhang; Niklas Muennighoff (CoRR, 2023)

The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI. - Shayne Longpre; Robert Mahari; Anthony Chen; Naana Obeng-Marnu; Damien Sileo; William Brannon; Niklas Muennighoff; Nathan Khazam; Jad Kabbara; Kartik Perisetla; Xinyi Wu; Enrico Shippole; et al. (CoRR, 2023)

StarCoder: may the source be with you! - Raymond Li; Loubna Ben Allal; Yangtian Zi; Niklas Muennighoff; Denis Kocetkov; Chenghao Mou; Marc Marone; Christopher Akiki; Jia Li 015; Jenny Chim; Qian Liu 012; Evgenii Zheltonozhskii; et al. (Trans. Mach. Learn. Res., 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)

OLMo: Accelerating the Science of Language Models. - Dirk Groeneveld; Iz Beltagy; Evan Pete Walsh; Akshita Bhagia; Rodney Kinney; Oyvind Tafjord; Ananya Harsh Jha; Hamish Ivison; Ian Magnusson; Yizhong Wang; Shane Arora; David Atkinson; et al. (ACL (1), 2024)

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning. - Shivalika Singh; Freddie Vargus; Daniel D'souza; Börje Karlsson 001; Abinaya Mahendiran; Wei-Yin Ko; Herumb Shandilya; Jay Patel; Deividas Mataciunas; Laura O'Mahony; Mike Zhang; Ramith Hettiarachchi; et al. (ACL (1), 2024)

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. - Luca Soldaini; Rodney Kinney; Akshita Bhagia; Dustin Schwenk; David Atkinson; Russell Authur; Ben Bogin; Khyathi Raghavi Chandu; Jennifer Dumas; Yanai Elazar; Valentin Hofmann; Ananya Harsh Jha; et al. (ACL (1), 2024)

Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model. - Ahmet Üstün; Viraat Aryabumi; Zheng Xin Yong; Wei-Yin Ko; Daniel D'souza; Gbemileke Onilude; Neel Bhandari; Shivalika Singh; Hui-Lee Ooi; Amr Kayid; Freddie Vargus; Phil Blunsom; et al. (ACL (1), 2024)

SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages. - Holy Lovenia; Rahmad Mahendra; Salsabil Maulana Akbar; Lester James V. Miranda; Jennifer Santoso; Elyanah Aco; Akhdan Fadhilah; Jonibek Mansurov; Joseph Marvin Imperial; Onno Kampman; Joel Ruben Antony Moniz; Muhammad Ravi Shulthan Habibi; et al. (EMNLP, 2024)

OctoPack: Instruction Tuning Code Large Language Models. - Niklas Muennighoff; Qian Liu 012; Armel Randy Zebaze; Qinkai Zheng; Binyuan Hui; Terry Yue Zhuo; Swayam Singh; Xiangru Tang; Leandro von Werra; Shayne Longpre (ICLR, 2024)

Model Alignment as Prospect Theoretic Optimization. - Kawin Ethayarajh; Winnie Xu; Niklas Muennighoff; Dan Jurafsky; Douwe Kiela (ICML, 2024)

The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embedding. - Kenneth C. Enevoldsen; Márton Kardos; Niklas Muennighoff; Kristoffer L. Nielbo (NeurIPS, 2024)

DataComp-LM: In search of the next generation of training sets for language models. - Jeffrey Li; Alex Fang; Georgios Smyrnis; Maor Ivgi; Matt Jordan; Samir Yitzhak Gadre; Hritik Bansal; Etash Kumar Guha; Sedrick Scott Keh; Kushal Arora; Saurabh Garg; Rui Xin; et al. (NeurIPS, 2024)

Consent in Crisis: The Rapid Decline of the AI Data Commons. - Shayne Longpre; Robert Mahari; Ariel Lee; Campbell Lund; Hamidah Oderinwale; William Brannon; Nayan Saxena; Naana Obeng-Marnu; Tobin South; Cole Hunter; Kevin Klyman; Christopher Klamm; et al. (NeurIPS, 2024)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies. - Chaofan Tao; Qian Liu 012; Longxu Dou; Niklas Muennighoff; Zhongwei Wan; Ping Luo 002; Min Lin; Ngai Wong 001 (NeurIPS, 2024)

C-Pack: Packed Resources For General Chinese Embeddings. - Shitao Xiao; Zheng Liu 011; Peitian Zhang; Niklas Muennighoff; Defu Lian; Jian-Yun Nie (SIGIR, 2024)

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models. - Terry Yue Zhuo; Armel Zebaze; Nitchakarn Suppattarachai; Leandro von Werra; Harm de Vries; Qian Liu 012; Niklas Muennighoff (CoRR, 2024)

OLMo: Accelerating the Science of Language Models. - Dirk Groeneveld; Iz Beltagy; Pete Walsh; Akshita Bhagia; Rodney Kinney; Oyvind Tafjord; Ananya Harsh Jha; Hamish Ivison; Ian Magnusson; Yizhong Wang; Shane Arora; David Atkinson; et al. (CoRR, 2024)

KTO: Model Alignment as Prospect Theoretic Optimization. - Kawin Ethayarajh; Winnie Xu; Niklas Muennighoff; Dan Jurafsky; Douwe Kiela (CoRR, 2024)

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning. - Shivalika Singh; Freddie Vargus; Daniel D'souza; Börje F. Karlsson; Abinaya Mahendiran; Wei-Yin Ko; Herumb Shandilya; Jay Patel; Deividas Mataciunas; Laura O'Mahony; Mike Zhang; Ramith Hettiarachchi; et al. (CoRR, 2024)

StarCoder 2 and The Stack v2: The Next Generation. - Anton Lozhkov; Raymond Li; Loubna Ben Allal; Federico Cassano; Joel Lamy-Poirier; Nouamane Tazi; Ao Tang; Dmytro Pykhtar; Jiawei Liu 004; Yuxiang Wei 003; Tianyang Liu 003; Max Tian; et al. (CoRR, 2024)

Language models scale reliably with over-training and on downstream tasks. - Samir Yitzhak Gadre; Georgios Smyrnis; Vaishaal Shankar; Suchin Gururangan; Mitchell Wortsman; Rulin Shao; Jean Mercat; Alex Fang; Jeffrey Li; Sedrick Keh; Rui Xin; Marianna Nezhurina; et al. (CoRR, 2024)

Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order. - Taishi Nakamura; Mayank Mishra; Simone Tedeschi; Yekun Chai; Jason T. Stillerman; Felix Friedrich; Prateek Yadav; Tanmay Laud; Minh Chien Vu; Terry Yue Zhuo; Diganta Misra; Ben Bogin; et al. (CoRR, 2024)

Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence. - Bo Peng; Daniel Goldstein; Quentin Anthony; Alon Albalak; Eric Alcaide; Stella Biderman; Eugene Cheah; Xingjian Du; Teddy Ferdinan; Haowen Hou; Przemyslaw Kazienko; Kranthi Kiran GV; et al. (CoRR, 2024)

Lessons from the Trenches on Reproducible Evaluation of Language Models. - Stella Biderman; Hailey Schoelkopf; Lintang Sutawika; Leo Gao; Jonathan Tow; Baber Abbasi; Alham Fikri Aji; Pawan Sasanka Ammanamanchi; Sidney Black; Jordan Clive; Anthony DiPofi; Julen Etxaniz; et al. (CoRR, 2024)

SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages. - Holy Lovenia; Rahmad Mahendra; Salsabil Maulana Akbar; Lester James V. Miranda; Jennifer Santoso; Elyanah Aco; Akhdan Fadhilah; Jonibek Mansurov; Joseph Marvin Imperial; Onno Pepijn Kampman; Joel Ruben Antony Moniz; Muhammad Ravi Shulthan Habibi; et al. (CoRR, 2024)

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions. - Terry Yue Zhuo; Minh Chien Vu; Jenny Chim; Han Hu; Wenhao Yu 002; Ratnadira Widyasari; Imam Nur Bani Yusuf; Haolan Zhan; Junda He; Indraneil Paul; Simon Brunner; Chen Gong 005; et al. (CoRR, 2024)

Consent in Crisis: The Rapid Decline of the AI Data Commons. - Shayne Longpre; Robert Mahari; Ariel Lee; Campbell Lund; Hamidah Oderinwale; William Brannon; Nayan Saxena; Naana Obeng-Marnu; Tobin South; Cole Hunter; Kevin Klyman; Christopher Klamm; et al. (CoRR, 2024)

OpenDevin: An Open Platform for AI Software Developers as Generalist Agents. - Xingyao Wang 002; Boxuan Li; Yufan Song; Frank F. Xu; Xiangru Tang; Mingchen Zhuge; Jiayi Pan; Yueqi Song; Bowen Li 002; Jaskirat Singh; Hoang H. Tran; Fuqiang Li; et al. (CoRR, 2024)

OLMoE: Open Mixture-of-Experts Language Models. - Niklas Muennighoff; Luca Soldaini; Dirk Groeneveld; Kyle Lo; Jacob Morrison; Sewon Min; Weijia Shi; Pete Walsh; Oyvind Tafjord; Nathan Lambert 001; Yuling Gu; Shane Arora; et al. (CoRR, 2024)

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models. - Matt Deitke; Christopher Clark; Sangho Lee 008; Rohun Tripathi; Yue Yang 006; Jae Sung Park; Mohammadreza Salehi; Niklas Muennighoff; Kyle Lo; Luca Soldaini; Jiasen Lu; Taira Anderson; et al. (CoRR, 2024)

Bridging the Data Provenance Gap Across Text, Speech and Video. - Shayne Longpre; Nikhil Singh 003; Manuel Cherep; Kushagra Tiwary; Joanna Materzynska; William Brannon; Robert Mahari; Manan Dey; Mohammed Hamdy; Nayan Saxena; Ahmad Mustafa Anis; Emad A. Alghamdi; et al. (CoRR, 2024)

A large-scale audit of dataset licensing and attribution in AI. - Shayne Longpre; Robert Mahari; Anthony Chen; Naana Obeng-Marnu; Damien Sileo; William Brannon; Niklas Muennighoff; Nathan Khazam; Jad Kabbara; Kartik Perisetla; Xinyi Wu; Enrico Shippole; et al. (Nat. Mac. Intell., 2024)

A Survey on Data Selection for Language Models. - Alon Albalak; Yanai Elazar; Sang Michael Xie; Shayne Longpre; Nathan Lambert 001; Xinyi Wang 003; Niklas Muennighoff; Bairu Hou; Liangming Pan; Haewon Jeong; Colin Raffel; Shiyu Chang; et al. (Trans. Mach. Learn. Res., 2024)

LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation. - Eunsu Kim; Juyoung Suk; Seungone Kim; Niklas Muennighoff; Dongkwan Kim 001; Alice Oh (ACL (Findings), 2025)

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code. - Taishi Nakamura; Mayank Mishra; Simone Tedeschi; Yekun Chai; Jason T. Stillerman; Felix Friedrich; Prateek Yadav; Tanmay Laud; Minh Chien Vu; Terry Yue Zhuo; Diganta Misra; Ben Bogin; et al. (COLING (Industry), 2025)

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models. - Matt Deitke; Christopher Clark; Sangho Lee 001; Rohun Tripathi; Yue Yang 006; Jae Sung Park; Mohammadreza Salehi; Niklas Muennighoff; Kyle Lo; Luca Soldaini; Jiasen Lu; Taira Anderson; et al. (CVPR, 2025)

OpenHands: An Open Platform for AI Software Developers as Generalist Agents. - Xingyao Wang 002; Boxuan Li; Yufan Song; Frank F. Xu; Xiangru Tang; Mingchen Zhuge; Jiayi Pan; Yueqi Song; Bowen Li 002; Jaskirat Singh; Hoang H. Tran; Fuqiang Li; et al. (ICLR, 2025)

Scaling Laws for Precision. - Tanishq Kumar; Zachary Ankner; Benjamin Frederick Spector; Blake Bordelon; Niklas Muennighoff; Mansheej Paul; Cengiz Pehlevan; Christopher Ré; Aditi Raghunathan (ICLR, 2025)

RegMix: Data Mixture as Regression for Language Model Pre-training. - Qian Liu 012; Xiaosen Zheng; Niklas Muennighoff; Guangtao Zeng; Longxu Dou; Tianyu Pang; Jing Jiang 001; Min Lin (ICLR, 2025)

Generative Representational Instruction Tuning. - Niklas Muennighoff; Hongjin Su; Liang Wang 046; Nan Yang 002; Furu Wei; Tao Yu 009; Amanpreet Singh; Douwe Kiela (ICLR, 2025)

OLMoE: Open Mixture-of-Experts Language Models. - Niklas Muennighoff; Luca Soldaini; Dirk Groeneveld; Kyle Lo; Jacob Morrison; Sewon Min; Weijia Shi; Evan Pete Walsh; Oyvind Tafjord; Nathan Lambert 001; Yuling Gu; Shane Arora; et al. (ICLR, 2025)

BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval. - Hongjin Su; Howard Yen; Mengzhou Xia; Weijia Shi; Niklas Muennighoff; Han-yu Wang; Haisu Liu; Quan Shi; Zachary S. Siegel; Michael Tang; Ruoxi Sun 002; Jinsung Yoon; et al. (ICLR, 2025)

SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? - John Yang 002; Carlos E. Jimenez; Alex L. Zhang; Kilian Lieret; Joyce Yang; Xindi Wu; Ori Press; Niklas Muennighoff; Gabriel Synnaeve; Karthik R. Narasimhan; Diyi Yang; Sida Wang 001; et al. (ICLR, 2025)

KMMLU: Measuring Massive Multitask Language Understanding in Korean. - Guijin Son; Hanwool Lee; Sungdong Kim; Seungone Kim; Niklas Muennighoff; Taekyoon Choi; Cheonbok Park; Kang Min Yoo; Stella Biderman (NAACL (Long Papers), 2025)

s1: Simple test-time scaling. - Niklas Muennighoff; Zitong Yang; Weijia Shi; Xiang Lisa Li; Li Fei-Fei 001; Hannaneh Hajishirzi; Luke Zettlemoyer; Percy Liang; Emmanuel J. Candès; Tatsunori Hashimoto (CoRR, 2025)

MMTEB: Massive Multilingual Text Embedding Benchmark. - Kenneth C. Enevoldsen; Isaac Chung; Imene Kerboua; Márton Kardos; Ashwin Mathur; David Stap; Jay Gala; Wissam Siblini; Dominik Krzeminski; Genta Indra Winata; Saba Sturua; Saiteja Utpala; et al. (CoRR, 2025)

Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning. - Ximing Lu; Seungju Han 002; David Acuna; Hyunwoo Kim 002; Jaehun Jung; Shrimai Prabhumoye; Niklas Muennighoff; Mostofa Patwary; Mohammad Shoeybi; Bryan Catanzaro; Yejin Choi 001 (CoRR, 2025)

MIEB: Massive Image Embedding Benchmark. - Chenghao Xiao; Isaac Chung; Imene Kerboua; Jamie Stirling; Xin Zhang; Márton Kardos; Roman Solomatin; Noura Al Moubayed; Kenneth C. Enevoldsen; Niklas Muennighoff (CoRR, 2025)

ReasonIR: Training Retrievers for Reasoning Tasks. - Rulin Shao; Rui Qiao 006; Varsha Kishore; Niklas Muennighoff; Xi Victoria Lin; Daniela Rus; Bryan Kian Hsiang Low; Sewon Min; Wen-tau Yih; Pang Wei Koh; Luke Zettlemoyer (CoRR, 2025)

Crosslingual Reasoning through Test-Time Scaling. - Zheng-Xin Yong; Muhammad Farid Adilazuarda; Jonibek Mansurov; Ruochen Zhang; Niklas Muennighoff; Carsten Eickhoff; Genta Indra Winata; Julia Kreutzer; Stephen H. Bach; Alham Fikri Aji (CoRR, 2025)

Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountability. - Genta Indra Winata; David Anugraha; Emmy Liu; Alham Fikri Aji; Shou-Yi Hung; Aditya Parashar; Patrick Amadeus Irawan; Ruochen Zhang; Zheng-Xin Yong; Jan Christian Blaise Cruz; Niklas Muennighoff; Seungone Kim; et al. (CoRR, 2025)

OpenThoughts: Data Recipes for Reasoning Models. - Etash Kumar Guha; Ryan Marten; Sedrick Keh; Negin Raoof; Georgios Smyrnis; Hritik Bansal; Marianna Nezhurina; Jean Mercat; Trung Vu 002; Zayne Sprague; Ashima Suvarna; Benjamin Feuer; et al. (CoRR, 2025)

FlexOlmo: Open Language Models for Flexible Data Use. - Weijia Shi; Akshita Bhagia; Kevin Farhat; Niklas Muennighoff; Pete Walsh; Jacob Morrison; Dustin Schwenk; Shayne Longpre; Jake Poznanski; Allyson Ettinger; Daogao Liu; Margaret Li; et al. (CoRR, 2025)