Continued Pretraining for Better Zero- and Few-Shot Promptability. - Zhaofeng Wu; Robert L. Logan IV; Pete Walsh; Akshita Bhagia; Dirk Groeneveld; Sameer Singh 001; Iz Beltagy (EMNLP, 2022)

Staged Training for Transformer Language Models. - Sheng Shen 001; Pete Walsh; Kurt Keutzer; Jesse Dodge; Matthew E. Peters; Iz Beltagy (ICML, 2022)

What's In My Big Data? - Yanai Elazar; Akshita Bhagia; Ian Magnusson; Abhilasha Ravichander; Dustin Schwenk; Alane Suhr; Pete Walsh; Dirk Groeneveld; Luca Soldaini; Sameer Singh 001; Hanna Hajishirzi; Noah A. Smith; et al. (CoRR, 2023)

Catwalk: A Unified Language Model Evaluation Framework for Many Datasets. - Dirk Groeneveld; Anas Awadalla; Iz Beltagy; Akshita Bhagia; Ian Magnusson; Hao Peng 009; Oyvind Tafjord; Pete Walsh; Kyle Richardson 001; Jesse Dodge (CoRR, 2023)

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. - Luca Soldaini; Rodney Kinney; Akshita Bhagia; Dustin Schwenk; David Atkinson; Russell Authur; Ben Bogin; Khyathi Raghavi Chandu; Jennifer Dumas; Yanai Elazar; Valentin Hofmann; Ananya Harsh Jha; et al. (ACL (1), 2024)

OLMo: Accelerating the Science of Language Models. - Dirk Groeneveld; Iz Beltagy; Pete Walsh; Akshita Bhagia; Rodney Kinney; Oyvind Tafjord; Ananya Harsh Jha; Hamish Ivison; Ian Magnusson; Yizhong Wang; Shane Arora; David Atkinson; et al. (CoRR, 2024)

OLMoE: Open Mixture-of-Experts Language Models. - Niklas Muennighoff; Luca Soldaini; Dirk Groeneveld; Kyle Lo; Jacob Morrison; Sewon Min; Weijia Shi; Pete Walsh; Oyvind Tafjord; Nathan Lambert 001; Yuling Gu; Shane Arora; et al. (CoRR, 2024)

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models. - Matt Deitke; Christopher Clark; Sangho Lee 008; Rohun Tripathi; Yue Yang 006; Jae Sung Park; Mohammadreza Salehi; Niklas Muennighoff; Kyle Lo; Luca Soldaini; Jiasen Lu; Taira Anderson; et al. (CoRR, 2024)

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models. - Matt Deitke; Christopher Clark; Sangho Lee 001; Rohun Tripathi; Yue Yang 006; Jae Sung Park; Mohammadreza Salehi; Niklas Muennighoff; Kyle Lo; Luca Soldaini; Jiasen Lu; Taira Anderson; et al. (CVPR, 2025)

2 OLMo 2 Furious. - Team OLMo; Pete Walsh; Luca Soldaini; Dirk Groeneveld; Kyle Lo; Shane Arora; Akshita Bhagia; Yuling Gu; Shengyi Huang; Matt Jordan; Nathan Lambert 001; Dustin Schwenk; et al. (CoRR, 2025)

FlexOlmo: Open Language Models for Flexible Data Use. - Weijia Shi; Akshita Bhagia; Kevin Farhat; Niklas Muennighoff; Pete Walsh; Jacob Morrison; Dustin Schwenk; Shayne Longpre; Jake Poznanski; Allyson Ettinger; Daogao Liu; Margaret Li; et al. (CoRR, 2025)