Ensemble Machine Learning Methods for Modeling COVID19 Deaths. - Rahil Bathwal; Pavan Chitta; Kushal Tirumala; Vignesh Varadarajan (CoRR, 2020)

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks. - Tristan Thrush; Kushal Tirumala; Anmol Gupta; Max Bartolo; Pedro Rodriguez 001; Tariq Kane; William Gaviria Rojas; Peter Mattson; Adina Williams; Douwe Kiela (ACL (demo), 2022)

Investigating Generalization by Controlling Normalized Margin. - Alexander R. Farhang; Jeremy D. Bernstein; Kushal Tirumala; Yang Liu; Yisong Yue (ICML, 2022)

Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models. - Kushal Tirumala; Aram H. Markosyan; Luke Zettlemoyer; Armen Aghajanyan (NeurIPS, 2022)

D4: Improving LLM Pretraining via Document De-Duplication and Diversification. - Kushal Tirumala; Daniel Simig; Armen Aghajanyan; Ari Morcos (NeurIPS, 2023)

SemDeDup: Data-efficient learning at web-scale through semantic deduplication. - Amro Abbas; Kushal Tirumala; Daniel Simig; Surya Ganguli; Ari S. Morcos (CoRR, 2023)

Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data. - Yu Yang; Aaditya K. Singh; Mostafa Elhoushi; Anas Mahmoud 002; Kushal Tirumala; Fabian Gloeckle; Baptiste Rozière; Carole-Jean Wu; Ari S. Morcos; Newsha Ardalani (CoRR, 2023)

Effective pruning of web-scale datasets based on complexity of concept clusters. - Amro Abbas; Evgenia Rusak; Kushal Tirumala; Wieland Brendel; Kamalika Chaudhuri; Ari S. Morcos (ICLR, 2024)

Text Quality-Based Pruning for Efficient Training of Language Models. - Vasu Sharma; Karthik Padthe; Newsha Ardalani; Kushal Tirumala; Russell Howes; Hu Xu 001; Po-Yao Huang 001; Shang-Wen Li 001; Armen Aghajanyan; Gargi Ghosh; Luke Zettlemoyer (CoRR, 2024)

An Introduction to Vision-Language Modeling. - Florian Bordes; Richard Yuanzhe Pang; Anurag Ajay; Alexander C. Li; Adrien Bardes; Suzanne Petryk; Oscar Mañas; Zhiqiu Lin; Anas Mahmoud 002; Bargav Jayaraman; Mark Ibrahim; Melissa Hall; et al. (CoRR, 2024)

Brevity is the soul of wit: Pruning long files for code generation. - Aaditya K. Singh; Yu Yang; Kushal Tirumala; Mostafa Elhoushi; Ari S. Morcos (CoRR, 2024)

When Worse is Better: Navigating the compression-generation tradeoff in visual tokenization. - Vivek Ramanujan; Kushal Tirumala; Armen Aghajanyan; Luke Zettlemoyer; Ali Farhadi (CoRR, 2024)

The Unreasonable Ineffectiveness of the Deeper Layers. - Andrey Gromov; Kushal Tirumala; Hassan Shapourian; Paolo Glorioso; Daniel A. Roberts (ICLR, 2025)

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model. - Chunting Zhou; Lili Yu; Arun Babu; Kushal Tirumala; Michihiro Yasunaga; Leonid Shamis; Jacob Kahn; Xuezhe Ma; Luke Zettlemoyer; Omer Levy (ICLR, 2025)

CAT: Content-Adaptive Image Tokenization. - Junhong Shen; Kushal Tirumala; Michihiro Yasunaga; Ishan Misra; Luke Zettlemoyer; Lili Yu; Chunting Zhou (CoRR, 2025)