Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel. - Stanislav Fort; Gintare Karolina Dziugaite; Mansheej Paul; Sepideh Kharaghani; Daniel M. Roy 001; Surya Ganguli (NeurIPS, 2020)

Deep Learning on a Data Diet: Finding Important Examples Early in Training. - Mansheej Paul; Surya Ganguli; Gintare Karolina Dziugaite (NeurIPS, 2021)

Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks. - Mansheej Paul; Brett W. Larsen; Surya Ganguli; Jonathan Frankle; Gintare Karolina Dziugaite (NeurIPS, 2022)

Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? - Mansheej Paul; Feng Chen 046; Brett W. Larsen; Jonathan Frankle; Surya Ganguli; Gintare Karolina Dziugaite (ICLR, 2023)

Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression. - Allan Raventós; Mansheej Paul; Feng Chen 046; Surya Ganguli (NeurIPS, 2023)

Does your data spark joy? Performance gains from domain upsampling at the end of training. - Cody Blakeney; Mansheej Paul; Brett W. Larsen; Sean Owen; Jonathan Frankle (CoRR, 2024)

Critique-out-Loud Reward Models. - Zachary Ankner; Mansheej Paul; Brandon Cui; Jonathan D. Chang; Prithviraj Ammanabrolu (CoRR, 2024)

LoRA Learns Less and Forgets Less. - Dan Biderman; Jacob P. Portes; Jose Javier Gonzalez Ortiz; Mansheej Paul; Philip Greengard; Connor Jennings; Daniel King; Sam Havens; Vitaliy Chiley; Jonathan Frankle; Cody Blakeney; John Patrick Cunningham (Trans. Mach. Learn. Res., 2024)

Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models. - Zachary Ankner; Cody Blakeney; Kartik Sreenivasan; Max Marion; Matthew L. Leavitt; Mansheej Paul (ICLR, 2025)

Scaling Laws for Precision. - Tanishq Kumar; Zachary Ankner; Benjamin Frederick Spector; Blake Bordelon; Niklas Muennighoff; Mansheej Paul; Cengiz Pehlevan; Christopher Ré; Aditi Raghunathan (ICLR, 2025)

Soup to go: mitigating forgetting during continual learning with model averaging. - Anat Kleiman; Gintare Karolina Dziugaite; Jonathan Frankle; Sham M. Kakade; Mansheej Paul (CoRR, 2025)

μnit Scaling: Simple and Scalable FP8 LLM Training. - Saaketh Narayan; Abhay Gupta; Mansheej Paul; Davis W. Blalock (CoRR, 2025)