Inductive Bias of Gradient Descent for Exponentially Weight Normalized Smooth Homogeneous Neural Nets. - Depen Morwani; Harish G. Ramaswamy (CoRR, 2020)

Using noise resilience for ranking generalization of deep neural networks. - Depen Morwani; Rahul Vashisht; Harish G. Ramaswamy (CoRR, 2020)

Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets. - Depen Morwani; Harish G. Ramaswamy (ALT, 2022)

Feature-Learning Networks Are Consistent Across Widths At Realistic Scales. - Nikhil Vyas 001; Alexander B. Atanasov; Blake Bordelon; Depen Morwani; Sabarish Sainathan; Cengiz Pehlevan (NeurIPS, 2023)

Simplicity Bias in 1-Hidden Layer Neural Networks. - Depen Morwani; Jatin Batra; Prateek Jain 002; Praneeth Netrapalli (NeurIPS, 2023)

Feature emergence via margin maximization: case studies in algebraic tasks. - Depen Morwani; Benjamin L. Edelman; Costin-Andrei Oncescu; Rosie Zhao; Sham M. Kakade (ICLR, 2024)

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning. - Nikhil Vyas 001; Depen Morwani; Rosie Zhao; Gal Kaplun; Sham M. Kakade; Boaz Barak (ICML, 2024)

Deconstructing What Makes a Good Optimizer for Language Models. - Rosie Zhao; Depen Morwani; David Brandfonbrener; Nikhil Vyas 001; Sham M. Kakade (CoRR, 2024)

SOAP: Improving and Stabilizing Shampoo using Adam. - Nikhil Vyas 001; Depen Morwani; Rosie Zhao; Itai Shapira; David Brandfonbrener; Lucas Janson; Sham M. Kakade (CoRR, 2024)

SOAP: Improving and Stabilizing Shampoo using Adam for Language Modeling. - Nikhil Vyas 001; Depen Morwani; Rosie Zhao; Itai Shapira; David Brandfonbrener; Lucas Janson; Sham M. Kakade (ICLR, 2025)

A New Perspective on Shampoo's Preconditioner. - Depen Morwani; Itai Shapira; Nikhil Vyas 001; Eran Malach; Sham M. Kakade; Lucas Janson (ICLR, 2025)

How Does Critical Batch Size Scale in Pre-training? - Hanlin Zhang 002; Depen Morwani; Nikhil Vyas 001; Jingfeng Wu; Difan Zou; Udaya Ghai; Dean P. Foster; Sham M. Kakade (ICLR, 2025)

Deconstructing What Makes a Good Optimizer for Autoregressive Language Models. - Rosie Zhao; Depen Morwani; David Brandfonbrener; Nikhil Vyas 001; Sham M. Kakade (ICLR, 2025)

Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants. - Depen Morwani; Nikhil Vyas 001; Hanlin Zhang 002; Sham M. Kakade (CoRR, 2025)

A Simplified Analysis of SGD for Linear Regression with Weight Averaging. - Alexandru Meterez; Depen Morwani; Costin-Andrei Oncescu; Jingfeng Wu; Cengiz Pehlevan; Sham M. Kakade (CoRR, 2025)