Learning the gravitational force law and other analytic functions. - Atish Agarwala; Abhimanyu Das; Rina Panigrahy; Qiuyi Zhang 001 (CoRR, 2020)

One Network Fits All? Modular versus Monolithic Task Formulations in Neural Networks. - Atish Agarwala; Abhimanyu Das; Brendan Juba; Rina Panigrahy; Vatsal Sharan; Xin Wang 116; Qiuyi Zhang 001 (ICLR, 2021)

Deep equilibrium networks are sensitive to initialization statistics. - Atish Agarwala; Samuel S. Schoenholz (ICML, 2022)

SAM operates far from home: eigenvalue regularization as a dynamical phenomenon. - Atish Agarwala; Yann N. Dauphin (ICML, 2023)

Second-order regression models exhibit progressive sharpening to the edge of stability. - Atish Agarwala; Fabian Pedregosa; Jeffrey Pennington (ICML, 2023)

On the Interplay Between Stepsize Tuning and Progressive Sharpening. - Vincent Roulet; Atish Agarwala; Fabian Pedregosa (CoRR, 2023)

Temperature check: theory and practice for training models with softmax-cross-entropy losses. - Atish Agarwala; Samuel Stern Schoenholz; Jeffrey Pennington; Yann N. Dauphin (Trans. Mach. Learn. Res., 2023)

Neglected Hessian component explains mysteries in sharpness regularization. - Yann N. Dauphin; Atish Agarwala; Hossein Mobahi (NeurIPS, 2024)

Stepping on the Edge: Curvature Aware Learning Rate Tuners. - Vincent Roulet; Atish Agarwala; Jean-Bastien Grill; Grzegorz Swirszcz; Mathieu Blondel; Fabian Pedregosa (NeurIPS, 2024)

Gradient descent induces alignment between weights and the empirical NTK for deep non-linear networks. - Daniel Beaglehole; Ioannis Mitliagkas; Atish Agarwala (CoRR, 2024)

High dimensional analysis reveals conservative sharpening and a stochastic edge of stability. - Atish Agarwala; Jeffrey Pennington (CoRR, 2024)

A Clipped Trip: the Dynamics of SGD with Gradient Clipping in High-Dimensions. - Noah Marshall; Ke Liang Xiao; Atish Agarwala; Elliot Paquette (CoRR, 2024)

Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects. - Ke Liang Xiao; Noah Marshall; Atish Agarwala; Elliot Paquette (CoRR, 2024)

Feature learning as alignment: a structural property of gradient descent in non-linear neural networks. - Daniel Beaglehole; Ioannis Mitliagkas; Atish Agarwala (Trans. Mach. Learn. Res., 2024)

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions. - Noah Marshall; Ke Liang Xiao; Atish Agarwala; Elliot Paquette (ICLR, 2025)

How far away are truly hyperparameter-free learning algorithms? - Priya Kasimbeg; Vincent Roulet; Naman Agarwal; Sourabh Medapati; Fabian Pedregosa; Atish Agarwala; George E. Dahl (Trans. Mach. Learn. Res., 2025)

Avoiding spurious sharpness minimization broadens applicability of SAM. - Sidak Pal Singh; Hossein Mobahi; Atish Agarwala; Yann N. Dauphin (CoRR, 2025)

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks. - Shikai Qiu; Lechao Xiao; Andrew Gordon Wilson; Jeffrey Pennington; Atish Agarwala (CoRR, 2025)