Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width. - Dayal Singh Kalra; Maissam Barkeshli (NeurIPS, 2023)

Phase diagram of training dynamics in deep neural networks: effect of learning rate, depth, and width. - Dayal Singh Kalra; Maissam Barkeshli (CoRR, 2023)

Why Warmup the Learning Rate? Underlying Mechanisms and Improvements. - Dayal Singh Kalra; Maissam Barkeshli (NeurIPS, 2024)

Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos. - Dayal Singh Kalra; Tianyu He; Maissam Barkeshli (ICLR, 2025)

(How) Can Transformers Predict Pseudo-Random Numbers? - Tao Tao; Darshil Doshi; Dayal Singh Kalra; Tianyu He; Maissam Barkeshli (CoRR, 2025)

When Can You Get Away with Low Memory Adam? - Dayal Singh Kalra; John Kirchenbauer; Maissam Barkeshli; Tom Goldstein (CoRR, 2025)