Learning in Games: Robustness of Fast Convergence. - Dylan J. Foster; Zhiyuan Li 005; Thodoris Lykouris; Karthik Sridharan; Éva Tardos (NIPS, 2016)

Solving Marginal MAP Problems with NP Oracles and Parity Constraints. - Yexiang Xue; Zhiyuan Li 005; Stefano Ermon; Carla P. Gomes; Bart Selman (NIPS, 2016)

Fast Convergence of Common Learning Algorithms in Games. - Dylan J. Foster; Zhiyuan Li 005; Thodoris Lykouris; Karthik Sridharan; Éva Tardos (CoRR, 2016)

Stability of Generalized Two-sided Markets with Transaction Thresholds. - Zhiyuan Li 005; Yicheng Liu; Pingzhong Tang; Tingting Xu; Wei Zhan (AAMAS, 2017)

Online Improper Learning with an Approximation Oracle. - Elad Hazan; Wei Hu 014; Yuanzhi Li; Zhiyuan Li 005 (NeurIPS, 2018)

Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks. - Behnam Neyshabur; Zhiyuan Li 005; Srinadh Bhojanapalli; Yann LeCun; Nathan Srebro (CoRR, 2018)

Theoretical Analysis of Auto Rate-Tuning by Batch Normalization. - Sanjeev Arora; Zhiyuan Li 005; Kaifeng Lyu (ICLR (Poster), 2019)

The role of over-parametrization in generalization of neural networks. - Behnam Neyshabur; Zhiyuan Li 005; Srinadh Bhojanapalli; Yann LeCun; Nathan Srebro (ICLR (Poster), 2019)

Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks. - Sanjeev Arora; Simon S. Du; Wei Hu 014; Zhiyuan Li 005; Ruosong Wang (ICML, 2019)

On Exact Computation with an Infinitely Wide Neural Net. - Sanjeev Arora; Simon S. Du; Wei Hu 014; Zhiyuan Li 005; Ruslan Salakhutdinov; Ruosong Wang (NeurIPS, 2019)

Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets. - Rohith Kuditipudi; Xiang Wang 011; Holden Lee; Yi Zhang 074; Zhiyuan Li 005; Wei Hu 014; Rong Ge 001; Sanjeev Arora (NeurIPS, 2019)

Understanding Generalization of Deep Neural Networks Trained with Noisy Labels. - Wei Hu 014; Zhiyuan Li 005; Dingli Yu (CoRR, 2019)

Enhanced Convolutional Neural Tangent Kernels. - Zhiyuan Li 005; Ruosong Wang; Dingli Yu; Simon S. Du; Wei Hu 014; Ruslan Salakhutdinov; Sanjeev Arora (CoRR, 2019)

An Exponential Learning Rate Schedule for Deep Learning. - Zhiyuan Li 005; Sanjeev Arora (ICLR, 2020)

Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks. - Sanjeev Arora; Simon S. Du; Zhiyuan Li 005; Ruslan Salakhutdinov; Ruosong Wang; Dingli Yu (ICLR, 2020)

Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee. - Wei Hu 014; Zhiyuan Li 005; Dingli Yu (ICLR, 2020)

Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate. - Zhiyuan Li 005; Kaifeng Lyu; Sanjeev Arora (NeurIPS, 2020)

When is Particle Filtering Efficient for POMDP Sequential Planning? - Simon S. Du; Wei Hu 014; Zhiyuan Li 005; Ruoqi Shen; Zhao Song 002; Jiajun Wu 001 (CoRR, 2020)

Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? - Zhiyuan Li 005; Yi Zhang 074; Sanjeev Arora (ICLR, 2021)

Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning. - Zhiyuan Li 005; Yuping Luo; Kaifeng Lyu (ICLR, 2021)

On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs). - Zhiyuan Li 005; Sadhika Malladi; Sanjeev Arora (NeurIPS, 2021)

Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias. - Kaifeng Lyu; Zhiyuan Li 005; Runzhe Wang; Sanjeev Arora (NeurIPS, 2021)

When is particle filtering efficient for planning in partially observed linear dynamical systems? - Simon S. Du; Wei Hu 014; Zhiyuan Li 005; Ruoqi Shen; Zhao Song 002; Jiajun Wu 001 (UAI, 2021)

What Happens after SGD Reaches Zero Loss? –A Mathematical Framework. - Zhiyuan Li 005; Tianhao Wang 017; Sanjeev Arora (ICLR, 2022)

Robust Training of Neural Networks Using Scale Invariant Architectures. - Zhiyuan Li 005; Srinadh Bhojanapalli; Manzil Zaheer; Sashank J. Reddi; Sanjiv Kumar (ICML, 2022)

Understanding Gradient Descent on the Edge of Stability in Deep Learning. - Sanjeev Arora; Zhiyuan Li 005; Abhishek Panigrahi (ICML, 2022)

Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent. - Zhiyuan Li 005; Tianhao Wang 017; Jason D. Lee; Sanjeev Arora (NeurIPS, 2022)

Fast Mixing of Stochastic Gradient Descent with Normalization and Weight Decay. - Zhiyuan Li 005; Tianhao Wang 017; Dingli Yu (NeurIPS, 2022)

Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction. - Kaifeng Lyu; Zhiyuan Li 005; Sanjeev Arora (NeurIPS, 2022)

How Does Sharpness-Aware Minimization Minimize Sharpness? - Kaiyue Wen; Tengyu Ma 001; Zhiyuan Li 005 (CoRR, 2022)

Bridging Theory and Practice in Deep Learning: Optimization and Generalization - Zhiyuan Li 005 (2022)

How Sharpness-Aware Minimization Minimizes Sharpness? - Kaiyue Wen; Tengyu Ma 001; Zhiyuan Li 005 (ICLR, 2023)

Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing. - Jikai Jin; Zhiyuan Li 005; Kaifeng Lyu; Simon Shaolei Du; Jason D. Lee (ICML, 2023)

Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models. - Hong Liu; Sang Michael Xie; Zhiyuan Li 005; Tengyu Ma 001 (ICML, 2023)

What is the Inductive Bias of Flatness Regularization? A Study of Deep Matrix Factorization Models. - Khashayar Gatmiry; Zhiyuan Li 005; Tengyu Ma 001; Sashank J. Reddi; Stefanie Jegelka; Ching-Yao Chuang (NeurIPS, 2023)

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization. - Kaiyue Wen; Zhiyuan Li 005; Tengyu Ma 001 (NeurIPS, 2023)

The Inductive Bias of Flatness Regularization for Deep Matrix Factorization. - Khashayar Gatmiry; Zhiyuan Li 005; Ching-Yao Chuang; Sashank J. Reddi; Tengyu Ma 001; Stefanie Jegelka (CoRR, 2023)

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training. - Hong Liu; Zhiyuan Li 005; David Leo Wright Hall; Percy Liang; Tengyu Ma 001 (ICLR, 2024)

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking. - Kaifeng Lyu; Jikai Jin; Zhiyuan Li 005; Simon Shaolei Du; Jason D. Lee; Wei Hu 014 (ICLR, 2024)

The Marginal Value of Momentum for Small Learning Rate SGD. - Runzhe Wang; Sadhika Malladi; Tianhao Wang 017; Kaifeng Lyu; Zhiyuan Li 005 (ICLR, 2024)

Simplicity Bias via Global Convergence of Sharpness Minimization. - Khashayar Gatmiry; Zhiyuan Li 005; Sashank J. Reddi; Stefanie Jegelka (ICML, 2024)

Chain of Thought Empowers Transformers to Solve Inherently Serial Problems. - Zhiyuan Li 005; Hong Liu; Denny Zhou; Tengyu Ma 001 (CoRR, 2024)

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective. - Kaiyue Wen; Zhiyuan Li 005; Jason S. Wang; David Hall 006; Percy Liang; Tengyu Ma 001 (CoRR, 2024)

A Theory of Learning with Autoregressive Chain of Thought. - Nirmit Joshi; Gal Vardi; Adam Block; Surbhi Goel; Zhiyuan Li 005; Theodor Misiakiewicz; Nathan Srebro (COLT, 2025)

Reasoning with Latent Thoughts: On the Power of Looped Transformers. - Nikunj Saunshi; Nishanth Dikkala; Zhiyuan Li 005; Sanjiv Kumar; Sashank J. Reddi (ICLR, 2025)

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View. - Kaiyue Wen; Zhiyuan Li 005; Jason S. Wang; David Leo Wright Hall; Percy Liang; Tengyu Ma 001 (ICLR, 2025)

Chain-of-Thought Provably Enables Learning the (Otherwise) Unlearnable. - Chenxiao Yang; Zhiyuan Li 005; David Wipf (ICLR, 2025)

Weak-to-Strong Generalization Even in Random Feature Networks, Provably. - Marko Medvedev; Kaifeng Lyu; Dingli Yu; Sanjeev Arora; Zhiyuan Li 005; Nathan Srebro (CoRR, 2025)

PENCIL: Long Thoughts with Short Memory. - Chenxiao Yang; Nathan Srebro; David McAllester; Zhiyuan Li 005 (CoRR, 2025)

Non-Asymptotic Length Generalization. - Thomas Chen; Tengyu Ma 001; Zhiyuan Li 005 (CoRR, 2025)