Zhiyuan Li

According to our database, Zhiyuan Li authored at least 38 papers between 2016 and 2023.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2023

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking.

Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon S. Du, Jason D. Lee, Wei Hu

CoRR, 2023

The Marginal Value of Momentum for Small Learning Rate SGD.

Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li

CoRR, 2023

The Inductive Bias of Flatness Regularization for Deep Matrix Factorization.

Khashayar Gatmiry, Zhiyuan Li, Ching-Yao Chuang, Sashank Reddi, Tengyu Ma, Stefanie Jegelka

CoRR, 2023

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training.

Hong Liu, Zhiyuan Li, David Hall, Percy Liang, Tengyu Ma

CoRR, 2023

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization.

Kaiyue Wen, Zhiyuan Li, Tengyu Ma

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models.

Hong Liu, Sang Michael Xie, Zhiyuan Li, Tengyu Ma

Proceedings of the International Conference on Machine Learning, 2023

Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing.

Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee

Proceedings of the International Conference on Machine Learning, 2023

How Sharpness-Aware Minimization Minimizes Sharpness?

Kaiyue Wen, Tengyu Ma, Zhiyuan Li

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

How Does Sharpness-Aware Minimization Minimize Sharpness?

Kaiyue Wen, Tengyu Ma, Zhiyuan Li

CoRR, 2022

Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction.

Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Fast Mixing of Stochastic Gradient Descent with Normalization and Weight Decay.

Zhiyuan Li, Tianhao Wang, Dingli Yu

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent.

Zhiyuan Li, Tianhao Wang, Jason D. Lee, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Understanding Gradient Descent on the Edge of Stability in Deep Learning.

Sanjeev Arora, Zhiyuan Li, Abhishek Panigrahi

Proceedings of the International Conference on Machine Learning, 2022

Robust Training of Neural Networks Using Scale Invariant Architectures.

Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar

Proceedings of the International Conference on Machine Learning, 2022

What Happens after SGD Reaches Zero Loss? --A Mathematical Framework.

Zhiyuan Li, Tianhao Wang, Sanjeev Arora

Proceedings of the Tenth International Conference on Learning Representations, 2022

2021

When is particle filtering efficient for planning in partially observed linear dynamical systems?

Simon S. Du, Wei Hu, Zhiyuan Li, Ruoqi Shen, Zhao Song, Jiajun Wu

Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, 2021

Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias.

Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs).

Zhiyuan Li, Sadhika Malladi, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning.

Zhiyuan Li, Yuping Luo, Kaifeng Lyu

Proceedings of the 9th International Conference on Learning Representations, 2021

Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?

Zhiyuan Li, Yi Zhang, Sanjeev Arora

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

When is Particle Filtering Efficient for POMDP Sequential Planning?

Simon S. Du, Wei Hu, Zhiyuan Li, Ruoqi Shen, Zhao Song, Jiajun Wu

CoRR, 2020

Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate.

Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee.

Wei Hu, Zhiyuan Li, Dingli Yu

Proceedings of the 8th International Conference on Learning Representations, 2020

Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks.

Sanjeev Arora, Simon S. Du, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang, Dingli Yu

Proceedings of the 8th International Conference on Learning Representations, 2020

An Exponential Learning Rate Schedule for Deep Learning.

Zhiyuan Li, Sanjeev Arora

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Enhanced Convolutional Neural Tangent Kernels.

Zhiyuan Li, Ruosong Wang, Dingli Yu, Simon S. Du, Wei Hu, Ruslan Salakhutdinov, Sanjeev Arora

CoRR, 2019

Understanding Generalization of Deep Neural Networks Trained with Noisy Labels.

Wei Hu, Zhiyuan Li, Dingli Yu

CoRR, 2019

Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets.

Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, Sanjeev Arora

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

On Exact Computation with an Infinitely Wide Neural Net.

Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, Ruosong Wang

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks.

Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruosong Wang

Proceedings of the 36th International Conference on Machine Learning, 2019

The role of over-parametrization in generalization of neural networks.

Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, Nathan Srebro

Proceedings of the 7th International Conference on Learning Representations, 2019

Theoretical Analysis of Auto Rate-Tuning by Batch Normalization.

Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks.

Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, Nathan Srebro

CoRR, 2018

Online Improper Learning with an Approximation Oracle.

Elad Hazan, Wei Hu, Yuanzhi Li, Zhiyuan Li

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

2017

Stability of Generalized Two-sided Markets with Transaction Thresholds.

Zhiyuan Li, Yicheng Liu, Pingzhong Tang, Tingting Xu, Wei Zhan

Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, 2017

2016

Fast Convergence of Common Learning Algorithms in Games.

Dylan J. Foster, Zhiyuan Li, Thodoris Lykouris, Karthik Sridharan, Éva Tardos

CoRR, 2016

Solving Marginal MAP Problems with NP Oracles and Parity Constraints.

Yexiang Xue, Zhiyuan Li, Stefano Ermon, Carla P. Gomes, Bart Selman

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Learning in Games: Robustness of Fast Convergence.

Dylan J. Foster, Zhiyuan Li, Thodoris Lykouris, Karthik Sridharan, Éva Tardos

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016