Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks. - Lechao Xiao; Yasaman Bahri; Jascha Sohl-Dickstein; Samuel S. Schoenholz; Jeffrey Pennington (ICML, 2018)

Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes. - Roman Novak; Lechao Xiao; Jaehoon Lee 001; Yasaman Bahri; Daniel A. Abolafia; Jeffrey Pennington; Jascha Sohl-Dickstein (CoRR, 2018)

Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes. - Roman Novak; Lechao Xiao; Yasaman Bahri; Jaehoon Lee 001; Greg Yang; Jiri Hron; Daniel A. Abolafia; Jeffrey Pennington; Jascha Sohl-Dickstein (ICLR (Poster), 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Roman Novak; Jascha Sohl-Dickstein; Jeffrey Pennington (NeurIPS, 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Jascha Sohl-Dickstein; Jeffrey Pennington (CoRR, 2019)

Disentangling trainability and generalization in deep learning. - Lechao Xiao; Jeffrey Pennington; Samuel S. Schoenholz (CoRR, 2019)

Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks. - Wei Hu; Lechao Xiao; Jeffrey Pennington (ICLR, 2020)

Neural Tangents: Fast and Easy Infinite Neural Networks in Python. - Roman Novak; Lechao Xiao; Jiri Hron; Jaehoon Lee 001; Alexander A. Alemi; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICLR, 2020)

Disentangling Trainability and Generalization in Deep Neural Networks. - Lechao Xiao; Jeffrey Pennington; Samuel Stern Schoenholz (ICML, 2020)

The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks. - Wei Hu; Lechao Xiao; Ben Adlam; Jeffrey Pennington (NeurIPS, 2020)

Finite Versus Infinite Neural Networks: an Empirical Study. - Jaehoon Lee 001; Samuel S. Schoenholz; Jeffrey Pennington; Ben Adlam; Lechao Xiao; Roman Novak; Jascha Sohl-Dickstein (NeurIPS, 2020)

Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit. - Ben Adlam; Jaehoon Lee 001; Lechao Xiao; Jeffrey Pennington; Jasper Snoek (ICLR, 2021)

Dataset Distillation with Infinitely Wide Convolutional Networks. - Timothy Nguyen; Roman Novak; Lechao Xiao; Jaehoon Lee 001 (NeurIPS, 2021)

Eigenspace Restructuring: A Principle of Space and Frequency in Neural Networks. - Lechao Xiao (COLT, 2022)

Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm. - Lechao Xiao; Jeffrey Pennington (ICML, 2022)

Fast Neural Kernel Embeddings for General Activations. - Insu Han; Amir Zandieh; Jaehoon Lee 001; Roman Novak; Lechao Xiao; Amin Karbasi (NeurIPS, 2022)

Precise Learning Curves and Higher-Order Scalings for Dot-product Kernel Regression. - Lechao Xiao; Hong Hu; Theodor Misiakiewicz; Yue Lu; Jeffrey Pennington (NeurIPS, 2022)

Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regression. - Lechao Xiao; Jeffrey Pennington (CoRR, 2022)

Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5? - C. Daniel Freeman; Laura Culp; Aaron Parisi; Maxwell L. Bileschi; Gamaleldin F. Elsayed; Alex Rizkowsky; Isabelle Simpson; Alex Alemi; Azade Nova; Ben Adlam; Bernd Bohnet; Gaurav Mishra; et al. (CoRR, 2023)

Small-scale proxies for large-scale Transformer training instabilities. - Mitchell Wortsman; Peter J. Liu; Lechao Xiao; Katie E. Everett; Alexander A. Alemi; Ben Adlam; John D. Co-Reyes; Izzeddin Gur; Abhishek Kumar; Roman Novak; Jeffrey Pennington; Jascha Sohl-Dickstein; et al. (ICLR, 2024)

Scaling Exponents Across Parameterizations and Optimizers. - Katie E. Everett; Lechao Xiao; Mitchell Wortsman; Alexander A. Alemi; Roman Novak; Peter J. Liu; Izzeddin Gur; Jascha Sohl-Dickstein; Leslie Pack Kaelbling; Jaehoon Lee 001; Jeffrey Pennington (ICML, 2024)

4+3 Phases of Compute-Optimal Neural Scaling Laws. - Elliot Paquette; Courtney Paquette; Lechao Xiao; Jeffrey Pennington (NeurIPS, 2024)

Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability. - Jiri Hron; Laura Culp; Gamaleldin F. Elsayed; Rosanne Liu; Ben Adlam; Maxwell L. Bileschi; Bernd Bohnet; JD Co-Reyes; Noah Fiedel; C. Daniel Freeman; Izzeddin Gur; Kathleen Kenealy; et al. (CoRR, 2024)

Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling. - Lechao Xiao (CoRR, 2024)

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models. - Avi Singh; John D. Co-Reyes; Rishabh Agarwal; Ankesh Anand; Piyush Patil; Xavier Garcia; Peter J. Liu; James Harrison; Jaehoon Lee 001; Kelvin Xu; Aaron T. Parisi; Abhishek Kumar; et al. (Trans. Mach. Learn. Res., 2024)

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks. - Shikai Qiu; Lechao Xiao; Andrew Gordon Wilson; Jeffrey Pennington; Atish Agarwala (CoRR, 2025)