Deep Neural Networks as Gaussian Processes. - Jaehoon Lee 001; Yasaman Bahri; Roman Novak; Samuel S. Schoenholz; Jeffrey Pennington; Jascha Sohl-Dickstein (ICLR (Poster), 2018)

Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes. - Roman Novak; Lechao Xiao; Jaehoon Lee 001; Yasaman Bahri; Daniel A. Abolafia; Jeffrey Pennington; Jascha Sohl-Dickstein (CoRR, 2018)

Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes. - Roman Novak; Lechao Xiao; Yasaman Bahri; Jaehoon Lee 001; Greg Yang; Jiri Hron; Daniel A. Abolafia; Jeffrey Pennington; Jascha Sohl-Dickstein (ICLR (Poster), 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Roman Novak; Jascha Sohl-Dickstein; Jeffrey Pennington (NeurIPS, 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Jascha Sohl-Dickstein; Jeffrey Pennington (CoRR, 2019)

On Empirical Comparisons of Optimizers for Deep Learning. - Dami Choi; Christopher J. Shallue; Zachary Nado; Jaehoon Lee 001; Chris J. Maddison; George E. Dahl (CoRR, 2019)

Measuring the Effects of Data Parallelism on Neural Network Training. - Christopher J. Shallue; Jaehoon Lee 001; Joseph M. Antognini; Jascha Sohl-Dickstein; Roy Frostig; George E. Dahl (J. Mach. Learn. Res., 2019)

Neural Tangents: Fast and Easy Infinite Neural Networks in Python. - Roman Novak; Lechao Xiao; Jiri Hron; Jaehoon Lee 001; Alexander A. Alemi; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICLR, 2020)

Finite Versus Infinite Neural Networks: an Empirical Study. - Jaehoon Lee 001; Samuel S. Schoenholz; Jeffrey Pennington; Ben Adlam; Lechao Xiao; Roman Novak; Jascha Sohl-Dickstein (NeurIPS, 2020)

On the infinite width limit of neural networks with a standard parameterization. - Jascha Sohl-Dickstein; Roman Novak; Samuel S. Schoenholz; Jaehoon Lee 001 (CoRR, 2020)

Towards NNGP-guided Neural Architecture Search. - Daniel S. Park; Jaehoon Lee 001; Daiyi Peng; Yuan Cao; Jascha Sohl-Dickstein (CoRR, 2020)

Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit. - Ben Adlam; Jaehoon Lee 001; Lechao Xiao; Jeffrey Pennington; Jasper Snoek (ICLR, 2021)

Dataset Distillation with Infinitely Wide Convolutional Networks. - Timothy Nguyen; Roman Novak; Lechao Xiao; Jaehoon Lee 001 (NeurIPS, 2021)

Explaining Neural Scaling Laws. - Yasaman Bahri; Ethan Dyer; Jared Kaplan; Jaehoon Lee 001; Utkarsh Sharma (CoRR, 2021)

Fast Neural Kernel Embeddings for General Activations. - Insu Han; Amir Zandieh; Jaehoon Lee 001; Roman Novak; Lechao Xiao; Amin Karbasi (NeurIPS, 2022)

Kernel Regression with Infinite-Width Neural Networks on Millions of Examples. - Ben Adlam; Jaehoon Lee 001; Shreyas Padhy; Zachary Nado; Jasper Snoek (CoRR, 2023)

Replacing softmax with ReLU in Vision Transformers. - Mitchell Wortsman; Jaehoon Lee 001; Justin Gilmer; Simon Kornblith (CoRR, 2023)

Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5? - C. Daniel Freeman; Laura Culp; Aaron Parisi; Maxwell L. Bileschi; Gamaleldin F. Elsayed; Alex Rizkowsky; Isabelle Simpson; Alex Alemi; Azade Nova; Ben Adlam; Bernd Bohnet; Gaurav Mishra; et al. (CoRR, 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)

Small-scale proxies for large-scale Transformer training instabilities. - Mitchell Wortsman; Peter J. Liu; Lechao Xiao; Katie E. Everett; Alexander A. Alemi; Ben Adlam; John D. Co-Reyes; Izzeddin Gur; Abhishek Kumar; Roman Novak; Jeffrey Pennington; Jascha Sohl-Dickstein; et al. (ICLR, 2024)

Scaling Exponents Across Parameterizations and Optimizers. - Katie E. Everett; Lechao Xiao; Mitchell Wortsman; Alexander A. Alemi; Roman Novak; Peter J. Liu; Izzeddin Gur; Jascha Sohl-Dickstein; Leslie Pack Kaelbling; Jaehoon Lee 001; Jeffrey Pennington (ICML, 2024)

Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. - Charlie Snell; Jaehoon Lee 001; Kelvin Xu; Aviral Kumar (CoRR, 2024)

Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability. - Jiri Hron; Laura Culp; Gamaleldin F. Elsayed; Rosanne Liu; Ben Adlam; Maxwell L. Bileschi; Bernd Bohnet; JD Co-Reyes; Noah Fiedel; C. Daniel Freeman; Izzeddin Gur; Kathleen Kenealy; et al. (CoRR, 2024)

Training LLMs over Neurally Compressed Text. - Brian Lester; Jaehoon Lee 001; Alexander A. Alemi; Jeffrey Pennington; Adam Roberts; Jascha Sohl-Dickstein; Noah Constant (Trans. Mach. Learn. Res., 2024)

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models. - Avi Singh; John D. Co-Reyes; Rishabh Agarwal; Ankesh Anand; Piyush Patil; Xavier Garcia; Peter J. Liu; James Harrison; Jaehoon Lee 001; Kelvin Xu; Aaron T. Parisi; Abhishek Kumar; et al. (Trans. Mach. Learn. Res., 2024)

Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Parameters for Reasoning. - Charlie Victor Snell; Jaehoon Lee 001; Kelvin Xu; Aviral Kumar (ICLR, 2025)