Gradient Descent Happens in a Tiny Subspace. - Guy Gur-Ari; Daniel A. Roberts; Ethan Dyer (CoRR, 2018)

Wider Networks Learn Better Features. - Dar Gilboa; Guy Gur-Ari (CoRR, 2019)

Asymptotics of Wide Networks from Feynman Diagrams. - Ethan Dyer; Guy Gur-Ari (ICLR, 2020)

On the training dynamics of deep networks with $L_2$ regularization. - Aitor Lewkowycz; Guy Gur-Ari (NeurIPS, 2020)

The large learning rate phase of deep learning: the catapult mechanism. - Aitor Lewkowycz; Yasaman Bahri; Ethan Dyer; Jascha Sohl-Dickstein; Guy Gur-Ari (CoRR, 2020)

On the asymptotics of wide networks with polynomial activations. - Kyle Aitken; Guy Gur-Ari (CoRR, 2020)

On the training dynamics of deep networks with L2 regularization. - Aitor Lewkowycz; Guy Gur-Ari (CoRR, 2020)

Are wider nets better given the same number of parameters? - Anna Golubeva; Guy Gur-Ari; Behnam Neyshabur (ICLR, 2021)

Show Your Work: Scratchpads for Intermediate Computation with Language Models. - Maxwell I. Nye; Anders Johan Andreassen; Guy Gur-Ari; Henryk Michalewski; Jacob Austin; David Bieber; David Dohan; Aitor Lewkowycz; Maarten Bosma; David Luan; Charles Sutton; Augustus Odena (CoRR, 2021)

Exploring Length Generalization in Large Language Models. - Cem Anil; Yuhuai Wu; Anders Andreassen; Aitor Lewkowycz; Vedant Misra; Vinay V. Ramasesh; Ambrose Slone; Guy Gur-Ari; Ethan Dyer; Behnam Neyshabur (NeurIPS, 2022)

Solving Quantitative Reasoning Problems with Language Models. - Aitor Lewkowycz; Anders Andreassen; David Dohan; Ethan Dyer; Henryk Michalewski; Vinay V. Ramasesh; Ambrose Slone; Cem Anil; Imanol Schlag; Theo Gutman-Solo; Yuhuai Wu; Behnam Neyshabur; et al. (NeurIPS, 2022)

PaLM: Scaling Language Modeling with Pathways. - Aakanksha Chowdhery; Sharan Narang; Jacob Devlin; Maarten Bosma; Gaurav Mishra; Adam Roberts; Paul Barham 001; Hyung Won Chung; Charles Sutton; Sebastian Gehrmann; Parker Schuh; Kensen Shi; et al. (J. Mach. Learn. Res., 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)

Towards Understanding Inductive Bias in Transformers: A View From Infinity. - Itay Lavie; Guy Gur-Ari; Zohar Ringel (ICML, 2024)