Specialization as an optimal strategy under varying external conditions. - M. Ani Hsieh; Ádám M. Halász; Ekin Dogus Cubuk; Samuel S. Schoenholz; Alcherio Martinoli (ICRA, 2009)

Explaining the Learning Dynamics of Direct Feedback Alignment. - Justin Gilmer; Colin Raffel; Samuel S. Schoenholz; Maithra Raghu; Jascha Sohl-Dickstein (ICLR (Workshop), 2017)

Deep Information Propagation. - Samuel S. Schoenholz; Justin Gilmer; Surya Ganguli; Jascha Sohl-Dickstein (ICLR (Poster), 2017)

Neural Message Passing for Quantum Chemistry. - Justin Gilmer; Samuel S. Schoenholz; Patrick F. Riley; Oriol Vinyals; George E. Dahl (ICML, 2017)

Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice. - Jeffrey Pennington; Samuel S. Schoenholz; Surya Ganguli (NIPS, 2017)

Mean Field Residual Networks: On the Edge of Chaos. - Greg Yang; Samuel S. Schoenholz (NIPS, 2017)

A Correspondence Between Random Neural Networks and Statistical Field Theory. - Samuel S. Schoenholz; Jeffrey Pennington; Jascha Sohl-Dickstein (CoRR, 2017)

The emergence of spectral universality in deep networks. - Jeffrey Pennington; Samuel S. Schoenholz; Surya Ganguli (AISTATS, 2018)

Intriguing Properties of Adversarial Examples. - Ekin Dogus Cubuk; Barret Zoph; Samuel S. Schoenholz; Quoc V. Le (ICLR (Workshop), 2018)

Adversarial Spheres. - Justin Gilmer; Luke Metz; Fartash Faghri; Samuel S. Schoenholz; Maithra Raghu; Martin Wattenberg; Ian J. Goodfellow (ICLR (Workshop), 2018)

Deep Neural Networks as Gaussian Processes. - Jaehoon Lee 001; Yasaman Bahri; Roman Novak; Samuel S. Schoenholz; Jeffrey Pennington; Jascha Sohl-Dickstein (ICLR (Poster), 2018)

Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks. - Minmin Chen; Jeffrey Pennington; Samuel S. Schoenholz (ICML, 2018)

Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks. - Lechao Xiao; Yasaman Bahri; Jascha Sohl-Dickstein; Samuel S. Schoenholz; Jeffrey Pennington (ICML, 2018)

A Mean Field Theory of Batch Normalization. - Greg Yang; Jeffrey Pennington; Vinay Rao; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICLR (Poster), 2019)

MetaInit: Initializing learning by learning to initialize. - Yann N. Dauphin; Samuel S. Schoenholz (NeurIPS, 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Roman Novak; Jascha Sohl-Dickstein; Jeffrey Pennington (NeurIPS, 2019)

Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs. - Dar Gilboa; Bo Chang 002; Minmin Chen; Greg Yang; Samuel S. Schoenholz; Ed H. Chi; Jeffrey Pennington (CoRR, 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Jascha Sohl-Dickstein; Jeffrey Pennington (CoRR, 2019)

Disentangling trainability and generalization in deep learning. - Lechao Xiao; Jeffrey Pennington; Samuel S. Schoenholz (CoRR, 2019)

Neural Tangents: Fast and Easy Infinite Neural Networks in Python. - Roman Novak; Lechao Xiao; Jiri Hron; Jaehoon Lee 001; Alexander A. Alemi; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICLR, 2020)

Disentangling Trainability and Generalization in Deep Neural Networks. - Lechao Xiao; Jeffrey Pennington; Samuel Stern Schoenholz (ICML, 2020)

Finite Versus Infinite Neural Networks: an Empirical Study. - Jaehoon Lee 001; Samuel S. Schoenholz; Jeffrey Pennington; Ben Adlam; Lechao Xiao; Roman Novak; Jascha Sohl-Dickstein (NeurIPS, 2020)

JAX MD: A Framework for Differentiable Physics. - Samuel S. Schoenholz; Ekin Dogus Cubuk (NeurIPS, 2020)

On the infinite width limit of neural networks with a standard parameterization. - Jascha Sohl-Dickstein; Roman Novak; Samuel S. Schoenholz; Jaehoon Lee 001 (CoRR, 2020)

Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible. - Neha S. Wadia; Daniel Duckworth; Samuel S. Schoenholz; Ethan Dyer; Jascha Sohl-Dickstein (CoRR, 2020)

Learn2Hop: Learned Optimization on Rough Landscapes. - Amil Merchant; Luke Metz; Samuel S. Schoenholz; Ekin D. Cubuk (ICML, 2021)

Tilting the playing field: Dynamical loss functions for machine learning. - Miguel Ruiz-Garcia; Ge Zhang; Samuel S. Schoenholz; Andrea J. Liu (ICML, 2021)

Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization. - Neha S. Wadia; Daniel Duckworth; Samuel S. Schoenholz; Ethan Dyer; Jascha Sohl-Dickstein (ICML, 2021)

Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping. - James Martens; Andy Ballard; Guillaume Desjardins; Grzegorz Swirszcz; Valentin Dalibard; Jascha Sohl-Dickstein; Samuel S. Schoenholz (CoRR, 2021)

Gradients are Not All You Need. - Luke Metz; C. Daniel Freeman; Samuel S. Schoenholz; Tal Kachman (CoRR, 2021)

Deep equilibrium networks are sensitive to initialization statistics. - Atish Agarwala; Samuel S. Schoenholz (ICML, 2022)

Fast Finite Width Neural Tangent Kernel. - Roman Novak; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICML, 2022)

What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries. - Stanislav Fort; Ekin Dogus Cubuk; Surya Ganguli; Samuel S. Schoenholz (CoRR, 2022)

∂PV: An end-to-end differentiable solar-cell simulator. - Sean Mann; Eric Fadel; Samuel S. Schoenholz; Ekin D. Cubuk; Steven G. Johnson; Giuseppe Romano (Comput. Phys. Commun., 2022)

End-to-End Differentiable Reactive Molecular Dynamics Simulations Using JAX. - Mehmet Cagri Kaymak; Samuel S. Schoenholz; Ekin D. Cubuk; Kurt A. O'Hearn; Kenneth M. Merz Jr.; Hasan Metin Aktulga (ISC, 2023)

Scaling deep learning for materials discovery. - Amil Merchant; Simon L. Batzner; Samuel S. Schoenholz; Muratahan Aykol; Gowoon Cheon; Ekin Dogus Cubuk (Nat., 2023)

Temperature check: theory and practice for training models with softmax-cross-entropy losses. - Atish Agarwala; Samuel Stern Schoenholz; Jeffrey Pennington; Yann N. Dauphin (Trans. Mach. Learn. Res., 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)