Monte Carlo Sort for unreliable human comparisons. - Samuel L. Smith (CoRR, 2016)

Offline bilingual word vectors, orthogonal transformations and the inverted softmax. - Samuel L. Smith; David H. P. Turban; Steven Hamblin; Nils Y. Hammerla (ICLR (Poster), 2017)

Don't Decay the Learning Rate, Increase the Batch Size. - Samuel L. Smith; Pieter-Jan Kindermans; Quoc V. Le (CoRR, 2017)

Don't Decay the Learning Rate, Increase the Batch Size. - Samuel L. Smith; Pieter-Jan Kindermans; Chris Ying; Quoc V. Le (ICLR (Poster), 2018)

A Bayesian Perspective on Generalization and Stochastic Gradient Descent. - Samuel L. Smith; Quoc V. Le (ICLR (Poster), 2018)

Decoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity Tasks. - Vitalii Zhelezniak; Dan Busbridge; April Shen; Samuel L. Smith; Nils Y. Hammerla (ICLR (Workshop), 2018)

Stochastic natural gradient descent draws posterior samples in function space. - Samuel L. Smith; Daniel Duckworth; Quoc V. Le; Jascha Sohl-Dickstein (CoRR, 2018)

The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study. - Daniel S. Park; Jascha Sohl-Dickstein; Quoc V. Le; Samuel L. Smith (ICML, 2019)

On the Generalization Benefit of Noise in Stochastic Gradient Descent. - Samuel L. Smith; Erich Elsen; Soham De (ICML, 2020)

Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks. - Soham De; Samuel L. Smith (NeurIPS, 2020)

Batch Normalization Biases Deep Residual Networks Towards Shallow Paths. - Soham De; Samuel L. Smith (CoRR, 2020)

Cold Posteriors and Aleatoric Uncertainty. - Ben Adlam; Jasper Snoek; Samuel L. Smith (CoRR, 2020)

BYOL works even without batch statistics. - Pierre H. Richemond; Jean-Bastien Grill; Florent Altché; Corentin Tallec; Florian Strub; Andrew Brock; Samuel L. Smith; Soham De; Razvan Pascanu; Bilal Piot; Michal Valko (CoRR, 2020)

Characterizing signal propagation to close the performance gap in unnormalized ResNets. - Andrew Brock; Soham De; Samuel L. Smith (ICLR, 2021)

On the Origin of Implicit Regularization in Stochastic Gradient Descent. - Samuel L. Smith; Benoit Dherin; David G. T. Barrett; Soham De (ICLR, 2021)

High-Performance Large-Scale Image Recognition Without Normalization. - Andy Brock; Soham De; Samuel L. Smith; Karen Simonyan (ICML, 2021)

Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error. - Stanislav Fort; Andrew Brock; Razvan Pascanu; Soham De; Samuel L. Smith (CoRR, 2021)

A study on the plasticity of neural networks. - Tudor Berariu; Wojciech Czarnecki 001; Soham De; Jörg Bornschein; Samuel L. Smith; Razvan Pascanu; Claudia Clopath (CoRR, 2021)

Unlocking High-Accuracy Differentially Private Image Classification through Scale. - Soham De; Leonard Berrada; Jamie Hayes; Samuel L. Smith; Borja Balle (CoRR, 2022)

Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation. - Bobby He; James Martens; Guodong Zhang; Aleksandar Botev; Andrew Brock; Samuel L. Smith; Yee Whye Teh (ICLR, 2023)

Resurrecting Recurrent Neural Networks for Long Sequences. - Antonio Orvieto; Samuel L. Smith; Albert Gu; Anushan Fernando; Çaglar Gülçehre; Razvan Pascanu; Soham De (ICML, 2023)

Differentially Private Diffusion Models Generate Useful Synthetic Images. - Sahra Ghalebikesabi; Leonard Berrada; Sven Gowal; Ira Ktena; Robert Stanforth; Jamie Hayes; Soham De; Samuel L. Smith; Olivia Wiles; Borja Balle (CoRR, 2023)

On the Universality of Linear Recurrences Followed by Nonlinear Projections. - Antonio Orvieto; Soham De; Çaglar Gülçehre; Razvan Pascanu; Samuel L. Smith (CoRR, 2023)

Unlocking Accuracy and Fairness in Differentially Private Image Classification. - Leonard Berrada; Soham De; Judy Hanwen Shen; Jamie Hayes; Robert Stanforth; David Stutz; Pushmeet Kohli; Samuel L. Smith; Borja Balle (CoRR, 2023)

ConvNets Match Vision Transformers at Scale. - Samuel L. Smith; Andrew Brock; Leonard Berrada; Soham De (CoRR, 2023)

Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues. - Antonio Orvieto; Soham De; Caglar Gulcehre; Razvan Pascanu; Samuel L. Smith (ICML, 2024)

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models. - Soham De; Samuel L. Smith; Anushan Fernando; Aleksandar Botev; George-Cristian Muraru; Albert Gu; Ruba Haroun; Leonard Berrada; Yutian Chen 001; Srivatsan Srinivasan; Guillaume Desjardins; Arnaud Doucet; et al. (CoRR, 2024)

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models. - Aleksandar Botev; Soham De; Samuel L. Smith; Anushan Fernando; George-Cristian Muraru; Ruba Haroun; Leonard Berrada; Razvan Pascanu; Pier Giuseppe Sessa; Robert Dadashi; Léonard Hussenot; Johan Ferret; et al. (CoRR, 2024)