Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes. - Jack W. Rae; Jonathan J. Hunt; Ivo Danihelka; Timothy Harley; Andrew W. Senior; Gregory Wayne; Alex Graves; Tim Lillicrap (NIPS, 2016)

Model-Free Episodic Control. - Charles Blundell; Benigno Uria; Alexander Pritzel; Yazhe Li; Avraham Ruderman; Joel Z. Leibo; Jack W. Rae; Daan Wierstra; Demis Hassabis (CoRR, 2016)

Memory-based Parameter Adaptation. - Pablo Sprechmann; Siddhant M. Jayakumar; Jack W. Rae; Alexander Pritzel; Adrià Puigdomènech Badia; Benigno Uria; Oriol Vinyals; Demis Hassabis; Razvan Pascanu; Charles Blundell (ICLR (Poster), 2018)

Fast Parametric Learning with Activation Memorization. - Jack W. Rae; Chris Dyer; Peter Dayan; Timothy P. Lillicrap (ICML, 2018)

Relational recurrent neural networks. - Adam Santoro; Ryan Faulkner 001; David Raposo; Jack W. Rae; Mike Chrzanowski; Theophane Weber; Daan Wierstra; Oriol Vinyals; Razvan Pascanu; Timothy P. Lillicrap (NeurIPS, 2018)

Neural Arithmetic Logic Units. - Andrew Trask; Felix Hill; Scott E. Reed; Jack W. Rae; Chris Dyer; Phil Blunsom (NeurIPS, 2018)

Unsupervised Predictive Memory in a Goal-Directed Agent. - Greg Wayne; Chia-Chun Hung; David Amos; Mehdi Mirza; Arun Ahuja; Agnieszka Grabska-Barwinska; Jack W. Rae; Piotr Mirowski; Joel Z. Leibo; Adam Santoro; Mevlana Gemici; Malcolm Reynolds; et al. (CoRR, 2018)

Meta-Learning Neural Bloom Filters. - Jack W. Rae; Sergey Bartunov; Timothy P. Lillicrap (ICML, 2019)

Training Language GANs from Scratch. - Cyprien de Masson d'Autume; Shakir Mohamed; Mihaela Rosca; Jack W. Rae (NeurIPS, 2019)

Compressive Transformers for Long-Range Sequence Modelling. - Jack W. Rae; Anna Potapenko; Siddhant M. Jayakumar; Timothy P. Lillicrap (CoRR, 2019)

Do Transformers Need Deep Long-Range Memory? - Jack W. Rae; Ali Razavi (ACL, 2020)

Meta-Learning Deep Energy-Based Memory Models. - Sergey Bartunov; Jack W. Rae; Simon Osindero; Timothy P. Lillicrap (ICLR, 2020)

Multiplicative Interactions and Where to Find Them. - Siddhant M. Jayakumar; Wojciech M. Czarnecki; Jacob Menick; Jonathan Schwarz; Jack W. Rae; Simon Osindero; Yee Whye Teh; Tim Harley; Razvan Pascanu (ICLR, 2020)

Compressive Transformers for Long-Range Sequence Modelling. - Jack W. Rae; Anna Potapenko; Siddhant M. Jayakumar; Chloe Hillier; Timothy P. Lillicrap (ICLR, 2020)

V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control. - H. Francis Song; Abbas Abdolmaleki; Jost Tobias Springenberg; Aidan Clark; Hubert Soyer; Jack W. Rae; Seb Noury; Arun Ahuja; Siqi Liu 002; Dhruva Tirumala; Nicolas Heess; Dan Belov; et al. (ICLR, 2020)

Stabilizing Transformers for Reinforcement Learning. - Emilio Parisotto; H. Francis Song; Jack W. Rae; Razvan Pascanu; Çaglar Gülçehre; Siddhant M. Jayakumar; Max Jaderberg; Raphaël Lopez Kaufman; Aidan Clark; Seb Noury; Matthew M. Botvinick; Nicolas Heess; et al. (ICML, 2020)

Top-KAST: Top-K Always Sparse Training. - Siddhant M. Jayakumar; Razvan Pascanu; Jack W. Rae; Simon Osindero; Erich Elsen (NeurIPS, 2020)

Scaling Language Models: Methods, Analysis & Insights from Training Gopher. - Jack W. Rae; Sebastian Borgeaud; Trevor Cai; Katie Millican; Jordan Hoffmann; H. Francis Song; John Aslanides; Sarah Henderson; Roman Ring; Susannah Young; Eliza Rutherford; Tom Hennigan; et al. (CoRR, 2021)

Towards lifelong reasoning with sparse and compressive memory systems - Jack W. Rae (2021)

Improving Language Models by Retrieving from Trillions of Tokens. - Sebastian Borgeaud; Arthur Mensch; Jordan Hoffmann; Trevor Cai; Eliza Rutherford; Katie Millican; George van den Driessche 002; Jean-Baptiste Lespiau; Bogdan Damoc; Aidan Clark; Diego de Las Casas; Aurelia Guy; et al. (ICML, 2022)

Unified Scaling Laws for Routed Language Models. - Aidan Clark; Diego de Las Casas; Aurelia Guy; Arthur Mensch; Michela Paganini; Jordan Hoffmann; Bogdan Damoc; Blake A. Hechtman; Trevor Cai; Sebastian Borgeaud; George van den Driessche 002; Eliza Rutherford; et al. (ICML, 2022)

An empirical analysis of compute-optimal large language model training. - Jordan Hoffmann; Sebastian Borgeaud; Arthur Mensch; Elena Buchatskaya; Trevor Cai; Eliza Rutherford; Diego de Las Casas; Lisa Anne Hendricks; Johannes Welbl; Aidan Clark; Tom Hennigan; Eric Noland; et al. (NeurIPS, 2022)

Unified Scaling Laws for Routed Language Models. - Aidan Clark; Diego de Las Casas; Aurelia Guy; Arthur Mensch; Michela Paganini; Jordan Hoffmann; Bogdan Damoc; Blake A. Hechtman; Trevor Cai; Sebastian Borgeaud; George van den Driessche 002; Eliza Rutherford; et al. (CoRR, 2022)

Training Compute-Optimal Large Language Models. - Jordan Hoffmann; Sebastian Borgeaud; Arthur Mensch; Elena Buchatskaya; Trevor Cai; Eliza Rutherford; Diego de Las Casas; Lisa Anne Hendricks; Johannes Welbl; Aidan Clark; Tom Hennigan; Eric Noland; et al. (CoRR, 2022)