Continual learning with hypernetworks. - Johannes von Oswald; Christian Henning; João Sacramento; Benjamin F. Grewe (ICLR, 2020)

Continual Learning in Recurrent Neural Networks with Hypernetworks. - Benjamin Ehret; Christian Henning; Maria R. Cervera; Alexander Meulemans; Johannes von Oswald; Benjamin F. Grewe (CoRR, 2020)

Economical ensembles with hypernetworks. - João Sacramento; Johannes von Oswald; Seijin Kobayashi; Christian Henning; Benjamin F. Grewe (CoRR, 2020)

Continual learning in recurrent neural networks. - Benjamin Ehret; Christian Henning; Maria R. Cervera; Alexander Meulemans; Johannes von Oswald; Benjamin F. Grewe (ICLR, 2021)

Neural networks with late-phase weights. - Johannes von Oswald; Seijin Kobayashi; João Sacramento; Alexander Meulemans; Christian Henning; Benjamin F. Grewe (ICLR, 2021)

Posterior Meta-Replay for Continual Learning. - Christian Henning; Maria R. Cervera; Francesco D'Angelo; Johannes von Oswald; Regina Traber; Benjamin Ehret; Seijin Kobayashi; Benjamin F. Grewe; João Sacramento (NeurIPS, 2021)

Learning where to learn: Gradient sparsity in meta and continual learning. - Johannes von Oswald; Dominic Zhao; Seijin Kobayashi; Simon Schug; Massimo Caccia; Nicolas Zucchet; João Sacramento (NeurIPS, 2021)

Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel. - Seijin Kobayashi; Pau Vilimelis Aceituno; Johannes von Oswald (NeurIPS, 2022)

The least-control principle for local learning at equilibrium. - Alexander Meulemans; Nicolas Zucchet; Seijin Kobayashi; Johannes von Oswald; João Sacramento (NeurIPS, 2022)

A contrastive rule for meta-learning. - Nicolas Zucchet; Simon Schug; Johannes von Oswald; Dominic Zhao; João Sacramento (NeurIPS, 2022)

The least-control principle for learning at equilibrium. - Alexander Meulemans; Nicolas Zucchet; Seijin Kobayashi; Johannes von Oswald; João Sacramento (CoRR, 2022)

Random initialisations performing above chance and how to find them. - Frederik Benzing; Simon Schug; Robert Meier; Johannes von Oswald; Yassir Akram; Nicolas Zucchet; Laurence Aitchison; Angelika Steger (CoRR, 2022)

Transformers Learn In-Context by Gradient Descent. - Johannes von Oswald; Eyvind Niklasson; Ettore Randazzo; João Sacramento; Alexander Mordvintsev; Andrey Zhmoginov; Max Vladymyrov (ICML, 2023)

Gated recurrent neural networks discover attention. - Nicolas Zucchet; Seijin Kobayashi; Yassir Akram; Johannes von Oswald; Maxime Larcher; Angelika Steger; João Sacramento (CoRR, 2023)

Uncovering mesa-optimization algorithms in Transformers. - Johannes von Oswald; Eyvind Niklasson; Maximilian Schlegel; Seijin Kobayashi; Nicolas Zucchet; Nino Scherrer; Nolan Miller; Mark Sandler 002; Blaise Agüera y Arcas; Max Vladymyrov; Razvan Pascanu; João Sacramento (CoRR, 2023)

Discovering modular solutions that generalize compositionally. - Simon Schug; Seijin Kobayashi; Yassir Akram; Maciej Wolczyk; Alexandra Proca; Johannes von Oswald; Razvan Pascanu; João Sacramento; Angelika Steger (ICLR, 2024)

Weight decay induces low-rank attention layers. - Seijin Kobayashi; Yassir Akram; Johannes von Oswald (NeurIPS, 2024)

Linear Transformers are Versatile In-Context Learners. - Max Vladymyrov; Johannes von Oswald; Mark Sandler 002; Rong Ge (NeurIPS, 2024)

State Soup: In-Context Skill Learning, Retrieval and Mixing. - Maciej Pióro; Maciej Wolczyk; Razvan Pascanu; Johannes von Oswald; João Sacramento (CoRR, 2024)

When can transformers compositionally generalize in-context? - Seijin Kobayashi; Simon Schug; Yassir Akram; Florian Redhardt; Johannes von Oswald; Razvan Pascanu; Guillaume Lajoie; João Sacramento (CoRR, 2024)

Adversarial Robustness of In-Context Learning in Transformers for Linear Regression. - Usman Anwar; Johannes von Oswald; Louis Kirsch; David Krueger 001; Spencer Frei (CoRR, 2024)

Interpretability of Learning Algorithms Encoded in Deep Neural Networks. - Johannes von Oswald (2024)

Multi-agent cooperation through learning-aware policy gradients. - Alexander Meulemans; Seijin Kobayashi; Johannes von Oswald; Nino Scherrer; Eric Elmoznino; Blake Aaron Richards; Guillaume Lajoie; Blaise Agüera y Arcas; João Sacramento (ICLR, 2025)

Learning Randomized Algorithms with Transformers. - Johannes von Oswald; Seijin Kobayashi; Yassir Akram; Angelika Steger (ICLR, 2025)

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training. - Johannes von Oswald; Nino Scherrer; Seijin Kobayashi; Luca Versari; Songlin Yang; Maximilian Schlegel; Kaitlin Maile; Yanick Schimpf; Oliver Sieberling; Alexander Meulemans; Rif A. Saurous; Guillaume Lajoie; et al. (CoRR, 2025)