Johannes von Oswald

According to our database, Johannes von Oswald authored at least 17 papers between 2020 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Linear Transformers are Versatile In-Context Learners.

Max Vladymyrov, Johannes von Oswald, Mark Sandler, Rong Ge

CoRR, 2024

2023

Discovering modular solutions that generalize compositionally.

Simon Schug, Seijin Kobayashi, Yassir Akram, Maciej Wolczyk, Alexandra Proca, Johannes von Oswald, Razvan Pascanu, João Sacramento, Angelika Steger

CoRR, 2023

Uncovering mesa-optimization algorithms in Transformers.

Johannes von Oswald, Eyvind Niklasson, Maximilian Schlegel, Seijin Kobayashi, Nicolas Zucchet, Nino Scherrer, Nolan Miller, Mark Sandler, Blaise Agüera y Arcas, Max Vladymyrov, Razvan Pascanu, João Sacramento

CoRR, 2023

Gated recurrent neural networks discover attention.

Nicolas Zucchet, Seijin Kobayashi, Yassir Akram, Johannes von Oswald, Maxime Larcher, Angelika Steger, João Sacramento

CoRR, 2023

Transformers Learn In-Context by Gradient Descent.

Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov

Proceedings of the International Conference on Machine Learning, 2023

2022

Random initialisations performing above chance and how to find them.

Frederik Benzing, Simon Schug, Robert Meier, Johannes von Oswald, Yassir Akram, Nicolas Zucchet, Laurence Aitchison, Angelika Steger

CoRR, 2022

The least-control principle for learning at equilibrium.

Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento

CoRR, 2022

A contrastive rule for meta-learning.

Nicolas Zucchet, Simon Schug, Johannes von Oswald, Dominic Zhao, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

The least-control principle for local learning at equilibrium.

Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel.

Seijin Kobayashi, Pau Vilimelis Aceituno, Johannes von Oswald

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

2021

Learning where to learn: Gradient sparsity in meta and continual learning.

Johannes von Oswald, Dominic Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Posterior Meta-Replay for Continual Learning.

Christian Henning, Maria R. Cervera, Francesco D'Angelo, Johannes von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F. Grewe, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Neural networks with late-phase weights.

Johannes von Oswald, Seijin Kobayashi, João Sacramento, Alexander Meulemans, Christian Henning, Benjamin F. Grewe

Proceedings of the 9th International Conference on Learning Representations, 2021

Continual learning in recurrent neural networks.

Benjamin Ehret, Christian Henning, Maria R. Cervera, Alexander Meulemans, Johannes von Oswald, Benjamin F. Grewe

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

Economical ensembles with hypernetworks.

João Sacramento, Johannes von Oswald, Seijin Kobayashi, Christian Henning, Benjamin F. Grewe

CoRR, 2020

Continual Learning in Recurrent Neural Networks with Hypernetworks.

Benjamin Ehret, Christian Henning, Maria R. Cervera, Alexander Meulemans, Johannes von Oswald, Benjamin F. Grewe

CoRR, 2020

Continual learning with hypernetworks.

Johannes von Oswald, Christian Henning, João Sacramento, Benjamin F. Grewe

Proceedings of the 8th International Conference on Learning Representations, 2020