Johannes von Oswald

Stefan Bauer

CoRR, February, 2026

From Growing to Looping: A Unified View of Iterative Computation in LLMs.

[BibT_eX]

[DOI]

CoRR, February, 2026

2025

Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning.

[BibT_eX]

[DOI]

Alexander Meulemans

CoRR, December, 2025

Do Depth-Grown Models Overcome the Curse of Depth? An In-Depth Analysis.

[BibT_eX]

[DOI]

CoRR, December, 2025

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training.

[BibT_eX]

[DOI]

CoRR, June, 2025

Understanding In-Context Learning of Linear Models in Transformers Through an Adversarial Lens.

[BibT_eX]

[DOI]

Trans. Mach. Learn. Res., 2025

Learning Randomized Algorithms with Transformers.

[BibT_eX]

[DOI]

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

Multi-agent cooperation through learning-aware policy gradients.

[BibT_eX]

[DOI]

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

2024

Interpretability of Learning Algorithms Encoded in Deep Neural Networks.

[BibT_eX]

[DOI]

PhD thesis, 2024

Adversarial Robustness of In-Context Learning in Transformers for Linear Regression.

[BibT_eX]

[DOI]

CoRR, 2024

When can transformers compositionally generalize in-context?

[BibT_eX]

[DOI]

CoRR, 2024

State Soup: In-Context Skill Learning, Retrieval and Mixing.

[BibT_eX]

[DOI]

CoRR, 2024

Linear Transformers are Versatile In-Context Learners.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, 2024

Weight decay induces low-rank attention layers.

[BibT_eX]

[DOI]

Seijin Kobayashi

Yassir Akram

Proceedings of the Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, 2024

Discovering modular solutions that generalize compositionally.

[BibT_eX]

[DOI]

Alexandra Maria Proca

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Uncovering mesa-optimization algorithms in Transformers.

[BibT_eX]

[DOI]

Max Vladymyrov

Razvan Pascanu

CoRR, 2023

Gated recurrent neural networks discover attention.

[BibT_eX]

[DOI]

CoRR, 2023

Transformers Learn In-Context by Gradient Descent.

[BibT_eX]

[DOI]

Alexander Mordvintsev

Andrey Zhmoginov

Max Vladymyrov

Proceedings of the International Conference on Machine Learning, 2023

2022

Random initialisations performing above chance and how to find them.

[BibT_eX]

[DOI]

CoRR, 2022

The least-control principle for learning at equilibrium.

[BibT_eX]

[DOI]

CoRR, 2022

A contrastive rule for meta-learning.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

The least-control principle for local learning at equilibrium.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel.

[BibT_eX]

[DOI]

Seijin Kobayashi

Pau Vilimelis Aceituno