Nicolas Zucchet

According to our database, Nicolas Zucchet authored at least 13 papers between 2021 and 2025.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2025

The emergence of sparse attention: impact of data distribution and benefits of repetition.

Nicolas Zucchet, Francesco D'Angelo, Andrew K. Lampinen, Stephanie C. Y. Chan

CoRR, May, 2025

How do language models learn facts? Dynamics, curricula and hallucinations.

Nicolas Zucchet, Jörg Bornschein, Stephanie C. Y. Chan, Andrew K. Lampinen, Razvan Pascanu, Soham De

CoRR, March, 2025

2024

Recurrent neural networks: vanishing and exploding gradients are not the end of the story.

Nicolas Zucchet, Antonio Orvieto

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

2023

Uncovering mesa-optimization algorithms in Transformers.

Johannes von Oswald, Eyvind Niklasson, Maximilian Schlegel, Seijin Kobayashi, Nicolas Zucchet, Nino Scherrer, Nolan Miller, Mark Sandler, Blaise Agüera y Arcas, Max Vladymyrov, Razvan Pascanu, João Sacramento

CoRR, 2023

Gated recurrent neural networks discover attention.

Nicolas Zucchet, Seijin Kobayashi, Yassir Akram, Johannes von Oswald, Maxime Larcher, Angelika Steger, João Sacramento

CoRR, 2023

Online learning of long-range dependencies.

Nicolas Zucchet, Robert Meier, Simon Schug, Asier Mujika, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

2022

Beyond Backpropagation: Bilevel Optimization Through Implicit Differentiation and Equilibrium Propagation.

Nicolas Zucchet, João Sacramento

Neural Comput., 2022

Random initialisations performing above chance and how to find them.

Frederik Benzing, Simon Schug, Robert Meier, Johannes von Oswald, Yassir Akram, Nicolas Zucchet, Laurence Aitchison, Angelika Steger

CoRR, 2022

The least-control principle for learning at equilibrium.

Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento

CoRR, 2022

Beyond backpropagation: implicit gradients for bilevel optimization.

Nicolas Zucchet, João Sacramento

CoRR, 2022

A contrastive rule for meta-learning.

Nicolas Zucchet, Simon Schug, Johannes von Oswald, Dominic Zhao, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

The least-control principle for local learning at equilibrium.

Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

2021

Learning where to learn: Gradient sparsity in meta and continual learning.

Johannes von Oswald, Dominic Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, João Sacramento

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021