Felix Dangel

According to our database, Felix Dangel authored at least 24 papers between 2019 and 2025.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2025

Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator.

YuXin Li, Felix Dangel, Derek Tam, Colin Raffel

CoRR, July, 2025

Kronecker-factored Approximate Curvature (KFAC) From Scratch.

Felix Dangel, Bálint Mucsányi, Tobias Weber, Runa Eschenhagen

CoRR, July, 2025

Collapsing Taylor Mode Automatic Differentiation.

Felix Dangel, Tim Siebert, Marius Zeinhofer, Andrea Walther

CoRR, May, 2025

Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization.

Andrés Guzmán-Cordero, Felix Dangel, Gil Goldshlager, Marius Zeinhofer

CoRR, May, 2025

Hide & Seek: Transformer Symmetries Obscure Sharpness & Riemannian Geometry Finds It.

Marvin F. da Silva, Felix Dangel, Sageev Oore

CoRR, May, 2025

Spectral-factorized Positive-definite Curvature Learning for NN Training.

Wu Lin, Felix Dangel, Runa Eschenhagen, Juhan Bae, Richard E. Turner, Roger B. Grosse

CoRR, February, 2025

Position: Curvature Matrices Should Be Democratized via Linear Operators.

Felix Dangel, Runa Eschenhagen, Weronika Ormaniec, Andres Fernandez, Lukas Tatzel, Agustinus Kristiadi

CoRR, January, 2025

What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis.

Weronika Ormaniec, Felix Dangel, Sidak Pal Singh

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

2024

Lowering PyTorch's Memory Consumption for Selective Differentiation.

Samarth Bhatia, Felix Dangel

CoRR, 2024

Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks.

Felix Dangel, Johannes Müller, Marius Zeinhofer

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods.

Felix Dangel

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC.

Wu Lin, Felix Dangel, Runa Eschenhagen, Kirill Neklyudov, Agustinus Kristiadi, Richard E. Turner, Alireza Makhzani

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective.

Wu Lin, Felix Dangel, Runa Eschenhagen, Juhan Bae, Richard E. Turner, Alireza Makhzani

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning.

Mohamed Elsayed, Homayoon Farrahi, Felix Dangel, A. Rupam Mahmood

Proceedings of the Forty-first International Conference on Machine Learning, 2024

2023

Backpropagation Beyond the Gradient.

Felix Dangel

PhD thesis, 2023

ViViT: Curvature Access Through The Generalized Gauss-Newton's Low-Rank Structure.

Felix Dangel, Lukas Tatzel, Philipp Hennig

Trans. Mach. Learn. Res., 2023

Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC for Large Neural Nets.

Wu Lin, Felix Dangel, Runa Eschenhagen, Kirill Neklyudov, Agustinus Kristiadi, Richard E. Turner, Alireza Makhzani

CoRR, 2023

On the Disconnect Between Theory and Practice of Overparametrized Neural Networks.

Jonathan Wenger, Felix Dangel, Agustinus Kristiadi

CoRR, 2023

Convolutions Through the Lens of Tensor Networks.

Felix Dangel

CoRR, 2023

The Geometry of Neural Nets' Parameter Spaces Under Reparametrization.

Agustinus Kristiadi, Felix Dangel, Philipp Hennig

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

2021

Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks.

Frank Schneider, Felix Dangel, Philipp Hennig

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

2020

BackPACK: Packing more into Backprop.

Felix Dangel, Frederik Kunstner, Philipp Hennig

Proceedings of the 8th International Conference on Learning Representations, 2020

Modular Block-diagonal Curvature Approximations for Feedforward Architectures.

Felix Dangel, Stefan Harmeling, Philipp Hennig

Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, 2020

2019

A Modular Approach to Block-diagonal Hessian Approximations for Second-order Optimization Methods.

Felix Dangel, Philipp Hennig

CoRR, 2019