A Modular Approach to Block-diagonal Hessian Approximations for Second-order Optimization Methods. - Felix Dangel; Philipp Hennig (CoRR, 2019)

Modular Block-diagonal Curvature Approximations for Feedforward Architectures. - Felix Dangel; Stefan Harmeling; Philipp Hennig (AISTATS, 2020)

BackPACK: Packing more into Backprop. - Felix Dangel; Frederik Kunstner; Philipp Hennig (ICLR, 2020)

Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks. - Frank Schneider 001; Felix Dangel; Philipp Hennig (NeurIPS, 2021)

The Geometry of Neural Nets' Parameter Spaces Under Reparametrization. - Agustinus Kristiadi; Felix Dangel; Philipp Hennig (NeurIPS, 2023)

Convolutions Through the Lens of Tensor Networks. - Felix Dangel (CoRR, 2023)

On the Disconnect Between Theory and Practice of Overparametrized Neural Networks. - Jonathan Wenger; Felix Dangel; Agustinus Kristiadi (CoRR, 2023)

Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC for Large Neural Nets. - Wu Lin; Felix Dangel; Runa Eschenhagen; Kirill Neklyudov; Agustinus Kristiadi; Richard E. Turner; Alireza Makhzani (CoRR, 2023)

ViViT: Curvature Access Through The Generalized Gauss-Newton's Low-Rank Structure. - Felix Dangel; Lukas Tatzel; Philipp Hennig (Trans. Mach. Learn. Res., 2023)

Backpropagation Beyond the Gradient. - Felix Dangel (2023)

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning. - Mohamed Elsayed 003; Homayoon Farrahi; Felix Dangel; A. Rupam Mahmood (ICML, 2024)

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective. - Wu Lin; Felix Dangel; Runa Eschenhagen; Juhan Bae; Richard E. Turner; Alireza Makhzani (ICML, 2024)

Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC. - Wu Lin; Felix Dangel; Runa Eschenhagen; Kirill Neklyudov; Agustinus Kristiadi; Richard E. Turner; Alireza Makhzani (ICML, 2024)

Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods. - Felix Dangel (NeurIPS, 2024)

Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks. - Felix Dangel; Johannes Müller; Marius Zeinhofer (NeurIPS, 2024)

Lowering PyTorch's Memory Consumption for Selective Differentiation. - Samarth Bhatia; Felix Dangel (CoRR, 2024)

What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis. - Weronika Ormaniec; Felix Dangel; Sidak Pal Singh (ICLR, 2025)

Position: Curvature Matrices Should Be Democratized via Linear Operators. - Felix Dangel; Runa Eschenhagen; Weronika Ormaniec; Andres Fernandez; Lukas Tatzel; Agustinus Kristiadi (CoRR, 2025)

Spectral-factorized Positive-definite Curvature Learning for NN Training. - Wu Lin; Felix Dangel; Runa Eschenhagen; Juhan Bae; Richard E. Turner; Roger B. Grosse (CoRR, 2025)

Hide & Seek: Transformer Symmetries Obscure Sharpness & Riemannian Geometry Finds It. - Marvin F. da Silva; Felix Dangel; Sageev Oore (CoRR, 2025)

Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization. - Andrés Guzmán-Cordero; Felix Dangel; Gil Goldshlager; Marius Zeinhofer (CoRR, 2025)

Collapsing Taylor Mode Automatic Differentiation. - Felix Dangel; Tim Siebert; Marius Zeinhofer; Andrea Walther (CoRR, 2025)

Kronecker-factored Approximate Curvature (KFAC) From Scratch. - Felix Dangel; Bálint Mucsányi; Tobias Weber; Runa Eschenhagen (CoRR, 2025)

Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator. - YuXin Li; Felix Dangel; Derek Tam; Colin Raffel (CoRR, 2025)