Frederik Kunstner

According to our database, Frederik Kunstner authored at least 11 papers between 2018 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models.

Frederik Kunstner, Robin Yadav, Alan Milligan, Mark Schmidt, Alberto Bietti

CoRR, 2024

2023

Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking.

Frederik Kunstner, Victor Sanches Portella, Mark Schmidt, Nicholas J. A. Harvey

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be.

Frederik Kunstner, Jacques Chen, Jonathan Wilder Lavington, Mark Schmidt

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent (Extended Abstract).

Frederik Kunstner, Raunak Kumar, Mark Schmidt

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, 2022

2021

Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent - an Open Problem.

Rémi Le Priol, Frederik Kunstner, Damien Scieur, Simon Lacoste-Julien

CoRR, 2021

Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent.

Frederik Kunstner, Raunak Kumar, Mark Schmidt

Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, 2021

2020

Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search).

Sharan Vaswani, Frederik Kunstner, Issam H. Laradji, Si Yi Meng, Mark Schmidt, Simon Lacoste-Julien

CoRR, 2020

BackPACK: Packing more into Backprop.

Felix Dangel, Frederik Kunstner, Philipp Hennig

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Limitations of the Empirical Fisher Approximation.

Frederik Kunstner, Lukas Balles, Philipp Hennig

CoRR, 2019

Limitations of the empirical Fisher approximation for natural gradient descent.

Frederik Kunstner, Philipp Hennig, Lukas Balles

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

2018

SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient.

Aaron Mishkin, Frederik Kunstner, Didrik Nielsen, Mark Schmidt, Mohammad Emtiyaz Khan

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018