Hadi Daneshmand

According to our database, Hadi Daneshmand authored at least 24 papers between 2014 and 2023.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2023

Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion.

Alexandru Meterez, Amir Joudaki, Francesco Orabona, Alexander Immer, Gunnar Rätsch, Hadi Daneshmand

CoRR, 2023

On the impact of activation and normalization in obtaining isometric embeddings at initialization.

Amir Joudaki, Hadi Daneshmand, Francis R. Bach

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Transformers learn to implement preconditioned gradient descent for in-context learning.

Kwangjun Ahn, Xiang Cheng, Hadi Daneshmand, Suvrit Sra

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

On Bridging the Gap between Mean Field and Finite Width Deep Random Multilayer Perceptron with Batch Normalization.

Amir Joudaki, Hadi Daneshmand, Francis R. Bach

Proceedings of the International Conference on Machine Learning, 2023

Efficient displacement convex optimization with particle gradient descent.

Hadi Daneshmand, Jason D. Lee, Chi Jin

Proceedings of the International Conference on Machine Learning, 2023

2022

Entropy Maximization with Depth: A Variational Principle for Random Neural Networks.

Amir Joudaki, Hadi Daneshmand, Francis R. Bach

CoRR, 2022

Polynomial-time sparse measure recovery.

Hadi Daneshmand, Francis R. Bach

CoRR, 2022

2021

Rethinking the Variational Interpretation of Accelerated Optimization Methods.

Peiyuan Zhang, Antonio Orvieto, Hadi Daneshmand

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Batch Normalization Orthogonalizes Representations in Deep Random Networks.

Hadi Daneshmand, Amir Joudaki, Francis R. Bach

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization.

Peiyuan Zhang, Antonio Orvieto, Hadi Daneshmand, Thomas Hofmann, Roy S. Smith

Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, 2021

2020

Optimization for Neural Networks: Quest for Theoretical Understandings.

Hadi Daneshmand

PhD thesis, 2020

Theoretical Understanding of Batch-normalization: A Markov Chain Perspective.

Hadi Daneshmand, Jonas Moritz Kohler, Francis R. Bach, Thomas Hofmann, Aurélien Lucchi

CoRR, 2020

Batch normalization provably avoids ranks collapse for randomly initialised deep networks.

Hadi Daneshmand, Jonas Moritz Kohler, Francis R. Bach, Thomas Hofmann, Aurélien Lucchi

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

2019

Mixing of Stochastic Accelerated Gradient Descent.

Peiyuan Zhang, Hadi Daneshmand, Thomas Hofmann

CoRR, 2019

Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization.

Jonas Moritz Kohler, Hadi Daneshmand, Aurélien Lucchi, Thomas Hofmann, Ming Zhou, Klaus Neymeyr

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

Local Saddle Point Optimization: A Curvature Exploitation Approach.

Leonard Adolphs, Hadi Daneshmand, Aurélien Lucchi, Thomas Hofmann

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

2018

Towards a Theoretical Understanding of Batch Normalization.

Jonas Moritz Kohler, Hadi Daneshmand, Aurélien Lucchi, Ming Zhou, Klaus Neymeyr, Thomas Hofmann

CoRR, 2018

Escaping Saddles with Stochastic Gradients.

Hadi Daneshmand, Jonas Moritz Kohler, Aurélien Lucchi, Thomas Hofmann

Proceedings of the 35th International Conference on Machine Learning, 2018

2017

Accelerated Dual Learning by Homotopic Initialization.

Hadi Daneshmand, Hamed Hassani, Thomas Hofmann

CoRR, 2017

2016

Estimating Diffusion Networks: Recovery Conditions, Sample Complexity and Soft-thresholding Algorithm.

Manuel Gomez-Rodriguez, Le Song, Hadi Daneshmand, Bernhard Schölkopf

J. Mach. Learn. Res., 2016

DynaNewton - Accelerating Newton's Method for Machine Learning.

Hadi Daneshmand, Aurélien Lucchi, Thomas Hofmann

CoRR, 2016

Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy.

Aryan Mokhtari, Hadi Daneshmand, Aurélien Lucchi, Thomas Hofmann, Alejandro Ribeiro

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Starting Small - Learning with Adaptive Sample Sizes.

Hadi Daneshmand, Aurélien Lucchi, Thomas Hofmann

Proceedings of the 33nd International Conference on Machine Learning, 2016

2014

Estimating Diffusion Network Structures: Recovery Conditions, Sample Complexity & Soft-thresholding Algorithm.

Hadi Daneshmand, Manuel Gomez-Rodriguez, Le Song, Bernhard Schölkopf

Proceedings of the 31th International Conference on Machine Learning, 2014