On the Selection of Initialization and Activation Function for Deep Neural Networks. - Soufiane Hayou; Arnaud Doucet; Judith Rousseau (CoRR, 2018)

On the Impact of the Activation function on Deep Neural Networks Training. - Soufiane Hayou; Arnaud Doucet; Judith Rousseau (ICML, 2019)

Training Dynamics of Deep Networks using Stochastic Gradient Descent via Neural Tangent Kernel. - Soufiane Hayou; Arnaud Doucet; Judith Rousseau (CoRR, 2019)

Pruning untrained neural networks: Principles and Analysis. - Soufiane Hayou; Jean-Francois Ton; Arnaud Doucet; Yee Whye Teh (CoRR, 2020)

Stable ResNet. - Soufiane Hayou; Eugenio Clerico; Bobby He; George Deligiannidis; Arnaud Doucet; Judith Rousseau (AISTATS, 2021)

The Curse of Depth in Kernel Regime. - Soufiane Hayou; Arnaud Doucet; Judith Rousseau (ICBINB@NeurIPS, 2021)

Robust Pruning at Initialization. - Soufiane Hayou; Jean-Francois Ton; Arnaud Doucet; Yee Whye Teh (ICLR, 2021)

Regularization in ResNet with Stochastic Depth. - Soufiane Hayou; Fadhel Ayed (NeurIPS, 2021)

The Equilibrium Hypothesis: Rethinking implicit regularization in Deep Neural Networks. - Yizhang Lou; Chris Mingard; Soufiane Hayou (CoRR, 2021)

Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning. - Soufiane Hayou; Bobby He; Gintare Karolina Dziugaite (CoRR, 2021)

Wide deep neural networks. - Soufiane Hayou (2021)

Feature Learning and Signal Propagation in Deep Neural Networks. - Yizhang Lou; Chris E. Mingard; Soufiane Hayou (ICML, 2022)

Connecting Optimization and Generalization via Gradient Flow Path Length. - Fusheng Liu; Haizhao Yang; Soufiane Hayou; Qianxiao Li (CoRR, 2022)

From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality. - Fusheng Liu; Haizhao Yang; Soufiane Hayou; Qianxiao Li (Trans. Mach. Learn. Res., 2022)

Width and Depth Limits Commute in Residual Networks. - Soufiane Hayou; Greg Yang (ICML, 2023)

On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks. - Soufiane Hayou (CoRR, 2023)

Commutative Width and Depth Scaling in Deep Neural Networks. - Soufiane Hayou (CoRR, 2023)

Data pruning and neural scaling laws: fundamental limitations of score-based algorithms. - Fadhel Ayed; Soufiane Hayou (Trans. Mach. Learn. Res., 2023)

On the infinite-depth limit of finite-width neural networks. - Soufiane Hayou (Trans. Mach. Learn. Res., 2023)

Leave-one-out Distinguishability in Machine Learning. - Jiayuan Ye 001; Anastasia Borovykh; Soufiane Hayou; Reza Shokri (ICLR, 2024)

Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks. - Greg Yang; Dingli Yu; Chen Zhu; Soufiane Hayou (ICLR, 2024)

LoRA+: Efficient Low Rank Adaptation of Large Models. - Soufiane Hayou; Nikhil Ghosh; Bin Yu 001 (ICML, 2024)

The Impact of Initialization on LoRA Finetuning Dynamics. - Soufiane Hayou; Nikhil Ghosh; Bin Yu 001 (NeurIPS, 2024)

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse. - Mohamed El Amine Seddik; Suei-Wen Chen; Soufiane Hayou; Pierre Youssef; Mérouane Debbah (CoRR, 2024)

Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map. - Yoonsoo Nam; Chris Mingard; Seok Hyeong Lee; Soufiane Hayou; Ard A. Louis (CoRR, 2024)

Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory. - Aymane El Firdoussi; Mohamed El Amine Seddik; Soufiane Hayou; Réda Alami; Ahmed Alzubaidi; Hakim Hacid (ICLR, 2025)

On the Stability of the Jacobian Matrix in Deep Neural Networks. - Benjamin Dadoun; Soufiane Hayou; Hanan Salam; Mohamed El Amine Seddik; Pierre Youssef (CoRR, 2025)

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size. - Soufiane Hayou; Liyuan Liu (CoRR, 2025)

PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models. - Soufiane Hayou; Nikhil Ghosh; Bin Yu 001 (CoRR, 2025)