Online Normalization for Training Neural Networks. - Vitaliy Chiley; Ilya Sharapov; Atli Kosson; Urs Köster; Ryan Reece; Sofia Samaniego de la Fuente; Vishal Subbiah; Michael James 002 (NeurIPS, 2019)

Adaptive Braking for Mitigating Gradient Delay. - Abhinav Venigalla; Atli Kosson; Vitaliy Chiley; Urs Köster (CoRR, 2020)

Pipelined Backpropagation at Scale: Training Large Models without Batches. - Atli Kosson; Vitaliy Chiley; Abhinav Venigalla; Joel Hestness; Urs Köster (MLSys, 2021)

Multiplication-Free Transformer Training via Piecewise Affine Operations. - Atli Kosson; Martin Jaggi (NeurIPS, 2023)

Hardware-Efficient Transformer Training via Piecewise Affine Operations. - Atli Kosson; Martin Jaggi (CoRR, 2023)

Rotational Optimizers: Simple & Robust DNN Training. - Atli Kosson; Bettina Messmer; Martin Jaggi (CoRR, 2023)

Memory Efficient Mixed-Precision Optimizers. - Basile Lewandowski; Atli Kosson (CoRR, 2023)

Ghost Noise for Regularizing Deep Neural Networks. - Atli Kosson; Dongyang Fan; Martin Jaggi (AAAI, 2024)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks. - Atli Kosson; Bettina Messmer; Martin Jaggi (ICML, 2024)

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations. - Alexander Hägele; Elie Bakouch; Atli Kosson; Loubna Ben Allal; Leandro von Werra; Martin Jaggi (NeurIPS, 2024)

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training. - Atli Kosson; Bettina Messmer; Martin Jaggi (NeurIPS, 2024)

Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler. - Aleksandr Dremov; Alexander Hägele; Atli Kosson; Martin Jaggi (Trans. Mach. Learn. Res., 2025)