Atli Kosson

According to our database, Atli Kosson authored at least 12 papers between 2019 and 2025.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2025

Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler.

Aleksandr Dremov, Alexander Hägele, Atli Kosson, Martin Jaggi

Trans. Mach. Learn. Res., 2025

2024

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training.

Atli Kosson, Bettina Messmer, Martin Jaggi

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations.

Alexander Hägele, Elie Bakouch, Atli Kosson, Loubna Ben Allal, Leandro von Werra, Martin Jaggi

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks.

Atli Kosson, Bettina Messmer, Martin Jaggi

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Ghost Noise for Regularizing Deep Neural Networks.

Atli Kosson, Dongyang Fan, Martin Jaggi

Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

2023

Memory Efficient Mixed-Precision Optimizers.

Basile Lewandowski, Atli Kosson

CoRR, 2023

Rotational Optimizers: Simple & Robust DNN Training.

Atli Kosson, Bettina Messmer, Martin Jaggi

CoRR, 2023

Hardware-Efficient Transformer Training via Piecewise Affine Operations.

Atli Kosson, Martin Jaggi

CoRR, 2023

Multiplication-Free Transformer Training via Piecewise Affine Operations.

Atli Kosson, Martin Jaggi

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

2021

Pipelined Backpropagation at Scale: Training Large Models without Batches.

Atli Kosson, Vitaliy Chiley, Abhinav Venigalla, Joel Hestness, Urs Köster

Proceedings of the Fourth Conference on Machine Learning and Systems, 2021

2020

Adaptive Braking for Mitigating Gradient Delay.

Abhinav Venigalla, Atli Kosson, Vitaliy Chiley, Urs Köster

CoRR, 2020

2019

Online Normalization for Training Neural Networks.

Vitaliy Chiley, Ilya Sharapov, Atli Kosson, Urs Köster, Ryan Reece, Sofia Samaniego de la Fuente, Vishal Subbiah, Michael James

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019