Rotational Optimizers: Simple & Robust DNN Training. - Atli Kosson; Bettina Messmer; Martin Jaggi (CoRR, 2023)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks. - Atli Kosson; Bettina Messmer; Martin Jaggi (ICML, 2024)

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training. - Atli Kosson; Bettina Messmer; Martin Jaggi (NeurIPS, 2024)

Towards an empirical understanding of MoE design choices. - Dongyang Fan; Bettina Messmer; Martin Jaggi (CoRR, 2024)

On-device Collaborative Language Modeling via a Mixture of Generalists and Specialists. - Dongyang Fan; Bettina Messmer; Martin Jaggi (CoRR, 2024)

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection. - Bettina Messmer; Vinko Sabolcec; Martin Jaggi (CoRR, 2025)

FineWeb2: One Pipeline to Scale Them All - Adapting Pre-Training Data Processing to Every Language. - Guilherme Penedo; Hynek Kydlícek; Vinko Sabolcec; Bettina Messmer; Negar Foroutan; Amir Hossein Kargaran; Colin Raffel; Martin Jaggi; Leandro von Werra; Thomas Wolf 008 (CoRR, 2025)