Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates. - Sebastian U. Stich; Amirkeivan Mohtashami; Martin Jaggi (AISTATS, 2021)

Simultaneous Training of Partially Masked Neural Networks. - Amirkeivan Mohtashami; Martin Jaggi; Sebastian U. Stich (CoRR, 2021)

Masked Training of Neural Networks with Partial Gradients. - Amirkeivan Mohtashami; Martin Jaggi; Sebastian U. Stich (AISTATS, 2022)

Characterizing & Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methods. - Amirkeivan Mohtashami; Sebastian U. Stich; Martin Jaggi (CoRR, 2022)

On Avoiding Local Minima Using Gradient Descent With Large Learning Rates. - Amirkeivan Mohtashami; Martin Jaggi; Sebastian U. Stich (CoRR, 2022)

Special Properties of Gradient Descent with Large Learning Rates. - Amirkeivan Mohtashami; Martin Jaggi; Sebastian U. Stich (ICML, 2023)

Random-Access Infinite Context Length for Transformers. - Amirkeivan Mohtashami; Martin Jaggi (NeurIPS, 2023)

Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models. - Amirkeivan Mohtashami; Mauro Verzetti; Paul K. Rubenstein (CoRR, 2023)

Landmark Attention: Random-Access Infinite Context Length for Transformers. - Amirkeivan Mohtashami; Martin Jaggi (CoRR, 2023)

CoTFormer: More Tokens With Attention Make Up For Less Depth. - Amirkeivan Mohtashami; Matteo Pagliardini; Martin Jaggi (CoRR, 2023)

MEDITRON-70B: Scaling Medical Pretraining for Large Language Models. - Zeming Chen 001; Alejandro Hernández-Cano; Angelika Romanou; Antoine Bonnet; Kyle Matoba; Francesco Salvi; Matteo Pagliardini; Simin Fan; Andreas Köpf; Amirkeivan Mohtashami; Alexandre Sallinen; Alireza Sakhaeirad; et al. (CoRR, 2023)

Social Learning: Towards Collaborative Learning with Large Language Models. - Amirkeivan Mohtashami; Florian Hartmann; Sian Gooding; Lukas Zilka; Matthew Sharifi; Blaise Agüera y Arcas (CoRR, 2023)

The splay-list: a distribution-adaptive concurrent skip-list. - Vitaly Aksenov; Dan Alistarh; Alexandra Drozdova; Amirkeivan Mohtashami (Distributed Comput., 2023)

QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. - Saleh Ashkboos; Amirkeivan Mohtashami; Maximilian L. Croci; Bo Li; Pashmina Cameron; Martin Jaggi; Dan Alistarh; Torsten Hoefler; James Hensman (NeurIPS, 2024)

DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging. - Matteo Pagliardini; Amirkeivan Mohtashami; François Fleuret; Martin Jaggi (NeurIPS, 2024)

QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. - Saleh Ashkboos; Amirkeivan Mohtashami; Maximilian L. Croci; Bo Li; Martin Jaggi; Dan Alistarh; Torsten Hoefler; James Hensman (CoRR, 2024)

CoTFormer: A Chain of Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference. - Amirkeivan Mohtashami; Matteo Pagliardini; Martin Jaggi (ICLR, 2025)