8-Bit Approximations for Parallelism in Deep Learning. - Tim Dettmers (ICLR (Poster), 2016)

Convolutional 2D Knowledge Graph Embeddings. - Tim Dettmers; Pasquale Minervini; Pontus Stenetorp; Sebastian Riedel 001 (AAAI, 2018)

Jack the Reader - A Machine Reading Framework. - Dirk Weissenborn; Pasquale Minervini; Isabelle Augenstein; Johannes Welbl; Tim Rocktäschel; Matko Bosnjak; Jeff Mitchell 001; Thomas Demeester; Tim Dettmers; Pontus Stenetorp; Sebastian Riedel 001 (ACL (4), 2018)

Sparse Networks from Scratch: Faster Training without Losing Performance. - Tim Dettmers; Luke Zettlemoyer (CoRR, 2019)

High Performance Natural Language Processing. - Gabriel Ilharco; Cesar Ilharco; Iulia Turc; Tim Dettmers; Felipe Ferreira; Kenton Lee (EMNLP (Tutorial Abstracts), 2020)

BASE Layers: Simplifying Training of Large, Sparse Models. - Mike Lewis; Shruti Bhosale; Tim Dettmers; Naman Goyal 001; Luke Zettlemoyer (ICML, 2021)

Training Transformers Together. - Alexander Borzunov; Max Ryabinin; Tim Dettmers; Quentin Lhoest; Lucile Saulnier; Michael Diskin; Yacine Jernite; Thomas Wolf 008 (NeurIPS (Competition and Demos), 2021)

8-bit Optimizers via Block-wise Quantization. - Tim Dettmers; Mike Lewis; Sam Shleifer; Luke Zettlemoyer (ICLR, 2022)

GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale. - Tim Dettmers; Mike Lewis; Younes Belkada; Luke Zettlemoyer (NeurIPS, 2022)

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models. - Margaret Li; Suchin Gururangan; Tim Dettmers; Mike Lewis; Tim Althoff; Noah A. Smith; Luke Zettlemoyer (CoRR, 2022)

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. - Tim Dettmers; Mike Lewis; Younes Belkada; Luke Zettlemoyer (CoRR, 2022)

Petals: Collaborative Inference and Fine-tuning of Large Models. - Alexander Borzunov; Dmitry Baranchuk; Tim Dettmers; Max Ryabinin; Younes Belkada; Artem Chumachenko; Pavel Samygin; Colin Raffel (CoRR, 2022)

Petals: Collaborative Inference and Fine-tuning of Large Models. - Alexander Borzunov; Dmitry Baranchuk; Tim Dettmers; Maksim Riabinin; Younes Belkada; Artem Chumachenko; Pavel Samygin; Colin Raffel (ACL (demo), 2023)

Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model. - Zeyu Liu; Tim Dettmers; Xi Lin; Veselin Stoyanov; Xian Li 003 (EMNLP, 2023)

The case for 4-bit precision: k-bit Inference Scaling Laws. - Tim Dettmers; Luke Zettlemoyer (ICML, 2023)

SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient. - Max Ryabinin; Tim Dettmers; Michael Diskin; Alexander Borzunov (ICML, 2023)

Distributed Inference and Fine-tuning of Large Language Models Over The Internet. - Alexander Borzunov; Max Ryabinin; Artem Chumachenko; Dmitry Baranchuk; Tim Dettmers; Younes Belkada; Pavel Samygin; Colin A. Raffel (NeurIPS, 2023)

QLoRA: Efficient Finetuning of Quantized LLMs. - Tim Dettmers; Artidoro Pagnoni; Ari Holtzman; Luke Zettlemoyer (NeurIPS, 2023)

Stable and low-precision training for large-scale vision-language models. - Mitchell Wortsman; Tim Dettmers; Luke Zettlemoyer; Ari Morcos; Ali Farhadi; Ludwig Schmidt (NeurIPS, 2023)

Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model. - Leo Z. Liu; Tim Dettmers; Xi Victoria Lin; Veselin Stoyanov; Xian Li 003 (CoRR, 2023)

SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression. - Tim Dettmers; Ruslan Svirschevski; Vage Egiazarian; Denis Kuznedelev; Elias Frantar; Saleh Ashkboos; Alexander Borzunov; Torsten Hoefler; Dan Alistarh (ICLR, 2024)

MatFormer: Nested Transformer for Elastic Inference. - Devvrit; Sneha Kudugunta; Aditya Kusupati; Tim Dettmers; Kaifeng Chen; Inderjit S. Dhillon; Yulia Tsvetkov; Hanna Hajishirzi; Sham M. Kakade; Ali Farhadi; Prateek Jain 002 (NeurIPS, 2024)

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore. - Rulin Shao; Jacqueline He; Akari Asai; Weijia Shi; Tim Dettmers; Sewon Min; Luke Zettlemoyer; Pang Wei Koh (NeurIPS, 2024)

OLMoE: Open Mixture-of-Experts Language Models. - Niklas Muennighoff; Luca Soldaini; Dirk Groeneveld; Kyle Lo; Jacob Morrison; Sewon Min; Weijia Shi; Pete Walsh; Oyvind Tafjord; Nathan Lambert 001; Yuling Gu; Shane Arora; et al. (CoRR, 2024)

Accessible Foundation Models: Systems, Algorithms, and Science - Tim Dettmers (2024)

Holistically Evaluating the Environmental Impact of Creating Language Models. - Jacob Morrison; Clara Na; Jared Fernandez; Tim Dettmers; Emma Strubell; Jesse Dodge (ICLR, 2025)

OLMoE: Open Mixture-of-Experts Language Models. - Niklas Muennighoff; Luca Soldaini; Dirk Groeneveld; Kyle Lo; Jacob Morrison; Sewon Min; Weijia Shi; Evan Pete Walsh; Oyvind Tafjord; Nathan Lambert 001; Yuling Gu; Shane Arora; et al. (ICLR, 2025)