Training Transformers Together. - Alexander Borzunov; Max Ryabinin; Tim Dettmers; Quentin Lhoest; Lucile Saulnier; Michael Diskin; Yacine Jernite; Thomas Wolf 008 (NeurIPS (Competition and Demos), 2021)

Distributed Deep Learning In Open Collaborations. - Michael Diskin; Alexey Bukhtiyarov; Max Ryabinin; Lucile Saulnier; Quentin Lhoest; Anton Sinitsin; Dmitry Popov 003; Dmitry V. Pyrkin; Maxim Kashirin; Alexander Borzunov; Albert Villanova del Moral; Denis Mazur; et al. (NeurIPS, 2021)

Secure Distributed Training at Scale. - Eduard Gorbunov; Alexander Borzunov; Michael Diskin; Max Ryabinin (ICML, 2022)

Petals: Collaborative Inference and Fine-tuning of Large Models. - Alexander Borzunov; Dmitry Baranchuk; Tim Dettmers; Max Ryabinin; Younes Belkada; Artem Chumachenko; Pavel Samygin; Colin Raffel (CoRR, 2022)

Petals: Collaborative Inference and Fine-tuning of Large Models. - Alexander Borzunov; Dmitry Baranchuk; Tim Dettmers; Maksim Riabinin; Younes Belkada; Artem Chumachenko; Pavel Samygin; Colin Raffel (ACL (demo), 2023)

SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient. - Max Ryabinin; Tim Dettmers; Michael Diskin; Alexander Borzunov (ICML, 2023)

Distributed Inference and Fine-tuning of Large Language Models Over The Internet. - Alexander Borzunov; Max Ryabinin; Artem Chumachenko; Dmitry Baranchuk; Tim Dettmers; Younes Belkada; Pavel Samygin; Colin A. Raffel (NeurIPS, 2023)

SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression. - Tim Dettmers; Ruslan Svirschevski; Vage Egiazarian; Denis Kuznedelev; Elias Frantar; Saleh Ashkboos; Alexander Borzunov; Torsten Hoefler; Dan Alistarh (ICLR, 2024)