Training Transformers Together. - Alexander Borzunov; Max Ryabinin; Tim Dettmers; Quentin Lhoest; Lucile Saulnier; Michael Diskin; Yacine Jernite; Thomas Wolf 008 (NeurIPS (Competition and Demos), 2021)

Distributed Deep Learning In Open Collaborations. - Michael Diskin; Alexey Bukhtiyarov; Max Ryabinin; Lucile Saulnier; Quentin Lhoest; Anton Sinitsin; Dmitry Popov 003; Dmitry V. Pyrkin; Maxim Kashirin; Alexander Borzunov; Albert Villanova del Moral; Denis Mazur; et al. (NeurIPS, 2021)

Secure Distributed Training at Scale. - Eduard Gorbunov; Alexander Borzunov; Michael Diskin; Max Ryabinin (ICML, 2022)

Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees. - Aleksandr Beznosikov; Peter Richtárik; Michael Diskin; Max Ryabinin; Alexander V. Gasnikov (NeurIPS, 2022)

A critical look at the evaluation of GNNs under heterophily: Are we really making progress? - Oleg Platonov; Denis Kuznedelev; Michael Diskin; Artem Babenko; Liudmila Prokhorenkova (ICLR, 2023)

SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient. - Max Ryabinin; Tim Dettmers; Michael Diskin; Alexander Borzunov (ICML, 2023)