Continuous Learning in a Hierarchical Multiscale Neural Network. - Thomas Wolf 008; Julien Chaumond; Clement Delangue (ACL (2), 2018)

Meta-Learning a Dynamical Language Model. - Thomas Wolf 008; Julien Chaumond; Clement Delangue (ICLR (Workshop), 2018)

A Hierarchical Multi-Task Approach for Learning Embeddings from Semantic Tasks. - Victor Sanh; Thomas Wolf 008; Sebastian Ruder (AAAI, 2019)

Large-Scale Transfer Learning for Natural Language Generation. - Sergey Golovanov; Rauf Kurbanov; Sergey I. Nikolenko; Kyryl Truskovskyi; Alexander Tselousov; Thomas Wolf 008 (ACL (1), 2019)

Transfer Learning in Natural Language Processing. - Sebastian Ruder; Matthew E. Peters; Swabha Swayamdipta; Thomas Wolf 008 (NAACL-HLT (Tutorial Abstracts), 2019)

TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents. - Thomas Wolf 008; Victor Sanh; Julien Chaumond; Clement Delangue (CoRR, 2019)

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. - Victor Sanh; Lysandre Debut; Julien Chaumond; Thomas Wolf 008 (CoRR, 2019)

HuggingFace's Transformers: State-of-the-art Natural Language Processing. - Thomas Wolf 008; Lysandre Debut; Victor Sanh; Julien Chaumond; Clement Delangue; Anthony Moi; Pierric Cistac; Tim Rault; Rémi Louf; Morgan Funtowicz; Jamie Brew (CoRR, 2019)

The Amazing World of Neural Language Generation. - Yangfeng Ji; Antoine Bosselut; Thomas Wolf 008; Asli Celikyilmaz (EMNLP (Tutorial Abstracts), 2020)

Overview of the SustaiNLP 2020 Shared Task. - Alex Wang; Thomas Wolf 008 (SustaiNLP@EMNLP, 2020)

Transformers: State-of-the-Art Natural Language Processing. - Thomas Wolf 008; Lysandre Debut; Victor Sanh; Julien Chaumond; Clement Delangue; Anthony Moi; Pierric Cistac; Tim Rault; Rémi Louf; Morgan Funtowicz; Joe Davison; Sam Shleifer; et al. (EMNLP (Demos), 2020)

Movement Pruning: Adaptive Sparsity by Fine-Tuning. - Victor Sanh; Thomas Wolf 008; Alexander M. Rush (NeurIPS, 2020)

TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation. - Shaojie Jiang; Thomas Wolf 008; Christof Monz; Maarten de Rijke (CoRR, 2020)

Datasets: A Community Library for Natural Language Processing. - Quentin Lhoest; Albert Villanova del Moral; Yacine Jernite; Abhishek Thakur; Patrick von Platen; Suraj Patil; Julien Chaumond; Mariama Drame; Julien Plu; Lewis Tunstall; Joe Davison; Mario Sasko; et al. (EMNLP (Demos), 2021)

Learning from others' mistakes: Avoiding dataset biases without modeling them. - Victor Sanh; Thomas Wolf 008; Yonatan Belinkov; Alexander M. Rush (ICLR, 2021)

Training Transformers Together. - Alexander Borzunov; Max Ryabinin; Tim Dettmers; Quentin Lhoest; Lucile Saulnier; Michael Diskin; Yacine Jernite; Thomas Wolf 008 (NeurIPS (Competition and Demos), 2021)

Distributed Deep Learning In Open Collaborations. - Michael Diskin; Alexey Bukhtiyarov; Max Ryabinin; Lucile Saulnier; Quentin Lhoest; Anton Sinitsin; Dmitry Popov 003; Dmitry V. Pyrkin; Maxim Kashirin; Alexander Borzunov; Albert Villanova del Moral; Denis Mazur; et al. (NeurIPS, 2021)

VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning. - Hao Tan 002; Jie Lei 003; Thomas Wolf 008; Mohit Bansal (CoRR, 2021)

Multitask Prompted Training Enables Zero-Shot Task Generalization. - Victor Sanh; Albert Webson; Colin Raffel; Stephen H. Bach; Lintang Sutawika; Zaid Alyafeai; Antoine Chaffin; Arnaud Stiegler; Teven Le Scao; Arun Raja; Manan Dey; M. Saiful Bari; et al. (CoRR, 2021)

Multitask Prompted Training Enables Zero-Shot Task Generalization. - Victor Sanh; Albert Webson; Colin Raffel; Stephen H. Bach; Lintang Sutawika; Zaid Alyafeai; Antoine Chaffin; Arnaud Stiegler; Arun Raja; Manan Dey; M Saiful Bari; Canwen Xu; et al. (ICLR, 2022)

Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements. - Leandro von Werra; Lewis Tunstall; Abhishek Thakur; Alexandra Sasha Luccioni; Tristan Thrush; Aleksandra Piktus; Felix Marty; Nazneen Rajani; Victor Mustar; Helen Ngo; Omar Sanseviero; Mario Sasko; et al. (CoRR, 2022)

BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model. - Christopher Akiki; Giada Pistilli; Margot Mieskes; Matthias Gallé; Thomas Wolf 008; Suzana Ilic; Yacine Jernite (CoRR, 2022)

FinGPT: Large Generative Models for a Small Language. - Risto Luukkonen; Ville Komulainen; Jouni Luoma; Anni Eskelinen; Jenna Kanerva; Hanna-Mari Kupari; Filip Ginter; Veronika Laippala; Niklas Muennighoff; Aleksandra Piktus; Thomas Wang; Nouamane Tazi; et al. (EMNLP, 2023)

Scaling Data-Constrained Language Models. - Niklas Muennighoff; Alexander M. Rush; Boaz Barak; Teven Le Scao; Nouamane Tazi; Aleksandra Piktus; Sampo Pyysalo; Thomas Wolf 008; Colin A. Raffel (NeurIPS, 2023)

Zephyr: Direct Distillation of LM Alignment. - Lewis Tunstall; Edward Beeching; Nathan Lambert 001; Nazneen Rajani; Kashif Rasul; Younes Belkada; Shengyi Huang; Leandro von Werra; Clémentine Fourrier; Nathan Habib; Nathan Sarrazin; Omar Sanseviero; et al. (CoRR, 2023)

GAIA: a benchmark for General AI Assistants. - Grégoire Mialon; Clémentine Fourrier; Craig Swift; Thomas Wolf 008; Yann LeCun; Thomas Scialom (CoRR, 2023)

The Stack: 3 TB of permissively licensed source code. - Denis Kocetkov; Raymond Li; Loubna Ben Allal; Jia Li 015; Chenghao Mou; Yacine Jernite; Margaret Mitchell; Carlos Muñoz Ferrandis; Sean Hughes; Thomas Wolf 008; Dzmitry Bahdanau; Leandro von Werra; et al. (Trans. Mach. Learn. Res., 2023)

StarCoder: may the source be with you! - Raymond Li; Loubna Ben Allal; Yangtian Zi; Niklas Muennighoff; Denis Kocetkov; Chenghao Mou; Marc Marone; Christopher Akiki; Jia Li 015; Jenny Chim; Qian Liu 012; Evgenii Zheltonozhskii; et al. (Trans. Mach. Learn. Res., 2023)

Federated benchmarking of medical artificial intelligence with MedPerf. - Alexandros Karargyris; Renato Umeton; Micah J. Sheller; Alejandro Aristizabal; Johnu George; Anna Wuest; Sarthak Pati; Hasan Kassem; Maximilian Zenk; Ujjwal Baid; Prakash Narayana Moorthy; Alexander Chowdhury; et al. (Nat. Mac. Intell., 2023)

GAIA: a benchmark for General AI Assistants. - Grégoire Mialon; Clémentine Fourrier; Thomas Wolf 008; Yann LeCun; Thomas Scialom (ICLR, 2024)

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale. - Guilherme Penedo; Hynek Kydlícek; Loubna Ben Allal; Anton Lozhkov; Margaret Mitchell; Colin A. Raffel; Leandro von Werra; Thomas Wolf 008 (NeurIPS, 2024)

Towards Best Practices for Open Datasets for LLM Training. - Stefan Baack; Stella Biderman; Kasia Odrozek; Aviya Skowron; Ayah Bdeir; Jillian Bommarito; Jennifer Ding; Maximilian Gahntz; Paul Keller; Pierre-Carl Langlais; Greg Lindahl; Sebastian Majstorovic; et al. (CoRR, 2025)

SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model. - Loubna Ben Allal; Anton Lozhkov; Elie Bakouch; Gabriel Martín Blázquez; Guilherme Penedo; Lewis Tunstall; Andrés Marafioti; Hynek Kydlícek; Agustín Piqueres Lajarín; Vaibhav Srivastav; Joshua Lochner; Caleb Fahlgren; et al. (CoRR, 2025)

SmolVLM: Redefining small and efficient multimodal models. - Andrés Marafioti; Orr Zohar; Miquel Farré; Merve Noyan; Elie Bakouch; Pedro Cuenca 002; Cyril Zakka; Loubna Ben Allal; Anton Lozhkov; Nouamane Tazi; Vaibhav Srivastav; Joshua Lochner; et al. (CoRR, 2025)

FineWeb2: One Pipeline to Scale Them All - Adapting Pre-Training Data Processing to Every Language. - Guilherme Penedo; Hynek Kydlícek; Vinko Sabolcec; Bettina Messmer; Negar Foroutan; Amir Hossein Kargaran; Colin Raffel; Martin Jaggi; Leandro von Werra; Thomas Wolf 008 (CoRR, 2025)

DABstep: Data Agent Benchmark for Multi-step Reasoning. - Alex Egg; Martin Iglesias Goyanes; Friso Kingma; Andreu Mora; Leandro von Werra; Thomas Wolf 008 (CoRR, 2025)