CamemBERT: a Tasty French Language Model. - Louis Martin; Benjamin Muller; Pedro Javier Ortiz Suárez; Yoann Dupont; Laurent Romary; Éric de la Clergerie; Djamé Seddah; Benoît Sagot (ACL, 2020)

Building a User-Generated Content North-African Arabizi Treebank: Tackling Hell. - Djamé Seddah; Farah Essaidi; Amal Fethi; Matthieu Futeral; Benjamin Muller; Pedro Javier Ortiz Suárez; Benoît Sagot; Abhishek Srivastava (ACL, 2020)

A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages. - Pedro Javier Ortiz Suárez; Laurent Romary; Benoît Sagot (ACL, 2020)

SinNer@Clef-Hipe2020 : Sinful adaptation of SotA models for Named Entity Recognition in French and German. - Pedro Javier Ortiz Suárez; Yoann Dupont; Gaël Lejeune; Tian Tian (CLEF (Working Notes), 2020)

Establishing a New State-of-the-Art for French Named Entity Recognition. - Pedro Javier Ortiz Suárez; Yoann Dupont; Benjamin Muller; Laurent Romary; Benoît Sagot (LREC, 2020)

Les modèles de langue contextuels Camembert pour le français : impact de la taille et de l'hétérogénéité des données d'entrainement (C AMEM BERT Contextual Language Models for French: Impact of Training Data Size and Heterogeneity ). - Louis Martin; Benjamin Muller; Pedro Javier Ortiz Suárez; Yoann Dupont; Laurent Romary; Éric Villemonte de la Clergerie; Benoît Sagot; Djamé Seddah (JEP-TALN-RECITAL (2), 2020)

A Data-driven Approach to Named Entity Recognition for Early Modern French. - Pedro Ortiz Suarez; Simon Gabay (COLING, 2022)

Towards a Cleaner Document-Oriented Multilingual Crawled Corpus. - Julien Abadji; Pedro Javier Ortiz Suárez; Laurent Romary; Benoît Sagot (LREC, 2022)

From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French. - Simon Gabay; Pedro Ortiz Suarez; Alexandre Bartz; Alix Chagué; Rachel Bawden; Philippe Gambette; Benoît Sagot (LREC, 2022)

BERTrade: Using Contextual Embeddings to Parse Old French. - Loïc Grobol; Mathilde Regnault; Pedro Javier Ortiz Suárez; Benoît Sagot; Laurent Romary; Benoît Crabbé (LREC, 2022)

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset. - Hugo Laurençon; Lucile Saulnier; Thomas Wang; Christopher Akiki; Albert Villanova del Moral; Teven Le Scao; Leandro von Werra; Chenghao Mou; Eduardo González Ponferrada; Huu Nguyen; Jörg Frohberg; Mario Sasko; et al. (NeurIPS, 2022)

Le projet FREEM : ressources, outils et enjeux pour l'étude du français d'Ancien Régime (The F RE EM project: Resources, tools and challenges for the study of Ancien Régime French). - Simon Gabay; Pedro Javier Ortiz Suárez; Rachel Bawden; Alexandre Bartz; Philippe Gambette; Benoît Sagot (TALN-RECITAL, 2022)

Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources. - Angelina McMillan-Major; Zaid Alyafeai; Stella Biderman; Kimbo Chen; Francesco De Toni; Gérard Dupont; Hady Elsahar; Chris Emezue; Alham Fikri Aji; Suzana Ilic; Nurulaqilla Khamis; Colin Leong; et al. (CoRR, 2022)

Automatic Extraction of Materials and Properties from Superconductors Scientific Literature. - Luca Foppiano; Pedro Baptista de Castro; Pedro Ortiz Suarez; Kensei Terashima; Yoshihiko Takano; Masashi Ishii (CoRR, 2022)

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. - Teven Le Scao; Angela Fan; Christopher Akiki; Ellie Pavlick; Suzana Ilic; Daniel Hesslow; Roman Castagné; Alexandra Sasha Luccioni; François Yvon; Matthias Gallé; Jonathan Tow; Alexander M. Rush; et al. (CoRR, 2022)

Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Data. - Tim Jansen; Yangling Tong; Victoria Zevallos; Pedro Ortiz Suarez (CoRR, 2022)

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets. - Julia Kreutzer; Isaac Caswell; Lisa Wang; Ahsan Wahab; Daan van Esch; Nasanbayar Ulzii-Orshikh; Allahsera Tapo; Nishant Subramani; Artem Sokolov; Claytone Sikasote; Monang Setyawan; Supheakmungkol Sarin; et al. (Trans. Assoc. Comput. Linguistics, 2022)

A Data-driven Approach to Natural Language Processing for Contemporary and Historical French. (Une approche basée sur les données pour le traitement automatique du langage naturel en français contemporain et historique). - Pedro Javier Ortiz Suárez (2022)

Semi-automatic staging area for high-quality structured data extraction from scientific literature. - Luca Foppiano; Tomoya Mato; Kensei Terashima; Pedro Ortiz Suarez; Taku Tou; Chikako Sakai; Wei-Sheng Wang; Toshiyuki Amagasa; Yoshihiko Takano; Masashi Ishii (CoRR, 2023)

A CURATEd CATalog: Rethinking the Extraction of Pretraining Corpora for Mid-Resourced Languages. - Jorge Palomar-Giner; José Javier Saiz; Ferran Espuña; Mario Mina; Severino Da Dalt; Joan Llop; Malte Ostendorff; Pedro Ortiz Suarez; Georg Rehm; Aitor Gonzalez-Agirre; Marta Villegas (LREC/COLING, 2024)

Tokenizer Choice For LLM Training: Negligible or Crucial? - Mehdi Ali; Michael Fromm 001; Klaudia Thellmann; Richard Rutmann; Max Lübbering; Johannes Leveling; Katrin Klug; Jan Ebert; Niclas Doll; Jasper Schulze Buschhoff; Charvi Jain; Alexander Arno Weber; et al. (NAACL-HLT (Findings), 2024)

Occiglot at WMT24: European Open-source Large Language Models Evaluated on Translation. - Eleftherios Avramidis; Annika Grützner-Zahn; Manuel Brack; Patrick Schramowski; Pedro Ortiz Suarez; Malte Ostendorff; Fabio Barth; Shushen Manakhimova; Vivien Macketanz; Georg Rehm; Kristian Kersting (WMT, 2024)

Molyé: A Corpus-based Approach to Language Contact in Colonial France. - Rasul Dent; Juliette Janes; Thibault Clérice; Pedro Ortiz Suarez; Benoît Sagot (CoRR, 2024)

Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs. - Mehdi Ali; Michael Fromm 001; Klaudia Thellmann; Jan Ebert; Alexander Arno Weber; Richard Rutmann; Charvi Jain; Max Lübbering; Daniel Steinigen; Johannes Leveling; Katrin Klug; Jasper Schulze Buschhoff; et al. (CoRR, 2024)

Data Processing for the OpenGPT-X Model Family. - Nicolo' Brandizzi; Hammam Abdelwahab; Anirban Bhowmick; Lennard Helmer; Benny Jörg Stein; Pavel Denisov; Qasid Saleem; Michael Fromm 001; Mehdi Ali; Richard Rutmann; Farzad Naderi; Mohamad Saif Agy; et al. (CoRR, 2024)

mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus. - Matthieu Futeral; Armel Randy Zebaze; Pedro Ortiz Suarez; Julien Abadji; Rémi Lacroix; Cordelia Schmid; Rachel Bawden; Benoît Sagot (ACL (Findings), 2025)

KréyoLID From Language Identification Towards Language Mining. - Rasul Dent; Pedro Ortiz Suarez; Thibault Clérice; Benoît Sagot (CoRR, 2025)

Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data. - Ekaterina Borisova 002; Fabio Barth; Nils Feldhus; Raia Abu Ahmad; Malte Ostendorff; Pedro Ortiz Suarez; Georg Rehm; Sebastian Möller 001 (CoRR, 2025)