MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling. - Nathan Godey; Roman Castagné; Éric de la Clergerie; Benoît Sagot (EMNLP (Findings), 2022)

MANTa: Efficient Gradient-Based Tokenization for Robust End-to-End Language Modeling. - Nathan Godey; Roman Castagné; Éric de la Clergerie; Benoît Sagot (CoRR, 2022)

Is Anisotropy Inherent to Transformers? - Nathan Godey; Éric de la Clergerie; Benoît Sagot (CoRR, 2023)

On the Scaling Laws of Geographical Representation in Language Models. - Nathan Godey; Éric de la Clergerie; Benoît Sagot (LREC/COLING, 2024)

Anisotropy Is Inherent to Self-Attention in Transformers. - Nathan Godey; Éric Villemonte de la Clergerie; Benoît Sagot (EACL (1), 2024)

Headless Language Models: Learning without Predicting with Contrastive Weight Tying. - Nathan Godey; Éric Villemonte de la Clergerie; Benoît Sagot (ICLR, 2024)

Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck. - Nathan Godey; Éric de la Clergerie; Benoît Sagot (CoRR, 2024)

Improving Representations for Language Modeling. (Amélioration des représentations pour la modélisation du langage). - Nathan Godey (2024)

Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression. - Nathan Godey; Alessio Devoto; Yu Zhao 043; Simone Scardapane; Pasquale Minervini; Éric de la Clergerie; Benoît Sagot (CoRR, 2025)

Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content. - Rian Touchent; Nathan Godey; Éric de la Clergerie (CoRR, 2025)