Nathan Godey

According to our database, Nathan Godey authored at least 10 papers between 2022 and 2025.

Collaborative distances:

• Dijkstra number of five.
• Erdős number of four.

2025

Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content.

Rian Touchent, Nathan Godey, Éric de la Clergerie

CoRR, June, 2025

Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression.

Nathan Godey, Alessio Devoto, Yu Zhao, Simone Scardapane, Pasquale Minervini, Éric de la Clergerie, Benoît Sagot

CoRR, March, 2025

2024

Improving Representations for Language Modeling. (Amélioration des représentations pour la modélisation du langage).

Nathan Godey

PhD thesis, 2024

Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck.

Nathan Godey, Éric de la Clergerie, Benoît Sagot

CoRR, 2024

Headless Language Models: Learning without Predicting with Contrastive Weight Tying.

Nathan Godey, Éric Villemonte de la Clergerie, Benoît Sagot

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Anisotropy Is Inherent to Self-Attention in Transformers.

Nathan Godey, Éric Villemonte de la Clergerie, Benoît Sagot

Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, 2024

On the Scaling Laws of Geographical Representation in Language Models.

Nathan Godey, Éric de la Clergerie, Benoît Sagot

Proceedings of the 2024 Joint International Conference on Computational Linguistics, 2024

2023

Is Anisotropy Inherent to Transformers?

Nathan Godey, Éric de la Clergerie, Benoît Sagot

CoRR, 2023

2022

MANTa: Efficient Gradient-Based Tokenization for Robust End-to-End Language Modeling.

Nathan Godey, Roman Castagné, Éric de la Clergerie, Benoît Sagot

CoRR, 2022

MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling.

Nathan Godey, Roman Castagné, Éric de la Clergerie, Benoît Sagot

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, 2022