Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization. - Tristan Thrush; Ethan Wilcox; Roger Levy (BlackboxNLP@EMNLP, 2020)

Compositional Neural Machine Translation by Removing the Lexicon from Syntax. - Tristan Thrush (CogSci, 2020)

ANLIzing the Adversarial Natural Language Inference Dataset. - Adina Williams; Tristan Thrush; Douwe Kiela (CoRR, 2020)

Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection. - Bertie Vidgen; Tristan Thrush; Zeerak Waseem; Douwe Kiela (ACL/IJCNLP (1), 2021)

Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation. - Max Bartolo; Tristan Thrush; Robin Jia; Sebastian Riedel 001; Pontus Stenetorp; Douwe Kiela (EMNLP (1), 2021)

Dynabench: Rethinking Benchmarking in NLP. - Douwe Kiela; Max Bartolo; Yixin Nie; Divyansh Kaushik; Atticus Geiger; Zhengxuan Wu; Bertie Vidgen; Grusha Prasad; Amanpreet Singh; Pratik Ringshia; Zhiyi Ma; Tristan Thrush; et al. (NAACL-HLT, 2021)

Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking. - Zhiyi Ma; Kawin Ethayarajh; Tristan Thrush; Somya Jain; Ledell Wu; Robin Jia; Christopher Potts; Adina Williams; Douwe Kiela (NeurIPS, 2021)

Human-Adversarial Visual Question Answering. - Sasha Sheng; Amanpreet Singh; Vedanuj Goswami; Jose Alberto Lopez Magana; Tristan Thrush; Wojciech Galuba; Devi Parikh; Douwe Kiela (NeurIPS, 2021)

Findings of the WMT 2021 Shared Task on Large-Scale Multilingual Machine Translation. - Guillaume Wenzek; Vishrav Chaudhary; Angela Fan; Sahir Gomez; Naman Goyal 001; Somya Jain; Douwe Kiela; Tristan Thrush; Francisco Guzmán (WMT@EMNLP, 2021)

Rover Relocalization for Mars Sample Return by Virtual Template Synthesis and Matching. - Tu-Hoa Pham; William Seto; Shreyansh Daftry; Barry Ridge; Johanna Hansen; Tristan Thrush; Mark Van der Merwe; Gerard Maggiolino; Alexander Brinkman; John Mayo; Yang Cheng; Curtis Padgett; et al. (IEEE Robotics Autom. Lett., 2021)

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks. - Tristan Thrush; Kushal Tirumala; Anmol Gupta; Max Bartolo; Pedro Rodriguez 001; Tariq Kane; William Gaviria Rojas; Peter Mattson; Adina Williams; Douwe Kiela (ACL (demo), 2022)

Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality. - Tristan Thrush; Ryan Jiang; Max Bartolo; Amanpreet Singh; Adina Williams; Douwe Kiela; Candace Ross (CVPR, 2022)

Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements. - Leandro von Werra; Lewis Tunstall; Abhishek Thakur; Sasha Luccioni; Tristan Thrush; Aleksandra Piktus; Felix Marty; Nazneen Rajani; Victor Mustar; Helen Ngo (EMNLP (Demos), 2022)

Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants. - Max Bartolo; Tristan Thrush; Sebastian Riedel 001; Pontus Stenetorp; Robin Jia; Douwe Kiela (NAACL-HLT, 2022)

Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-Based Hate. - Hannah Kirk; Bertie Vidgen; Paul Röttger; Tristan Thrush; Scott Hale (NAACL-HLT, 2022)

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset. - Hugo Laurençon; Lucile Saulnier; Thomas Wang; Christopher Akiki; Albert Villanova del Moral; Teven Le Scao; Leandro von Werra; Chenghao Mou; Eduardo González Ponferrada; Huu Nguyen; Jörg Frohberg; Mario Sasko; et al. (NeurIPS, 2022)

DataPerf: Benchmarks for Data-Centric AI Development. - Mark Mazumder; Colby R. Banbury; Xiaozhe Yao; Bojan Karlas; William Gaviria Rojas; Sudnya Frederick Diamos; Greg Diamos; Lynn He; Douwe Kiela; David Jurado; David Kanter; Rafael Mosquera; et al. (CoRR, 2022)

Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements. - Leandro von Werra; Lewis Tunstall; Abhishek Thakur; Alexandra Sasha Luccioni; Tristan Thrush; Aleksandra Piktus; Felix Marty; Nazneen Rajani; Victor Mustar; Helen Ngo; Omar Sanseviero; Mario Sasko; et al. (CoRR, 2022)

Measuring Data. - Margaret Mitchell; Alexandra Sasha Luccioni; Nathan Lambert 001; Marissa Gerchick; Angelina McMillan-Major; Ezinwanne Ozoani; Nazneen Rajani; Tristan Thrush; Yacine Jernite; Douwe Kiela (CoRR, 2022)

DataPerf: Benchmarks for Data-Centric AI Development. - Mark Mazumder; Colby R. Banbury; Xiaozhe Yao; Bojan Karlas; William Gaviria Rojas; Sudnya Frederick Diamos; Greg Diamos; Lynn He; Alicia Parrish; Hannah Rose Kirk; Jessica Quaye; Charvi Rastogi; et al. (NeurIPS, 2023)

Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language. - William Berrios; Gautam Mittal; Tristan Thrush; Douwe Kiela; Amanpreet Singh (CoRR, 2023)

ColorSwap: A Color and Word Order Dataset for Multimodal Evaluation. - Jirayu Burapacheep; Ishan Gaur; Agam Bhatia; Tristan Thrush (ACL (Findings), 2024)

I am a Strange Dataset: Metalinguistic Tests for Language Models. - Tristan Thrush; Jared Moore; Miguel Monares; Christopher Potts; Douwe Kiela (ACL (1), 2024)

Nearest Neighbor Normalization Improves Multimodal Retrieval. - Neil Chowdhury; Franklin Wang; Sumedh Shenoy; Douwe Kiela; Sarah Schwettmann; Tristan Thrush (EMNLP, 2024)

Improving Pretraining Data Using Perplexity Correlations. - Tristan Thrush; Christopher Potts; Tatsunori Hashimoto (ICLR, 2025)

MixMin: Finding Data Mixtures via Convex Minimization. - Anvith Thudi; Evianne Rovers; Yangjun Ruan; Tristan Thrush; Chris J. Maddison (CoRR, 2025)