Universal Sentence Encoder for English. - Daniel Cer; Yinfei Yang; Sheng-yi Kong; Nan Hua; Nicole Limtiaco; Rhomni St. John; Noah Constant; Mario Guajardo-Cespedes; Steve Yuan; Chris Tar; Brian Strope; Ray Kurzweil (EMNLP (Demonstration), 2018)

Learning Semantic Textual Similarity from Conversations. - Yinfei Yang; Steve Yuan; Daniel Cer; Sheng-yi Kong; Noah Constant; Petr Pilar; Heming Ge; Yun-Hsuan Sung; Brian Strope; Ray Kurzweil (Rep4NLP@ACL, 2018)

Effective Parallel Corpus Mining using Bilingual Sentence Embeddings. - Mandy Guo; Qinlan Shen; Yinfei Yang; Heming Ge; Daniel Cer; Gustavo Hernández Ábrego; Keith Stevens; Noah Constant; Yun-Hsuan Sung; Brian Strope; Ray Kurzweil (WMT, 2018)

Universal Sentence Encoder. - Daniel Cer; Yinfei Yang; Sheng-yi Kong; Nan Hua; Nicole Limtiaco; Rhomni St. John; Noah Constant; Mario Guajardo-Cespedes; Steve Yuan; Chris Tar; Yun-Hsuan Sung; Brian Strope; et al. (CoRR, 2018)

Character-Level Language Modeling with Deeper Self-Attention. - Rami Al-Rfou; Dokook Choe; Noah Constant; Mandy Guo; Llion Jones (AAAI, 2019)

ReQA: An Evaluation for End-to-End Answer Retrieval Models. - Amin Ahmad; Noah Constant; Yinfei Yang; Daniel Cer (MRQA@EMNLP, 2019)

Bridging the Gap for Tokenizer-Free Language Models. - Dokook Choe; Rami Al-Rfou; Mandy Guo; Heeyoung Lee; Noah Constant (CoRR, 2019)

Multilingual Universal Sentence Encoder for Semantic Retrieval. - Yinfei Yang; Daniel Cer; Amin Ahmad; Mandy Guo; Jax Law; Noah Constant; Gustavo Hernández Ábrego; Steve Yuan; Chris Tar; Yun-Hsuan Sung; Brian Strope; Ray Kurzweil (ACL (demo), 2020)

LAReQA: Language-Agnostic Answer Retrieval from a Multilingual Pool. - Uma Roy; Noah Constant; Rami Al-Rfou; Aditya Barua; Aaron Phillips; Yinfei Yang (EMNLP (1), 2020)

MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models. - Mandy Guo; Yinfei Yang; Daniel Cer; Qinlan Shen; Noah Constant (CoRR, 2020)

TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling. - Parker Riley; Noah Constant; Mandy Guo; Girish Kumar; David C. Uthus; Zarana Parekh (CoRR, 2020)

Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension. - Siamak Shakeri; Noah Constant; Mihir Sanjay Kale; Linting Xue (CoRR, 2020)

nmT5 - Is parallel data still relevant for pre-training massively multilingual language models? - Mihir Kale; Aditya Siddhant; Rami Al-Rfou; Linting Xue; Noah Constant; Melvin Johnson (ACL/IJCNLP (2), 2021)

TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling. - Parker Riley; Noah Constant; Mandy Guo; Girish Kumar; David C. Uthus; Zarana Parekh (ACL/IJCNLP (1), 2021)

The Power of Scale for Parameter-Efficient Prompt Tuning. - Brian Lester; Rami Al-Rfou; Noah Constant (EMNLP (1), 2021)

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation. - Sebastian Ruder; Noah Constant; Jan A. Botha; Aditya Siddhant; Orhan Firat; Jinlan Fu; Pengfei Liu 003; Junjie Hu 001; Dan Garrette; Graham Neubig; Melvin Johnson (EMNLP (1), 2021)

Towards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension. - Siamak Shakeri; Noah Constant; Mihir Kale; Linting Xue (INLG, 2021)

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution. - Xavier Garcia; Noah Constant; Ankur P. Parikh; Orhan Firat (NAACL-HLT, 2021)

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. - Linting Xue; Noah Constant; Adam Roberts; Mihir Kale; Rami Al-Rfou; Aditya Siddhant; Aditya Barua; Colin Raffel (NAACL-HLT, 2021)

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation. - Sebastian Ruder; Noah Constant; Jan A. Botha; Aditya Siddhant; Orhan Firat; Jinlan Fu; Pengfei Liu 003; Junjie Hu 001; Graham Neubig; Melvin Johnson (CoRR, 2021)

Towards Universality in Multilingual Text Rewriting. - Xavier Garcia; Noah Constant; Mandy Guo; Orhan Firat (CoRR, 2021)

Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models. - Jianmo Ni; Gustavo Hernández Ábrego; Noah Constant; Ji Ma 004; Keith B. Hall; Daniel Cer; Yinfei Yang (ACL (Findings), 2022)

SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer. - Tu Vu; Brian Lester; Noah Constant; Rami Al-Rfou'; Daniel Cer (ACL (1), 2022)

Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation. - Tu Vu; Aditya Barua; Brian Lester; Daniel Cer; Mohit Iyyer; Noah Constant (EMNLP, 2022)

Reducing Retraining by Recycling Parameter-Efficient Prompts. - Brian Lester; Joshua Yurtsever; Siamak Shakeri; Noah Constant (CoRR, 2022)

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models. - Linting Xue; Aditya Barua; Noah Constant; Rami Al-Rfou; Sharan Narang; Mihir Kale; Adam Roberts; Colin Raffel (Trans. Assoc. Comput. Linguistics, 2022)

Character-Aware Models Improve Visual Text Rendering. - Rosanne Liu; Dan Garrette; Chitwan Saharia; William Chan; Adam Roberts; Sharan Narang; Irina Blok; RJ Mical; Mohammad Norouzi 002; Noah Constant (ACL (1), 2023)

UniMax: Fairer and More Effective Language Sampling for Large-Scale Multilingual Pretraining. - Hyung Won Chung; Xavier Garcia; Adam Roberts; Yi Tay; Orhan Firat; Sharan Narang; Noah Constant (ICLR, 2023)

Bidirectional Language Models Are Also Few-shot Learners. - Ajay Patel; Bryan Li; Mohammad Sadegh Rasooli; Noah Constant; Colin Raffel; Chris Callison-Burch (ICLR, 2023)

Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5? - C. Daniel Freeman; Laura Culp; Aaron Parisi; Maxwell L. Bileschi; Gamaleldin F. Elsayed; Alex Rizkowsky; Isabelle Simpson; Alex Alemi; Azade Nova; Ben Adlam; Bernd Bohnet; Gaurav Mishra; et al. (CoRR, 2023)

FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation. - Parker Riley; Timothy Dozat; Jan A. Botha; Xavier Garcia; Dan Garrette; Jason Riesa; Orhan Firat; Noah Constant (Trans. Assoc. Comput. Linguistics, 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)

FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation. - Tu Vu; Mohit Iyyer; Xuezhi Wang 002; Noah Constant; Jerry W. Wei; Jason Wei; Chris Tar; Yun-Hsuan Sung; Denny Zhou; Quoc V. Le; Thang Luong (ACL (Findings), 2024)

Transfer Learning for Text Diffusion Models. - Kehang Han; Kathleen Kenealy; Aditya Barua; Noah Fiedel; Noah Constant (CoRR, 2024)

Training LLMs over Neurally Compressed Text. - Brian Lester; Jaehoon Lee 001; Alexander A. Alemi; Jeffrey Pennington; Adam Roberts; Jascha Sohl-Dickstein; Noah Constant (Trans. Mach. Learn. Res., 2024)

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models. - Avi Singh; John D. Co-Reyes; Rishabh Agarwal; Ankesh Anand; Piyush Patil; Xavier Garcia; Peter J. Liu; James Harrison; Jaehoon Lee 001; Kelvin Xu; Aaron T. Parisi; Abhishek Kumar; et al. (Trans. Mach. Learn. Res., 2024)

Live Music Models. - Antoine Caillon; Brian McWilliams; Cassie Tarakajian; Ian Simon; Ilaria Manco; Jesse H. Engel; Noah Constant; Yunpeng Li 008; Timo I. Denk; Alberto Lalama; Andrea Agostinelli; Cheng-Zhi Anna Huang; et al. (CoRR, 2025)