ETC: Encoding Long and Structured Inputs in Transformers. - Joshua Ainslie; Santiago Ontañón; Chris Alberti; Vaclav Cvicek; Zachary Fisher; Philip Pham; Anirudh Ravula; Sumit Sanghai; Qifan Wang 001; Li Yang (EMNLP (1), 2020)

Learning to Extract Attribute Value from Product via Question Answering: A Multi-task Approach. - Qifan Wang 001; Li Yang; Bhargav Kanagal; Sumit Sanghai; D. Sivakumar 001; Bin Shu; Zac Yu; Jon Elsas (KDD, 2020)

ETC: Encoding Long and Structured Data in Transformers. - Joshua Ainslie; Santiago Ontañón; Chris Alberti; Philip Pham; Anirudh Ravula; Sumit Sanghai (CoRR, 2020)

ShopTalk: A System for Conversational Faceted Search. - Gurmeet Manku; James Lee-Thorp; Bhargav Kanagal; Joshua Ainslie; Jingchen Feng; Zach Pearson; Ebenezer Anjorin; Sudeep Gandhe; Ilya Eckstein; Jim Rosswog; Sumit Sanghai; Michael Pohl; et al. (CoRR, 2021)

Generate-and-Retrieve: Use Your Predictions to Improve Retrieval for Semantic Parsing. - Yury Zemlyanskiy; Michiel de Jong; Joshua Ainslie; Panupong Pasupat; Peter Shaw 004; Linlu Qiu; Sumit Sanghai; Fei Sha (COLING, 2022)

MAVE: A Product Dataset for Multi-source Attribute Value Extraction. - Li Yang; Qifan Wang 001; Zac Yu; Anand Kulkarni; Sumit Sanghai; Bin Shu; Jon Elsas; Bhargav Kanagal (WSDM, 2022)

ImPaKT: A Dataset for Open-Schema Knowledge Base Construction. - Luke Vilnis; Zachary Fisher; Bhargav Kanagal; Patrick Murray; Sumit Sanghai (CoRR, 2022)

FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference. - Michiel de Jong; Yury Zemlyanskiy; Joshua Ainslie; Nicholas FitzGerald; Sumit Sanghai; Fei Sha; William W. Cohen (ACL (Findings), 2023)

CoLT5: Faster Long-Range Transformers with Conditional Computation. - Joshua Ainslie; Tao Lei; Michiel de Jong; Santiago Ontañón; Siddhartha Brahma; Yury Zemlyanskiy; David C. Uthus; Mandy Guo; James Lee-Thorp; Yi Tay; Yun-Hsuan Sung; Sumit Sanghai (EMNLP, 2023)

GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. - Joshua Ainslie; James Lee-Thorp; Michiel de Jong; Yury Zemlyanskiy; Federico Lebrón; Sumit Sanghai (EMNLP, 2023)

Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute. - Michiel de Jong; Yury Zemlyanskiy; Nicholas FitzGerald; Joshua Ainslie; Sumit Sanghai; Fei Sha; William W. Cohen (ICML, 2023)

Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models. - Luke Vilnis; Yury Zemlyanskiy; Patrick Murray; Alexandre Tachard Passos; Sumit Sanghai (ICML, 2023)

GLIMMER: generalized late-interaction memory reranker. - Michiel de Jong; Yury Zemlyanskiy; Nicholas FitzGerald; Sumit Sanghai; William W. Cohen; Joshua Ainslie (CoRR, 2023)

Functional Interpolation for Relative Positions improves Long Context Transformers. - Shanda Li; Chong You; Guru Guruganesh; Joshua Ainslie; Santiago Ontañón; Manzil Zaheer; Sumit Sanghai; Yiming Yang; Sanjiv Kumar; Srinadh Bhojanapalli (ICLR, 2024)

MEMORY-VQ: Compression for Tractable Internet-Scale Memory. - Yury Zemlyanskiy; Michiel de Jong; Luke Vilnis; Santiago Ontañón; William W. Cohen; Sumit Sanghai; Joshua Ainslie (NAACL (Short Papers), 2024)

Integrating Planning into Single-Turn Long-Form Text Generation. - Yi Liang; You Wu 001; Honglei Zhuang; Li Chen; Jiaming Shen; Yiling Jia; Zhen Qin 001; Sumit Sanghai; Xuanhui Wang; Carl Yang 001; Michael Bendersky (CoRR, 2024)