James Lee-Thorp

According to our database, James Lee-Thorp authored at least 8 papers between 2021 and 2023.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2023

Memory Augmented Language Models through Mixture of Word Experts.

Cícero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David C. Uthus

CoRR, 2023

Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints.

Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby

Proceedings of the Eleventh International Conference on Learning Representations, 2023

GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints.

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023

CoLT5: Faster Long-Range Transformers with Conditional Computation.

Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023

2022

Scaling Up Models and Data with t5x and seqio.

Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo

CoRR, 2022

FNet: Mixing Tokens with Fourier Transforms.

James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontañón

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022

Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT.

James Lee-Thorp, Joshua Ainslie

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, 2022

2021

ShopTalk: A System for Conversational Faceted Search.

Gurmeet Manku, James Lee-Thorp, Bhargav Kanagal, Joshua Ainslie, Jingchen Feng, Zach Pearson, Ebenezer Anjorin, Sudeep Gandhe, Ilya Eckstein, Jim Rosswog, Sumit Sanghai, Michael Pohl, Larry Adams, D. Sivakumar

CoRR, 2021