Stand-Alone Self-Attention in Vision Models. - Niki Parmar; Prajit Ramachandran; Ashish Vaswani; Irwan Bello; Anselm Levskaya; Jonathon Shlens (NeurIPS, 2019)

Reformer: The Efficient Transformer. - Nikita Kitaev; Lukasz Kaiser; Anselm Levskaya (ICLR, 2020)

Exploring the limits of Concurrency in ML Training on Google TPUs. - Sameer Kumar; James Bradbury; Cliff Young; Yu Emma Wang; Anselm Levskaya; Blake A. Hechtman; Dehao Chen; HyoukJoong Lee; Mehmet Deveci; Naveen Kumar; Pankaj Kanwar; Shibo Wang; et al. (CoRR, 2020)

Scaling Up Models and Data with t5x and seqio. - Adam Roberts; Hyung Won Chung; Anselm Levskaya; Gaurav Mishra; James Bradbury; Daniel Andor; Sharan Narang; Brian Lester; Colin Gaffney; Afroz Mohiuddin; Curtis Hawthorne; Aitor Lewkowycz; et al. (CoRR, 2022)

Efficiently Scaling Transformer Inference. - Reiner Pope; Sholto Douglas; Aakanksha Chowdhery; Jacob Devlin; James Bradbury; Anselm Levskaya; Jonathan Heek; Kefan Xiao; Shivani Agrawal; Jeff Dean (CoRR, 2022)

PaLM: Scaling Language Modeling with Pathways. - Aakanksha Chowdhery; Sharan Narang; Jacob Devlin; Maarten Bosma; Gaurav Mishra; Adam Roberts; Paul Barham 001; Hyung Won Chung; Charles Sutton; Sebastian Gehrmann; Parker Schuh; Kensen Shi; et al. (J. Mach. Learn. Res., 2023)

Scaling Up Models and Data with t5x and seqio. - Adam Roberts; Hyung Won Chung; Gaurav Mishra; Anselm Levskaya; James Bradbury; Daniel Andor; Sharan Narang; Brian Lester; Colin Gaffney; Afroz Mohiuddin; Curtis Hawthorne; Aitor Lewkowycz; et al. (J. Mach. Learn. Res., 2023)