Low Resource Text Classification with ULMFit and Backtranslation. - Sam Shleifer (CoRR, 2019)

Using Small Proxy Datasets to Accelerate Hyperparameter Search. - Sam Shleifer; Eric Prokop (CoRR, 2019)

Classification As Decoder: Trading Flexibility For Control In Neural Dialogue. - Sam Shleifer; Manish Chablani; Namit Katariya; Anitha Kannan; Xavier Amatriain (CoRR, 2019)

Classification as Decoder: Trading Flexibility for Control in Medical Dialogue. - Sam Shleifer; Manish Chablani; Anitha Kannan; Namit Katariya; Xavier Amatriain (CoRR, 2019)

Incrementally Improving Graph WaveNet Performance on Traffic Prediction. - Sam Shleifer; Clara McCreery; Vamsi Chitters (CoRR, 2019)

Transformers: State-of-the-Art Natural Language Processing. - Thomas Wolf 008; Lysandre Debut; Victor Sanh; Julien Chaumond; Clement Delangue; Anthony Moi; Pierric Cistac; Tim Rault; Rémi Louf; Morgan Funtowicz; Joe Davison; Sam Shleifer; et al. (EMNLP (Demos), 2020)

Pre-trained Summarization Distillation. - Sam Shleifer; Alexander M. Rush (CoRR, 2020)

NormFormer: Improved Transformer Pretraining with Extra Normalization. - Sam Shleifer; Jason Weston; Myle Ott (CoRR, 2021)

Few-shot Learning with Multilingual Language Models. - Xi Victoria Lin; Todor Mihaylov; Mikel Artetxe; Tianlu Wang; Shuohui Chen; Daniel Simig; Myle Ott; Naman Goyal 001; Shruti Bhosale; Jingfei Du; Ramakanth Pasunuru; Sam Shleifer; et al. (CoRR, 2021)

Efficient Large Scale Language Modeling with Mixtures of Experts. - Mikel Artetxe; Shruti Bhosale; Naman Goyal 001; Todor Mihaylov; Myle Ott; Sam Shleifer; Xi Victoria Lin; Jingfei Du; Srinivasan Iyer 001; Ramakanth Pasunuru; Giri Anantharaman; Xian Li 003; et al. (CoRR, 2021)

Automated coronary calcium scoring using deep learning with multicenter external validation. - David Eng; Christopher Chute; Nishith Khandwala; Pranav Rajpurkar; Jin Long; Sam Shleifer; Mohamed H. Khalaf; Alexander T. Sandhu; Fátima Rodriguez; David J. Maron; Saeed Seyyedi; Daniele Marin; et al. (npj Digit. Medicine, 2021)

Efficient Large Scale Language Modeling with Mixtures of Experts. - Mikel Artetxe; Shruti Bhosale; Naman Goyal 001; Todor Mihaylov; Myle Ott; Sam Shleifer; Xi Victoria Lin; Jingfei Du; Srinivasan Iyer 001; Ramakanth Pasunuru; Giridharan Anantharaman; Xian Li 003; et al. (EMNLP, 2022)

Few-shot Learning with Multilingual Generative Language Models. - Xi Victoria Lin; Todor Mihaylov; Mikel Artetxe; Tianlu Wang; Shuohui Chen; Daniel Simig; Myle Ott; Naman Goyal 001; Shruti Bhosale; Jingfei Du; Ramakanth Pasunuru; Sam Shleifer; et al. (EMNLP, 2022)

8-bit Optimizers via Block-wise Quantization. - Tim Dettmers; Mike Lewis; Sam Shleifer; Luke Zettlemoyer (ICLR, 2022)

Efficient Language Modeling with Sparse all-MLP. - Ping Yu; Mikel Artetxe; Myle Ott; Sam Shleifer; Hongyu Gong; Ves Stoyanov; Xian Li 003 (CoRR, 2022)

OPT: Open Pre-trained Transformer Language Models. - Susan Zhang; Stephen Roller; Naman Goyal 001; Mikel Artetxe; Moya Chen; Shuohui Chen; Christopher Dewan; Mona T. Diab; Xian Li 003; Xi Victoria Lin; Todor Mihaylov; Myle Ott; et al. (CoRR, 2022)

PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. - Yanli Zhao; Andrew Gu; Rohan Varma; Liang Luo; Chien-Chin Huang; Min Xu; Less Wright; Hamid Shojanazeri; Myle Ott; Sam Shleifer; Alban Desmaison; Can Balioglu; et al. (CoRR, 2023)

PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. - Yanli Zhao; Andrew Gu; Rohan Varma; Liang Luo; Chien-Chin Huang; Min Xu; Less Wright; Hamid Shojanazeri; Myle Ott; Sam Shleifer; Alban Desmaison; Can Balioglu; et al. (Proc. VLDB Endow., 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)