Deep Transfer Learning for Infectious Disease Case Detection Using Electronic Medical Records. - Ye Ye 002; Andrew Gu (CoRR, 2021)

PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. - Yanli Zhao; Andrew Gu; Rohan Varma; Liang Luo; Chien-Chin Huang; Min Xu; Less Wright; Hamid Shojanazeri; Myle Ott; Sam Shleifer; Alban Desmaison; Can Balioglu; et al. (CoRR, 2023)

PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. - Yanli Zhao; Andrew Gu; Rohan Varma; Liang Luo; Chien-Chin Huang; Min Xu; Less Wright; Hamid Shojanazeri; Myle Ott; Sam Shleifer; Alban Desmaison; Can Balioglu; et al. (Proc. VLDB Endow., 2023)

TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training. - Wanchao Liang; Tianyu Liu; Less Wright; Will Constable; Andrew Gu; Chien-Chin Huang; Iris Zhang; Wei Feng; Howard Huang; Junjie Wang; Sanket Purandare; Gokul Nadathur; et al. (CoRR, 2024)

SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile. - Ruisi Zhang; Tianyu Liu; Will Feng; Andrew Gu; Sanket Purandare; Wanchao Liang; Francisco Massa (CoRR, 2024)

TorchTitan: One-stop PyTorch native solution for production ready LLM pretraining. - Wanchao Liang; Tianyu Liu; Less Wright; Will Constable; Andrew Gu; Chien-Chin Huang; Iris Zhang; Wei Feng; Howard Huang; Junjie Wang; Sanket Purandare; Gokul Nadathur; et al. (ICLR, 2025)

Scaling Llama 3 Training with Efficient Parallelism Strategies. - Weiwei Chu; Xinfeng Xie; Jiecao Yu; Jie Wang 022; Amar Phanishayee; Chunqiang Tang; Yuchen Hao; Jianyu Huang; Mustafa Ozdal; Jun Wang; Vedanuj Goswami; Naman Goyal 001; et al. (ISCA, 2025)

GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection. - DiJia Su; Andrew Gu; Jane Xu; Yuandong Tian; Jiawei Zhao (CoRR, 2025)