SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models. - Vithursan Thangarasa; Abhay Gupta; William Marshall; Tianda Li; Kevin Leong; Dennis DeCoste; Sean Lie; Shreyas Saxena (UAI, 2023)

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models. - Vithursan Thangarasa; Mahmoud Salem; Shreyas Saxena; Kevin Leong; Joel Hestness; Sean Lie (CoRR, 2024)

Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment. - Abhinav Agarwalla; Abhay Gupta; Alexandre Noll Marques; Shubhra Pandit; Michael Goin; Eldar Kurtic; Kevin Leong; Tuan Nguyen; Mahmoud Salem; Dan Alistarh; Sean Lie; Mark Kurtz (CoRR, 2024)