Online evolutionary batch size orchestration for scheduling deep learning workloads in GPU clusters. - Zhengda Bian; Shenggui Li; Wei Wang; Yang You 001 (SC, 2021)

An Efficient 2D Method for Training Super-Large Deep Learning Models. - Qifan Xu; Shenggui Li; Chaoyu Gong; Yang You 001 (CoRR, 2021)

Sequence Parallelism: Making 4D Parallelism Possible. - Shenggui Li; Fuzhao Xue; Yongbin Li; Yang You 001 (CoRR, 2021)

PatrickStar: Parallel Training of Pre-trained Models via a Chunk-based Memory Management. - Jiarui Fang; Yang Yu; Shenggui Li; Yang You 001; Jie Zhou 016 (CoRR, 2021)

Sky Computing: Accelerating Geo-distributed Computing in Federated Learning. - Jie Zhu; Shenggui Li; Yang You 001 (CoRR, 2022)

A Frequency-aware Software Cache for Large Recommendation System Embeddings. - Jiarui Fang; Geng Zhang; Jiatong Han; Shenggui Li; Zhengda Bian; Yongbin Li; Jin Liu; Yang You 001 (CoRR, 2022)

EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models. - Jiangsu Du; Ziming Liu; Jiarui Fang; Shenggui Li; Yongbin Li; Yutong Lu; Yang You 001 (CoRR, 2022)

Elixir: Train a Large Language Model on a Small GPU Cluster. - Haichen Huang; Jiarui Fang; Hongxin Liu; Shenggui Li; Yang You 001 (CoRR, 2022)

Critique of "MemXCT: Memory-Centric X-Ray CT Reconstruction With Massive Parallelization" by SCC Team From Nanyang Technological University. - Shenggui Li; Bu-Sung Lee (IEEE Trans. Parallel Distributed Syst., 2022)

Sequence Parallelism: Long Sequence Training from System Perspective. - Shenggui Li; Fuzhao Xue; Chaitanya Baranwal; Yongbin Li; Yang You 001 (ACL (1), 2023)

Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training. - Shenggui Li; Hongxin Liu; Zhengda Bian; Jiarui Fang; Haichen Huang; Yuliang Liu; Boxiang Wang; Yang You 001 (ICPP, 2023)

Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models. - Yuliang Liu; Shenggui Li; Jiarui Fang; Yanjun Shao; Boyuan Yao; Yang You 001 (CoRR, 2023)

Parallel Training of Pre-Trained Models via Chunk-Based Dynamic Memory Management. - Jiarui Fang; Zilin Zhu; Shenggui Li; Hui Su; Yang Yu; Jie Zhou 016; Yang You 001 (IEEE Trans. Parallel Distributed Syst., 2023)

GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding. - Cunxiao Du; Jing Jiang 001; Yuanchen Xu; Jiawei Wu 003; Sicheng Yu; Yongqi Li 001; Shenggui Li; Kai Xu; Liqiang Nie; Zhaopeng Tu; Yang You (ICML, 2024)

Open-Sora: Democratizing Efficient Video Production for All. - Zangwei Zheng; Xiangyu Peng; Tianji Yang; Chenhui Shen; Shenggui Li; Hongxin Liu; Yukun Zhou; Tianyi Li; Yang You 001 (CoRR, 2024)

SPPO:Efficient Long-sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading. - Qiaoling Chen; Shenggui Li; Wei Gao; Peng Sun 006; Yonggang Wen 001; Tianwei Zhang 004 (CoRR, 2025)