X-former Elucidator: Reviving Efficient Attention for Long Context Language Modeling. - Xupeng Miao; Shenhan Zhu; Fangcheng Fu; Ziyu Guo; Zhi Yang 001; Yaofeng Tu; Zhihao Jia; Bin Cui 001 (IJCAI, 2024)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing. - Xiaonan Nie; Qibin Liu; Fangcheng Fu; Shenhan Zhu; Xupeng Miao; Xiaoyang Li; Yang Zhang; Shouda Liu; Bin Cui 001 (NeurIPS, 2024)

Efficient Multi-Task Large Model Training via Data Heterogeneity-aware Model Management. - Yujie Wang; Shenhan Zhu; Fangcheng Fu; Xupeng Miao; Jie Zhang; Juan Zhu; Fan Hong; Yong Li; Bin Cui 001 (CoRR, 2024)

Data-Centric and Heterogeneity-Adaptive Sequence Parallelism for Efficient LLM Training. - Yujie Wang; Shiju Wang; Shenhan Zhu; Fangcheng Fu; Xinyi Liu; Xuefeng Xiao 001; Huixia Li; Jiashi Li; Faming Wu; Bin Cui 001 (CoRR, 2024)

Improving Automatic Parallel Training via Balanced Memory Workload Optimization. - Yujie Wang; Youhe Jiang; Xupeng Miao; Fangcheng Fu; Shenhan Zhu; Xiaonan Nie; Yaofeng Tu; Bin Cui 001 (IEEE Trans. Knowl. Data Eng., 2024)

FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism. - Yujie Wang; Shiju Wang; Shenhan Zhu; Fangcheng Fu; Xinyi Liu; Xuefeng Xiao 001; Huixia Li; Jiashi Li; Faming Wu; Bin Cui 001 (ASPLOS (2), 2025)

Spindle: Efficient Distributed Training of Multi-Task Large Models via Wavefront Scheduling. - Yujie Wang; Shenhan Zhu; Fangcheng Fu; Xupeng Miao; Jie Zhang; Juan Zhu; Fan Hong; Yong Li; Bin Cui 001 (ASPLOS (2), 2025)

NetMoE: Accelerating MoE Training through Dynamic Sample Placement. - Xinyi Liu; Yujie Wang; Fangcheng Fu; Xupeng Miao; Shenhan Zhu; Xiaonan Nie; Bin Cui 001 (ICLR, 2025)

Galvatron: An Automatic Distributed System for Efficient Foundation Model Training. - Xinyi Liu; Yujie Wang; Shenhan Zhu; Fangcheng Fu; Qingshuo Liu; Guangming Lin; Bin Cui 001 (CoRR, 2025)