Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce. - Xupeng Miao; Xiaonan Nie; Yingxia Shao; Zhi Yang 001; Jiawei Jiang 001; Lingxiao Ma; Bin Cui 001 (SIGMOD Conference, 2021)

Dense-to-Sparse Gate for Mixture-of-Experts. - Xiaonan Nie; Shijie Cao; Xupeng Miao; Lingxiao Ma; Jilong Xue; Youshan Miao; Zichao Yang; Zhi Yang 001; Bin Cui 001 (CoRR, 2021)

HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework. - Xupeng Miao; Hailin Zhang 004; Yining Shi 001; Xiaonan Nie; Zhi Yang 001; Yangyu Tao; Bin Cui 001 (Proc. VLDB Endow., 2021)

TSPLIT: Fine-grained GPU Memory Management for Efficient DNN Training via Tensor Splitting. - Xiaonan Nie; Xupeng Miao; Zhi Yang 001; Bin Cui 001 (ICDE, 2022)

HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training. - Xupeng Miao; Yining Shi 001; Hailin Zhang 004; Xin Zhang; Xiaonan Nie; Zhi Yang 001; Bin Cui 001 (SIGMOD Conference, 2022)

HetuMoE: An Efficient Trillion-scale Mixture-of-Expert Distributed Training System. - Xiaonan Nie; Pinxue Zhao; Xupeng Miao; Tong Zhao; Bin Cui 001 (CoRR, 2022)

OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning. - Youhe Jiang; Xupeng Miao; Xiaonan Nie; Bin Cui 001 (CoRR, 2022)

Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism. - Xupeng Miao; Yujie Wang; Youhe Jiang; Chunan Shi; Xiaonan Nie; Hailin Zhang 004; Bin Cui 001 (Proc. VLDB Endow., 2022)

OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning. - Youhe Jiang; Fangcheng Fu; Xupeng Miao; Xiaonan Nie; Bin Cui 001 (IJCAI, 2023)

Improving Automatic Parallel Training via Balanced Memory Workload Optimization. - Yujie Wang; Youhe Jiang; Xupeng Miao; Fangcheng Fu; Xiaonan Nie; Bin Cui 001 (CoRR, 2023)

FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement. - Xiaonan Nie; Xupeng Miao; Zilong Wang; Zichao Yang; Jilong Xue; Lingxiao Ma; Gang Cao; Bin Cui 001 (Proc. ACM Manag. Data, 2023)

Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent. - Xiaonan Nie; Yi Liu; Fangcheng Fu; Jinbao Xue; Dian Jiao; Xupeng Miao; Yangyu Tao; Bin Cui 001 (Proc. VLDB Endow., 2023)

Hetu: a highly efficient automatic parallel distributed deep learning system. - Xupeng Miao; Xiaonan Nie; Hailin Zhang 004; Tong Zhao; Bin Cui 001 (Sci. China Inf. Sci., 2023)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing. - Xiaonan Nie; Qibin Liu; Fangcheng Fu; Shenhan Zhu; Xupeng Miao; Xiaoyang Li; Yang Zhang; Shouda Liu; Bin Cui 001 (NeurIPS, 2024)

Enabling Parallelism Hot Switching for Efficient Training of Large Language Models. - Hao Ge; Fangcheng Fu; Haoyang Li; Xuanyu Wang; Sheng Lin; Yujie Wang; Xiaonan Nie; Hailin Zhang 004; Xupeng Miao; Bin Cui 001 (SOSP, 2024)

Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge. - Bin Xiao; Chunan Shi; Xiaonan Nie; Fan Yang 024; Xiangwei Deng; Lei Su; Weipeng Chen; Bin Cui 001 (CoRR, 2024)

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs. - Pinxue Zhao; Hailin Zhang 004; Fangcheng Fu; Xiaonan Nie; Qibin Liu; Fang Yang; Yuanbo Peng; Dian Jiao; Shuaipeng Li; Jinbao Xue; Yangyu Tao; Bin Cui 001 (CoRR, 2024)

BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline. - Guosheng Dong; Da Pan 003; Yiding Sun; Shusen Zhang; Zheng Liang; Xin Wu; Yanjun Shen; Fan Yang 087; Haoze Sun; Tianpeng Li; Mingan Lin; Jianhua Xu; et al. (CoRR, 2024)

DataSculpt: Crafting Data Landscapes for LLM Post-Training through Multi-objective Partitioning. - Keer Lu; Zheng Liang; Xiaonan Nie; Da Pan 003; Shusen Zhang; Keshi Zhao; Weipeng Chen; Zenan Zhou; Guosheng Dong; Wentao Zhang 001; Bin Cui 001 (CoRR, 2024)

FlashFlex: Accommodating Large Language Model Training over Heterogeneous Environment. - Ran Yan; Youhe Jiang; Wangcheng Tao; Xiaonan Nie; Bin Cui 001; Binhang Yuan (CoRR, 2024)

Improving Automatic Parallel Training via Balanced Memory Workload Optimization. - Yujie Wang; Youhe Jiang; Xupeng Miao; Fangcheng Fu; Shenhan Zhu; Xiaonan Nie; Yaofeng Tu; Bin Cui 001 (IEEE Trans. Knowl. Data Eng., 2024)

DataSculpt: A Holistic Data Management Framework for Long-Context LLMs Training. - Keer Lu; Xiaonan Nie; Zheng Liang; Da Pan 003; Shusen Zhang; Keshi Zhao; Weipeng Chen; Zenan Zhou; Guosheng Dong; Bin Cui 001; Wentao Zhang 001 (ICDE, 2025)

NetMoE: Accelerating MoE Training through Dynamic Sample Placement. - Xinyi Liu; Yujie Wang; Fangcheng Fu; Xupeng Miao; Shenhan Zhu; Xiaonan Nie; Bin Cui 001 (ICLR, 2025)

ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUs. - Hao Ge; Junda Feng; Qi Huang; Fangcheng Fu; Xiaonan Nie; Lei Zuo; Haibin Lin; Bin Cui 001; Xin Liu (CoRR, 2025)

MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training. - Pinxue Zhao; Hailin Zhang 004; Fangcheng Fu; Xiaonan Nie; Qibin Liu; Fang Yang; Yuanbo Peng; Dian Jiao; Shuaipeng Li; Jinbao Xue; Yangyu Tao; Bin Cui 001 (Proc. ACM Manag. Data, 2025)

Emerging Properties in Unified Multimodal Pretraining. - Chaorui Deng; Deyao Zhu; Kunchang Li; Chenhui Gou; Feng Li; Zeyu Wang; Shu Zhong; Weihao Yu 007; Xiaonan Nie; Ziang Song; Shi Guang; Haoqi Fan 001 (CoRR, 2025)

Efficient and scalable huge embedding model training via distributed cache management. - Xupeng Miao; Hailin Zhang 004; Yining Shi 001; Xiaonan Nie; Zhi Yang 001; Yangyu Tao; Jie Jiang; Bin Cui 001 (VLDB J., 2025)

Seedance 1.0: Exploring the Boundaries of Video Generation Models. - Yu Gao; Haoyuan Guo; Tuyen Hoang; Weilin Huang; Lu Jiang; Fangyuan Kong; Huixia Li; Jiashi Li; Liang Li; Xiaojie Li; Xunsong Li; Yifu Li; et al. (CoRR, 2025)

Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization. - Haoyang Li; Fangcheng Fu; Hao Ge; Sheng Lin; Xuanyu Wang; Jiawen Niu; Yujie Wang; Hailin Zhang 004; Xiaonan Nie; Bin Cui 001 (Proc. ACM Manag. Data, 2025)

PQCache: Product Quantization-based KVCache for Long Context LLM Inference. - Hailin Zhang 004; Xiaodong Ji; Yilin Chen; Fangcheng Fu; Xupeng Miao; Xiaonan Nie; Weipeng Chen; Bin Cui 001 (Proc. ACM Manag. Data, 2025)

Acceleration Algorithms in GNNs: A Survey. - Lu Ma; Zeang Sheng; Xunkai Li; Xinyi Gao 001; Zhezheng Hao; Ling Yang 006; Xiaonan Nie; Jiawei Jiang 001; Wentao Zhang 001; Bin Cui 001 (IEEE Trans. Knowl. Data Eng., 2025)