HetuMoE: An Efficient Trillion-scale Mixture-of-Expert Distributed Training System. - Xiaonan Nie; Pinxue Zhao; Xupeng Miao; Tong Zhao; Bin Cui 001 (CoRR, 2022)

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs. - Pinxue Zhao; Hailin Zhang 004; Fangcheng Fu; Xiaonan Nie; Qibin Liu; Fang Yang; Yuanbo Peng; Dian Jiao; Shuaipeng Li; Jinbao Xue; Yangyu Tao; Bin Cui 001 (CoRR, 2024)

HMoE: Heterogeneous Mixture of Experts for Language Modeling. - An Wang; Xingwu Sun; Ruobing Xie; Shuaipeng Li; Jiaqi Zhu 004; Zhen Yang; Pinxue Zhao; J. N. Han; Zhanhui Kang; Di Wang; Naoaki Okazaki; Cheng-Zhong Xu 001 (CoRR, 2024)

MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training. - Pinxue Zhao; Hailin Zhang 004; Fangcheng Fu; Xiaonan Nie; Qibin Liu; Fang Yang; Yuanbo Peng; Dian Jiao; Shuaipeng Li; Jinbao Xue; Yangyu Tao; Bin Cui 001 (Proc. ACM Manag. Data, 2025)