Elastic Deep Learning Using Knowledge Distillation with Heterogeneous Computing Resources. - Daxiang Dong; Ji Liu 003; Xi Wang; Weibao Gong; An Qin 001; Xingjian Li 002; Dianhai Yu; Patrick Valduriez; Dejing Dou (Euro-Par Workshops, 2021)

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation. - Yu Sun 004; Shuohuan Wang; Shikun Feng; Siyu Ding; Chao Pang; Junyuan Shang; Jiaxiang Liu 004; Xuyi Chen; Yanbin Zhao; Yuxiang Lu; Weixin Liu; Zhihua Wu; et al. (CoRR, 2021)

End-to-end Adaptive Distributed Training on PaddlePaddle. - Yulong Ao; Zhihua Wu; Dianhai Yu; Weibao Gong; Zhiqing Kui; Minxu Zhang; Zilingfeng Ye; Liang Shen; Yanjun Ma; Tian Wu; Haifeng Wang; Wei Zeng; et al. (CoRR, 2021)

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation. - Shuohuan Wang; Yu Sun 004; Yang Xiang; Zhihua Wu; Siyu Ding; Weibao Gong; Shikun Feng; Junyuan Shang; Yanbin Zhao; Chao Pang; Jiaxiang Liu 004; Xuyi Chen; et al. (CoRR, 2021)

Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters. - Yang Xiang 001; Zhihua Wu; Weibao Gong; Siyu Ding; Xianjie Mo; Yuang Liu; Shuohuan Wang; Peng Liu 024; Yongshuai Hou; Long Li; Bin Wang; Shaohuai Shi; et al. (CoRR, 2022)

SE-MoE: A Scalable and Efficient Mixture-of-Experts Distributed Training and Inference System. - Liang Shen; Zhihua Wu; Weibao Gong; Hongxiang Hao; Yangfan Bai; HuaChao Wu; Xinxuan Wu; Haoyi Xiong; Dianhai Yu; Yanjun Ma (CoRR, 2022)

MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services. - Dianhai Yu; Liang Shen; Hongxiang Hao; Weibao Gong; HuaChao Wu; Jiang Bian 003; Lirong Dai 001; Haoyi Xiong (IEEE Trans. Serv. Comput., 2024)