CoLLiE: Collaborative Training of Large Language Models in an Efficient Way. - Kai Lv 001; Shuo Zhang 022; Tianle Gu; Shuhao Xing; Jiawei Hong; Keyu Chen; Xiaoran Liu; Yuqing Yang 004; Honglin Guo; Tengxiao Liu; Yu Sun 031; Qipeng Guo; et al. (EMNLP (Demos), 2023)

Scaling Laws of RoPE-based Extrapolation. - Xiaoran Liu; Hang Yan 001; Shuo Zhang 022; Chenxin An; Xipeng Qiu; Dahua Lin (CoRR, 2023)

InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning. - Huaiyuan Ying; Shuo Zhang 022; Linyang Li; Zhejian Zhou; Yunfan Shao; Zhaoye Fei; Yichuan Ma; Jiawei Hong; Kuikun Liu; Ziyi Wang 006; Yudong Wang 002; Zijian Wu 002; et al. (CoRR, 2024)

Efficient Training of Large Language Models on Distributed Infrastructures: A Survey. - Jiangfei Duan; Shuo Zhang 022; Zerui Wang; Lijuan Jiang; Wenwen Qu; Qinghao Hu 004; Guoteng Wang; Qizhen Weng; Hang Yan 001; Xingcheng Zhang; Xipeng Qiu; Dahua Lin; et al. (CoRR, 2024)