NEZHA: Neural Contextualized Representation for Chinese Language Understanding. - Junqiu Wei 001; Xiaozhe Ren; Xiaoguang Li; Wenyong Huang; Yi Liao; Yasheng Wang; Jiashu Lin; Xin Jiang 002; Xiao Chen 012; Qun Liu 001 (CoRR, 2019)

EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation. - Chenhe Dong; Guangrun Wang; Hang Xu 004; Jiefeng Peng; Xiaozhe Ren; Xiaodan Liang (EMNLP (Findings), 2021)

SparseBERT: Rethinking the Importance Analysis in Self-attention. - Han Shi; Jiahui Gao; Xiaozhe Ren; Hang Xu 004; Xiaodan Liang; Zhenguo Li; James Tin-Yau Kwok (ICML, 2021)

PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation. - Wei Zeng 006; Xiaozhe Ren; Teng Su; Hui Wang 030; Yi Liao; Zhiwei Wang; Xin Jiang 002; ZhenZhang Yang; Kaisheng Wang; Xiaoda Zhang; Chen Li 023; Ziyan Gong; et al. (CoRR, 2021)

NumGPT: Improving Numeracy Ability of Generative Pre-trained Models. - Zhihua Jin; Xin Jiang 002; Xingbo Wang 001; Qun Liu 001; Yong Wang 021; Xiaozhe Ren; Huamin Qu (CoRR, 2021)

Large-Scale Deep Learning Optimizations: A Comprehensive Survey. - Xiaoxin He; Fuzhao Xue; Xiaozhe Ren; Yang You 001 (CoRR, 2021)

AutoBERT-Zero: Evolving BERT Backbone from Scratch. - Jiahui Gao; Hang Xu 004; Han Shi; Xiaozhe Ren; Philip L. H. Yu; Xiaodan Liang; Xin Jiang 002; Zhenguo Li (AAAI, 2022)

Deeper vs Wider: A Revisit of Transformer Configuration. - Fuzhao Xue; Jianghai Chen; Aixin Sun; Xiaozhe Ren; Zangwei Zheng; Xiaoxin He; Xin Jiang 002; Yang You 001 (CoRR, 2022)

CAME: Confidence-guided Adaptive Memory Efficient Optimization. - Yang Luo; Xiaozhe Ren; Zangwei Zheng; Zhuo Jiang; Xin Jiang 002; Yang You 001 (ACL (1), 2023)

One Student Knows All Experts Know: From Sparse to Dense. - Fuzhao Xue; Xiaoxin He; Xiaozhe Ren; Yuxuan Lou; Yang You 001 (Tiny Papers @ ICLR, 2023)

A Study on Transformer Configuration and Training Objective. - Fuzhao Xue; Jianghai Chen; Aixin Sun; Xiaozhe Ren; Zangwei Zheng; Xiaoxin He; Yongming Chen; Xin Jiang 002; Yang You 001 (ICML, 2023)

Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline. - Zangwei Zheng; Xiaozhe Ren; Fuzhao Xue; Yang Luo; Xin Jiang 002; Yang You 001 (NeurIPS, 2023)

EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge. - Bufang Yang; Lixing He; Neiwen Ling; Zhenyu Yan 002; Guoliang Xing; Xian Shuai; Xiaozhe Ren; Xin Jiang 002 (SenSys, 2023)

PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing. - Xiaozhe Ren; Pingyi Zhou; Xinfan Meng; Xinjing Huang; Yadao Wang; Weichao Wang; Pengfei Li; Xiaoda Zhang; Alexander Podolskiy; Grigory Arshinov; Andrey Bout; Irina Piontkovskaya; et al. (CoRR, 2023)

A Survey of Reasoning with Foundation Models. - Jiankai Sun; Chuanyang Zheng; Enze Xie; Zhengying Liu; Ruihang Chu; Jianing Qiu; Jiaqi Xu; Mingyu Ding; Hongyang Li 001; Mengzhe Geng; Yue Wu 002; Wenhai Wang; et al. (CoRR, 2023)

PIXART-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation. - Junsong Chen; Chongjian Ge; Enze Xie; Yue Wu 002; Lewei Yao; Xiaozhe Ren; Zhongdao Wang; Ping Luo 002; Huchuan Lu; Zhenguo Li (ECCV (32), 2024)

ScheMoE: An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling. - Shaohuai Shi; Xinglin Pan; Qiang Wang 022; Chengjian Liu; Xiaozhe Ren; Zhongzhe Hu; Yu Yang; Bo Li 001; Xiaowen Chu 001 (EuroSys, 2024)

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation. - Chuanyang Zheng; Yihang Gao; Han Shi; Minbin Huang; Jingyao Li; Jing Xiong; Xiaozhe Ren; Michael K. Ng 001; Xin Jiang; Zhenguo Li; Yu Li (NeurIPS, 2024)

Poster Abstract: Tasking Heterogeneous Sensor Systems with LLMs. - Kaiwei Liu 001; Bufang Yang; Lilin Xu; Yunqi Guo; Neiwen Ling; Zhihe Zhao; Guoliang Xing; Xian Shuai; Xiaozhe Ren; Xin Jiang 002; Zhenyu Yan 002 (SenSys, 2024)

CAPE: Context-Adaptive Positional Encoding for Length Extrapolation. - Chuanyang Zheng; Yihang Gao; Han Shi; Minbin Huang; Jingyao Li; Jing Xiong; Xiaozhe Ren; Michael K. Ng 001; Xin Jiang; Zhenguo Li; Yu Li (CoRR, 2024)

Scaling Law for Language Models Training Considering Batch Size. - Xian Shuai; Yiding Wang; Yimeng Wu; Xin Jiang; Xiaozhe Ren (CoRR, 2024)

SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator. - Guoxuan Chen; Han Shi; Jiawei Li; Yihang Gao; Xiaozhe Ren; Yimeng Chen; Xin Jiang; Zhenguo Li; Weiyang Liu; Chao Huang (CoRR, 2024)

DAPE V2: Process Attention Score as Feature Map for Length Extrapolation. - Chuanyang Zheng; Yihang Gao; Han Shi; Jing Xiong; Jiankai Sun; Jingyao Li; Minbin Huang; Xiaozhe Ren; Michael Ng 001; Xin Jiang; Zhenguo Li; Yu Li (ACL (1), 2025)

TaskSense: A Translation-like Approach for Tasking Heterogeneous Sensor Systems with LLMs. - Kaiwei Liu 001; Bufang Yang; Lilin Xu; Yunqi Guo; Guoliang Xing; Xian Shuai; Xiaozhe Ren; Xin Jiang 002; Zhenyu Yan 002 (SenSys, 2025)

Self-Adjust Softmax. - Chuanyang Zheng; Yihang Gao; Guoxuan Chen; Han Shi; Jing Xiong; Xiaozhe Ren; Chao Huang; Xin Jiang; Zhenguo Li; Yu Li (CoRR, 2025)

Pangu DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning. - Wenxuan Shi; Haochen Tan; Chuqiao Kuang; Xiaoguang Li; Xiaozhe Ren; Chen Zhang; Hanting Chen; Yasheng Wang; Lifeng Shang; Fisher Yu 003; Yunhe Wang 001 (CoRR, 2025)

A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook. - Jiankai Sun; Chuanyang Zheng; Enze Xie; Zhengying Liu; Ruihang Chu; Jianing Qiu; Jiaqi Xu; Mingyu Ding; Hongyang Li 001; Mengzhe Geng; Yue Wu 012; Wenhai Wang; et al. (ACM Comput. Surv., 2025)