Data-Free Distillation of Language Model by Text-to-Text Transfer. - Zheyuan Bai; Xinduo Liu; Hailin Hu 002; Tianyu Guo 001; Qinghua Zhang; Yunhe Wang 001 (CoRR, 2023)

PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensation. - Yunhe Wang 001; Hanting Chen; Yehui Tang; Tianyu Guo 001; Kai Han 002; Ying Nie; Xutao Wang; Hailin Hu 002; Zheyuan Bai; Yun Wang; Fangcheng Liu; Zhicheng Liu 001; et al. (CoRR, 2023)

Multiscale Positive-Unlabeled Detection of AI-Generated Texts. - Yuchuan Tian; Hanting Chen; Xutao Wang; Zheyuan Bai; Qinghua Zhang; Ruifeng Li; Chao Xu 006; Yunhe Wang 001 (ICLR, 2024)

Rethinking Optimization and Architecture for Tiny Language Models. - Yehui Tang; Kai Han 002; Fangcheng Liu; Yunsheng Ni; Yuchuan Tian; Zheyuan Bai; Yi-Qi Hu; Sichao Liu; Shangling Jui; Yunhe Wang 001 (ICML, 2024)

Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs. - Hanting Chen; Jiarui Qin; Jialong Guo; Tao Yuan; Yichun Yin; Huiling Zhen; Yasheng Wang; Jinpeng Li 004; Xiaojun Meng; Meng Zhang; Rongju Ruan; Zheyuan Bai; et al. (CoRR, 2025)