Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing. - Haoming Jiang; Chen Liang 006; Chong Wang; Tuo Zhao (ACL, 2020)

BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision. - Chen Liang 006; Yue Yu 001; Haoming Jiang; Siawpeng Er; Ruijia Wang; Tuo Zhao; Chao Zhang 014 (KDD, 2020)

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization. - Chen Liang 006; Simiao Zuo; Minshuo Chen; Haoming Jiang; Xiaodong Liu 003; Pengcheng He; Tuo Zhao; Weizhu Chen (ACL/IJCNLP (1), 2021)

Token-wise Curriculum Learning for Neural Machine Translation. - Chen Liang 006; Haoming Jiang; Xiaodong Liu 003; Pengcheng He; Weizhu Chen; Jianfeng Gao 001; Tuo Zhao (EMNLP (Findings), 2021)

ARCH: Efficient Adversarial Regularized Training with Caching. - Simiao Zuo; Chen Liang 006; Haoming Jiang; Pengcheng He; Xiaodong Liu 003; Jianfeng Gao 001; Weizhu Chen; Tuo Zhao (EMNLP (Findings), 2021)

Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach. - Simiao Zuo; Chen Liang 006; Haoming Jiang; Xiaodong Liu 003; Pengcheng He; Jianfeng Gao 001; Weizhu Chen; Tuo Zhao (EMNLP (1), 2021)

Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach. - Simiao Zuo; Chen Liang 006; Haoming Jiang; Xiaodong Liu 003; Pengcheng He; Jianfeng Gao 001; Weizhu Chen; Tuo Zhao (CoRR, 2021)

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing. - Chen Liang 006; Pengcheng He; Yelong Shen; Weizhu Chen; Tuo Zhao (ACL (1), 2022)

No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models. - Chen Liang 006; Haoming Jiang; Simiao Zuo; Pengcheng He; Xiaodong Liu 003; Jianfeng Gao 001; Weizhu Chen; Tuo Zhao (ICLR, 2022)

PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance. - Qingru Zhang; Simiao Zuo; Chen Liang 006; Alexander Bukharin; Pengcheng He; Weizhu Chen; Tuo Zhao (ICML, 2022)

Self-Training with Differentiable Teacher. - Simiao Zuo; Yue Yu 001; Chen Liang 006; Haoming Jiang; Siawpeng Er; Chao Zhang 014; Tuo Zhao; Hongyuan Zha (NAACL-HLT (Findings), 2022)

MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation. - Simiao Zuo; Qingru Zhang; Chen Liang 006; Pengcheng He; Tuo Zhao; Weizhu Chen (NAACL-HLT, 2022)

HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers. - Chen Liang 006; Haoming Jiang; Zheng Li 018; Xianfeng Tang; Bing Yin; Tuo Zhao (ICLR, 2023)

LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation. - Yixiao Li; Yifan Yu 008; Qingru Zhang; Chen Liang 006; Pengcheng He; Weizhu Chen; Tuo Zhao (ICML, 2023)

Less is More: Task-aware Layer-wise Distillation for Language Model Compression. - Chen Liang 006; Simiao Zuo; Qingru Zhang; Pengcheng He; Weizhu Chen; Tuo Zhao (ICML, 2023)

Module-wise Adaptive Distillation for Multimodality Foundation Models. - Chen Liang 006; Jiahui Yu; Ming-Hsuan Yang 001; Matthew Brown 001; Yin Cui; Tuo Zhao; Boqing Gong; Tianyi Zhou 002 (NeurIPS, 2023)

HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers. - Chen Liang 006; Haoming Jiang; Zheng Li 018; Xianfeng Tang; Bin Yin; Tuo Zhao (CoRR, 2023)

LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models. - Yixiao Li; Yifan Yu 008; Chen Liang 006; Nikos Karampatziakis; Pengcheng He; Weizhu Chen; Tuo Zhao (ICLR, 2024)

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. - Marah I Abdin; Sam Ade Jacobs; Ammar Ahmad Awan; Jyoti Aneja; Ahmed Awadallah 001; Hany Awadalla; Nguyen Bach; Amit Bahree; Arash Bakhtiari; Harkirat S. Behl; Alon Benhaim; Misha Bilenko; et al. (CoRR, 2024)

GRIN: GRadient-INformed MoE. - Liyuan Liu; Young Jin Kim 001; Shuohang Wang; Chen Liang 006; Yelong Shen; Hao Cheng 002; Xiaodong Liu 003; Masahiro Tanaka; Xiaoxia Wu; Wenxiang Hu; Vishrav Chaudhary; Zeqi Lin; et al. (CoRR, 2024)

On Parameter Efficiency of Neural Language Models. - Chen Liang 006 (2024)

Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling. - Liliang Ren; Yang Liu; Yadong Lu; Yelong Shen; Chen Liang 006; Weizhu Chen (ICLR, 2025)

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs. - Abdelrahman Abouelenin; Atabak Ashfaq; Adam Atkinson; Hany Awadalla; Nguyen Bach; Jianmin Bao; Alon Benhaim; Martin Cai; Vishrav Chaudhary; Congcong Chen; Dong Chen 003; Dongdong Chen 001; et al. (CoRR, 2025)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation. - Zichong Li; Chen Liang 006; Zixuan Zhang; Ilgee Hong; Young Jin Kim 001; Weizhu Chen; Tuo Zhao (CoRR, 2025)