Neural Question Generation from Text: A Preliminary Study. - Qingyu Zhou; Nan Yang 002; Furu Wei; Chuanqi Tan; Hangbo Bao; Ming Zhou 001 (NLPCC, 2017)

Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension. - Hangbo Bao; Li Dong 004; Furu Wei; Wenhui Wang 003; Nan Yang 002; Lei Cui 001; Songhao Piao; Ming Zhou 001 (MRQA@EMNLP, 2019)

Neural Melody Composition from Lyrics. - Hangbo Bao; Shaohan Huang; Furu Wei; Lei Cui 001; Yu Wu 012; Chuanqi Tan; Songhao Piao; Ming Zhou 001 (NLPCC (1), 2019)

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training. - Hangbo Bao; Li Dong 004; Furu Wei; Wenhui Wang 003; Nan Yang 002; Xiaodong Liu 003; Yu Wang 009; Jianfeng Gao 001; Songhao Piao; Ming Zhou 001; Hsiao-Wuen Hon (ICML, 2020)

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. - Wenhui Wang 003; Furu Wei; Li Dong 004; Hangbo Bao; Nan Yang 002; Ming Zhou 001 (NeurIPS, 2020)

Learning to Sample Replacements for ELECTRA Pre-Training. - Yaru Hao; Li Dong 004; Hangbo Bao; Ke Xu 001; Furu Wei (ACL/IJCNLP (Findings), 2021)

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers. - Wenhui Wang 003; Hangbo Bao; Shaohan Huang; Li Dong 004; Furu Wei (ACL/IJCNLP (Findings), 2021)

BEiT: BERT Pre-Training of Image Transformers. - Hangbo Bao; Li Dong 004; Furu Wei (CoRR, 2021)

s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning. - Hangbo Bao; Li Dong 004; Wenhui Wang 003; Nan Yang 002; Furu Wei (CoRR, 2021)

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts. - Wenhui Wang 003; Hangbo Bao; Li Dong 004; Furu Wei (CoRR, 2021)

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption. - Tianyu Chen; Hangbo Bao; Shaohan Huang; Li Dong 004; Binxing Jiao; Daxin Jiang; Haoyi Zhou; Jianxin Li 002; Furu Wei (ACL (Findings), 2022)

Attention Temperature Matters in Abstractive Summarization Distillation. - Shengqiang Zhang; Xingxing Zhang 002; Hangbo Bao; Furu Wei (ACL (1), 2022)

BEiT: BERT Pre-Training of Image Transformers. - Hangbo Bao; Li Dong 004; Songhao Piao; Furu Wei (ICLR, 2022)

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts. - Hangbo Bao; Wenhui Wang 003; Li Dong 004; Qiang Liu; Owais Khan Mohammed; Kriti Aggarwal; Subhojit Som; Songhao Piao; Furu Wei (NeurIPS, 2022)

VL-BEiT: Generative Vision-Language Pretraining. - Hangbo Bao; Wenhui Wang 003; Li Dong 004; Furu Wei (CoRR, 2022)

BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers. - Zhiliang Peng; Li Dong 004; Hangbo Bao; Qixiang Ye; Furu Wei (CoRR, 2022)

Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks. - Wenhui Wang 003; Hangbo Bao; Li Dong 004; Johan Bjorck; Zhiliang Peng; Qiang Liu; Kriti Aggarwal; Owais Khan Mohammed; Saksham Singhal; Subhojit Som; Furu Wei (CoRR, 2022)

Image as a Foreign Language: BEIT Pretraining for Vision and Vision-Language Tasks. - Wenhui Wang 003; Hangbo Bao; Li Dong 004; Johan Bjorck; Zhiliang Peng; Qiang Liu; Kriti Aggarwal; Owais Khan Mohammed; Saksham Singhal; Subhojit Som; Furu Wei (CVPR, 2023)

Corrupted Image Modeling for Self-Supervised Visual Pre-Training. - Yuxin Fang; Li Dong 004; Hangbo Bao; Xinggang Wang; Furu Wei (ICLR, 2023)

A Unified View of Masked Image Modeling. - Zhiliang Peng; Li Dong 004; Hangbo Bao; Furu Wei; Qixiang Ye (Trans. Mach. Learn. Res., 2023)

Multimodal Latent Language Modeling with Next-Token Diffusion. - Yutao Sun; Hangbo Bao; Wenhui Wang 003; Zhiliang Peng; Li Dong 004; Shaohan Huang; Jianyong Wang 001; Furu Wei (CoRR, 2024)

Fine-tuning pretrained transformer encoders for sequence-to-sequence learning. - Hangbo Bao; Li Dong 004; Wenhui Wang 003; Nan Yang 002; Songhao Piao; Furu Wei (Int. J. Mach. Learn. Cybern., 2024)